2024-09-10T22:32:18.432 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T22:32:18.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:32:18.557 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899745' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0RHjmVgckUa8xEq/Ln5zZSIoh7fLYZX1xP71EpW3WkpArKSO1igUWWl1agndCMWona4dQXdo2wmrHNI09tAww= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFHffcnFyhucunupRdbMjElCumHZls5U6Url/DLVm13X0OtDr7AUjYu1Q8z/zCG2ZTs/1fGPfuSUAa8sBpjSlho= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMgRXD2GycNgNnv4MYtRTkY/gLngNhsaU7imgVbrBWnAeK8voqNb4QP0V2BicO7XE//A5akNtQIvC2q0AvhdvIg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T22:32:18.558 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T22:32:18.558 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T22:32:18.559 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T22:32:18.560 INFO:teuthology.task.internal:Checking packages... 2024-09-10T22:32:18.581 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T22:32:18.581 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T22:32:18.581 INFO:teuthology.packaging:ref: None 2024-09-10T22:32:18.581 INFO:teuthology.packaging:tag: None 2024-09-10T22:32:18.581 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:32:18.581 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:32:18.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:32:18.732 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T22:32:18.734 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T22:32:18.741 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T22:32:18.741 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T22:32:18.759 INFO:teuthology.task.internal:Saving configuration 2024-09-10T22:32:18.778 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T22:32:18.785 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T22:32:18.819 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:27:17.355218', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0RHjmVgckUa8xEq/Ln5zZSIoh7fLYZX1xP71EpW3WkpArKSO1igUWWl1agndCMWona4dQXdo2wmrHNI09tAww='} 2024-09-10T22:32:18.850 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:27:17.353576', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFHffcnFyhucunupRdbMjElCumHZls5U6Url/DLVm13X0OtDr7AUjYu1Q8z/zCG2ZTs/1fGPfuSUAa8sBpjSlho='} 2024-09-10T22:32:18.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:27:17.356607', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMgRXD2GycNgNnv4MYtRTkY/gLngNhsaU7imgVbrBWnAeK8voqNb4QP0V2BicO7XE//A5akNtQIvC2q0AvhdvIg='} 2024-09-10T22:32:18.884 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T22:32:18.891 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T22:32:18.891 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T22:32:18.891 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T22:32:18.892 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T22:32:19.005 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f55a9734c10>, signals=[15]) 2024-09-10T22:32:19.005 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T22:32:19.012 INFO:teuthology.task.internal:Opening connections... 2024-09-10T22:32:19.012 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T22:32:19.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:32:19.097 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-09-10T22:32:19.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:32:19.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-09-10T22:32:19.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:32:19.311 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T22:32:19.318 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-09-10T22:32:19.337 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-09-10T22:32:19.337 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-09-10T22:32:19.395 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-09-10T22:32:19.395 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-09-10T22:32:19.395 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:32:19.396 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:32:19.397 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-09-10T22:32:19.428 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-09-10T22:32:19.446 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-09-10T22:32:19.447 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:32:19.505 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:32:19.506 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:32:19.506 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:32:19.506 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-09-10T22:32:19.536 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-09-10T22:32:19.554 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-09-10T22:32:19.555 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2024-09-10T22:32:19.612 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:32:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:32:19.616 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:32:19.617 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-09-10T22:32:19.644 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T22:32:19.656 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T22:32:19.689 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T22:32:19.690 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:32:19.692 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:32:19.695 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:32:19.712 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T22:32:19.718 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T22:32:19.718 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:32:19.751 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:32:19.753 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:32:19.856 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T22:32:19.960 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T22:32:19.976 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T22:32:19.976 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T22:32:19.976 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T22:32:19.976 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:32:19.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:32:19.994 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-10T22:32:20.053 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:32:20.053 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-503.el9.x86_64 2024-09-10T22:32:20.053 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-09-10T22:32:22.620 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-10T22:32:24.529 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T22:32:28.822 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-10T22:32:30.298 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-10T22:32:31.560 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-10T22:32:37.008 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T22:32:39.039 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T22:32:39.040 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T22:32:39.147 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-10T22:32:39.157 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T22:32:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-09-10T22:32:39.157 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T22:32:39.157 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-10T22:32:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-09-10T22:32:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:32:39.159 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 79 M 2024-09-10T22:32:39.159 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 126 M 2024-09-10T22:32:39.159 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-10T22:32:39.376 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 227 kB/s | 23 kB 00:00 2024-09-10T22:32:39.768 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-10T22:32:40.127 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-10T22:32:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-10T22:32:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:32:40.444 INFO:teuthology.orchestra.run.smithi120.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-10T22:32:41.123 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-10T22:32:41.480 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-10T22:32:41.480 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-10T22:32:42.430 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-10T22:32:42.430 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-10T22:32:44.884 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-10T22:32:45.622 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:32:45.713 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:32:48.514 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:32:48.647 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:32:53.494 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:32:53.547 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:32:58.400 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:33:36.871 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:33:36.911 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:33:37.197 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:33:37.197 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:33:37.197 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:33:37.198 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:33:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:33:38.058 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:33:38.059 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-10T22:33:38.410 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:33:38.500 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.501 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:33:38.501 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T22:33:38.501 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T22:33:38.501 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T22:33:38.501 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T22:33:38.501 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T22:33:38.501 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:33:38.559 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:33:38.559 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:33:38.559 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 82.el9 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-09-10T22:33:38.596 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8274670 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-09-10T22:33:38.597 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:33:38.598 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:33:38.599 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:33:38.599 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-505.el9.x86_64 2024-09-10T22:33:38.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:33:39.306 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-09-10T22:33:40.580 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:33:40.601 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-09-10T22:33:40.603 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-09-10T22:33:40.619 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.liaSNA3keN 2024-09-10T22:33:40.619 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.liaSNA3keN 2024-09-10T22:33:40.687 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.liaSNA3keN 2024-09-10T22:33:40.828 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.liaSNA3keN is 6KB 2024-09-10T22:33:40.877 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.liaSNA3keN 2024-09-10T22:33:40.892 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:33:40.955 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:33:40.955 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:33:40.955 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:33:40.955 INFO:teuthology.orchestra.run.smithi120.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T22:33:40.957 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T22:33:41.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-09-10T22:34:11.115 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T22:34:11.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:34:11.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:34:29.569 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-10T22:34:38.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:34:38.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:34:41.665 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-09-10T22:34:53.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:34:53.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:35:25.486 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-10T22:35:25.814 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:35:25.814 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T22:35:25.814 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-09-10T22:35:25.829 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-505.el9.x86_64 2024-09-10T22:35:25.829 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T22:35:25.829 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:35:25.829 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:35:26.832 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:26.832 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T22:35:26.839 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T22:35:26.839 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:35:26.842 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:35:26.845 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:35:26.863 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T22:35:26.870 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T22:35:26.883 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T22:35:26.883 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:35:26.898 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:35:26.903 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:35:27.060 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T22:35:27.067 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T22:35:27.067 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:35:27.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:27.086 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:35:27.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:27.103 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:35:27.105 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:35:27.128 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:35:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.178 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.212 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.213 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.219 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.233 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:35:27.236 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T22:35:27.246 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T22:35:27.246 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:35:27.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:35:27.258 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:35:27.286 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T22:35:27.295 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T22:35:27.295 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:35:27.314 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:35:27.326 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:35:27.344 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:27.418 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:27.492 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:27.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:27.573 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:35:27.573 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:35:27.638 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:27.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:27.731 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:27.822 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:27.855 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:35:27.855 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:35:27.921 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:27.973 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:35:28.029 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:28.080 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:35:28.117 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:35:28.117 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:35:28.184 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-09-10T22:35:28.186 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-09-10T22:35:28.189 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-09-10T22:35:28.221 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:35:28.260 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:35:28.271 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:35:28.743 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T22:35:28.751 INFO:teuthology.task.internal:Starting timer... 2024-09-10T22:35:28.751 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T22:35:28.761 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T22:35:28.769 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T22:35:28.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-09-10T22:35:28.900 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-09-10T22:35:28.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-09-10T22:35:28.949 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-09-10T22:35:28.951 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2024-09-10T22:35:29.019 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2024-09-10T22:35:29.021 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T22:35:29.021 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-09-10T22:35:29.043 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-09-10T22:35:29.044 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-09-10T22:35:29.079 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-09-10T22:35:29.079 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2024-09-10T22:35:29.098 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2024-09-10T22:35:29.099 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi176.front.sepia.ceph.com': 'permissive'} 2024-09-10T22:35:29.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:35:29.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:29.124 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:35:29.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:29.153 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:35:29.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:29.179 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T22:35:29.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:35:29.209 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:35:29.240 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:35:29.271 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T22:35:29.281 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T22:35:29.286 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T22:35:29.440 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T22:35:29.452 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T22:35:29.453 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi120.front.sepia.ceph.com,smithi153.front.sepia.ceph.com,smithi176.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T22:39:28.145 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com')] 2024-09-10T22:39:28.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:39:28.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:39:28.233 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-09-10T22:39:28.311 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-09-10T22:39:28.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-10T22:39:28.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:39:28.394 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-09-10T22:39:28.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-09-10T22:39:28.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-10T22:39:28.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:39:28.551 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-10T22:39:28.631 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-10T22:39:28.632 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T22:39:28.642 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T22:39:28.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:39:28.643 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:39:28.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:39:28.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:39:28.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:39:28.647 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:39:28.674 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:39:28.675 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:39:28.689 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:39:28.691 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:39:28.701 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:39:28.713 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-09-10T22:39:28.716 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-09-10T22:39:28.716 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:39:28.725 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-09-10T22:39:28.727 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-09-10T22:39:28.739 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:39:28.741 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2024-09-10T22:39:28.742 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:39:28.752 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2024-09-10T22:39:28.753 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:39:28.756 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:39:28.767 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:39:28.781 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:39:28.811 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:39:28.811 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.815 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.817 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.840 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:39:28.843 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:39:28.844 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-09-10T22:39:28.844 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.844 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.844 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.844 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:39:28.845 INFO:teuthology.run_tasks:Running task install... 2024-09-10T22:39:28.853 DEBUG:teuthology.task.install:project ceph 2024-09-10T22:39:28.854 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T22:39:28.854 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T22:39:28.854 INFO:teuthology.task.install:Using flavor: default 2024-09-10T22:39:28.859 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T22:39:28.859 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T22:39:28.860 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:39:28.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:39:28.863 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:39:28.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:39:28.864 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:39:28.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:39:29.024 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:39:29.024 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:39:29.064 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:39:29.065 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:39:29.066 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:39:29.066 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:39:29.173 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:39:29.173 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:39:29.173 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:39:29.184 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:39:29.184 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:39:29.184 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:39:29.194 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:39:29.194 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:39:29.194 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:39:29.199 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:39:29.199 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:39:29.212 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:39:29.212 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:39:29.220 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:39:29.220 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:39:29.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:39:29.278 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:39:29.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:39:29.339 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:39:29.353 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:39:29.361 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:39:29.404 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-09-10T22:39:29.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-09-10T22:39:29.417 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2024-09-10T22:39:29.418 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2024-09-10T22:39:29.426 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-09-10T22:39:29.427 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-09-10T22:39:29.634 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-09-10T22:39:29.643 INFO:teuthology.orchestra.run.smithi176.stdout:45 files removed 2024-09-10T22:39:29.653 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-09-10T22:39:29.661 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:39:29.671 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:39:29.680 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:39:30.218 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-10T22:39:30.221 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-10T22:39:30.230 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-10T22:39:30.510 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-10T22:39:30.513 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 96 kB/s | 24 kB 00:00 2024-09-10T22:39:30.514 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-10T22:39:30.782 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-10T22:39:30.786 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T22:39:30.789 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T22:39:31.431 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-10T22:39:31.498 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T22:39:31.628 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-10T22:39:33.408 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T22:39:37.942 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-10T22:39:38.715 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 20 MB 00:06 2024-09-10T22:39:39.104 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-10T22:39:39.509 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 19 kB 00:00 2024-09-10T22:39:40.680 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T22:39:43.015 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-10T22:39:44.280 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.5 MB 00:01 2024-09-10T22:39:44.866 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-10T22:39:45.912 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-10T22:39:46.144 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-10T22:39:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-10T22:39:47.660 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T22:39:48.440 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:48.441 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:48.454 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:39:48.455 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:39:48.456 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:39:48.457 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:39:48.459 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:39:48.462 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:39:48.463 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:39:48.526 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-09-10T22:39:48.536 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-10T22:39:48.536 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:39:48.537 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:39:48.538 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:39:48.539 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:39:48.540 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:39:48.541 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:39:48.542 INFO:teuthology.orchestra.run.smithi153.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:39:48.543 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:39:48.544 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:39:48.545 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:39:48.546 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:39:48.547 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:39:48.548 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:39:48.549 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:Install 149 Packages 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 8 Packages 2024-09-10T22:39:48.550 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:39:48.551 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 240 M 2024-09-10T22:39:48.551 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-09-10T22:39:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 47 kB/s | 6.6 kB 00:00 2024-09-10T22:39:49.562 INFO:teuthology.orchestra.run.smithi153.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.1 MB/s | 900 kB 00:00 2024-09-10T22:39:49.663 INFO:teuthology.orchestra.run.smithi153.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.4 MB/s | 145 kB 00:00 2024-09-10T22:39:49.763 INFO:teuthology.orchestra.run.smithi153.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 9.9 MB/s | 5.5 MB 00:00 2024-09-10T22:39:49.930 INFO:teuthology.orchestra.run.smithi153.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 9.2 MB/s | 1.5 MB 00:00 2024-09-10T22:39:49.972 INFO:teuthology.orchestra.run.smithi153.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-09-10T22:39:50.155 INFO:teuthology.orchestra.run.smithi153.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 21 MB/s | 4.8 MB 00:00 2024-09-10T22:39:50.364 INFO:teuthology.orchestra.run.smithi153.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 19 MB/s | 22 MB 00:01 2024-09-10T22:39:50.406 INFO:teuthology.orchestra.run.smithi153.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 606 kB/s | 25 kB 00:00 2024-09-10T22:39:50.773 INFO:teuthology.orchestra.run.smithi153.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 18 MB/s | 11 MB 00:00 2024-09-10T22:39:50.831 INFO:teuthology.orchestra.run.smithi153.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.8 MB/s | 225 kB 00:00 2024-09-10T22:39:50.898 INFO:teuthology.orchestra.run.smithi153.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 407 kB/s | 27 kB 00:00 2024-09-10T22:39:50.965 INFO:teuthology.orchestra.run.smithi153.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 476 kB/s | 32 kB 00:00 2024-09-10T22:39:51.124 INFO:teuthology.orchestra.run.smithi153.stdout:(14/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 15 MB/s | 17 MB 00:01 2024-09-10T22:39:51.157 INFO:teuthology.orchestra.run.smithi153.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 3.8 MB/s | 755 kB 00:00 2024-09-10T22:39:51.183 INFO:teuthology.orchestra.run.smithi153.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.7 MB/s | 162 kB 00:00 2024-09-10T22:39:51.208 INFO:teuthology.orchestra.run.smithi153.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T22:39:51.233 INFO:teuthology.orchestra.run.smithi153.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 9.9 MB/s | 507 kB 00:00 2024-09-10T22:39:51.292 INFO:teuthology.orchestra.run.smithi153.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 773 kB/s | 45 kB 00:00 2024-09-10T22:39:51.384 INFO:teuthology.orchestra.run.smithi153.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.5 MB/s | 143 kB 00:00 2024-09-10T22:39:51.467 INFO:teuthology.orchestra.run.smithi153.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 21 MB/s | 5.5 MB 00:00 2024-09-10T22:39:51.468 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T22:39:51.492 INFO:teuthology.orchestra.run.smithi153.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.5 MB/s | 161 kB 00:00 2024-09-10T22:39:51.526 INFO:teuthology.orchestra.run.smithi153.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 5.4 MB/s | 321 kB 00:00 2024-09-10T22:39:51.551 INFO:teuthology.orchestra.run.smithi153.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T22:39:51.585 INFO:teuthology.orchestra.run.smithi153.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-10T22:39:51.610 INFO:teuthology.orchestra.run.smithi153.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.4 MB/s | 85 kB 00:00 2024-09-10T22:39:51.669 INFO:teuthology.orchestra.run.smithi153.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T22:39:51.744 INFO:teuthology.orchestra.run.smithi153.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 20 MB/s | 3.2 MB 00:00 2024-09-10T22:39:51.770 INFO:teuthology.orchestra.run.smithi153.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 342 kB/s | 34 kB 00:00 2024-09-10T22:39:51.786 INFO:teuthology.orchestra.run.smithi153.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.6 MB/s | 154 kB 00:00 2024-09-10T22:39:52.170 INFO:teuthology.orchestra.run.smithi153.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 19 MB/s | 7.4 MB 00:00 2024-09-10T22:39:52.254 INFO:teuthology.orchestra.run.smithi153.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.4 MB/s | 291 kB 00:00 2024-09-10T22:39:52.321 INFO:teuthology.orchestra.run.smithi153.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 740 kB/s | 49 kB 00:00 2024-09-10T22:39:52.387 INFO:teuthology.orchestra.run.smithi153.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 246 kB/s | 16 kB 00:00 2024-09-10T22:39:52.463 INFO:teuthology.orchestra.run.smithi153.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.7 MB/s | 285 kB 00:00 2024-09-10T22:39:52.630 INFO:teuthology.orchestra.run.smithi153.stdout:(36/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 16 MB/s | 14 MB 00:00 2024-09-10T22:39:52.663 INFO:teuthology.orchestra.run.smithi153.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 3.8 MB/s | 783 kB 00:00 2024-09-10T22:39:52.688 INFO:teuthology.orchestra.run.smithi153.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 314 kB/s | 18 kB 00:00 2024-09-10T22:39:53.091 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T22:39:53.264 INFO:teuthology.orchestra.run.smithi153.stdout:(39/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 18 MB/s | 50 MB 00:02 2024-09-10T22:39:53.297 INFO:teuthology.orchestra.run.smithi153.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 126 kB/s | 80 kB 00:00 2024-09-10T22:39:53.389 INFO:teuthology.orchestra.run.smithi153.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T22:39:53.415 INFO:teuthology.orchestra.run.smithi153.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-10T22:39:53.457 INFO:teuthology.orchestra.run.smithi153.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 995 kB/s | 191 kB 00:00 2024-09-10T22:39:53.507 INFO:teuthology.orchestra.run.smithi153.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-10T22:39:53.532 INFO:teuthology.orchestra.run.smithi153.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 909 kB/s | 106 kB 00:00 2024-09-10T22:39:53.558 INFO:teuthology.orchestra.run.smithi153.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-10T22:39:53.599 INFO:teuthology.orchestra.run.smithi153.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-10T22:39:53.641 INFO:teuthology.orchestra.run.smithi153.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-10T22:39:53.683 INFO:teuthology.orchestra.run.smithi153.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-09-10T22:39:53.725 INFO:teuthology.orchestra.run.smithi153.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-09-10T22:39:53.753 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:53.754 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:53.759 INFO:teuthology.orchestra.run.smithi153.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-10T22:39:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:39:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:39:53.767 INFO:teuthology.orchestra.run.smithi176.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:39:53.768 INFO:teuthology.orchestra.run.smithi176.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:39:53.769 INFO:teuthology.orchestra.run.smithi176.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:39:53.772 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:39:53.773 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:39:53.784 INFO:teuthology.orchestra.run.smithi153.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-09-10T22:39:53.826 INFO:teuthology.orchestra.run.smithi153.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-09-10T22:39:53.837 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-09-10T22:39:53.851 INFO:teuthology.orchestra.run.smithi153.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:39:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:39:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:39:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:39:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:39:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:39:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:39:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:39:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:39:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:39:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:39:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout:Install 149 Packages 2024-09-10T22:39:53.875 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 8 Packages 2024-09-10T22:39:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:39:53.876 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 240 M 2024-09-10T22:39:53.876 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-09-10T22:39:53.918 INFO:teuthology.orchestra.run.smithi153.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-09-10T22:39:53.943 INFO:teuthology.orchestra.run.smithi153.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-10T22:39:54.052 INFO:teuthology.orchestra.run.smithi153.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-10T22:39:54.077 INFO:teuthology.orchestra.run.smithi153.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2024-09-10T22:39:54.169 INFO:teuthology.orchestra.run.smithi153.stdout:(59/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.8 MB/s | 3.0 MB 00:00 2024-09-10T22:39:54.194 INFO:teuthology.orchestra.run.smithi153.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 318 kB/s | 45 kB 00:00 2024-09-10T22:39:54.219 INFO:teuthology.orchestra.run.smithi153.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-10T22:39:54.287 INFO:teuthology.orchestra.run.smithi153.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-10T22:39:54.311 INFO:teuthology.orchestra.run.smithi153.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 209 kB/s | 24 kB 00:00 2024-09-10T22:39:54.337 INFO:teuthology.orchestra.run.smithi153.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 404 kB/s | 47 kB 00:00 2024-09-10T22:39:54.387 INFO:teuthology.orchestra.run.smithi153.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T22:39:54.405 INFO:teuthology.orchestra.run.smithi153.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 574 kB/s | 38 kB 00:00 2024-09-10T22:39:54.430 INFO:teuthology.orchestra.run.smithi153.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T22:39:54.580 INFO:teuthology.orchestra.run.smithi153.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-10T22:39:54.622 INFO:teuthology.orchestra.run.smithi153.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-10T22:39:54.689 INFO:teuthology.orchestra.run.smithi153.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-10T22:39:54.714 INFO:teuthology.orchestra.run.smithi153.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T22:39:54.748 INFO:teuthology.orchestra.run.smithi153.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-10T22:39:54.831 INFO:teuthology.orchestra.run.smithi153.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-10T22:39:54.865 INFO:teuthology.orchestra.run.smithi153.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-09-10T22:39:54.894 INFO:teuthology.orchestra.run.smithi176.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 44 kB/s | 6.6 kB 00:00 2024-09-10T22:39:54.940 INFO:teuthology.orchestra.run.smithi153.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-10T22:39:54.965 INFO:teuthology.orchestra.run.smithi153.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 610 kB/s | 61 kB 00:00 2024-09-10T22:39:55.074 INFO:teuthology.orchestra.run.smithi153.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-09-10T22:39:55.099 INFO:teuthology.orchestra.run.smithi153.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-10T22:39:55.124 INFO:teuthology.orchestra.run.smithi153.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-10T22:39:55.149 INFO:teuthology.orchestra.run.smithi153.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-09-10T22:39:55.200 INFO:teuthology.orchestra.run.smithi153.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-09-10T22:39:55.225 INFO:teuthology.orchestra.run.smithi153.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-09-10T22:39:55.236 INFO:teuthology.orchestra.run.smithi176.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 2.6 MB/s | 900 kB 00:00 2024-09-10T22:39:55.250 INFO:teuthology.orchestra.run.smithi153.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-10T22:39:55.301 INFO:teuthology.orchestra.run.smithi153.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-10T22:39:55.342 INFO:teuthology.orchestra.run.smithi153.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-10T22:39:55.353 INFO:teuthology.orchestra.run.smithi176.stdout:(3/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 9.0 MB/s | 5.5 MB 00:00 2024-09-10T22:39:55.378 INFO:teuthology.orchestra.run.smithi176.stdout:(4/157): ceph-immutable-object-cache-19.3.0-485 1.0 MB/s | 145 kB 00:00 2024-09-10T22:39:55.384 INFO:teuthology.orchestra.run.smithi153.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 77 kB 00:00 2024-09-10T22:39:55.434 INFO:teuthology.orchestra.run.smithi153.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-10T22:39:55.437 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:55.438 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:39:55.451 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:39:55.452 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:39:55.453 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:39:55.454 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:39:55.456 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:39:55.459 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:39:55.459 INFO:teuthology.orchestra.run.smithi153.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-10T22:39:55.460 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:39:55.520 INFO:teuthology.orchestra.run.smithi176.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 14 MB/s | 2.4 MB 00:00 2024-09-10T22:39:55.524 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-09-10T22:39:55.527 INFO:teuthology.orchestra.run.smithi153.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:39:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:39:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:39:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:39:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:39:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:39:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:39:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:39:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:39:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:39:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:39:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:39:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:39:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:39:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout:Install 149 Packages 2024-09-10T22:39:55.564 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 8 Packages 2024-09-10T22:39:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:39:55.565 INFO:teuthology.orchestra.run.smithi153.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-10T22:39:55.565 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 240 M 2024-09-10T22:39:55.566 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-09-10T22:39:55.577 INFO:teuthology.orchestra.run.smithi153.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-10T22:39:55.627 INFO:teuthology.orchestra.run.smithi153.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-10T22:39:55.653 INFO:teuthology.orchestra.run.smithi153.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-09-10T22:39:55.703 INFO:teuthology.orchestra.run.smithi153.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-09-10T22:39:55.737 INFO:teuthology.orchestra.run.smithi153.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-09-10T22:39:55.737 INFO:teuthology.orchestra.run.smithi176.stdout:(6/157): ceph-common-19.3.0-4854.g8f219c47.el9. 22 MB/s | 22 MB 00:00 2024-09-10T22:39:55.779 INFO:teuthology.orchestra.run.smithi176.stdout:(7/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-09-10T22:39:55.795 INFO:teuthology.orchestra.run.smithi153.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-10T22:39:55.820 INFO:teuthology.orchestra.run.smithi153.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 430 kB/s | 36 kB 00:00 2024-09-10T22:39:55.879 INFO:teuthology.orchestra.run.smithi176.stdout:(8/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 4.8 MB 00:00 2024-09-10T22:39:55.929 INFO:teuthology.orchestra.run.smithi176.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 506 kB/s | 25 kB 00:00 2024-09-10T22:39:55.988 INFO:teuthology.orchestra.run.smithi153.stdout:(98/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-09-10T22:39:56.012 INFO:teuthology.orchestra.run.smithi153.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 218 kB/s | 47 kB 00:00 2024-09-10T22:39:56.038 INFO:teuthology.orchestra.run.smithi153.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-10T22:39:56.080 INFO:teuthology.orchestra.run.smithi153.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-10T22:39:56.122 INFO:teuthology.orchestra.run.smithi153.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-09-10T22:39:56.155 INFO:teuthology.orchestra.run.smithi153.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-09-10T22:39:56.189 INFO:teuthology.orchestra.run.smithi153.stdout:(104/157): uuid-1.6.2-55.el9.x86_64.rpm 287 kB/s | 58 kB 00:00 2024-09-10T22:39:56.214 INFO:teuthology.orchestra.run.smithi153.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 241 kB/s | 22 kB 00:00 2024-09-10T22:39:56.265 INFO:teuthology.orchestra.run.smithi153.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 7.7 MB/s | 863 kB 00:00 2024-09-10T22:39:56.306 INFO:teuthology.orchestra.run.smithi153.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-10T22:39:56.332 INFO:teuthology.orchestra.run.smithi153.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-10T22:39:56.357 INFO:teuthology.orchestra.run.smithi153.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-09-10T22:39:56.382 INFO:teuthology.orchestra.run.smithi153.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-10T22:39:56.407 INFO:teuthology.orchestra.run.smithi153.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-10T22:39:56.421 INFO:teuthology.orchestra.run.smithi176.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 25 MB/s | 17 MB 00:00 2024-09-10T22:39:56.433 INFO:teuthology.orchestra.run.smithi153.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T22:39:56.466 INFO:teuthology.orchestra.run.smithi153.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-10T22:39:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T22:39:56.525 INFO:teuthology.orchestra.run.smithi153.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-10T22:39:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 360 kB/s | 27 kB 00:00 2024-09-10T22:39:56.583 INFO:teuthology.orchestra.run.smithi153.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-10T22:39:56.609 INFO:teuthology.orchestra.run.smithi153.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-09-10T22:39:56.634 INFO:teuthology.orchestra.run.smithi153.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-10T22:39:56.638 INFO:teuthology.orchestra.run.smithi176.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 347 kB/s | 32 kB 00:00 2024-09-10T22:39:56.660 INFO:teuthology.orchestra.run.smithi153.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T22:39:56.684 INFO:teuthology.orchestra.run.smithi153.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-10T22:39:56.718 INFO:teuthology.orchestra.run.smithi153.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T22:39:56.752 INFO:teuthology.orchestra.run.smithi153.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T22:39:56.755 INFO:teuthology.orchestra.run.smithi176.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 6.3 MB/s | 755 kB 00:00 2024-09-10T22:39:56.777 INFO:teuthology.orchestra.run.smithi153.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T22:39:56.831 INFO:teuthology.orchestra.run.smithi176.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T22:39:56.844 INFO:teuthology.orchestra.run.smithi153.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-09-10T22:39:56.869 INFO:teuthology.orchestra.run.smithi153.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-10T22:39:56.892 INFO:teuthology.orchestra.run.smithi120.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-09-10T22:39:56.894 INFO:teuthology.orchestra.run.smithi153.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-10T22:39:56.911 INFO:teuthology.orchestra.run.smithi153.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-10T22:39:56.928 INFO:teuthology.orchestra.run.smithi153.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-09-10T22:39:56.945 INFO:teuthology.orchestra.run.smithi153.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 583 kB/s | 19 kB 00:00 2024-09-10T22:39:56.962 INFO:teuthology.orchestra.run.smithi153.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 585 kB/s | 20 kB 00:00 2024-09-10T22:39:56.973 INFO:teuthology.orchestra.run.smithi176.stdout:(16/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 9.2 MB/s | 11 MB 00:01 2024-09-10T22:39:56.987 INFO:teuthology.orchestra.run.smithi153.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:39:56.998 INFO:teuthology.orchestra.run.smithi176.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 760 kB/s | 127 kB 00:00 2024-09-10T22:39:57.021 INFO:teuthology.orchestra.run.smithi153.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T22:39:57.025 INFO:teuthology.orchestra.run.smithi176.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 9.9 MB/s | 507 kB 00:00 2024-09-10T22:39:57.038 INFO:teuthology.orchestra.run.smithi153.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-10T22:39:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-09-10T22:39:57.113 INFO:teuthology.orchestra.run.smithi153.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T22:39:57.115 INFO:teuthology.orchestra.run.smithi176.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 492 kB/s | 45 kB 00:00 2024-09-10T22:39:57.147 INFO:teuthology.orchestra.run.smithi153.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-09-10T22:39:57.173 INFO:teuthology.orchestra.run.smithi153.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-10T22:39:57.198 INFO:teuthology.orchestra.run.smithi153.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T22:39:57.216 INFO:teuthology.orchestra.run.smithi176.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.4 MB/s | 143 kB 00:00 2024-09-10T22:39:57.223 INFO:teuthology.orchestra.run.smithi153.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 823 kB/s | 90 kB 00:00 2024-09-10T22:39:57.248 INFO:teuthology.orchestra.run.smithi153.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-10T22:39:57.274 INFO:teuthology.orchestra.run.smithi153.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T22:39:57.299 INFO:teuthology.orchestra.run.smithi153.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-10T22:39:57.316 INFO:teuthology.orchestra.run.smithi176.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.6 MB/s | 161 kB 00:00 2024-09-10T22:39:57.324 INFO:teuthology.orchestra.run.smithi153.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T22:39:57.326 INFO:teuthology.orchestra.run.smithi120.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 2.0 MB/s | 900 kB 00:00 2024-09-10T22:39:57.349 INFO:teuthology.orchestra.run.smithi153.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-10T22:39:57.375 INFO:teuthology.orchestra.run.smithi153.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T22:39:57.400 INFO:teuthology.orchestra.run.smithi153.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T22:39:57.425 INFO:teuthology.orchestra.run.smithi153.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-10T22:39:57.450 INFO:teuthology.orchestra.run.smithi153.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-10T22:39:57.484 INFO:teuthology.orchestra.run.smithi153.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T22:39:57.501 INFO:teuthology.orchestra.run.smithi120.stdout:(3/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 6.4 MB/s | 5.5 MB 00:00 2024-09-10T22:39:57.508 INFO:teuthology.orchestra.run.smithi176.stdout:(22/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-10T22:39:57.536 INFO:teuthology.orchestra.run.smithi120.stdout:(4/157): ceph-immutable-object-cache-19.3.0-485 696 kB/s | 145 kB 00:00 2024-09-10T22:39:57.559 INFO:teuthology.orchestra.run.smithi153.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T22:39:57.593 INFO:teuthology.orchestra.run.smithi176.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.6 MB/s | 306 kB 00:00 2024-09-10T22:39:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 21 MB/s | 2.3 MB 00:00 2024-09-10T22:39:57.689 INFO:teuthology.orchestra.run.smithi176.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.1 MB/s | 99 kB 00:00 2024-09-10T22:39:57.726 INFO:teuthology.orchestra.run.smithi120.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 2.4 MB 00:00 2024-09-10T22:39:57.759 INFO:teuthology.orchestra.run.smithi153.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-09-10T22:39:57.780 INFO:teuthology.orchestra.run.smithi176.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.0 MB/s | 85 kB 00:00 2024-09-10T22:39:57.827 INFO:teuthology.orchestra.run.smithi153.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 611 kB/s | 41 kB 00:00 2024-09-10T22:39:57.943 INFO:teuthology.orchestra.run.smithi153.stdout:(153/157): librados2-19.3.0-4854.g8f219c47.el9. 7.0 MB/s | 3.5 MB 00:00 2024-09-10T22:39:57.977 INFO:teuthology.orchestra.run.smithi153.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-10T22:39:58.036 INFO:teuthology.orchestra.run.smithi153.stdout:(155/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 5.9 MB/s | 3.2 MB 00:00 2024-09-10T22:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-10T22:39:58.111 INFO:teuthology.orchestra.run.smithi120.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-09-10T22:39:58.136 INFO:teuthology.orchestra.run.smithi153.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-10T22:39:58.142 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:39:58.142 INFO:teuthology.orchestra.run.smithi153.stdout:Total 25 MB/s | 240 MB 00:09 2024-09-10T22:39:58.177 INFO:teuthology.orchestra.run.smithi120.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 4.8 MB 00:00 2024-09-10T22:39:58.210 INFO:teuthology.orchestra.run.smithi176.stdout:(26/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 22 MB/s | 50 MB 00:02 2024-09-10T22:39:58.234 INFO:teuthology.orchestra.run.smithi176.stdout:(27/157): python3-rados-19.3.0-4854.g8f219c47.e 350 kB/s | 321 kB 00:00 2024-09-10T22:39:58.260 INFO:teuthology.orchestra.run.smithi176.stdout:(28/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.3 MB/s | 171 kB 00:00 2024-09-10T22:39:58.285 INFO:teuthology.orchestra.run.smithi176.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 681 kB/s | 34 kB 00:00 2024-09-10T22:39:58.318 INFO:teuthology.orchestra.run.smithi176.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.6 MB/s | 154 kB 00:00 2024-09-10T22:39:58.377 INFO:teuthology.orchestra.run.smithi176.stdout:(31/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 5.2 MB/s | 3.2 MB 00:00 2024-09-10T22:39:58.502 INFO:teuthology.orchestra.run.smithi176.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.3 MB/s | 291 kB 00:00 2024-09-10T22:39:58.544 INFO:teuthology.orchestra.run.smithi120.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 11 MB/s | 22 MB 00:01 2024-09-10T22:39:58.569 INFO:teuthology.orchestra.run.smithi176.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 744 kB/s | 49 kB 00:00 2024-09-10T22:39:58.619 INFO:teuthology.orchestra.run.smithi120.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 336 kB/s | 25 kB 00:00 2024-09-10T22:39:58.694 INFO:teuthology.orchestra.run.smithi176.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 20 MB/s | 7.4 MB 00:00 2024-09-10T22:39:58.711 INFO:teuthology.orchestra.run.smithi176.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 116 kB/s | 16 kB 00:00 2024-09-10T22:39:58.761 INFO:teuthology.orchestra.run.smithi176.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.2 MB/s | 285 kB 00:00 2024-09-10T22:39:58.837 INFO:teuthology.orchestra.run.smithi176.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 6.1 MB/s | 783 kB 00:00 2024-09-10T22:39:58.853 INFO:teuthology.orchestra.run.smithi176.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 198 kB/s | 18 kB 00:00 2024-09-10T22:39:59.011 INFO:teuthology.orchestra.run.smithi120.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 13 MB/s | 11 MB 00:00 2024-09-10T22:39:59.062 INFO:teuthology.orchestra.run.smithi120.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T22:39:59.104 INFO:teuthology.orchestra.run.smithi120.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 644 kB/s | 27 kB 00:00 2024-09-10T22:39:59.146 INFO:teuthology.orchestra.run.smithi120.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 766 kB/s | 32 kB 00:00 2024-09-10T22:39:59.230 INFO:teuthology.orchestra.run.smithi120.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.8 MB/s | 755 kB 00:00 2024-09-10T22:39:59.287 INFO:teuthology.orchestra.run.smithi176.stdout:(39/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 14 MB/s | 14 MB 00:01 2024-09-10T22:39:59.288 INFO:teuthology.orchestra.run.smithi120.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.7 MB/s | 162 kB 00:00 2024-09-10T22:39:59.313 INFO:teuthology.orchestra.run.smithi176.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-10T22:39:59.338 INFO:teuthology.orchestra.run.smithi120.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T22:39:59.383 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-09-10T22:39:59.406 INFO:teuthology.orchestra.run.smithi120.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 7.4 MB/s | 507 kB 00:00 2024-09-10T22:39:59.537 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-09-10T22:39:59.537 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-09-10T22:39:59.671 INFO:teuthology.orchestra.run.smithi176.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 981 kB/s | 801 kB 00:00 2024-09-10T22:39:59.696 INFO:teuthology.orchestra.run.smithi176.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 660 kB/s | 253 kB 00:00 2024-09-10T22:39:59.739 INFO:teuthology.orchestra.run.smithi120.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 5.5 MB 00:00 2024-09-10T22:39:59.805 INFO:teuthology.orchestra.run.smithi176.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-09-10T22:39:59.806 INFO:teuthology.orchestra.run.smithi120.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 679 kB/s | 45 kB 00:00 2024-09-10T22:39:59.847 INFO:teuthology.orchestra.run.smithi176.stdout:(44/157): libquadmath-11.5.0-2.el9.x86_64.rpm 342 kB/s | 191 kB 00:00 2024-09-10T22:39:59.865 INFO:teuthology.orchestra.run.smithi120.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-10T22:39:59.914 INFO:teuthology.orchestra.run.smithi176.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-10T22:39:59.956 INFO:teuthology.orchestra.run.smithi176.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-10T22:40:00.032 INFO:teuthology.orchestra.run.smithi176.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-10T22:40:00.032 INFO:teuthology.orchestra.run.smithi120.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 969 kB/s | 161 kB 00:00 2024-09-10T22:40:00.091 INFO:teuthology.orchestra.run.smithi120.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 5.4 MB/s | 321 kB 00:00 2024-09-10T22:40:00.149 INFO:teuthology.orchestra.run.smithi120.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T22:40:00.157 INFO:teuthology.orchestra.run.smithi176.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-09-10T22:40:00.208 INFO:teuthology.orchestra.run.smithi120.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-10T22:40:00.259 INFO:teuthology.orchestra.run.smithi120.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T22:40:00.492 INFO:teuthology.orchestra.run.smithi120.stdout:(26/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 14 MB/s | 3.2 MB 00:00 2024-09-10T22:40:00.542 INFO:teuthology.orchestra.run.smithi120.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.4 MB/s | 171 kB 00:00 2024-09-10T22:40:00.593 INFO:teuthology.orchestra.run.smithi120.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 685 kB/s | 34 kB 00:00 2024-09-10T22:40:00.824 INFO:teuthology.orchestra.run.smithi176.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-10T22:40:01.040 INFO:teuthology.orchestra.run.smithi176.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 935 kB/s | 943 kB 00:01 2024-09-10T22:40:01.068 INFO:teuthology.orchestra.run.smithi120.stdout:(29/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 21 MB/s | 50 MB 00:02 2024-09-10T22:40:01.093 INFO:teuthology.orchestra.run.smithi120.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 308 kB/s | 154 kB 00:00 2024-09-10T22:40:01.349 INFO:teuthology.orchestra.run.smithi176.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 495 kB/s | 689 kB 00:01 2024-09-10T22:40:01.374 INFO:teuthology.orchestra.run.smithi176.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-10T22:40:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-09-10T22:40:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-09-10T22:40:01.428 INFO:teuthology.orchestra.run.smithi120.stdout:(31/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 5.2 MB/s | 17 MB 00:03 2024-09-10T22:40:01.519 INFO:teuthology.orchestra.run.smithi120.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.1 MB/s | 291 kB 00:00 2024-09-10T22:40:01.578 INFO:teuthology.orchestra.run.smithi120.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 846 kB/s | 49 kB 00:00 2024-09-10T22:40:01.670 INFO:teuthology.orchestra.run.smithi120.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-10T22:40:01.695 INFO:teuthology.orchestra.run.smithi120.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 140 kB/s | 16 kB 00:00 2024-09-10T22:40:01.721 INFO:teuthology.orchestra.run.smithi120.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.5 MB/s | 285 kB 00:00 2024-09-10T22:40:01.841 INFO:teuthology.orchestra.run.smithi176.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 396 kB/s | 313 kB 00:00 2024-09-10T22:40:01.879 INFO:teuthology.orchestra.run.smithi120.stdout:(37/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 17 MB/s | 14 MB 00:00 2024-09-10T22:40:01.899 INFO:teuthology.orchestra.run.smithi176.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-10T22:40:01.905 INFO:teuthology.orchestra.run.smithi120.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 99 kB/s | 18 kB 00:00 2024-09-10T22:40:01.938 INFO:teuthology.orchestra.run.smithi120.stdout:(39/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 3.2 MB/s | 783 kB 00:00 2024-09-10T22:40:02.014 INFO:teuthology.orchestra.run.smithi120.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 596 kB/s | 80 kB 00:00 2024-09-10T22:40:02.081 INFO:teuthology.orchestra.run.smithi120.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-10T22:40:02.123 INFO:teuthology.orchestra.run.smithi120.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-10T22:40:02.156 INFO:teuthology.orchestra.run.smithi120.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-10T22:40:02.182 INFO:teuthology.orchestra.run.smithi120.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T22:40:02.207 INFO:teuthology.orchestra.run.smithi120.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T22:40:02.232 INFO:teuthology.orchestra.run.smithi120.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T22:40:02.266 INFO:teuthology.orchestra.run.smithi120.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-10T22:40:02.299 INFO:teuthology.orchestra.run.smithi120.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-10T22:40:02.350 INFO:teuthology.orchestra.run.smithi176.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-10T22:40:02.375 INFO:teuthology.orchestra.run.smithi176.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-10T22:40:02.542 INFO:teuthology.orchestra.run.smithi176.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-10T22:40:03.041 INFO:teuthology.orchestra.run.smithi120.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 144 kB/s | 107 kB 00:00 2024-09-10T22:40:03.125 INFO:teuthology.orchestra.run.smithi176.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 214 kB/s | 160 kB 00:00 2024-09-10T22:40:03.152 INFO:teuthology.orchestra.run.smithi176.stdout:(59/157): libnbd-1.20.2-2.el9.x86_64.rpm 213 kB/s | 171 kB 00:00 2024-09-10T22:40:03.176 INFO:teuthology.orchestra.run.smithi176.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-10T22:40:03.208 INFO:teuthology.orchestra.run.smithi120.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 706 kB/s | 689 kB 00:00 2024-09-10T22:40:03.251 INFO:teuthology.orchestra.run.smithi120.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 958 kB/s | 943 kB 00:00 2024-09-10T22:40:03.634 INFO:teuthology.orchestra.run.smithi120.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-10T22:40:03.718 INFO:teuthology.orchestra.run.smithi176.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-10T22:40:03.793 INFO:teuthology.orchestra.run.smithi120.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-10T22:40:04.118 INFO:teuthology.orchestra.run.smithi120.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 345 kB/s | 313 kB 00:00 2024-09-10T22:40:04.210 INFO:teuthology.orchestra.run.smithi176.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 562 kB/s | 581 kB 00:01 2024-09-10T22:40:04.243 INFO:teuthology.orchestra.run.smithi176.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 594 kB/s | 662 kB 00:01 2024-09-10T22:40:04.268 INFO:teuthology.orchestra.run.smithi120.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-10T22:40:04.293 INFO:teuthology.orchestra.run.smithi176.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-10T22:40:04.602 INFO:teuthology.orchestra.run.smithi120.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-10T22:40:04.860 INFO:teuthology.orchestra.run.smithi176.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-10T22:40:05.002 INFO:teuthology.orchestra.run.smithi120.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 233 kB/s | 171 kB 00:00 2024-09-10T22:40:05.027 INFO:teuthology.orchestra.run.smithi176.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 358 kB/s | 292 kB 00:00 2024-09-10T22:40:05.153 INFO:teuthology.orchestra.run.smithi176.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 207 kB/s | 188 kB 00:00 2024-09-10T22:40:05.369 INFO:teuthology.orchestra.run.smithi120.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 209 kB/s | 160 kB 00:00 2024-09-10T22:40:05.619 INFO:teuthology.orchestra.run.smithi120.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-09-10T22:40:06.078 INFO:teuthology.orchestra.run.smithi176.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-10T22:40:06.204 INFO:teuthology.orchestra.run.smithi120.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.2 MB/s | 3.0 MB 00:02 2024-09-10T22:40:06.336 INFO:teuthology.orchestra.run.smithi176.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-10T22:40:06.721 INFO:teuthology.orchestra.run.smithi120.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 490 kB/s | 662 kB 00:01 2024-09-10T22:40:06.746 INFO:teuthology.orchestra.run.smithi120.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-10T22:40:06.779 INFO:teuthology.orchestra.run.smithi120.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 501 kB/s | 581 kB 00:01 2024-09-10T22:40:07.112 INFO:teuthology.orchestra.run.smithi176.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 179 kB/s | 137 kB 00:00 2024-09-10T22:40:07.321 INFO:teuthology.orchestra.run.smithi120.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-09-10T22:40:07.522 INFO:teuthology.orchestra.run.smithi120.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 254 kB/s | 188 kB 00:00 2024-09-10T22:40:07.603 INFO:teuthology.orchestra.run.smithi176.stdout:(71/157): postgresql-13.16-1.el9.x86_64.rpm 665 kB/s | 1.6 MB 00:02 2024-09-10T22:40:07.630 INFO:teuthology.orchestra.run.smithi120.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 331 kB/s | 292 kB 00:00 2024-09-10T22:40:07.753 INFO:teuthology.orchestra.run.smithi176.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 543 kB/s | 910 kB 00:01 2024-09-10T22:40:07.795 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-09-10T22:40:07.906 INFO:teuthology.orchestra.run.smithi120.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 66 kB/s | 38 kB 00:00 2024-09-10T22:40:07.931 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:40:07.970 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:40:08.023 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:40:08.057 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:40:08.158 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:40:08.345 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:40:08.395 INFO:teuthology.orchestra.run.smithi176.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 136 kB/s | 87 kB 00:00 2024-09-10T22:40:08.688 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:40:08.771 INFO:teuthology.orchestra.run.smithi176.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-09-10T22:40:08.800 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:08.923 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:08.951 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:09.033 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:09.063 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:40:09.093 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:40:09.246 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:40:09.294 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:40:09.323 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:09.396 INFO:teuthology.orchestra.run.smithi176.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-10T22:40:09.419 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:09.438 INFO:teuthology.orchestra.run.smithi176.stdout:(76/157): protobuf-3.14.0-13.el9.x86_64.rpm 563 kB/s | 1.0 MB 00:01 2024-09-10T22:40:09.449 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:09.494 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:09.516 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:09.556 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:09.743 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:09.876 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:10.029 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:40:10.072 INFO:teuthology.orchestra.run.smithi120.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 750 kB/s | 1.6 MB 00:02 2024-09-10T22:40:10.125 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:10.207 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:10.280 INFO:teuthology.orchestra.run.smithi176.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 277 kB/s | 245 kB 00:00 2024-09-10T22:40:10.305 INFO:teuthology.orchestra.run.smithi176.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 286 kB/s | 248 kB 00:00 2024-09-10T22:40:10.306 INFO:teuthology.orchestra.run.smithi120.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 1.8 MB/s | 4.9 MB 00:02 2024-09-10T22:40:10.398 INFO:teuthology.orchestra.run.smithi120.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-09-10T22:40:10.416 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:10.529 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:40:10.561 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:40:10.596 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:40:10.668 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:40:10.697 INFO:teuthology.orchestra.run.smithi176.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.6 MB/s | 6.0 MB 00:02 2024-09-10T22:40:10.714 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:40:10.745 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:40:10.789 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:40:10.830 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:40:10.866 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:40:10.881 INFO:teuthology.orchestra.run.smithi176.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-09-10T22:40:10.905 INFO:teuthology.orchestra.run.smithi176.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 32 kB/s | 19 kB 00:00 2024-09-10T22:40:10.934 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:40:10.988 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:40:11.019 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:40:11.048 INFO:teuthology.orchestra.run.smithi120.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 187 kB/s | 137 kB 00:00 2024-09-10T22:40:11.053 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:40:11.130 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:40:11.156 INFO:teuthology.orchestra.run.smithi120.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 840 kB/s | 910 kB 00:01 2024-09-10T22:40:11.165 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:40:11.208 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:40:11.245 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:40:11.314 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:40:11.368 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:40:11.439 INFO:teuthology.orchestra.run.smithi176.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-10T22:40:11.443 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:40:11.473 INFO:teuthology.orchestra.run.smithi176.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-09-10T22:40:11.583 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:40:11.637 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:40:11.807 INFO:teuthology.orchestra.run.smithi120.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 135 kB/s | 87 kB 00:00 2024-09-10T22:40:12.159 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:40:12.241 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:40:12.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:40:12.290 INFO:teuthology.orchestra.run.smithi176.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-09-10T22:40:12.316 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:40:12.363 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:40:12.404 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:40:12.406 INFO:teuthology.orchestra.run.smithi176.stdout:(85/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 259 kB/s | 250 kB 00:00 2024-09-10T22:40:12.564 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:40:12.623 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:40:12.724 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:40:12.789 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:40:12.861 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:40:12.892 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:40:12.928 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:40:12.932 INFO:teuthology.orchestra.run.smithi176.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-09-10T22:40:12.957 INFO:teuthology.orchestra.run.smithi176.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 116 kB/s | 77 kB 00:00 2024-09-10T22:40:12.975 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:40:12.982 INFO:teuthology.orchestra.run.smithi176.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-10T22:40:13.011 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:40:13.015 INFO:teuthology.orchestra.run.smithi120.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 525 kB/s | 1.0 MB 00:01 2024-09-10T22:40:13.062 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:40:13.113 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:40:13.257 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:40:13.298 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:40:13.427 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:40:13.534 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:40:13.572 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:40:13.612 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:40:13.624 INFO:teuthology.orchestra.run.smithi120.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 101 kB/s | 61 kB 00:00 2024-09-10T22:40:13.646 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:40:13.679 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:40:13.724 INFO:teuthology.orchestra.run.smithi176.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-09-10T22:40:13.754 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:40:13.874 INFO:teuthology.orchestra.run.smithi176.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 313 kB/s | 279 kB 00:00 2024-09-10T22:40:13.888 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:40:13.899 INFO:teuthology.orchestra.run.smithi176.stdout:(91/157): python3-protobuf-3.14.0-13.el9.noarch 278 kB/s | 269 kB 00:00 2024-09-10T22:40:13.928 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:40:13.956 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:40:13.990 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:40:14.016 INFO:teuthology.orchestra.run.smithi120.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 1.6 MB/s | 5.8 MB 00:03 2024-09-10T22:40:14.029 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:40:14.061 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:40:14.090 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:40:14.125 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:40:14.167 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:40:14.217 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:40:14.251 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:40:14.333 INFO:teuthology.orchestra.run.smithi176.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-10T22:40:14.337 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:40:14.381 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:40:14.414 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:40:14.483 INFO:teuthology.orchestra.run.smithi176.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 88 kB/s | 54 kB 00:00 2024-09-10T22:40:14.518 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:40:14.525 INFO:teuthology.orchestra.run.smithi120.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 272 kB/s | 245 kB 00:00 2024-09-10T22:40:14.566 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:40:14.842 INFO:teuthology.orchestra.run.smithi120.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 301 kB/s | 248 kB 00:00 2024-09-10T22:40:14.950 INFO:teuthology.orchestra.run.smithi176.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-09-10T22:40:15.067 INFO:teuthology.orchestra.run.smithi176.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 76 kB/s | 44 kB 00:00 2024-09-10T22:40:15.092 INFO:teuthology.orchestra.run.smithi120.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-09-10T22:40:15.287 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:40:15.328 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:40:15.342 INFO:teuthology.orchestra.run.smithi120.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.7 MB/s | 6.0 MB 00:03 2024-09-10T22:40:15.362 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:40:15.470 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:40:15.476 INFO:teuthology.orchestra.run.smithi120.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 30 kB/s | 19 kB 00:00 2024-09-10T22:40:15.518 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:40:15.562 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:40:15.626 INFO:teuthology.orchestra.run.smithi176.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 64 kB/s | 36 kB 00:00 2024-09-10T22:40:15.650 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:40:15.698 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:40:15.852 INFO:teuthology.orchestra.run.smithi176.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.3 MB/s | 16 MB 00:01 2024-09-10T22:40:15.877 INFO:teuthology.orchestra.run.smithi176.stdout:(98/157): qatlib-24.02.0-1.el9.x86_64.rpm 238 kB/s | 221 kB 00:00 2024-09-10T22:40:15.885 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:40:15.909 INFO:teuthology.orchestra.run.smithi120.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 61 kB/s | 35 kB 00:00 2024-09-10T22:40:15.912 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:40:15.991 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:40:16.052 INFO:teuthology.orchestra.run.smithi120.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 179 kB/s | 172 kB 00:00 2024-09-10T22:40:16.285 INFO:teuthology.orchestra.run.smithi176.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-09-10T22:40:16.486 INFO:teuthology.orchestra.run.smithi176.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-10T22:40:16.630 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:40:16.686 INFO:teuthology.orchestra.run.smithi176.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 365 kB/s | 304 kB 00:00 2024-09-10T22:40:16.702 INFO:teuthology.orchestra.run.smithi120.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 316 kB/s | 250 kB 00:00 2024-09-10T22:40:16.736 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:40:16.811 INFO:teuthology.orchestra.run.smithi120.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 293 kB/s | 222 kB 00:00 2024-09-10T22:40:16.936 INFO:teuthology.orchestra.run.smithi176.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 147 kB/s | 96 kB 00:00 2024-09-10T22:40:17.020 INFO:teuthology.orchestra.run.smithi176.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-09-10T22:40:17.069 INFO:teuthology.orchestra.run.smithi120.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-10T22:40:17.137 INFO:teuthology.orchestra.run.smithi176.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 98 kB/s | 64 kB 00:00 2024-09-10T22:40:17.195 INFO:teuthology.orchestra.run.smithi176.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-10T22:40:17.245 INFO:teuthology.orchestra.run.smithi176.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.0 MB/s | 548 kB 00:00 2024-09-10T22:40:17.262 INFO:teuthology.orchestra.run.smithi176.stdout:(107/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-10T22:40:17.279 INFO:teuthology.orchestra.run.smithi176.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 593 kB/s | 19 kB 00:00 2024-09-10T22:40:17.305 INFO:teuthology.orchestra.run.smithi176.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T22:40:17.322 INFO:teuthology.orchestra.run.smithi176.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-10T22:40:17.347 INFO:teuthology.orchestra.run.smithi176.stdout:(111/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-09-10T22:40:17.353 INFO:teuthology.orchestra.run.smithi120.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 119 kB/s | 77 kB 00:00 2024-09-10T22:40:17.372 INFO:teuthology.orchestra.run.smithi176.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T22:40:17.398 INFO:teuthology.orchestra.run.smithi176.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-10T22:40:17.445 INFO:teuthology.orchestra.run.smithi120.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 67 kB/s | 42 kB 00:00 2024-09-10T22:40:17.464 INFO:teuthology.orchestra.run.smithi176.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-10T22:40:17.499 INFO:teuthology.orchestra.run.smithi176.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-09-10T22:40:17.523 INFO:teuthology.orchestra.run.smithi176.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-10T22:40:17.548 INFO:teuthology.orchestra.run.smithi176.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-10T22:40:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T22:40:17.607 INFO:teuthology.orchestra.run.smithi176.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T22:40:17.649 INFO:teuthology.orchestra.run.smithi176.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-10T22:40:17.675 INFO:teuthology.orchestra.run.smithi176.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T22:40:17.708 INFO:teuthology.orchestra.run.smithi176.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T22:40:17.742 INFO:teuthology.orchestra.run.smithi176.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T22:40:17.767 INFO:teuthology.orchestra.run.smithi176.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-10T22:40:17.809 INFO:teuthology.orchestra.run.smithi176.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-09-10T22:40:17.834 INFO:teuthology.orchestra.run.smithi176.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-10T22:40:17.860 INFO:teuthology.orchestra.run.smithi176.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-10T22:40:17.870 INFO:teuthology.orchestra.run.smithi120.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 340 kB/s | 269 kB 00:00 2024-09-10T22:40:17.885 INFO:teuthology.orchestra.run.smithi176.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-10T22:40:17.943 INFO:teuthology.orchestra.run.smithi176.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-10T22:40:17.966 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:40:17.969 INFO:teuthology.orchestra.run.smithi176.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:40:18.019 INFO:teuthology.orchestra.run.smithi176.stdout:(131/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-09-10T22:40:18.041 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:40:18.044 INFO:teuthology.orchestra.run.smithi176.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-10T22:40:18.069 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:40:18.069 INFO:teuthology.orchestra.run.smithi176.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T22:40:18.087 INFO:teuthology.orchestra.run.smithi120.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-10T22:40:18.095 INFO:teuthology.orchestra.run.smithi176.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-10T22:40:18.102 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:40:18.112 INFO:teuthology.orchestra.run.smithi176.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 393 kB/s | 16 kB 00:00 2024-09-10T22:40:18.145 INFO:teuthology.orchestra.run.smithi176.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-09-10T22:40:18.162 INFO:teuthology.orchestra.run.smithi176.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T22:40:18.188 INFO:teuthology.orchestra.run.smithi176.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 971 kB/s | 90 kB 00:00 2024-09-10T22:40:18.205 INFO:teuthology.orchestra.run.smithi176.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-09-10T22:40:18.221 INFO:teuthology.orchestra.run.smithi120.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 359 kB/s | 279 kB 00:00 2024-09-10T22:40:18.230 INFO:teuthology.orchestra.run.smithi176.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-10T22:40:18.255 INFO:teuthology.orchestra.run.smithi176.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T22:40:18.281 INFO:teuthology.orchestra.run.smithi176.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-10T22:40:18.306 INFO:teuthology.orchestra.run.smithi176.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-10T22:40:18.331 INFO:teuthology.orchestra.run.smithi176.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T22:40:18.356 INFO:teuthology.orchestra.run.smithi176.stdout:(145/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T22:40:18.381 INFO:teuthology.orchestra.run.smithi176.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-10T22:40:18.406 INFO:teuthology.orchestra.run.smithi176.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-10T22:40:18.432 INFO:teuthology.orchestra.run.smithi176.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T22:40:18.469 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:40:18.471 INFO:teuthology.orchestra.run.smithi120.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-10T22:40:18.499 INFO:teuthology.orchestra.run.smithi176.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T22:40:18.705 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:18.721 INFO:teuthology.orchestra.run.smithi120.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 86 kB/s | 54 kB 00:00 2024-09-10T22:40:18.791 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:18.829 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:40:18.882 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:40:18.941 INFO:teuthology.orchestra.run.smithi176.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 6.4 MB/s | 3.2 MB 00:00 2024-09-10T22:40:18.991 INFO:teuthology.orchestra.run.smithi176.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 5.9 MB/s | 3.5 MB 00:00 2024-09-10T22:40:19.047 INFO:teuthology.orchestra.run.smithi120.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-10T22:40:19.266 INFO:teuthology.orchestra.run.smithi176.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 148 kB/s | 41 kB 00:00 2024-09-10T22:40:19.363 INFO:teuthology.orchestra.run.smithi120.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-09-10T22:40:19.377 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:40:19.410 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:19.464 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:19.486 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:40:19.508 INFO:teuthology.orchestra.run.smithi176.stdout:(153/157): NetworkManager-1.49.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-09-10T22:40:19.641 INFO:teuthology.orchestra.run.smithi176.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-10T22:40:19.667 INFO:teuthology.orchestra.run.smithi176.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 621 kB/s | 249 kB 00:00 2024-09-10T22:40:19.680 INFO:teuthology.orchestra.run.smithi120.stdout:(96/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T22:40:19.800 INFO:teuthology.orchestra.run.smithi176.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-10T22:40:19.939 INFO:teuthology.orchestra.run.smithi120.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 248 kB/s | 221 kB 00:00 2024-09-10T22:40:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:(157/157): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-10T22:40:19.974 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:40:19.974 INFO:teuthology.orchestra.run.smithi176.stdout:Total 9.2 MB/s | 240 MB 00:26 2024-09-10T22:40:20.056 INFO:teuthology.orchestra.run.smithi120.stdout:(98/157): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-09-10T22:40:20.273 INFO:teuthology.orchestra.run.smithi120.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-10T22:40:20.673 INFO:teuthology.orchestra.run.smithi120.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-10T22:40:20.973 INFO:teuthology.orchestra.run.smithi120.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 294 kB/s | 304 kB 00:01 2024-09-10T22:40:21.140 INFO:teuthology.orchestra.run.smithi120.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 111 kB/s | 96 kB 00:00 2024-09-10T22:40:21.224 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-09-10T22:40:21.377 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-09-10T22:40:21.377 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-09-10T22:40:21.474 INFO:teuthology.orchestra.run.smithi120.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 80 kB/s | 64 kB 00:00 2024-09-10T22:40:21.641 INFO:teuthology.orchestra.run.smithi120.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-09-10T22:40:21.675 INFO:teuthology.orchestra.run.smithi120.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-09-10T22:40:21.742 INFO:teuthology.orchestra.run.smithi120.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-10T22:40:21.767 INFO:teuthology.orchestra.run.smithi120.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-10T22:40:21.800 INFO:teuthology.orchestra.run.smithi120.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-09-10T22:40:21.826 INFO:teuthology.orchestra.run.smithi120.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T22:40:21.851 INFO:teuthology.orchestra.run.smithi120.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T22:40:21.877 INFO:teuthology.orchestra.run.smithi120.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T22:40:21.919 INFO:teuthology.orchestra.run.smithi120.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-10T22:40:22.019 INFO:teuthology.orchestra.run.smithi120.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-10T22:40:22.094 INFO:teuthology.orchestra.run.smithi120.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-10T22:40:22.120 INFO:teuthology.orchestra.run.smithi120.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T22:40:22.145 INFO:teuthology.orchestra.run.smithi120.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T22:40:22.157 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:22.190 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:22.204 INFO:teuthology.orchestra.run.smithi120.stdout:(117/157): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-10T22:40:22.229 INFO:teuthology.orchestra.run.smithi120.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 168 kB/s | 14 kB 00:00 2024-09-10T22:40:22.254 INFO:teuthology.orchestra.run.smithi120.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T22:40:22.288 INFO:teuthology.orchestra.run.smithi120.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T22:40:22.313 INFO:teuthology.orchestra.run.smithi120.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T22:40:22.339 INFO:teuthology.orchestra.run.smithi120.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T22:40:22.372 INFO:teuthology.orchestra.run.smithi120.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T22:40:22.398 INFO:teuthology.orchestra.run.smithi120.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T22:40:22.423 INFO:teuthology.orchestra.run.smithi120.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-10T22:40:22.448 INFO:teuthology.orchestra.run.smithi120.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-10T22:40:22.474 INFO:teuthology.orchestra.run.smithi120.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-10T22:40:22.499 INFO:teuthology.orchestra.run.smithi120.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-10T22:40:22.550 INFO:teuthology.orchestra.run.smithi120.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-10T22:40:22.575 INFO:teuthology.orchestra.run.smithi120.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:40:22.600 INFO:teuthology.orchestra.run.smithi120.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T22:40:22.642 INFO:teuthology.orchestra.run.smithi120.stdout:(132/157): protobuf-compiler-3.14.0-13.el9.x86_ 739 kB/s | 863 kB 00:01 2024-09-10T22:40:22.659 INFO:teuthology.orchestra.run.smithi120.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-09-10T22:40:22.701 INFO:teuthology.orchestra.run.smithi120.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 481 kB/s | 16 kB 00:00 2024-09-10T22:40:22.727 INFO:teuthology.orchestra.run.smithi120.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.8 MB/s | 90 kB 00:00 2024-09-10T22:40:22.744 INFO:teuthology.orchestra.run.smithi120.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-10T22:40:22.749 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:22.760 INFO:teuthology.orchestra.run.smithi120.stdout:(137/157): python3-repoze-lru-0.7-16.el9.noarch 1.8 MB/s | 31 kB 00:00 2024-09-10T22:40:22.785 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:22.794 INFO:teuthology.orchestra.run.smithi120.stdout:(138/157): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-10T22:40:22.820 INFO:teuthology.orchestra.run.smithi120.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-10T22:40:22.862 INFO:teuthology.orchestra.run.smithi120.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 882 kB/s | 59 kB 00:00 2024-09-10T22:40:22.887 INFO:teuthology.orchestra.run.smithi120.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-09-10T22:40:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:22.901 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:40:22.901 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:40:22.901 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:22.912 INFO:teuthology.orchestra.run.smithi120.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T22:40:22.928 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:40:22.937 INFO:teuthology.orchestra.run.smithi120.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-10T22:40:22.963 INFO:teuthology.orchestra.run.smithi120.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T22:40:22.988 INFO:teuthology.orchestra.run.smithi120.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-10T22:40:23.014 INFO:teuthology.orchestra.run.smithi120.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T22:40:23.039 INFO:teuthology.orchestra.run.smithi120.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T22:40:23.064 INFO:teuthology.orchestra.run.smithi120.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T22:40:23.131 INFO:teuthology.orchestra.run.smithi120.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T22:40:23.355 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-09-10T22:40:23.355 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-09-10T22:40:23.631 INFO:teuthology.orchestra.run.smithi120.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 5.8 MB/s | 3.2 MB 00:00 2024-09-10T22:40:23.674 INFO:teuthology.orchestra.run.smithi120.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 4.3 MB/s | 2.3 MB 00:00 2024-09-10T22:40:23.715 INFO:teuthology.orchestra.run.smithi120.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 984 kB/s | 41 kB 00:00 2024-09-10T22:40:23.774 INFO:teuthology.orchestra.run.smithi120.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T22:40:23.858 INFO:teuthology.orchestra.run.smithi120.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-10T22:40:23.899 INFO:teuthology.orchestra.run.smithi120.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 6.5 MB/s | 1.8 MB 00:00 2024-09-10T22:40:23.958 INFO:teuthology.orchestra.run.smithi120.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-10T22:40:24.108 INFO:teuthology.orchestra.run.smithi120.stdout:(157/157): librados2-19.3.0-4854.g8f219c47.el9. 3.2 MB/s | 3.5 MB 00:01 2024-09-10T22:40:24.115 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:40:24.116 INFO:teuthology.orchestra.run.smithi120.stdout:Total 8.4 MB/s | 240 MB 00:28 2024-09-10T22:40:25.374 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-09-10T22:40:25.528 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-09-10T22:40:25.528 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-09-10T22:40:27.852 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-09-10T22:40:27.852 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-09-10T22:40:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-09-10T22:40:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:40:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:40:30.046 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:40:30.088 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:40:30.114 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:40:30.156 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:40:30.482 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:40:30.606 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:30.730 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:30.757 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:30.849 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:30.878 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:40:30.908 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:40:31.062 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:40:31.118 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:40:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:31.251 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:31.280 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:31.335 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:31.356 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:31.379 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:31.566 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:31.691 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:31.804 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:40:31.899 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:31.964 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:32.292 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:40:32.335 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:40:32.369 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:40:32.467 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:40:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:40:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:40:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:40:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:40:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:40:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:40:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:40:32.809 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:40:32.844 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:40:32.921 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:40:32.955 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:40:32.998 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:40:33.027 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:40:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:40:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:40:33.215 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:40:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:40:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:40:33.924 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:40:34.015 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:40:34.044 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:40:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:40:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:40:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:40:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:40:34.370 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-09-10T22:40:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:40:34.463 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:40:34.513 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:40:34.528 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:40:34.557 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:40:34.601 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:40:34.619 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:40:34.641 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:40:34.652 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-09-10T22:40:34.671 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:34.677 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:40:34.686 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:40:34.724 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:40:34.745 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:40:34.759 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:40:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:40:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:40:34.855 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:40:34.872 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:40:34.873 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:34.873 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:40:34.873 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:34.980 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:40:35.022 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:40:35.063 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:40:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:40:35.165 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:40:35.188 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:35.205 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:40:35.244 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:40:35.270 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:40:35.294 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:35.295 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:40:35.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:40:35.423 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:35.461 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:40:35.490 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:40:35.496 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:40:35.527 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:40:35.555 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:40:35.599 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:40:35.638 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:40:35.664 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:40:35.669 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:40:35.699 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:40:35.717 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:40:35.733 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:40:35.753 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:35.766 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:40:35.808 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:40:35.844 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:40:35.858 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:35.888 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:40:35.942 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:40:35.972 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:35.981 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:40:35.993 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:40:36.046 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:40:36.149 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:40:36.183 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:36.197 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:40:36.344 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:40:36.384 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:40:36.410 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:40:36.410 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:36.411 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:40:36.411 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:40:36.411 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:40:36.411 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:36.488 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:40:36.595 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:36.651 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:40:36.673 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:36.674 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:40:36.675 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:36.675 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:40:36.675 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:40:36.675 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:40:36.675 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:36.899 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:40:36.920 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:40:36.977 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:40:37.004 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:40:37.011 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:40:37.036 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:40:37.071 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:40:37.110 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:40:37.127 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:40:37.150 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:40:37.150 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:37.150 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:40:37.150 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:40:37.150 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:40:37.151 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:37.166 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:40:37.177 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:40:37.211 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:40:37.222 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:40:37.254 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:40:37.257 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:40:37.298 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:40:37.305 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:40:37.313 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:40:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:40:37.348 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:40:37.375 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:40:37.377 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:40:37.443 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:40:37.497 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:40:37.529 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:40:37.529 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:40:37.553 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:40:37.562 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:40:37.571 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:40:37.610 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:40:37.630 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:40:37.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:40:37.682 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:40:37.688 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:40:37.733 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:40:37.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:40:37.777 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:40:37.843 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:40:37.911 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:40:37.924 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:40:37.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:40:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:40:37.993 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:40:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:40:38.119 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:40:38.171 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:40:38.276 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:40:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:40:38.483 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:40:38.510 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:40:38.542 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:40:38.608 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:40:38.633 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:40:38.692 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:40:38.776 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:40:38.805 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:40:38.849 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:40:38.895 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:40:38.945 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:40:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:40:39.182 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:40:39.273 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:40:39.338 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:40:39.402 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:40:39.443 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:40:39.479 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:40:39.525 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:40:39.561 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:40:39.605 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:40:39.633 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:40:39.647 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:40:39.723 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:40:39.751 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:40:39.783 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:40:39.784 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:40:39.832 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:40:39.882 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:40:39.984 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:40:40.017 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:40:40.057 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:40:40.090 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:40:40.103 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:40:40.115 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:40:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:40.200 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:40:40.246 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:40:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:40:40.324 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:40:40.359 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:40:40.390 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:40:40.428 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:40:40.470 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:40:40.518 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:40:40.556 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:40:40.590 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:40:40.624 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:40:40.666 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:40:40.700 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:40:40.788 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:40:40.813 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:40:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:40:40.841 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:40.870 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:40:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:40:40.973 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:40:41.022 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:40:41.730 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:40:41.777 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:40:41.814 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:40:41.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:40:41.872 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:40:41.926 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:40:41.983 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:40:42.028 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:40:42.117 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:40:42.156 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:40:42.339 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:40:42.368 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:40:42.456 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:40:42.840 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:40:42.873 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:40:42.986 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:40:43.104 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:40:43.153 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:40:43.183 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:40:43.208 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:43.226 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:40:43.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:40:43.271 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:40:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:43.376 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:40:43.423 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:40:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:44.061 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:44.061 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:40:44.061 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:40:44.061 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:44.095 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:40:44.375 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:40:44.411 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:40:44.435 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:40:44.436 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:44.436 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:40:44.436 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:40:44.436 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:40:44.436 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:44.465 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:40:44.478 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:40:44.501 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:40:44.502 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:44.502 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:40:44.502 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:44.546 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:40:44.559 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:40:44.569 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:44.596 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:40:44.629 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:40:44.880 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:40:44.903 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:40:44.904 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:44.904 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:40:44.904 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:40:44.904 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:40:44.904 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:40:44.958 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:40:45.000 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:45.092 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:40:45.122 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:40:45.158 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:40:45.652 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:40:45.687 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:45.758 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:40:45.788 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:40:48.125 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:48.177 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:48.734 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:40:48.781 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:48.875 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:40:48.875 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:40:48.875 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:40:48.875 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:40:48.916 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:40:49.592 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:40:49.633 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:40:49.673 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:40:49.706 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:40:49.733 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:40:49.914 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:40:49.971 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:40:50.002 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:40:50.032 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:40:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:40:50.599 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:40:50.633 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:40:50.653 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:40:51.493 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:40:51.588 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:40:51.628 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:40:51.709 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:40:51.749 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:40:51.774 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:40:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:40:51.801 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:40:51.885 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:40:51.885 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:40:51.977 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:40:52.002 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:40:52.007 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:40:52.008 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:40:52.189 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:40:52.189 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:40:52.271 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:40:52.334 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:40:55.720 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:40:55.720 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /root 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2024-09-10T22:40:55.721 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:55.907 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:40:55.923 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:40:55.923 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:55.924 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:40:55.924 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:57.434 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:40:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:40:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:58.152 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:40:58.176 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:40:58.176 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:40:58.177 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:40:58.177 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:40:58.177 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:40:58.177 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:40:58.283 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:40:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:40:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:40:58.583 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:40:58.620 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:40:58.620 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:40:58.621 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:40:58.622 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:40:58.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:40:58.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:40:58.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:40:58.630 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:40:58.635 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:40:58.636 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:40:58.637 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:40:58.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:40:58.639 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:40:58.640 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:40:58.641 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:40:58.642 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:40:58.643 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:40:58.644 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:40:58.644 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:40:58.644 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:40:58.644 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:40:58.644 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:40:58.706 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:40:58.795 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:40:58.941 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:40:58.986 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:40:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:40:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:40:59.511 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:40:59.532 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:40:59.560 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:40:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:40:59.859 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-09-10T22:41:01.182 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-09-10T22:41:01.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-09-10T22:41:01.183 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-09-10T22:41:01.183 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:01.309 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.310 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:41:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:41:01.313 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:41:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:41:01.315 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:41:01.316 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:41:01.317 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:41:01.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:41:01.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:01.320 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:41:01.321 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:41:01.322 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-09-10T22:41:01.364 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:41:01.395 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:41:01.395 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:01.754 DEBUG:teuthology.parallel:result is None 2024-09-10T22:41:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:41:02.890 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:41:02.901 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:41:02.928 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:41:02.929 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:02.929 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:41:02.929 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:41:02.929 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:41:02.929 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:03.161 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:41:03.184 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:41:03.184 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:03.184 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:41:03.184 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:41:03.184 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:41:03.185 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:03.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:41:03.660 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:03.775 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:41:03.831 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:41:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:41:03.874 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:41:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:41:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:41:04.072 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:41:04.121 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:41:04.143 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:41:04.167 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:41:04.190 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:41:04.192 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:04.243 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:41:04.255 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:41:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:41:04.353 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:41:04.400 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:41:04.436 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:41:04.480 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:41:04.512 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:41:04.540 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:41:04.978 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:41:05.005 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:41:05.037 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:41:05.111 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:41:05.145 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:41:05.348 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:41:05.396 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:41:05.420 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:05.463 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:41:05.486 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:41:05.486 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:05.486 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:41:05.486 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:05.538 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:41:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:41:05.562 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:05.562 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:41:05.562 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:41:05.563 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:41:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:05.873 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:41:05.896 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:41:05.896 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:05.896 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:41:05.897 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:41:05.897 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:41:05.897 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:08.314 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:41:08.377 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:41:09.342 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:41:09.369 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:41:09.484 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:41:09.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:41:09.680 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:41:09.704 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:41:09.704 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:09.704 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:41:09.704 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:41:09.704 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:41:09.705 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:09.757 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:41:09.768 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:41:09.874 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:41:09.933 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:41:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:41:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:41:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:41:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:41:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:41:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:41:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:41:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:10.980 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:41:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:41:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:41:11.008 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:11.008 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:41:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:11.031 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:41:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:41:11.061 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:41:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:11.090 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:41:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:41:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:41:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:11.742 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:11.784 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:11.805 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:12.639 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:41:12.764 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:41:12.805 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:41:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:41:12.925 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:41:12.951 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:41:12.977 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:41:12.977 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:41:13.069 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:41:13.069 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:41:13.153 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:41:13.178 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:41:13.184 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:13.184 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:13.363 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:13.363 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:41:13.447 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:41:13.511 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:16.215 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:41:16.264 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:41:16.305 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:41:16.338 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:41:16.365 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:41:16.551 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:41:16.596 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:41:16.627 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:41:16.655 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:41:16.721 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:41:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:17.309 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:17.329 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:41:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:41:18.232 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:41:18.277 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:41:18.362 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:41:18.402 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:41:18.427 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:41:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:41:18.453 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:41:18.521 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:41:18.521 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:41:18.605 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:41:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:41:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:18.635 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:18.815 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:41:18.816 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:41:18.900 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:41:18.973 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:41:19.725 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:41:19.726 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:41:19.727 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:41:19.740 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:41:19.740 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:41:19.740 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:41:19.740 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:41:19.741 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:41:19.742 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:41:19.743 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:41:19.744 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:41:19.745 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:41:19.746 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:41:19.747 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:41:19.748 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:41:19.749 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:41:19.750 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:41:19.751 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:41:19.751 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:41:19.751 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:41:19.751 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:41:19.751 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:41:22.297 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:22.297 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:22.297 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-09-10T22:41:22.297 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:22.297 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:41:22.298 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.299 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:41:22.300 INFO:teuthology.orchestra.run.smithi176.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.301 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:41:22.302 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:41:22.303 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:41:22.304 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:41:22.305 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:41:22.306 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:41:22.307 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:41:22.308 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:41:22.309 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:41:22.310 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:41:22.310 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:41:22.310 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-09-10T22:41:22.719 DEBUG:teuthology.parallel:result is None 2024-09-10T22:41:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:41:25.582 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:41:25.583 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:41:25.584 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:41:25.585 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:41:25.586 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:41:25.587 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:41:25.588 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:41:25.589 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:41:25.590 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:41:25.591 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:41:25.592 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:41:25.593 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:41:25.594 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:41:25.595 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:41:25.596 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:41:28.251 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:41:28.251 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:28.251 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-09-10T22:41:28.251 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:41:28.252 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.253 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:41:28.254 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:41:28.255 INFO:teuthology.orchestra.run.smithi120.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:41:28.256 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:41:28.257 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:41:28.258 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:41:28.259 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:41:28.260 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:41:28.261 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:41:28.262 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:41:28.263 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:41:28.264 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:41:28.264 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:41:28.264 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-09-10T22:41:28.716 DEBUG:teuthology.parallel:result is None 2024-09-10T22:41:28.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:28.877 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:41:28.900 INFO:teuthology.orchestra.run.smithi120.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:28.900 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:28.900 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:41:28.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:29.050 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:41:29.076 INFO:teuthology.orchestra.run.smithi153.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:29.076 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:29.077 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:41:29.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:29.225 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:41:29.252 INFO:teuthology.orchestra.run.smithi176.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:29.252 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:41:29.252 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:41:29.254 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T22:41:29.254 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:41:29.254 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:41:29.283 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:41:29.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:41:29.317 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:41:29.317 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:41:29.350 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T22:41:29.350 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:41:29.350 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:41:29.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:41:29.442 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:41:29.442 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:41:29.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:41:29.538 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:41:29.538 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:41:29.570 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:41:29.637 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T22:41:29.637 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:41:29.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:41:29.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:41:29.729 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:41:29.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:41:29.758 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:41:29.825 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:41:29.825 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:41:29.852 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:41:29.916 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T22:41:29.916 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:41:29.917 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:41:29.942 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:41:30.008 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:41:30.008 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:41:30.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:41:30.099 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:41:30.099 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:41:30.124 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:41:30.189 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T22:41:30.284 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T22:41:30.284 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:30.285 INFO:tasks.cephadm:Cluster fsid is d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:41:30.285 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T22:41:30.285 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.120', 'mon.b': '172.21.15.153', 'mon.c': '172.21.15.176'} 2024-09-10T22:41:30.286 INFO:tasks.cephadm:First mon is mon.a on smithi120 2024-09-10T22:41:30.286 INFO:tasks.cephadm:First mgr is x 2024-09-10T22:41:30.286 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T22:41:30.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-09-10T22:41:30.312 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-09-10T22:41:30.338 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2024-09-10T22:41:30.364 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T22:41:30.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:30.522 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T22:41:30.671 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:41:30.673 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:41:30.674 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:41:30.674 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.051 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:41 /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.051 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.448 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:41 /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.449 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.894 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:41 /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.895 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.915 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.932 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:41:31.966 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T22:41:31.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:41:31.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:41:31.974 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:41:33.588 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:41:33.653 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:41:33.915 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:42:02.846 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-10T22:42:02.847 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-10T22:42:02.954 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:42:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:42:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:42:02.954 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-09-10T22:42:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:42:02.955 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T22:42:02.955 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:42:06.725 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-09-10T22:42:06.726 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-09-10T22:42:06.751 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-09-10T22:42:06.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-09-10T22:42:06.812 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2024-09-10T22:42:06.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-09-10T22:42:06.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-09-10T22:42:06.899 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2024-09-10T22:42:06.926 INFO:tasks.cephadm:Writing seed config... 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-10T22:42:06.928 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T22:42:06.929 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-10T22:42:06.930 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:42:06.930 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T22:42:06.947 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d2838936-6fc5-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T22:42:06.948 DEBUG:teuthology.orchestra.run.smithi120:mon.a> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T22:42:06.991 DEBUG:teuthology.orchestra.run.smithi120:mgr.x> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T22:42:07.034 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T22:42:07.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.120 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:42:07.238 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:42:07.238 INFO:teuthology.orchestra.run.smithi120.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.120', '--skip-admin-label'] 2024-09-10T22:42:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T22:42:07.239 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying podman|docker is present... 2024-09-10T22:42:07.263 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:42:07.263 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying lvm2 is present... 2024-09-10T22:42:07.263 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying time synchronization is in place... 2024-09-10T22:42:07.271 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:42:07.271 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:42:07.277 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:42:07.278 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:42:07.284 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2024-09-10T22:42:07.291 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2024-09-10T22:42:07.291 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:42:07.291 INFO:teuthology.orchestra.run.smithi120.stdout:Repeating the final host check... 2024-09-10T22:42:07.313 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:42:07.314 INFO:teuthology.orchestra.run.smithi120.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T22:42:07.314 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl is present 2024-09-10T22:42:07.314 INFO:teuthology.orchestra.run.smithi120.stdout:lvcreate is present 2024-09-10T22:42:07.320 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:42:07.320 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:42:07.326 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:42:07.326 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:42:07.333 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout enabled 2024-09-10T22:42:07.339 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout active 2024-09-10T22:42:07.339 INFO:teuthology.orchestra.run.smithi120.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:42:07.339 INFO:teuthology.orchestra.run.smithi120.stdout:Host looks OK 2024-09-10T22:42:07.339 INFO:teuthology.orchestra.run.smithi120.stdout:Cluster fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:07.340 INFO:teuthology.orchestra.run.smithi120.stdout:Acquiring lock 140128750457712 on /run/cephadm/d2838936-6fc5-11ef-bcea-c7b262605968.lock 2024-09-10T22:42:07.340 INFO:teuthology.orchestra.run.smithi120.stdout:Lock 140128750457712 acquired on /run/cephadm/d2838936-6fc5-11ef-bcea-c7b262605968.lock 2024-09-10T22:42:07.340 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 3300 ... 2024-09-10T22:42:07.340 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying IP 172.21.15.120 port 6789 ... 2024-09-10T22:42:07.340 INFO:teuthology.orchestra.run.smithi120.stdout:Base mon IP(s) is [172.21.15.120:3300, 172.21.15.120:6789], mon addrv is [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-10T22:42:07.343 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.120 metric 100 2024-09-10T22:42:07.344 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.120 metric 100 2024-09-10T22:42:07.346 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T22:42:07.346 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T22:42:07.346 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T22:42:07.348 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:42:07.349 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2024-09-10T22:42:07.350 INFO:teuthology.orchestra.run.smithi120.stdout:Mon IP `172.21.15.120` is in CIDR network `172.21.0.0/20` 2024-09-10T22:42:07.350 INFO:teuthology.orchestra.run.smithi120.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T22:42:07.350 INFO:teuthology.orchestra.run.smithi120.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T22:42:07.350 INFO:teuthology.orchestra.run.smithi120.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:42:07.731 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:42:07.731 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:42:07.731 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T22:42:07.731 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T22:42:07.732 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T22:42:07.732 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:42:07.732 INFO:teuthology.orchestra.run.smithi120.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T22:42:08.710 INFO:teuthology.orchestra.run.smithi120.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:42:08.711 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:42:08.711 INFO:teuthology.orchestra.run.smithi120.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T22:42:09.624 INFO:teuthology.orchestra.run.smithi120.stdout:stat: stdout 167 167 2024-09-10T22:42:09.624 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial keys... 2024-09-10T22:42:10.505 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBBy+Bms8r9OBAAixE0RdOg5rvJ0Ut9MmB3ZA== 2024-09-10T22:42:11.385 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBCy+BmktvgLxAAnYH0dMHIhKhzIxp/BE5kyQ== 2024-09-10T22:42:12.291 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph-authtool: stdout AQBDy+BmsmdFKRAAfEqEHpjaSSI95MyEDfyrVg== 2024-09-10T22:42:12.292 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial monmap... 2024-09-10T22:42:20.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:42:20.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T22:42:20.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:20.701 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:42:20.701 INFO:teuthology.orchestra.run.smithi120.stdout:monmaptool for a [v2:172.21.15.120:3300,v1:172.21.15.120:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:42:20.702 INFO:teuthology.orchestra.run.smithi120.stdout:setting min_mon_release = quincy 2024-09-10T22:42:20.702 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: set fsid to d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:20.702 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:42:20.702 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:42:20.702 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mon... 2024-09-10T22:42:25.261 INFO:teuthology.orchestra.run.smithi120.stdout:create mon.a on 2024-09-10T22:42:25.484 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T22:42:25.626 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T22:42:25.802 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target. 2024-09-10T22:42:25.802 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target. 2024-09-10T22:42:26.314 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a 2024-09-10T22:42:26.314 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T22:42:26.474 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target.wants/ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968@.service. 2024-09-10T22:42:27.026 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:42:27.026 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-10T22:42:27.033 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:42:27.033 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:42:27.033 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-10T22:42:27.033 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:42:27.033 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon to start... 2024-09-10T22:42:27.034 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mon... 2024-09-10T22:42:27.138 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 systemd[1]: Started Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:42:28.021 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T22:42:28.022 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout id: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout services: 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.509324s) 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout data: 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T22:42:28.023 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:28.024 INFO:teuthology.orchestra.run.smithi120.stdout:mon is available 2024-09-10T22:42:28.024 INFO:teuthology.orchestra.run.smithi120.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T22:42:28.127 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: monmap epoch 1 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: last_changed 2024-09-10T22:42:13.658772+0000 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: min_mon_release 19 (squid) 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: election_strategy: 1 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: fsmap 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: mgrmap e1: no daemons active 2024-09-10T22:42:28.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:27 smithi120 ceph-mon[30877]: from='client.? 172.21.15.120:0/3259130135' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T22:42:29.091 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:28 smithi120 ceph-mon[30877]: from='client.? 172.21.15.120:0/3444001939' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:42:29.091 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:28 smithi120 ceph-mon[30877]: from='client.? 172.21.15.120:0/3444001939' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T22:42:29.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:29.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:42:29.095 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:42:29.096 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:42:29.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:42:29.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:42:29.097 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:42:29.097 INFO:teuthology.orchestra.run.smithi120.stdout:Generating new minimal ceph.conf... 2024-09-10T22:42:30.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:29 smithi120 ceph-mon[30877]: from='client.? 172.21.15.120:0/3999820160' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:31.482 INFO:teuthology.orchestra.run.smithi120.stdout:Restarting the monitor... 2024-09-10T22:42:31.764 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:31 smithi120 systemd[1]: Stopping Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:42:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a[30873]: 2024-09-10T22:42:31.762+0000 7fbeb03f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:42:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a[30873]: 2024-09-10T22:42:31.762+0000 7fbeb03f1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T22:42:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:31 smithi120 podman[31121]: 2024-09-10 22:42:31.946999902 +0000 UTC m=+0.299813967 container died 45ca949fef11ed0dbcf10b272bb76762377cef5fdc079d6c694a89b02e7705fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T22:42:32.678 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 podman[31121]: 2024-09-10 22:42:32.304852441 +0000 UTC m=+0.657666514 container cleanup 45ca949fef11ed0dbcf10b272bb76762377cef5fdc079d6c694a89b02e7705fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:42:32.678 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 bash[31121]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a 2024-09-10T22:42:32.678 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 podman[31133]: 2024-09-10 22:42:32.485144431 +0000 UTC m=+0.530783723 container remove 45ca949fef11ed0dbcf10b272bb76762377cef5fdc079d6c694a89b02e7705fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T22:42:33.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 systemd[1]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T22:42:33.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 systemd[1]: Stopped Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:42:33.041 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 systemd[1]: Starting Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:42:33.042 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 podman[31228]: 2024-09-10 22:42:32.816451025 +0000 UTC m=+0.015813250 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:42:33.042 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:32 smithi120 podman[31228]: 2024-09-10 22:42:32.914988702 +0000 UTC m=+0.114350919 container create 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T22:42:33.214 INFO:teuthology.orchestra.run.smithi120.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 podman[31228]: 2024-09-10 22:42:33.133909185 +0000 UTC m=+0.333271411 container init 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 podman[31228]: 2024-09-10 22:42:33.136987718 +0000 UTC m=+0.336349935 container start 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: pidfile_write: ignore empty --pid-file 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: load: jerasure load: lrc 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Git sha 0 2024-09-10T22:42:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: DB SUMMARY 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: DB Session ID: VS19AL0OR55PL36QK1MU 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95504 ; 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:42:33.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:42:33.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:42:33.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.env: 0x5576ebe69180 2024-09-10T22:42:33.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:42:33.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.info_log: 0x5576ed47d700 2024-09-10T22:42:33.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:42:33.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.statistics: (nil) 2024-09-10T22:42:33.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.use_fsync: 0 2024-09-10T22:42:33.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:42:33.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.db_log_dir: 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.wal_dir: 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:42:33.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.write_buffer_manager: 0x5576ed481b80 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:42:33.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.unordered_write: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.row_cache: None 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.wal_filter: None 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:42:33.453 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.wal_compression: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:42:33.454 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_open_files: -1 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:42:33.455 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Compression algorithms supported: 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kZSTD supported: 0 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:42:33.456 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.merge_operator: 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_filter: None 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5576ed47d320) 2024-09-10T22:42:33.457 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_type: 0 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_index_type: 0 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_shortening: 1 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: checksum: 4 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: no_block_cache: 0 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache: 0x5576ed471610 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_options: 2024-09-10T22:42:33.458 INFO:journalctl@ceph.mon.a.smithi120.stdout: capacity : 536870912 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: num_shard_bits : 4 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: strict_capacity_limit : 0 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_cache_compressed: (nil) 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: persistent_cache: (nil) 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size: 4096 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_size_deviation: 10 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_restart_interval: 16 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: index_block_restart_interval: 1 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: metadata_block_size: 4096 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: partition_filters: 0 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: use_delta_encoding: 1 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: filter_policy: bloomfilter 2024-09-10T22:42:33.459 INFO:journalctl@ceph.mon.a.smithi120.stdout: whole_key_filtering: 1 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: verify_compression: 0 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: format_version: 5 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_index_compression: 1 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: block_align: 0 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: max_auto_readahead_size: 262144 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: prepopulate_block_cache: 0 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression: NoCompression 2024-09-10T22:42:33.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.num_levels: 7 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:42:33.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:42:33.462 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:42:33.463 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:42:33.464 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:42:33.465 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.ttl: 2592000 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:42:33.466 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1df01695-c7e9-46b8-a5a8-e14474927615 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008153163743, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008153165075, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 287, "table_properties": {"data_size": 89676, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11789, "raw_average_key_size": 47, "raw_value_size": 83032, "raw_average_value_size": 337, "num_data_blocks": 12, "num_entries": 246, "num_filter_entries": 246, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726008153, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1df01695-c7e9-46b8-a5a8-e14474927615", "db_session_id": "VS19AL0OR55PL36QK1MU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726008153165201, "job": 1, "event": "recovery_finished"} 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5576ed4a0e00 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: DB pointer 0x5576ed5a2000 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** DB Stats ** 2024-09-10T22:42:33.467 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: L0 2/0 91.34 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Sum 2/0 91.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:42:33.468 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** Compaction Stats [default] ** 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 109.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Cumulative compaction: 0.00 GB write, 20.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:42:33.469 INFO:journalctl@ceph.mon.a.smithi120.stdout: Interval compaction: 0.00 GB write, 20.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: Block cache BinnedLRUCache@0x5576ed471610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: starting mon.a rank 0 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???) e1 preinit fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).mds e1 new map 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).mds e1 print_map 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: e1 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: btime 2024-09-10T22:42:26:976507+0000 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: legacy client fscid: -1 2024-09-10T22:42:33.470 INFO:journalctl@ceph.mon.a.smithi120.stdout: 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout: No filesystems configured 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 bash[31228]: 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 systemd[1]: Started Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: monmap epoch 1 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: last_changed 2024-09-10T22:42:13.658772+0000 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:42:33.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: min_mon_release 19 (squid) 2024-09-10T22:42:33.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: election_strategy: 1 2024-09-10T22:42:33.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:42:33.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: fsmap 2024-09-10T22:42:33.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T22:42:33.472 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:33 smithi120 ceph-mon[31247]: mgrmap e1: no daemons active 2024-09-10T22:42:34.260 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T22:42:34.282 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:42:34.282 INFO:teuthology.orchestra.run.smithi120.stdout:Creating mgr... 2024-09-10T22:42:34.282 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T22:42:34.283 INFO:teuthology.orchestra.run.smithi120.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T22:42:34.542 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x 2024-09-10T22:42:34.542 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T22:42:34.692 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968.target.wants/ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d2838936-6fc5-11ef-bcea-c7b262605968@.service. 2024-09-10T22:42:34.918 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:34 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1017645114' entity='client.admin' 2024-09-10T22:42:35.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:35 smithi120 podman[31460]: 2024-09-10 22:42:35.138601923 +0000 UTC m=+0.331675795 container start cbfc6d50a4ef89faeca222bda3492d8aa498700b60869eade594e6290cd6d959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:42:35.240 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:42:35.240 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-10T22:42:35.248 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:42:35.249 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:42:35.249 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-10T22:42:35.249 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:42:35.258 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:42:35.258 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr to start... 2024-09-10T22:42:35.266 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr... 2024-09-10T22:42:35.505 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:35 smithi120 bash[31460]: cbfc6d50a4ef89faeca222bda3492d8aa498700b60869eade594e6290cd6d959 2024-09-10T22:42:35.505 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:35 smithi120 systemd[1]: Started Ceph mgr.x for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:42:36.101 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:35 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3800028682' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:42:36.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:36.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:42:36.403 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "d2838936-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:36.404 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:42:36.405 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:42:26:976507+0000", 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:42:36.406 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:42:26.977148+0000", 2024-09-10T22:42:36.407 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:42:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:42:36.408 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (1/15)... 2024-09-10T22:42:39.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:38 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/102187146' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "d2838936-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:42:39.516 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.517 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:42:39.518 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:42:26:976507+0000", 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:42:39.519 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:42:26.977148+0000", 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:39.520 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:42:39.521 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:42:39.521 INFO:teuthology.orchestra.run.smithi120.stdout:mgr not available, waiting (2/15)... 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: Activating manager daemon x 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: mgrmap e2: x(active, starting, since 0.00313839s) 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:42:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: Manager daemon x is now available 2024-09-10T22:42:40.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:42:40.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:42:40.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' 2024-09-10T22:42:40.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' 2024-09-10T22:42:40.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:40 smithi120 ceph-mon[31247]: from='mgr.14100 172.21.15.120:0/447705754' entity='mgr.x' 2024-09-10T22:42:42.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:42 smithi120 ceph-mon[31247]: mgrmap e3: x(active, since 1.00724s) 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsid": "d2838936-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:42:42.772 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:42:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:42:42.774 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:42:26:976507+0000", 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:42:42.775 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:42:26.977148+0000", 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:42:42.776 INFO:teuthology.orchestra.run.smithi120.stdout:mgr is available 2024-09-10T22:42:43.336 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:43 smithi120 ceph-mon[31247]: mgrmap e4: x(active, since 2s) 2024-09-10T22:42:43.336 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:43 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2591527118' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:42:43.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:43.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:42:43.935 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout fsid = d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.120:3300,v1:172.21.15.120:6789] 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 2024-09-10T22:42:43.936 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:42:43.937 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling cephadm module... 2024-09-10T22:42:44.316 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:44 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1383454892' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:42:45.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:45 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1867239848' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T22:42:45.447 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ignoring --setuser ceph since I am not root 2024-09-10T22:42:45.447 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ignoring --setgroup ceph since I am not root 2024-09-10T22:42:46.384 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1867239848' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T22:42:46.384 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:46 smithi120 ceph-mon[31247]: mgrmap e5: x(active, since 5s) 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2024-09-10T22:42:47.164 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 5... 2024-09-10T22:42:47.408 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:47 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/489746174' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:42:50.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: Active manager daemon x restarted 2024-09-10T22:42:50.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: Activating manager daemon x 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: mgrmap e6: x(active, starting, since 0.0033271s) 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:42:50.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:50 smithi120 ceph-mon[31247]: Manager daemon x is now available 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 5 is available 2024-09-10T22:42:51.779 INFO:teuthology.orchestra.run.smithi120.stdout:Setting orchestrator backend to cephadm... 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: Found migration_current of "None". Setting to last migration. 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: mgrmap e7: x(active, since 1.00634s) 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:42:51.795+0000 7f91f9476640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:42:51] ENGINE Error in HTTPServer.serve 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: Traceback (most recent call last): 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:42:52.105 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._connections.run(self.expiration_interval) 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._run(expiration_interval) 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: s = self.context.wrap_socket( 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: return self.sslsocket_class._create( 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self.do_handshake() 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:42:52.106 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._sslobj.do_handshake() 2024-09-10T22:42:52.107 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:42:52.107 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:42:52.957 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: [10/Sep/2024:22:42:51] ENGINE Bus STARTING 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: [10/Sep/2024:22:42:51] ENGINE Serving on http://172.21.15.120:8765 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: [10/Sep/2024:22:42:51] ENGINE Serving on https://172.21.15.120:7150 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: [10/Sep/2024:22:42:51] ENGINE Bus STARTED 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: [10/Sep/2024:22:42:51] ENGINE Error in HTTPServer.serve 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout: Traceback (most recent call last): 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:42:52.958 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._run(expiration_interval) 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: s = self.context.wrap_socket( 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: return self.sslsocket_class._create( 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:42:52.959 INFO:journalctl@ceph.mon.a.smithi120.stdout: self.do_handshake() 2024-09-10T22:42:52.960 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:42:52.960 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._sslobj.do_handshake() 2024-09-10T22:42:52.960 INFO:journalctl@ceph.mon.a.smithi120.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:42:52.960 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:52.960 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:52 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:54.139 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:53 smithi120 ceph-mon[31247]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:54.139 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:53 smithi120 ceph-mon[31247]: mgrmap e8: x(active, since 2s) 2024-09-10T22:42:54.142 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T22:42:54.142 INFO:teuthology.orchestra.run.smithi120.stdout:Generating ssh key... 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:54 smithi120 ceph-mon[31247]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: Generating public/private rsa key pair. 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: Your identification has been saved in /tmp/tmp53eeroc5/key 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: Your public key has been saved in /tmp/tmp53eeroc5/key.pub 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: The key fingerprint is: 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: SHA256:ig2PApAOgZhiFNEx7YtPISoPr12YrcDtZnduNe16KXk ceph-d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: The key's randomart image is: 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: +---[RSA 3072]----+ 2024-09-10T22:42:55.197 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |==+oo | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |=+ ... | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |* . | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |+ . o | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |... + o S . | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |+o.= O . o . | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |o=+.B + . + . | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: | o+* o o o E | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: |..=.. +. .= | 2024-09-10T22:42:55.198 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:42:55 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: +----[SHA256]-----+ 2024-09-10T22:42:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:56 smithi120 ceph-mon[31247]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:56 smithi120 ceph-mon[31247]: Generating ssh key... 2024-09-10T22:42:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:56 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:56 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:42:56.698 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:42:56.698 INFO:teuthology.orchestra.run.smithi120.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:42:56.698 INFO:teuthology.orchestra.run.smithi120.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T22:42:56.708 INFO:teuthology.orchestra.run.smithi120.stdout:Adding host smithi120... 2024-09-10T22:42:57.413 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:57 smithi120 ceph-mon[31247]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:58.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:58 smithi120 ceph-mon[31247]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "addr": "172.21.15.120", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:58.311 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:42:58 smithi120 ceph-mon[31247]: Deploying cephadm binary to smithi120 2024-09-10T22:43:01.318 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Added host 'smithi120' with addr '172.21.15.120' 2024-09-10T22:43:01.319 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mon service... 2024-09-10T22:43:01.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:01 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:01.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:01 smithi120 ceph-mon[31247]: Added host smithi120 2024-09-10T22:43:01.577 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:01 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:03.297 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:02 smithi120 ceph-mon[31247]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:03.297 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:02 smithi120 ceph-mon[31247]: Saving service mon spec with placement count:5 2024-09-10T22:43:03.298 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:02 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:03.298 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:02 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:03.299 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T22:43:03.299 INFO:teuthology.orchestra.run.smithi120.stdout:Deploying unmanaged mgr service... 2024-09-10T22:43:04.580 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:04 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:04.580 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:04 smithi120 ceph-mon[31247]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:04.580 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:04 smithi120 ceph-mon[31247]: Saving service mgr spec with placement count:2 2024-09-10T22:43:04.580 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:04 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:05.228 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T22:43:06.139 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:05 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:06.140 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:05 smithi120 ceph-mon[31247]: from='mgr.14118 172.21.15.120:0/3041046220' entity='mgr.x' 2024-09-10T22:43:07.044 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3776933902' entity='client.admin' 2024-09-10T22:43:08.072 INFO:teuthology.orchestra.run.smithi120.stdout:Enabling the dashboard module... 2024-09-10T22:43:08.194 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:08 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/940657327' entity='client.admin' 2024-09-10T22:43:09.089 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:09 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1440440114' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T22:43:09.362 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:09 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ignoring --setuser ceph since I am not root 2024-09-10T22:43:09.362 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:09 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ignoring --setgroup ceph since I am not root 2024-09-10T22:43:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1440440114' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T22:43:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:10 smithi120 ceph-mon[31247]: mgrmap e9: x(active, since 18s) 2024-09-10T22:43:10.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:43:10.914 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for the mgr to restart... 2024-09-10T22:43:10.915 INFO:teuthology.orchestra.run.smithi120.stdout:Waiting for mgr epoch 9... 2024-09-10T22:43:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:11 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/546686188' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: Active manager daemon x restarted 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: Activating manager daemon x 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: mgrmap e10: x(active, starting, since 0.00331066s) 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:43:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:43:14.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:14 smithi120 ceph-mon[31247]: Manager daemon x is now available 2024-09-10T22:43:15.443 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:15.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:43:15.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:43:15.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:43:15.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: mgrmap e11: x(active, since 1.00596s) 2024-09-10T22:43:15.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:15 smithi120 ceph-mon[31247]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:43:16.181 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:43:16.081+0000 7f3f6e268640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:43:16] ENGINE Error in HTTPServer.serve 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: Traceback (most recent call last): 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._connections.run(self.expiration_interval) 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._run(expiration_interval) 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:43:16.182 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: s = self.context.wrap_socket( 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: return self.sslsocket_class._create( 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self.do_handshake() 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: self._sslobj.do_handshake() 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:43:16.183 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:mgr epoch 9 is available 2024-09-10T22:43:16.184 INFO:teuthology.orchestra.run.smithi120.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: [10/Sep/2024:22:43:15] ENGINE Bus STARTING 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: [10/Sep/2024:22:43:15] ENGINE Serving on http://172.21.15.120:8765 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: [10/Sep/2024:22:43:16] ENGINE Serving on https://172.21.15.120:7150 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: [10/Sep/2024:22:43:16] ENGINE Bus STARTED 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:16 smithi120 ceph-mon[31247]: [10/Sep/2024:22:43:16] ENGINE Error in HTTPServer.serve 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout: Traceback (most recent call last): 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:43:17.197 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._run(expiration_interval) 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: s = self.context.wrap_socket( 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: return self.sslsocket_class._create( 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: self.do_handshake() 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:43:17.198 INFO:journalctl@ceph.mon.a.smithi120.stdout: self._sslobj.do_handshake() 2024-09-10T22:43:17.199 INFO:journalctl@ceph.mon.a.smithi120.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:43:18.050 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:17 smithi120 ceph-mon[31247]: mgrmap e12: x(active, since 2s) 2024-09-10T22:43:18.050 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:18.050 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:17 smithi120 ceph-mon[31247]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:18.052 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFETCCAvmgAwIBAgIUNCiVgwmAEJDPn+2M5KUzTeyykLMwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIyNDMxN1oXDTM0\nMDkxMTIyNDMxN1owGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjEyMDCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBALp/T1A0M6BNDMjRbDGB0NEsXaRjnBFiOMtL\nChJpN7+8piQRzqvVYLFHgaeJumhaYsb9hMJvne0iArckVGT/7Fr/FrxQof+EEDVf\nkiwmwf3uft0PImanHoEf8PhEWnMcVGH53lfh1Dw/4/t6whmrHPlkppxpR7WkB//t\njkUOtqG/jMapHs7RvLFyoVK17a9oTlk1wEoBJiDLfghYhl6YXZJPoCGi6rhRhNB6\nqTFwE5llqo/NFDL0MydRh5be39UvZsLArndKCOEcspDuVXjCeAEgs9z4hE/IHGOw\nlcMjSiFrGms21akmnlzLqIyYsHPWKjIdLWVLYS/bwHepdi1FUr4ehBdobiyWAkmJ\n5jZjl9X2utCdlqhwQBYV11WNzvzQzRBowZNe2JEG4c3nQSpN3PBLlx5XTmulMF0R\nWZCH0lz5OQGy1gbdQnPzW9aymVwf6GP1wypMD3e8Y78FtXPPB1Z2KYAi3kR+P7Sj\nGgyhmeJGuA70/foY9JqaxaGo3DGRYyrdzLbOzUyiLW8jOeTzuHsQfC+LjemLQuCV\n1raEQlFEAjFH0DGq703zn7/gButg8uL2eYTiN5WuKL8q08lhrJG3Z7m2jbKxGB5t\nR7AqZ4ex/WAAQLgS4ZWEVhzTuim6A9LtPgEAl7mET002qlaFJy7DkwmdhKpYcSkO\nGHBGvJYTAgMBAAGjVDBSMEIGA1UdEQQ7MDmCHnNtaXRoaTEyMC5mcm9udC5zZXBp\nYS5jZXBoLmNvbYIRZGFzaGJvYXJkX3NlcnZlcnOHBKwVD3gwDAYDVR0TAQH/BAIw\nADANBgkqhkiG9w0BAQsFAAOCAgEAXZXXbAVH4sRitXVu2CMc83lbfvWDq1fSlckq\nuiEs1qyRiFG0i4VIelqu/zkWtFonhufv8snS50SHTSd7V5BFgDdzMaC+aQlzCt9T\n/Nxa6nd6Q7t4FwhLuDF+Vpmm8DhvDmyz/cbK1RaRAbgfDGS0z2hwv94C47ifWTis\ng4BrzLZhjVGtLVCelNnXXlsRoIbSVhQ/lXbpad0L5sQTntd788BYColInFcF3HCQ\nx5rfi4USc+4caCP+UVali2WrsGWXN8QxWAq/WyKwLZEDv5amtH6oW/7Kp8/nn+m1\nYsGLWU7YBsouZ2LpP09yxNJ7B9Z1cEUIUFMtju82IpQakdRxPRnh0XxlVsVnlhHD\nGbjKYJ5SA3wa9Y8DgQHhNIEhkZnxzPELuxEdtajiukhI7lg3LB4kn/J0xiFpzsED\na/FaUeJr35JAGe4CnChQEKEX2+HTfeJoXECKRJd+PInoAH1D7MPGLdWBzQ8IKhOe\nroROa+WFhAMQOMKDqlJ0aXGbJ1el55R5W98jSs8e2IpWZW1X1opT9evd8d0b1Xiz\n4xSsi8D45GQ+DZH/BJYwSR2dZbnDa2cxq/Hz4xlI9TnheOjx7zUBoq03lEuw8XFe\njpgBZH6uUOaa8ElhALvD5tq/O8tTxlPXW902k88YH5BjuDZy+ng9ExAuWFUQvwYi\neAqhLS0=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAun9PUDQzoE0MyNFsMYHQ0SxdpGOcEWI4y0sKEmk3v7ymJBHO\nq9VgsUeBp4m6aFpixv2Ewm+d7SICtyRUZP/sWv8WvFCh/4QQNV+SLCbB/e5+3Q8i\nZqcegR/w+ERacxxUYfneV+HUPD/j+3rCGasc+WSmnGlHtaQH/+2ORQ62ob+Mxqke\nztG8sXKhUrXtr2hOWTXASgEmIMt+CFiGXphdkk+gIaLquFGE0HqpMXATmWWqj80U\nMvQzJ1GHlt7f1S9mwsCud0oI4RyykO5VeMJ4ASCz3PiET8gcY7CVwyNKIWsaazbV\nqSaeXMuojJiwc9YqMh0tZUthL9vAd6l2LUVSvh6EF2huLJYCSYnmNmOX1fa60J2W\nqHBAFhXXVY3O/NDNEGjBk17YkQbhzedBKk3c8EuXHldOa6UwXRFZkIfSXPk5AbLW\nBt1Cc/Nb1rKZXB/oY/XDKkwPd7xjvwW1c88HVnYpgCLeRH4/tKMaDKGZ4ka4DvT9\n+hj0mprFoajcMZFjKt3Mts7NTKItbyM55PO4exB8L4uN6YtC4JXWtoRCUUQCMUfQ\nMarvTfOfv+AG62Dy4vZ5hOI3la4ovyrTyWGskbdnubaNsrEYHm1HsCpnh7H9YABA\nuBLhlYRWHNO6KboD0u0+AQCXuYRPTTaqVoUnLsOTCZ2EqlhxKQ4YcEa8lhMCAwEA\nAQKCAgAFXmzgw4qeRBmTc76dnj3DXSJE/T5yRwBDj0whmOMrTeRVMtNe9wGVmO1j\n7qnDH0xWANkK5T5dTpatbF+m7gcr3MlsGmt6I58j1OCBt4vrd4DcyaFqkHJnB+MJ\nV/kSn0TT8pGIDiRB/Ro0adBJ/4xnUZJ0+5g+ORP8i3uGWWoV4KHOMPLbbrDz8CIy\nHgf9qLm4r7zID0hoYUimN4YxavQmpiGCAQ1Tc7qoax+RIMeE/vxDWjAJV3uxghNZ\nMIlZEHLw4Fh6w/2FctdhJ2X71/1ZwEtCl+pB3GI/gcc4mud4wXs9OD2RD7TiDwAx\n1qNzHOiSEMJMduMRXE4qI7gXODZYM4d+ai7SH2TmsnonKfcSWeK6RIb+PBJZ2FUg\naL0bR7/GHiOMZLwSeKYCieDXKPl3fI/CL+8Se62Kv0ldU9MbhtH0tIdHl3fR0zPo\ngyvGAx/K6ehUKC1ghZjeJL/xcabKRM9X9LCGIuHxY+m1jWl+Qug7X8VUrKHik6nt\nbP+YGNbj1V7evy8msrwcwciCjktH6HfrNoUaNC46C1y2RpEJ2X09DJfFeLln4HPJ\nUoVI8MM5MaQ39PWSicNsaRE1Q5Q6PCoiUukdUl97WVEqZZ4WX2DW6XkxNmu8VaPS\n5jZB7+ePnrCUtaj9bQKbeWuLg1R0bfY2FF7PWH38l1DEvRpyYQKCAQEA5nPFeaR9\n/nU1SlURsNR+2z7VUxqUOhaQM7nc/g1oS4N3asAbrCMUtPh/H0rh7HyqhyyTA/d4\n8TWqDy9tu8m0+m93Wvvxjj8RVTTxMWXAphOQme//xlZhJ0Pmb92eDkUKmvhuH66S\njMdrXVnKgoAq3ZPiK+ZG7olCNwANyWxRvNqJyaKUP6opdRs6jK0PPdlKG3R5r5nY\n7e04PSglwufUqHu0bB1sYeUUOjoBzbnjajwjIRAPvASXkzQbF2GZ0pTyOBytlPbC\nkCXGYP4nbmKmplruqa6DNZSoUbvaX23P2fWA3TtkB18L8PA3mszSS4iRqXr2asP8\n0O6EJgjTzNczRwKCAQEAzywZQjRh8Zmj+20ZSM2cFqJQySM8I/2pV08fejO8v9pC\nnYfojY6yIs6kXu65Y6SY5UPRBV+qgVGSsUdch5ZnU54vL1QBk9ebvWWIdNw+c8wL\nwWxOGD1tBRr/FRuQEFTZ7oMagtmOYf2WF47ZfSB0apy4F7JTMX1dEHsSexHUMU0y\nUjb23CHL5fNWV/YVYAxNk4nVjQev1HTV5zvSWIN/s7YWNa875pdxA56sB0XeExiK\ntQRJv/QZ4ECxlApiSlJbPNiMzbgyDXq0NUk2Ce3nXBHCQcBWkV3rvpy4Ixb1tvcL\n7Ck6+Sv08rYKY17tfdSPYtOunnWilobW2CL2Ude01QKCAQAh9DzMR2go7awEX7RT\nkXt/YiqcK/Za+KgWdvQQmnkbZytZFyT8n86VO2e/fJD8/wFBGVgv5W50W6vKwPoy\nCO4ZWAMF7MkSZEhxfXNsC8nurkNH6wyMdH2sA10YblazU4jUfHX9RQNiqp8vCePE\nho3KR9ezECKzUpXm9KMTYlpIsQpMIyAkoTWqg3CE+ZqdSj6Y/PhQ3yIPLDx1Bk1K\nAk5a5J8Yrt0hL1djnNhXL2RKasHiRTOMSEDs92hmcYLwe/GUQuVwZUsVYftwnwHA\nSdiz+gHgDk5WvqhuIndySAOv4XQzJGUAkQh5X2tR64uwhVaw2eUkpevB/OKLmBFc\nxPhRAoIBAHVCepF0YLOEyMH4gEUqy1wsRxlxbgrY+k5GpIRgYEce3HGZiVEGIWF+\nIJ7y88+oTDoH5njqKvHDIApjX8YGpTk44XAiVjo94RKUM2Yhet/D8Ctsh89EhT5M\nh3CFB0eXHzrWzdyeEu73v0paOlOOWRY22iPfdSMf19sKLe2mq7mz/fV20sbKAkSH\nNKnYyURx8j691zdZ0HWqI6AbxfT/iPpI580+/+KGeRsZZjk/nHCmhWvrn2qcWUEK\nnPgz4Go8rxODZj7SMNJcTz+RzppUI42DJ7ZbjSP+Sv66aabCyhTjQ8ZJl+2Mqbnu\nu3Z/NmDwPLAtLDrYTkxwyY3QTCcKtnkCggEAGDbffotwOqeFi9ys/tpDO3/jjACH\n0qGK1kUmSRdNgOuocvfcmJ2uoWsSB9M65S37od1Fyas65b5RD/823jWbUD/lKhU0\ntRkNBBFq5iHkjc/WSkwNHcPPBr2q8rMgojixTBdAFQqRSfKxSLqJL2r4c6JI/7qG\nfBTF3JsYlQcTYFs1TFY0UaX/IYdWhtn8s/mGaP0jCxzWDD7mvD/Ga/01Ua6OAnYh\n5gNmjc2vNCrpkOTAoOoGclP/t+/1aE1Bbb91wvlZT4k4+RnkSrXrvS5czL3aYB30\nfA/SOrBSHAw9KuUahlLhXHiC1If84mKERNvPUph8/hYE2MG3AVBIPNszeQ==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T22:43:19.897 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T22:43:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:20 smithi120 ceph-mon[31247]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T22:43:21.088 INFO:teuthology.orchestra.run.smithi120.stdout:Creating initial admin user... 2024-09-10T22:43:21.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:21 smithi120 ceph-mon[31247]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:21.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:22.503 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$uAdJj.gJ2If.gJtLp04.l.gUea7zp5uzMBzb.MAEKQ73zXTfdiSee", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726008201, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T22:43:22.503 INFO:teuthology.orchestra.run.smithi120.stdout:Fetching dashboard port number... 2024-09-10T22:43:23.125 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:22 smithi120 ceph-mon[31247]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:23.125 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:22 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:23.694 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T22:43:23.705 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:43:23.705 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout disabled 2024-09-10T22:43:23.712 INFO:teuthology.orchestra.run.smithi120.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:43:23.712 INFO:teuthology.orchestra.run.smithi120.stdout:systemctl: stdout inactive 2024-09-10T22:43:23.712 INFO:teuthology.orchestra.run.smithi120.stdout:firewalld.service is not enabled 2024-09-10T22:43:23.712 INFO:teuthology.orchestra.run.smithi120.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T22:43:23.714 INFO:teuthology.orchestra.run.smithi120.stdout:Ceph Dashboard is now available at: 2024-09-10T22:43:23.714 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:23.714 INFO:teuthology.orchestra.run.smithi120.stdout: URL: https://smithi120.front.sepia.ceph.com:8443/ 2024-09-10T22:43:23.714 INFO:teuthology.orchestra.run.smithi120.stdout: User: admin 2024-09-10T22:43:23.714 INFO:teuthology.orchestra.run.smithi120.stdout: Password: 2p57nfkhlq 2024-09-10T22:43:23.715 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:23.715 INFO:teuthology.orchestra.run.smithi120.stdout:Saving cluster configuration to /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config directory 2024-09-10T22:43:24.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3608608303' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T22:43:25.094 INFO:teuthology.orchestra.run.smithi120.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T22:43:25.094 INFO:teuthology.orchestra.run.smithi120.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: ceph telemetry on 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout:For more information see: 2024-09-10T22:43:25.095 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T22:43:25.096 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:25.096 INFO:teuthology.orchestra.run.smithi120.stdout:Bootstrap complete. 2024-09-10T22:43:25.135 INFO:tasks.cephadm:Fetching config... 2024-09-10T22:43:25.136 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:43:25.136 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T22:43:25.151 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T22:43:25.151 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:43:25.151 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T22:43:25.207 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T22:43:25.208 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:43:25.208 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T22:43:25.273 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T22:43:25.273 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:43:25.273 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T22:43:25.328 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T22:43:25.329 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:43:25.415 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:43:25.428 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:43:25.470 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:43:25.483 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:43:25.525 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWrY2hqxJeHKJKyWGHyz0QpRh7TzgbtlJLw52cuoYHroueqFD/Rj9nItZRDw79NXHw5PyhAQKD4Wx9sZ+p5lg2fTThJ5BS9/FL6Za2Qq2aG8c8cu4AlhsW5c9NFE4ED/VgUd2/x0m1uq69REoxS7zi/es3ogUjlFdhHwtLwPzYAe3vn2bJU3C6Lk2OeZYb8GBrZeBM+lkNp2MT1MOAzZ0aT8Mk+miaHGx9/9JSqQMKBp2QNGgbpJJIOtbvqHvlbrmyjWKzDLIpiMdoYjWcnwcu4xpUEmvmEk6iq8a64ijnUUTwXWY6njjpnTc1F5kpfLJemrz4T5pSnwk/sdf0/KLy9MiO4yF4qEDkCKAdTRK6W6GRaqrpCNScUFk/YzorW6ookNn+QM1milXy3NuvwoQVlTjegnt68Sqr4yZxMSG1xzxqN2TGNImR7/IZIp3Jh3Tqlf2ZlqhnzzXoWvZVRsC7FHJTOyq8167xBUyO0fKbW9A2ovPAsFGSr3auZEUWpQM= ceph-d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:43:25.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T22:43:25.662 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4204138830' entity='client.admin' 2024-09-10T22:43:25.785 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:27.055 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T22:43:27.056 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T22:43:27.535 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:27 smithi120 ceph-mon[31247]: mgrmap e13: x(active, since 12s) 2024-09-10T22:43:27.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/731510541' entity='client.admin' 2024-09-10T22:43:29.232 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2024-09-10T22:43:29.232 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:43:29.232 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:43:29.252 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:43:29.252 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:29.311 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2024-09-10T22:43:29.311 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch host add smithi153 2024-09-10T22:43:29.739 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:29.860 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:29 smithi120 ceph-mon[31247]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:29.860 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: Updating smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:31.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:31.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:31 smithi120 ceph-mon[31247]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:31 smithi120 ceph-mon[31247]: Updating smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:43:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:31 smithi120 ceph-mon[31247]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:33.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:32 smithi120 ceph-mon[31247]: Deploying cephadm binary to smithi153 2024-09-10T22:43:33.676 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi153' with addr '172.21.15.153' 2024-09-10T22:43:34.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:43:34.533 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: Added host smithi153 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:35.149 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:35.150 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2024-09-10T22:43:35.783 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi176 2024-09-10T22:43:35.783 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:43:35.783 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:43:35.803 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:43:35.803 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:35.862 INFO:tasks.cephadm:Adding host smithi176 to orchestrator... 2024-09-10T22:43:35.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch host add smithi176 2024-09-10T22:43:36.113 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:36.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:37 smithi120 ceph-mon[31247]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:43:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:37 smithi120 ceph-mon[31247]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:38 smithi120 ceph-mon[31247]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: Deploying cephadm binary to smithi176 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-10T22:43:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:39 smithi120 ceph-mon[31247]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:39.944 INFO:teuthology.orchestra.run.smithi120.stdout:Added host 'smithi176' with addr '172.21.15.176' 2024-09-10T22:43:40.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: Updating smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:40.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:40.628 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:43:40.867 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:41.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:41 smithi120 ceph-mon[31247]: Updating smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:43:41.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:41 smithi120 ceph-mon[31247]: Added host smithi176 2024-09-10T22:43:41.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:41 smithi120 ceph-mon[31247]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:41.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:41.267 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:41.534 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:43:41.535 INFO:teuthology.orchestra.run.smithi120.stdout:[{"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}] 2024-09-10T22:43:42.149 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T22:43:42.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T22:43:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:42 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:42 smithi120 ceph-mon[31247]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:43:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:42 smithi120 ceph-mon[31247]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:43.294 INFO:teuthology.orchestra.run.smithi120.stderr:adjusted tunables profile to default 2024-09-10T22:43:43.593 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:43 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/109282717' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:43:43.915 INFO:tasks.cephadm:Adding mon.a on smithi120 2024-09-10T22:43:43.915 INFO:tasks.cephadm:Adding mon.b on smithi153 2024-09-10T22:43:43.915 INFO:tasks.cephadm:Adding mon.c on smithi176 2024-09-10T22:43:43.915 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi176:172.21.15.176=c' 2024-09-10T22:43:44.291 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T22:43:44.335 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/109282717' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:44.997 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mon update... 2024-09-10T22:43:45.612 DEBUG:teuthology.orchestra.run.smithi153:mon.b> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T22:43:45.615 DEBUG:teuthology.orchestra.run.smithi176:mon.c> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T22:43:45.618 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:43:45.618 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: Updating smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: Updating smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi176:172.21.15.176=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: Saving service mon spec with placement smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi176:172.21.15.176=c;count:3 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:46.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:46.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:46.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:43:46.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:46.743 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:43:47.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:46 smithi120 ceph-mon[31247]: Deploying daemon mon.c on smithi176 2024-09-10T22:43:47.349 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:46 smithi120 ceph-mon[31247]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:48 smithi120 ceph-mon[31247]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:48.569 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:43:48.569 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:42:13.658772Z","created":"2024-09-10T22:42:13.658772Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T22:43:48.570 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-09-10T22:43:49.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:49 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/3456092740' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:43:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:49 smithi176 systemd[1]: Started Ceph mon.c for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:43:50.192 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:43:50.193 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:43:50.440 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:43:55.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:43:55.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: mon.a calling monitor election 2024-09-10T22:43:55.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: mon.c calling monitor election 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: monmap epoch 2 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: last_changed 2024-09-10T22:43:49.635083+0000 2024-09-10T22:43:55.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: min_mon_release 19 (squid) 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: election_strategy: 1 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: fsmap 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: mgrmap e13: x(active, since 40s) 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: overall HEALTH_OK 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:43:55.076 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: mon.a calling monitor election 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: mon.c calling monitor election 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: monmap epoch 2 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: last_changed 2024-09-10T22:43:49.635083+0000 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: min_mon_release 19 (squid) 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: election_strategy: 1 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:43:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: fsmap 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: mgrmap e13: x(active, since 40s) 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: overall HEALTH_OK 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:43:55.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:55.577 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:43:55.577 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":2,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:43:49.635083Z","created":"2024-09-10T22:42:13.658772Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:43:55.577 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 2 2024-09-10T22:43:55.929 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:55 smithi176 ceph-mon[34982]: Deploying daemon mon.b on smithi153 2024-09-10T22:43:55.929 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:55 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/1292400082' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:43:55.930 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:55 smithi120 ceph-mon[31247]: Deploying daemon mon.b on smithi153 2024-09-10T22:43:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:55 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/1292400082' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:43:56.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:43:57.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:56 smithi176 ceph-mon[34982]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:56 smithi120 ceph-mon[31247]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:57.220 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:43:57.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:43:57.474 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:43:58.113 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:43:58.113 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":2,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:43:49.635083Z","created":"2024-09-10T22:42:13.658772Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:43:58.113 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 2 2024-09-10T22:43:58.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:58 smithi120 ceph-mon[31247]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:58.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:43:58 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/296604029' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:43:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:58 smithi176 ceph-mon[34982]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:43:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:43:58 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/296604029' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:43:59.629 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:43:59.630 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:43:59.878 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: mon.a calling monitor election 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: mon.c calling monitor election 2024-09-10T22:44:03.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: mon.b calling monitor election 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: monmap epoch 3 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: last_changed 2024-09-10T22:43:58.338409+0000 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:44:03.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: min_mon_release 19 (squid) 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: election_strategy: 1 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: fsmap 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: mgrmap e13: x(active, since 49s) 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: overall HEALTH_OK 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.733 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:44:03.733 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":3,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:43:58.338409Z","created":"2024-09-10T22:42:13.658772Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T22:44:03.733 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 3 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: mon.a calling monitor election 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: mon.c calling monitor election 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: mon.b calling monitor election 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:03.734 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: monmap epoch 3 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: last_changed 2024-09-10T22:43:58.338409+0000 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: created 2024-09-10T22:42:13.658772+0000 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: min_mon_release 19 (squid) 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: election_strategy: 1 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: 0: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.a 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.c 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: 2: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: fsmap 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: mgrmap e13: x(active, since 49s) 2024-09-10T22:44:03.735 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: overall HEALTH_OK 2024-09-10T22:44:03.736 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.736 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.736 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:03.736 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:04.319 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T22:44:04.320 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T22:44:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:04 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/2824494036' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:44:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:04 smithi176 ceph-mon[34982]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:04.576 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:04.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:04 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/2824494036' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:44:04.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:04 smithi120 ceph-mon[31247]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:04.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:04.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:04 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/2824494036' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:44:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:04 smithi153 ceph-mon[34951]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:44:05.280 INFO:teuthology.orchestra.run.smithi120.stdout:# minimal ceph.conf for d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:44:05.280 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-09-10T22:44:05.280 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:44:05.281 INFO:teuthology.orchestra.run.smithi120.stdout: mon_host = [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T22:44:05.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-10T22:44:05.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-10T22:44:05.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:05 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/956346499' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-10T22:44:05.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:05 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/956346499' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:05.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:05.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: Updating smithi120:/etc/ceph/ceph.conf 2024-09-10T22:44:05.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: Updating smithi153:/etc/ceph/ceph.conf 2024-09-10T22:44:05.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: Updating smithi176:/etc/ceph/ceph.conf 2024-09-10T22:44:05.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:05 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/956346499' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:05.974 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T22:44:05.974 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:44:05.974 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:44:06.001 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:44:06.002 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:44:06.068 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:44:06.068 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:44:06.097 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:44:06.097 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:44:06.163 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:44:06.164 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:44:06.191 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:44:06.191 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:44:06.257 INFO:tasks.cephadm:Adding mgr.x on smithi120 2024-09-10T22:44:06.257 INFO:tasks.cephadm:Adding mgr.y on smithi153 2024-09-10T22:44:06.257 INFO:tasks.cephadm:Adding mgr.z on smithi176 2024-09-10T22:44:06.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi120=x;smithi153=y;smithi176=z' 2024-09-10T22:44:06.524 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: Updating smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.524 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: Updating smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.524 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: Updating smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.524 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.524 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:06.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:06.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: Reconfiguring daemon mon.a on smithi120 2024-09-10T22:44:06.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:06 smithi176 ceph-mon[34982]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:06.531 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:44:06.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: Updating smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: Updating smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: Updating smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: Reconfiguring daemon mon.a on smithi120 2024-09-10T22:44:06.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:06 smithi120 ceph-mon[31247]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:06.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: Updating smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: Updating smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: Updating smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:44:06.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:06.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:06.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: Reconfiguring daemon mon.a on smithi120 2024-09-10T22:44:06.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:06 smithi153 ceph-mon[34951]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:07.145 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mgr update... 2024-09-10T22:44:07.787 DEBUG:teuthology.orchestra.run.smithi153:mgr.y> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T22:44:07.790 DEBUG:teuthology.orchestra.run.smithi176:mgr.z> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T22:44:07.793 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T22:44:07.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T22:44:07.796 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T22:44:07.796 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T22:44:07.838 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T22:44:07.839 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T22:44:07.882 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T22:44:07.882 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T22:44:07.926 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T22:44:07.926 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T22:44:07.969 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T22:44:07.969 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T22:44:08.012 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T22:44:08.012 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T22:44:08.055 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T22:44:08.055 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T22:44:08.100 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T22:44:08.101 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T22:44:08.101 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:08.150 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T22:44:08.150 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T22:44:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:08.166 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T22:44:08.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T22:44:08.210 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T22:44:08.210 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T22:44:08.253 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T22:44:08.253 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T22:44:08.302 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T22:44:08.302 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T22:44:08.343 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.343 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.343 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:44:08.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:08.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:08.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:08.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: Reconfiguring daemon mon.b on smithi153 2024-09-10T22:44:08.344 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.345 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T22:44:08.345 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T22:44:08.388 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T22:44:08.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T22:44:08.432 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T22:44:08.432 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: Reconfiguring daemon mon.b on smithi153 2024-09-10T22:44:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.483 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T22:44:08.483 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T22:44:08.531 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T22:44:08.531 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T22:44:08.572 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T22:44:08.572 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T22:44:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: Reconfiguring daemon mon.b on smithi153 2024-09-10T22:44:08.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:08.615 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T22:44:08.615 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T22:44:08.659 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T22:44:08.659 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T22:44:08.711 DEBUG:tasks.cephadm:set 22 configs 2024-09-10T22:44:08.711 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi120') 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi176=z;count:3 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:09.246 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi176=z;count:3 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:09.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi120=x;smithi153=y;smithi176=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: Saving service mgr spec with placement smithi120=x;smithi153=y;smithi176=z;count:3 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:44:09.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.213 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi120') 2024-09-10T22:44:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:44:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: Reconfiguring daemon mon.c on smithi176 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/357683855' entity='client.admin' 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/605909671' entity='client.admin' 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:10.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: Reconfiguring daemon mon.c on smithi176 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/357683855' entity='client.admin' 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/605909671' entity='client.admin' 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:10.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:10.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: Reconfiguring daemon mon.c on smithi176 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/357683855' entity='client.admin' 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/605909671' entity='client.admin' 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:10.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:10.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:10.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:10.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:10.557 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:10.557 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:10.559 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:10.561 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:10.570 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:10.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi120') 2024-09-10T22:44:11.190 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.191 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:11 smithi120 ceph-mon[31247]: Deploying daemon mgr.z on smithi176 2024-09-10T22:44:11.191 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:11 smithi120 ceph-mon[31247]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:11.191 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.196 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.203 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.204 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:11.229 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:11 smithi176 ceph-mon[34982]: Deploying daemon mgr.z on smithi176 2024-09-10T22:44:11.230 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:11 smithi176 ceph-mon[34982]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:11 smithi153 ceph-mon[34951]: Deploying daemon mgr.z on smithi176 2024-09-10T22:44:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:11 smithi153 ceph-mon[34951]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:11.823 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:11 smithi176 systemd[1]: Starting Ceph mgr.z for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:12 smithi176 ceph-mon[34982]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:11 smithi176 podman[36547]: 2024-09-10 22:44:11.780918868 +0000 UTC m=+0.015254218 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:11 smithi176 podman[36547]: 2024-09-10 22:44:11.881052177 +0000 UTC m=+0.115387518 container create 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:12 smithi176 podman[36547]: 2024-09-10 22:44:12.066590112 +0000 UTC m=+0.300925455 container init 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:12 smithi176 podman[36547]: 2024-09-10 22:44:12.069359667 +0000 UTC m=+0.303695012 container start 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:12 smithi176 ceph-mgr[36566]: -- 172.21.15.176:0/2591989734 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f85f4750e0 con 0x55f85f453400 2024-09-10T22:44:12.134 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:12 smithi176 bash[36547]: 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 2024-09-10T22:44:12.369 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:12 smithi120 ceph-mon[31247]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:12 smithi153 ceph-mon[34951]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:12.573 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 22:44:12 smithi176 systemd[1]: Started Ceph mgr.z for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:13.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:13.294 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: Deploying daemon mgr.y on smithi153 2024-09-10T22:44:13.294 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1495184132' entity='client.admin' 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda7803f510 msgr2=0x7fda780419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda7803f510 0x7fda780419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fda84009bd0 tx=0x7fda84005f90 comp rx=0 tx=0).stop 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda90101910 msgr2=0x7fda9006b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda90101910 0x7fda9006b350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda8000b560 tx=0x7fda8000ba20 comp rx=0 tx=0).stop 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 shutdown_connections 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda7803f510 0x7fda780419d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda90103430 0x7fda901a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda90102a70 0x7fda9019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 --2- 172.21.15.120:0/2594969147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda90101910 0x7fda9006b350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 >> 172.21.15.120:0/2594969147 conn(0x7fda900fd1e0 msgr2=0x7fda90105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:13.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 shutdown_connections 2024-09-10T22:44:13.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.294+0000 7fda9639b640 1 -- 172.21.15.120:0/2594969147 wait complete. 2024-09-10T22:44:13.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.479+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f12600fed00 con 0x7f1260105be0 2024-09-10T22:44:13.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.482+0000 7f12527fc640 1 -- 172.21.15.120:0/3942302511 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v14)=0 v14) ==== 116+0+0 (secure 0 0 0) 0x7f125c003d80 con 0x7f1260105be0 2024-09-10T22:44:13.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f124c03f510 msgr2=0x7f124c0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f124c03f510 0x7f124c0419d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1254005fd0 tx=0x7f125400b390 comp rx=0 tx=0).stop 2024-09-10T22:44:13.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1260105be0 msgr2=0x7f1260199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1260105be0 0x7f1260199230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f125c005940 tx=0x7f125c004360 comp rx=0 tx=0).stop 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 shutdown_connections 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f124c03f510 0x7f124c0419d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1260108730 0x7f1260199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1260105be0 0x7f1260199230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 --2- 172.21.15.120:0/3942302511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f126006bfc0 0x7f1260198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 >> 172.21.15.120:0/3942302511 conn(0x7f12600fd210 msgr2=0x7f1260103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 shutdown_connections 2024-09-10T22:44:13.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.484+0000 7f1267a15640 1 -- 172.21.15.120:0/3942302511 wait complete. 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: Deploying daemon mgr.y on smithi153 2024-09-10T22:44:13.541 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1495184132' entity='client.admin' 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:44:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:13.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:13.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: Deploying daemon mgr.y on smithi153 2024-09-10T22:44:13.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1495184132' entity='client.admin' 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 -- 172.21.15.120:0/2177432909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8103430 msgr2=0x7f5ae8109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 --2- 172.21.15.120:0/2177432909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8103430 0x7f5ae8109db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009cd0 tx=0x7f5ae002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 -- 172.21.15.120:0/2177432909 shutdown_connections 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 --2- 172.21.15.120:0/2177432909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8103430 0x7f5ae8109db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 --2- 172.21.15.120:0/2177432909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 --2- 172.21.15.120:0/2177432909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8101910 0x7f5ae8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 -- 172.21.15.120:0/2177432909 >> 172.21.15.120:0/2177432909 conn(0x7f5ae80fd1e0 msgr2=0x7f5ae80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 -- 172.21.15.120:0/2177432909 shutdown_connections 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.666+0000 7f5aee53f640 1 -- 172.21.15.120:0/2177432909 wait complete. 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 Processor -- start 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 -- start start 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8103430 0x7f5ae8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810d390 con 0x7f5ae8101910 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810d210 con 0x7f5ae8103430 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aee53f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810d510 con 0x7f5ae8102a70 2024-09-10T22:44:13.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44210/0 (socket says 172.21.15.120:44210) 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5ae7fff640 1 -- 172.21.15.120:0/444443740 learned_addr learned my addr 172.21.15.120:0/444443740 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5ae77fe640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.667+0000 7f5aecadc640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8103430 0x7f5ae8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 -- 172.21.15.120:0/444443740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 msgr2=0x7f5ae81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 -- 172.21.15.120:0/444443740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8103430 msgr2=0x7f5ae8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8103430 0x7f5ae8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 -- 172.21.15.120:0/444443740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae81996b0 con 0x7f5ae8101910 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae77fe640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae7fff640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b4a0 tx=0x7f5ad800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:13.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8014070 con 0x7f5ae8101910 2024-09-10T22:44:13.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5ad8010a80 con 0x7f5ae8101910 2024-09-10T22:44:13.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017ad0 con 0x7f5ae8101910 2024-09-10T22:44:13.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae81999a0 con 0x7f5ae8101910 2024-09-10T22:44:13.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.668+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a0080 con 0x7f5ae8101910 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.669+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f5ad8017c70 con 0x7f5ae8101910 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.669+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae8101d10 con 0x7f5ae8101910 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.669+0000 7f5ae57fa640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 0x7f5ad00419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.669+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ad8053590 con 0x7f5ae8101910 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.670+0000 7f5ae77fe640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 0x7f5ad00419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:13.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.670+0000 7f5ae77fe640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 0x7f5ad00419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fc0 tx=0x7f5adc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:13.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.674+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ad8010c30 con 0x7f5ae8101910 2024-09-10T22:44:13.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.849+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f5ae802e930 con 0x7f5ae8101910 2024-09-10T22:44:13.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.855+0000 7f5ae57fa640 1 -- 172.21.15.120:0/444443740 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7f5ad8010610 con 0x7f5ae8101910 2024-09-10T22:44:13.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 msgr2=0x7f5ad00419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 0x7f5ad00419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fc0 tx=0x7f5adc005f90 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 msgr2=0x7f5ae8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b4a0 tx=0x7f5ad800b960 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 shutdown_connections 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad003f510 0x7f5ad00419d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae8103430 0x7f5ae8198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae8102a70 0x7f5ae81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 --2- 172.21.15.120:0/444443740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ae8101910 0x7f5ae8197b80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 >> 172.21.15.120:0/444443740 conn(0x7f5ae80fd1e0 msgr2=0x7f5ae8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 shutdown_connections 2024-09-10T22:44:13.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:13.857+0000 7f5aee53f640 1 -- 172.21.15.120:0/444443740 wait complete. 2024-09-10T22:44:14.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 -- 172.21.15.120:0/945480175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0101910 msgr2=0x7f36c0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 --2- 172.21.15.120:0/945480175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0101910 0x7f36c0101d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009930 tx=0x7f36bc02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:14.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 -- 172.21.15.120:0/945480175 shutdown_connections 2024-09-10T22:44:14.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 --2- 172.21.15.120:0/945480175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0103430 0x7f36c0109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 --2- 172.21.15.120:0/945480175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0102a70 0x7f36c0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 --2- 172.21.15.120:0/945480175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0101910 0x7f36c0101d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.120+0000 7f36c7fda640 1 -- 172.21.15.120:0/945480175 >> 172.21.15.120:0/945480175 conn(0x7f36c00fd1e0 msgr2=0x7f36c00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- 172.21.15.120:0/945480175 shutdown_connections 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- 172.21.15.120:0/945480175 wait complete. 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 Processor -- start 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- start start 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0101910 0x7f36c006b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0102a70 0x7f36c019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010d3e0 con 0x7f36c0103430 2024-09-10T22:44:14.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010d260 con 0x7f36c0101910 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c7fda640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010d560 con 0x7f36c0102a70 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c6577640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c6577640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44226/0 (socket says 172.21.15.120:44226) 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.121+0000 7f36c6577640 1 -- 172.21.15.120:0/3707113978 learned_addr learned my addr 172.21.15.120:0/3707113978 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c5575640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0102a70 0x7f36c019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c5d76640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0101910 0x7f36c006b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 -- 172.21.15.120:0/3707113978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0102a70 msgr2=0x7f36c019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0102a70 0x7f36c019d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 -- 172.21.15.120:0/3707113978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0101910 msgr2=0x7f36c006b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0101910 0x7f36c006b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 -- 172.21.15.120:0/3707113978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01a08a0 con 0x7f36c0103430 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c6577640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36b800eaa0 tx=0x7f36b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:14.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8004d20 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f36b8005020 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c01a0b90 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80116d0 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.122+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01a1120 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.123+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f36b800f040 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.123+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c0101d10 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.123+0000 7f36b6ffd640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 0x7f36ac0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.123+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f36b8053a70 con 0x7f36c0103430 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.124+0000 7f36c5d76640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 0x7f36ac0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.124+0000 7f36c5d76640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 0x7f36ac0419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36bc005ab0 tx=0x7f36bc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:14.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.128+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f36b8016030 con 0x7f36c0103430 2024-09-10T22:44:14.179 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:13 smithi153 systemd[1]: Starting Ceph mgr.y for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:14.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.310+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f36c0063d50 con 0x7f36c0103430 2024-09-10T22:44:14.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.315+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f36b80051d0 con 0x7f36c0103430 2024-09-10T22:44:14.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.316+0000 7f36b6ffd640 1 -- 172.21.15.120:0/3707113978 <== mon.0 v2:172.21.15.120:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f36b8055e50 con 0x7f36c0103430 2024-09-10T22:44:14.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 msgr2=0x7f36ac0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 0x7f36ac0419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36bc005ab0 tx=0x7f36bc005a40 comp rx=0 tx=0).stop 2024-09-10T22:44:14.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 msgr2=0x7f36c019ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36b800eaa0 tx=0x7f36b800eea0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 shutdown_connections 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36ac03f510 0x7f36ac0419d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36c0103430 0x7f36c019ffe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36c0102a70 0x7f36c019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 --2- 172.21.15.120:0/3707113978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c0101910 0x7f36c006b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 >> 172.21.15.120:0/3707113978 conn(0x7f36c00fd1e0 msgr2=0x7f36c0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 shutdown_connections 2024-09-10T22:44:14.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.318+0000 7f36c7fda640 1 -- 172.21.15.120:0/3707113978 wait complete. 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.350+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/4285111291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 msgr2=0x7f7a78102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.350+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/4285111291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a78102ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009930 tx=0x7f7a6802a420 comp rx=0 tx=0).stop 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/4285111291 shutdown_connections 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/4285111291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 0x7f7a78109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/4285111291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a78102ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/4285111291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a78101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/4285111291 >> 172.21.15.120:0/4285111291 conn(0x7f7a780fd1e0 msgr2=0x7f7a780ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:14.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/4285111291 shutdown_connections 2024-09-10T22:44:14.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/4285111291 wait complete. 2024-09-10T22:44:14.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.351+0000 7f7a7fd8a640 1 Processor -- start 2024-09-10T22:44:14.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 -- start start 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a7810c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 0x7f7a7810af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7810e600 con 0x7f7a78102a70 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7810e480 con 0x7f7a78103430 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7fd8a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7810e780 con 0x7f7a78101910 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7db26640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a7810c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44248/0 (socket says 172.21.15.120:44248) 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 -- 172.21.15.120:0/2981550270 learned_addr learned my addr 172.21.15.120:0/2981550270 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7e327640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 0x7f7a7810af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 -- 172.21.15.120:0/2981550270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 msgr2=0x7f7a7810c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a7810c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 -- 172.21.15.120:0/2981550270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 msgr2=0x7f7a7810af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 0x7f7a7810af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 -- 172.21.15.120:0/2981550270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7810b800 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7db26640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a7810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.352+0000 7f7a7d325640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005ab0 tx=0x7f7a68003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.353+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6803f070 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.353+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a68031b00 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.353+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a781a2dd0 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.353+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68035030 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.353+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a781a3270 con 0x7f7a78102a70 2024-09-10T22:44:14.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f7a6803bb80 con 0x7f7a78102a70 2024-09-10T22:44:14.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a6effd640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 0x7f7a640419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:14.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a78101d10 con 0x7f7a78102a70 2024-09-10T22:44:14.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7a680747d0 con 0x7f7a78102a70 2024-09-10T22:44:14.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a7db26640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 0x7f7a640419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:14.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.354+0000 7f7a7db26640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 0x7f7a640419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009fc0 tx=0x7f7a74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:14.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.358+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7a6803fe40 con 0x7f7a78102a70 2024-09-10T22:44:14.445 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:14 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2594969147' entity='client.admin' 2024-09-10T22:44:14.445 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:14 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3942302511' entity='client.admin' 2024-09-10T22:44:14.445 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:14 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/444443740' entity='client.admin' 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:14 smithi153 ceph-mon[34951]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 podman[36069]: 2024-09-10 22:44:14.174809486 +0000 UTC m=+0.111672924 container create ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 podman[36069]: 2024-09-10 22:44:14.083847902 +0000 UTC m=+0.020711348 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 podman[36069]: 2024-09-10 22:44:14.369029452 +0000 UTC m=+0.305892890 container init ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 podman[36069]: 2024-09-10 22:44:14.371800179 +0000 UTC m=+0.308663620 container start ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T22:44:14.446 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 ceph-mgr[36088]: -- 172.21.15.153:0/2930285221 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec4eb450e0 con 0x55ec4eb23000 2024-09-10T22:44:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:14 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2594969147' entity='client.admin' 2024-09-10T22:44:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:14 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3942302511' entity='client.admin' 2024-09-10T22:44:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:14 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/444443740' entity='client.admin' 2024-09-10T22:44:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:14 smithi120 ceph-mon[31247]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:14.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.544+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f7a781a3a40 con 0x7f7a78102a70 2024-09-10T22:44:14.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.550+0000 7f7a6effd640 1 -- 172.21.15.120:0/2981550270 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7f7a6803b460 con 0x7f7a78102a70 2024-09-10T22:44:14.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 msgr2=0x7f7a640419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 0x7f7a640419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009fc0 tx=0x7f7a74005f90 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 msgr2=0x7f7a7810aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005ab0 tx=0x7f7a68003b90 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 shutdown_connections 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7a6403f510 0x7f7a640419d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a78103430 0x7f7a7810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7a78102a70 0x7f7a7810aa00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 --2- 172.21.15.120:0/2981550270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a78101910 0x7f7a7810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 >> 172.21.15.120:0/2981550270 conn(0x7f7a780fd1e0 msgr2=0x7f7a78103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 shutdown_connections 2024-09-10T22:44:14.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:14.552+0000 7f7a7fd8a640 1 -- 172.21.15.120:0/2981550270 wait complete. 2024-09-10T22:44:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:14 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2594969147' entity='client.admin' 2024-09-10T22:44:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:14 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3942302511' entity='client.admin' 2024-09-10T22:44:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:14 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/444443740' entity='client.admin' 2024-09-10T22:44:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:14 smithi176 ceph-mon[34982]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:14.699 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 bash[36069]: ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 2024-09-10T22:44:14.699 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 22:44:14 smithi153 systemd[1]: Started Ceph mgr.y for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:15.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3605102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 msgr2=0x7fbbf8102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:15.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3605102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8102d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009900 tx=0x7fbbec02a410 comp rx=0 tx=0).stop 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3605102470 shutdown_connections 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3605102470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8103430 0x7fbbf8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3605102470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8102d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3605102470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8101f50 0x7fbbf8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3605102470 >> 172.21.15.120:0/3605102470 conn(0x7fbbf80fc9b0 msgr2=0x7fbbf80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3605102470 shutdown_connections 2024-09-10T22:44:15.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3605102470 wait complete. 2024-09-10T22:44:15.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.068+0000 7fbbfe54a640 1 Processor -- start 2024-09-10T22:44:15.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 -- start start 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8101f50 0x7fbbf8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8103430 0x7fbbf8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf810d4c0 con 0x7fbbf8102890 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf810d340 con 0x7fbbf8101f50 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfe54a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf810d640 con 0x7fbbf8103430 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44268/0 (socket says 172.21.15.120:44268) 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 -- 172.21.15.120:0/3939358804 learned_addr learned my addr 172.21.15.120:0/3939358804 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf7fff640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8101f50 0x7fbbf8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbfcae7640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8103430 0x7fbbf8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 -- 172.21.15.120:0/3939358804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8103430 msgr2=0x7fbbf8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:15.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8103430 0x7fbbf8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 -- 172.21.15.120:0/3939358804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8101f50 msgr2=0x7fbbf8198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8101f50 0x7fbbf8198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.069+0000 7fbbf77fe640 1 -- 172.21.15.120:0/3939358804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf819a870 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbf77fe640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbbec005ab0 tx=0x7fbbec003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec03f070 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbbec031b70 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf819ab00 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec035030 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.070+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81040f0 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7fbbec03bb00 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf810dc80 con 0x7fbbf8102890 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbf57fa640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 0x7fbbe00419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:15.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbbec0747d0 con 0x7fbbf8102890 2024-09-10T22:44:15.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbf7fff640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 0x7fbbe00419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:15.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.071+0000 7fbbf7fff640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 0x7fbbe00419d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009bd0 tx=0x7fbbe8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:15.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.076+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbbec03be20 con 0x7fbbf8102890 2024-09-10T22:44:15.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.252+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fbbf8102350 con 0x7fbbf8102890 2024-09-10T22:44:15.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.257+0000 7fbbf57fa640 1 -- 172.21.15.120:0/3939358804 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v18)=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbbec031d20 con 0x7fbbf8102890 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 msgr2=0x7fbbe00419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 0x7fbbe00419d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009bd0 tx=0x7fbbe8005f90 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 msgr2=0x7fbbf8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbbec005ab0 tx=0x7fbbec003b90 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 shutdown_connections 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbbe003f510 0x7fbbe00419d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbf8103430 0x7fbbf8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbf8102890 0x7fbbf8199390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 --2- 172.21.15.120:0/3939358804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbf8101f50 0x7fbbf8198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 >> 172.21.15.120:0/3939358804 conn(0x7fbbf80fc9b0 msgr2=0x7fbbf80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 shutdown_connections 2024-09-10T22:44:15.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:15.259+0000 7fbbfe54a640 1 -- 172.21.15.120:0/3939358804 wait complete. 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3707113978' entity='client.admin' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2981550270' entity='client.admin' 2024-09-10T22:44:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:15 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3939358804' entity='client.admin' 2024-09-10T22:44:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3707113978' entity='client.admin' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2981550270' entity='client.admin' 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:15 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3939358804' entity='client.admin' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3707113978' entity='client.admin' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2981550270' entity='client.admin' 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:15 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3939358804' entity='client.admin' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:16.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:16.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: Reconfiguring daemon mgr.x on smithi120 2024-09-10T22:44:16.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:16 smithi120 ceph-mon[31247]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:16.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:16.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:16.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: Reconfiguring daemon mgr.x on smithi120 2024-09-10T22:44:16.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:16 smithi153 ceph-mon[34951]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:17.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:17.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:17.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:17.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: Reconfiguring daemon mgr.x on smithi120 2024-09-10T22:44:17.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:16 smithi176 ceph-mon[34982]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:17 smithi176 ceph-mon[34982]: Standby manager daemon z started 2024-09-10T22:44:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:17 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:44:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:17 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:17 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:44:18.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:17 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:18.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:17 smithi120 ceph-mon[31247]: Standby manager daemon z started 2024-09-10T22:44:18.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:17 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:44:18.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:17 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:18.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:17 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:44:18.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:17 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:17 smithi153 ceph-mon[34951]: Standby manager daemon z started 2024-09-10T22:44:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:17 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:44:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:17 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:17 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:44:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:17 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.176:0/265093408' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:18.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- 172.21.15.120:0/1953197336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814108730 msgr2=0x7f781410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:18.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- 172.21.15.120:0/1953197336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814108730 0x7f781410abf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7804009ca0 tx=0x7f780402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- 172.21.15.120:0/1953197336 shutdown_connections 2024-09-10T22:44:18.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- 172.21.15.120:0/1953197336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814108730 0x7f781410abf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- 172.21.15.120:0/1953197336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7814105b80 0x7f7814107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- 172.21.15.120:0/1953197336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781406b6e0 0x7f78141055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- 172.21.15.120:0/1953197336 >> 172.21.15.120:0/1953197336 conn(0x7f78140fd1e0 msgr2=0x7f78140ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- 172.21.15.120:0/1953197336 shutdown_connections 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- 172.21.15.120:0/1953197336 wait complete. 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 Processor -- start 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- start start 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 0x7f7814198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7814108730 0x7f7814199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e570 con 0x7f7814105b80 2024-09-10T22:44:18.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e3f0 con 0x7f7814108730 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.783+0000 7f781b4c0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e6f0 con 0x7f781406b6e0 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44296/0 (socket says 172.21.15.120:44296) 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 -- 172.21.15.120:0/4211406230 learned_addr learned my addr 172.21.15.120:0/4211406230 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7819a5d640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7814108730 0x7f7814199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f781925c640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 0x7f7814198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 -- 172.21.15.120:0/4211406230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 msgr2=0x7f7814198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 0x7f7814198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 -- 172.21.15.120:0/4211406230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7814108730 msgr2=0x7f7814199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7814108730 0x7f7814199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 -- 172.21.15.120:0/4211406230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781419a7d0 con 0x7f7814105b80 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f781925c640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 0x7f7814198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:44:18.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f7818a5b640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f780800d740 tx=0x7f780800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808017070 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f780800de50 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808004c70 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781419aac0 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.784+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a11a0 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.785+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781410ed30 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.786+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 14) ==== 102338+0+0 (secure 0 0 0) 0x7f7808004e10 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.786+0000 7f78127fc640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 0x7f77fc07c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.786+0000 7f781925c640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 0x7f77fc07c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.786+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f780809b6d0 con 0x7f7814105b80 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.787+0000 7f781925c640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 0x7f77fc07c870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f780c0058c0 tx=0x7f780c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:18.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.791+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7808063db0 con 0x7f7814105b80 2024-09-10T22:44:18.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:18 smithi120 ceph-mon[31247]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:44:18.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:18 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:44:18.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:18 smithi120 ceph-mon[31247]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:18.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.976+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f781402e750 con 0x7f7814105b80 2024-09-10T22:44:18.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.979+0000 7f78127fc640 1 -- 172.21.15.120:0/4211406230 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f78080687e0 con 0x7f7814105b80 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 msgr2=0x7f77fc07c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 0x7f77fc07c870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f780c0058c0 tx=0x7f780c005850 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 msgr2=0x7f78141992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f780800d740 tx=0x7f780800dc00 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 shutdown_connections 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f77fc07a3b0 0x7f77fc07c870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7814108730 0x7f7814199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7814105b80 0x7f78141992f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 --2- 172.21.15.120:0/4211406230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781406b6e0 0x7f7814198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.981+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 >> 172.21.15.120:0/4211406230 conn(0x7f78140fd1e0 msgr2=0x7f7814103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.982+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 shutdown_connections 2024-09-10T22:44:18.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:18.982+0000 7f781b4c0640 1 -- 172.21.15.120:0/4211406230 wait complete. 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 -- 172.21.15.120:0/3548228485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688100e60 msgr2=0x7f1688101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 --2- 172.21.15.120:0/3548228485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688100e60 0x7f1688101260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1684009900 tx=0x7f168402a410 comp rx=0 tx=0).stop 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 -- 172.21.15.120:0/3548228485 shutdown_connections 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 --2- 172.21.15.120:0/3548228485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688108860 0x7f168810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 --2- 172.21.15.120:0/3548228485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 0x7f1688108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 --2- 172.21.15.120:0/3548228485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688100e60 0x7f1688101260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 -- 172.21.15.120:0/3548228485 >> 172.21.15.120:0/3548228485 conn(0x7f16880fc9b0 msgr2=0x7f16880fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 -- 172.21.15.120:0/3548228485 shutdown_connections 2024-09-10T22:44:19.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.002+0000 7f1690678640 1 -- 172.21.15.120:0/3548228485 wait complete. 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 Processor -- start 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 -- start start 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 0x7f16881992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e540 con 0x7f1688108860 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e3c0 con 0x7f1688100e60 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f1690678640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f168810e6c0 con 0x7f1688105f30 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f168ec15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f168e414640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f168e414640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42418/0 (socket says 172.21.15.120:42418) 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f168ec15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44310/0 (socket says 172.21.15.120:44310) 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.003+0000 7f168e414640 1 -- 172.21.15.120:0/1436898396 learned_addr learned my addr 172.21.15.120:0/1436898396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:19.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168dc13640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 0x7f16881992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 -- 172.21.15.120:0/1436898396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 msgr2=0x7f16881992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 0x7f16881992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 -- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 msgr2=0x7f1688199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 -- 172.21.15.120:0/1436898396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168819a7a0 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168ec15640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f168e414640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1684004770 tx=0x7f16840042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168403f070 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168819aa30 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f168403b460 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684035070 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.004+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881a1170 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.006+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168810ed00 con 0x7f1688100e60 2024-09-10T22:44:19.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.007+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 14) ==== 102338+0+0 (secure 0 0 0) 0x7f168404b060 con 0x7f1688100e60 2024-09-10T22:44:19.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.007+0000 7f167f7fe640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 0x7f167407c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:19.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.007+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f16840bbe70 con 0x7f1688100e60 2024-09-10T22:44:19.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.007+0000 7f168dc13640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 0x7f167407c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:19.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.008+0000 7f168dc13640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 0x7f167407c870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1678005e10 tx=0x7f1678005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:19.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.016+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1684084550 con 0x7f1688100e60 2024-09-10T22:44:19.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:18 smithi176 ceph-mon[34982]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:44:19.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:18 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:44:19.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:18 smithi176 ceph-mon[34982]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:19.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.196+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f168802e710 con 0x7f1688100e60 2024-09-10T22:44:19.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:18 smithi153 ceph-mon[34951]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:44:19.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:18 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:44:19.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:18 smithi153 ceph-mon[34951]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:19.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.201+0000 7f167f7fe640 1 -- 172.21.15.120:0/1436898396 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7f168408d2c0 con 0x7f1688100e60 2024-09-10T22:44:19.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 msgr2=0x7f167407c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:19.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 0x7f167407c870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1678005e10 tx=0x7f1678005da0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 msgr2=0x7f1688198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1684004770 tx=0x7f16840042f0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 shutdown_connections 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f167407a3b0 0x7f167407c870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1688108860 0x7f1688199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1688105f30 0x7f16881992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 --2- 172.21.15.120:0/1436898396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1688100e60 0x7f1688198c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:19.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.203+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 >> 172.21.15.120:0/1436898396 conn(0x7f16880fc9b0 msgr2=0x7f1688106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:19.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.204+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 shutdown_connections 2024-09-10T22:44:19.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:19.204+0000 7f1690678640 1 -- 172.21.15.120:0/1436898396 wait complete. 2024-09-10T22:44:19.895 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:19.896 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:19.933 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi120') 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4211406230' entity='client.admin' 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='client.? ' entity='client.admin' 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: Standby manager daemon y started 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:44:20.153 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:19 smithi120 ceph-mon[31247]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4211406230' entity='client.admin' 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='client.? ' entity='client.admin' 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: Standby manager daemon y started 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:44:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:44:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:19 smithi176 ceph-mon[34982]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4211406230' entity='client.admin' 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='client.? ' entity='client.admin' 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: Standby manager daemon y started 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:44:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:19 smithi153 ceph-mon[34951]: from='mgr.? 172.21.15.153:0/3252638556' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 -- 172.21.15.120:0/126214324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea006b640 msgr2=0x7f7ea0105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/126214324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea006b640 0x7f7ea0105690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009900 tx=0x7f7e9002a410 comp rx=0 tx=0).stop 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 -- 172.21.15.120:0/126214324 shutdown_connections 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/126214324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea0108720 0x7f7ea010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/126214324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/126214324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea006b640 0x7f7ea0105690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 -- 172.21.15.120:0/126214324 >> 172.21.15.120:0/126214324 conn(0x7f7ea00fd1e0 msgr2=0x7f7ea00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 -- 172.21.15.120:0/126214324 shutdown_connections 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.793+0000 7f7ea63ee640 1 -- 172.21.15.120:0/126214324 wait complete. 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 Processor -- start 2024-09-10T22:44:20.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 -- start start 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea006b640 0x7f7ea0102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010e610 con 0x7f7ea0108720 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010e490 con 0x7f7ea006b640 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea63ee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010e790 con 0x7f7ea0105bd0 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea498b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea498b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44334/0 (socket says 172.21.15.120:44334) 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7ea498b640 1 -- 172.21.15.120:0/897683346 learned_addr learned my addr 172.21.15.120:0/897683346 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.794+0000 7f7e9ffff640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea006b640 0x7f7ea0102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7e9f7fe640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 -- 172.21.15.120:0/897683346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 msgr2=0x7f7ea0101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:20.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 -- 172.21.15.120:0/897683346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea006b640 msgr2=0x7f7ea0102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea006b640 0x7f7ea0102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 -- 172.21.15.120:0/897683346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea0102370 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7e9f7fe640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea498b640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b970 tx=0x7f7e9800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98019070 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e98014440 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea01a2f60 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980075d0 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.795+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea01a3440 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.796+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea0106050 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.797+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f7e98014930 con 0x7f7ea0108720 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.798+0000 7f7e9d7fa640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 0x7f7e880b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:20.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.798+0000 7f7e9ffff640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 0x7f7e880b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:20.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.798+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e980e3650 con 0x7f7ea0108720 2024-09-10T22:44:20.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.798+0000 7f7e9ffff640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 0x7f7e880b7710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:20.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.802+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7e980abb90 con 0x7f7ea0108720 2024-09-10T22:44:20.812 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:20.812 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:20.812 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:20.813 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:21.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.008+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f7ea002e930 con 0x7f7ea0108720 2024-09-10T22:44:21.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.011+0000 7f7e9d7fa640 1 -- 172.21.15.120:0/897683346 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f7e980b05c0 con 0x7f7ea0108720 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 msgr2=0x7f7e880b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 0x7f7e880b7710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90000bc0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 msgr2=0x7f7ea0101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b970 tx=0x7f7e9800be30 comp rx=0 tx=0).stop 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 shutdown_connections 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7e880b5250 0x7f7e880b7710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ea0108720 0x7f7ea0101b70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ea0105bd0 0x7f7ea0101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 --2- 172.21.15.120:0/897683346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea006b640 0x7f7ea0102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 >> 172.21.15.120:0/897683346 conn(0x7f7ea00fd1e0 msgr2=0x7f7ea0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 shutdown_connections 2024-09-10T22:44:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:21.014+0000 7f7ea63ee640 1 -- 172.21.15.120:0/897683346 wait complete. 2024-09-10T22:44:21.235 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:20 smithi120 ceph-mon[31247]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:21.235 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:20 smithi120 ceph-mon[31247]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:44:21.235 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:44:21.235 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:20 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2812865066' entity='client.admin' 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- 172.21.15.120:0/1809352607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 msgr2=0x7f8f1c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- 172.21.15.120:0/1809352607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c102ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009930 tx=0x7f8f0c02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- 172.21.15.120:0/1809352607 shutdown_connections 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- 172.21.15.120:0/1809352607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- 172.21.15.120:0/1809352607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c102ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- 172.21.15.120:0/1809352607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 0x7f8f1c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- 172.21.15.120:0/1809352607 >> 172.21.15.120:0/1809352607 conn(0x7f8f1c0fd1e0 msgr2=0x7f8f1c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- 172.21.15.120:0/1809352607 shutdown_connections 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- 172.21.15.120:0/1809352607 wait complete. 2024-09-10T22:44:21.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 Processor -- start 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- start start 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 0x7f8f1c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.238+0000 7f8f23749640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10d450 con 0x7f8f1c102a70 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f23749640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10d2d0 con 0x7f8f1c101910 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f23749640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c10d5d0 con 0x7f8f1c103430 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44326/0 (socket says 172.21.15.120:44326) 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 -- 172.21.15.120:0/2812865066 learned_addr learned my addr 172.21.15.120:0/2812865066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f214e5640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 0x7f8f1c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f21ce6640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:21.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 -- 172.21.15.120:0/2812865066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 msgr2=0x7f8f1c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 -- 172.21.15.120:0/2812865066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 msgr2=0x7f8f1c197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 0x7f8f1c197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 -- 172.21.15.120:0/2812865066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c199750 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f21ce6640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f20ce4640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009a50 tx=0x7f8f0c032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c03f030 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f0c0044b0 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1999e0 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c035070 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.239+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1a0120 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.240+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c10dc10 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.241+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 14) ==== 102338+0+0 (secure 0 0 0) 0x7f8f0c03bb10 con 0x7f8f1c102a70 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.241+0000 7f8f127fc640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 0x7f8f0807c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.242+0000 7f8f214e5640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 0x7f8f0807c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:21.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.242+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f0c0bc770 con 0x7f8f1c102a70 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.242+0000 7f8f214e5640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 0x7f8f0807c870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f180058c0 tx=0x7f8f1800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.246+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f0c084e50 con 0x7f8f1c102a70 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.346+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f8f0c0810c0 con 0x7f8f1c102a70 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.445+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f8f1c1a05d0 con 0x7f8f1c102a70 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.448+0000 7f8f127fc640 1 -- 172.21.15.120:0/2812865066 <== mon.0 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v21)=0 v21) ==== 108+0+0 (secure 0 0 0) 0x7f8f0c0b9640 con 0x7f8f1c102a70 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 msgr2=0x7f8f0807c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 0x7f8f0807c870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f180058c0 tx=0x7f8f1800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 msgr2=0x7f8f1c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009a50 tx=0x7f8f0c032f90 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 shutdown_connections 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8f0807a3b0 0x7f8f0807c870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f1c103430 0x7f8f1c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f1c102a70 0x7f8f1c198270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 --2- 172.21.15.120:0/2812865066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f1c101910 0x7f8f1c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:21.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 >> 172.21.15.120:0/2812865066 conn(0x7f8f1c0fd1e0 msgr2=0x7f8f1c103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:21.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 shutdown_connections 2024-09-10T22:44:21.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:20.451+0000 7f8f23749640 1 -- 172.21.15.120:0/2812865066 wait complete. 2024-09-10T22:44:21.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:20 smithi176 ceph-mon[34982]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:21.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:20 smithi176 ceph-mon[34982]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:44:21.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:44:21.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:20 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2812865066' entity='client.admin' 2024-09-10T22:44:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:20 smithi153 ceph-mon[34951]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:20 smithi153 ceph-mon[34951]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:44:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:44:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:20 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2812865066' entity='client.admin' 2024-09-10T22:44:22.156 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/897683346' entity='client.admin' 2024-09-10T22:44:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/897683346' entity='client.admin' 2024-09-10T22:44:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/897683346' entity='client.admin' 2024-09-10T22:44:22.657 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:22.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 -- 172.21.15.120:0/3691601820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae54107f70 msgr2=0x7fae5410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 --2- 172.21.15.120:0/3691601820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae54107f70 0x7fae5410a360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009900 tx=0x7fae3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 -- 172.21.15.120:0/3691601820 shutdown_connections 2024-09-10T22:44:22.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 --2- 172.21.15.120:0/3691601820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae5410a980 0x7fae5410ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 --2- 172.21.15.120:0/3691601820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae54107f70 0x7fae5410a360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 --2- 172.21.15.120:0/3691601820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 0x7fae54107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.653+0000 7fae5968f640 1 -- 172.21.15.120:0/3691601820 >> 172.21.15.120:0/3691601820 conn(0x7fae540fd1e0 msgr2=0x7fae540ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- 172.21.15.120:0/3691601820 shutdown_connections 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- 172.21.15.120:0/3691601820 wait complete. 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 Processor -- start 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- start start 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 0x7fae5419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae54107f70 0x7fae5419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae54110740 con 0x7fae5410a980 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fae541105c0 con 0x7fae5406b590 2024-09-10T22:44:22.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.654+0000 7fae5968f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fae541108c0 con 0x7fae54107f70 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44350/0 (socket says 172.21.15.120:44350) 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 -- 172.21.15.120:0/2899221235 learned_addr learned my addr 172.21.15.120:0/2899221235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae4bfff640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae54107f70 0x7fae5419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae52ffd640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 0x7fae5419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 -- 172.21.15.120:0/2899221235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae54107f70 msgr2=0x7fae5419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae54107f70 0x7fae5419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 -- 172.21.15.120:0/2899221235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 msgr2=0x7fae5419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 0x7fae5419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 -- 172.21.15.120:0/2899221235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5419ca20 con 0x7fae5410a980 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae537fe640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00eb10 tx=0x7fae4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004b20 con 0x7fae5410a980 2024-09-10T22:44:22.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae4c004cc0 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c010690 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5419cd10 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.655+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae541a33f0 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.657+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae54107a30 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.659+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fae4c005000 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.660+0000 7fae50ff9640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 0x7fae400b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.660+0000 7fae52ffd640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 0x7fae400b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.660+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fae4c0e4600 con 0x7fae5410a980 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.661+0000 7fae52ffd640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 0x7fae400b7790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae440058c0 tx=0x7fae4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:22.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.663+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fae4c056220 con 0x7fae5410a980 2024-09-10T22:44:22.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.847+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fae5402e950 con 0x7fae5410a980 2024-09-10T22:44:22.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.851+0000 7fae50ff9640 1 -- 172.21.15.120:0/2899221235 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v23)=0 v23) ==== 127+0+0 (secure 0 0 0) 0x7fae4c0ad190 con 0x7fae5410a980 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 msgr2=0x7fae400b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 0x7fae400b7790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae440058c0 tx=0x7fae4400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 msgr2=0x7fae5419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00eb10 tx=0x7fae4c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 shutdown_connections 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fae400b52d0 0x7fae400b7790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae5410a980 0x7fae5419c180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fae54107f70 0x7fae5419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 --2- 172.21.15.120:0/2899221235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae5406b590 0x7fae5419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.853+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 >> 172.21.15.120:0/2899221235 conn(0x7fae540fd1e0 msgr2=0x7fae54108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.854+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 shutdown_connections 2024-09-10T22:44:22.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:22.854+0000 7fae5968f640 1 -- 172.21.15.120:0/2899221235 wait complete. 2024-09-10T22:44:23.128 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:23 smithi120 ceph-mon[31247]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:23.129 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2899221235' entity='client.admin' 2024-09-10T22:44:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:23 smithi176 ceph-mon[34982]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:23 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2899221235' entity='client.admin' 2024-09-10T22:44:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:23 smithi153 ceph-mon[34951]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:23 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2899221235' entity='client.admin' 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.293+0000 7f096ce93640 1 -- 172.21.15.120:0/2490645097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 msgr2=0x7f09681056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.293+0000 7f096ce93640 1 --2- 172.21.15.120:0/2490645097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f09681056e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0958009900 tx=0x7f095802a410 comp rx=0 tx=0).stop 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- 172.21.15.120:0/2490645097 shutdown_connections 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- 172.21.15.120:0/2490645097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968108750 0x7f096810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- 172.21.15.120:0/2490645097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0968105c20 0x7f0968108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- 172.21.15.120:0/2490645097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f09681056e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- 172.21.15.120:0/2490645097 >> 172.21.15.120:0/2490645097 conn(0x7f09680fd210 msgr2=0x7f09680ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- 172.21.15.120:0/2490645097 shutdown_connections 2024-09-10T22:44:24.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- 172.21.15.120:0/2490645097 wait complete. 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 Processor -- start 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- start start 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968105c20 0x7f0968101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0968108750 0x7f0968101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f096810e5c0 con 0x7f096806c000 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f096810e440 con 0x7f0968105c20 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.294+0000 7f096ce93640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f096810e740 con 0x7f0968108750 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53084/0 (socket says 172.21.15.120:53084) 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 -- 172.21.15.120:0/3180324811 learned_addr learned my addr 172.21.15.120:0/3180324811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f0965d9b640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968105c20 0x7f0968101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 -- 172.21.15.120:0/3180324811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0968108750 msgr2=0x7f0968101bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0968108750 0x7f0968101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 -- 172.21.15.120:0/3180324811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968105c20 msgr2=0x7f0968101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968105c20 0x7f0968101680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 -- 172.21.15.120:0/3180324811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0968102480 con 0x7f096806c000 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096659c640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0958005940 tx=0x7f09580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095803f030 con 0x7f096806c000 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681a2e70 con 0x7f096806c000 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0958003c90 con 0x7f096806c000 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958035070 con 0x7f096806c000 2024-09-10T22:44:24.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.295+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a3340 con 0x7f096806c000 2024-09-10T22:44:24.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.296+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0968106000 con 0x7f096806c000 2024-09-10T22:44:24.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.299+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f0958003800 con 0x7f096806c000 2024-09-10T22:44:24.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.299+0000 7f09577fe640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 0x7f09500b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.299+0000 7f0965d9b640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 0x7f09500b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.300+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0958104da0 con 0x7f096806c000 2024-09-10T22:44:24.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.300+0000 7f0965d9b640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 0x7f09500b7790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.302+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09580cd930 con 0x7f096806c000 2024-09-10T22:44:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:24 smithi120 ceph-mon[31247]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:24 smithi153 ceph-mon[34951]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:24.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.490+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f09680fed00 con 0x7f096806c000 2024-09-10T22:44:24.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.494+0000 7f09577fe640 1 -- 172.21.15.120:0/3180324811 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7f09580cd230 con 0x7f096806c000 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 msgr2=0x7f09500b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 0x7f09500b7790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c005850 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 msgr2=0x7f0968102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0958005940 tx=0x7f09580042f0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 shutdown_connections 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f09500b52d0 0x7f09500b7790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0968108750 0x7f0968101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0968105c20 0x7f0968101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 --2- 172.21.15.120:0/3180324811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f096806c000 0x7f0968102e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.496+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 >> 172.21.15.120:0/3180324811 conn(0x7f09680fd210 msgr2=0x7f0968104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.497+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 shutdown_connections 2024-09-10T22:44:24.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.497+0000 7f096ce93640 1 -- 172.21.15.120:0/3180324811 wait complete. 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.478+0000 7f6bffec8640 1 -- 172.21.15.120:0/1931931476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 msgr2=0x7f6bf8107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.478+0000 7f6bffec8640 1 --2- 172.21.15.120:0/1931931476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8107ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009900 tx=0x7f6bf402a410 comp rx=0 tx=0).stop 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- 172.21.15.120:0/1931931476 shutdown_connections 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- 172.21.15.120:0/1931931476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- 172.21.15.120:0/1931931476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8107ff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- 172.21.15.120:0/1931931476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 0x7f6bf81055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- 172.21.15.120:0/1931931476 >> 172.21.15.120:0/1931931476 conn(0x7f6bf80fd1e0 msgr2=0x7f6bf80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- 172.21.15.120:0/1931931476 shutdown_connections 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- 172.21.15.120:0/1931931476 wait complete. 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 Processor -- start 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- start start 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 0x7f6bf8102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810e540 con 0x7f6bf8105b80 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810e3c0 con 0x7f6bf8108730 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bffec8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810e6c0 con 0x7f6bf806b6e0 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.479+0000 7f6bfd463640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53094/0 (socket says 172.21.15.120:53094) 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 -- 172.21.15.120:0/2891857194 learned_addr learned my addr 172.21.15.120:0/2891857194 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:24.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfdc64640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 0x7f6bf8102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfe465640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 -- 172.21.15.120:0/2891857194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 msgr2=0x7f6bf8102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 0x7f6bf8102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 -- 172.21.15.120:0/2891857194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 msgr2=0x7f6bf8101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf8101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 -- 172.21.15.120:0/2891857194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf81022d0 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfe465640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bfd463640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009230 tx=0x7f6bf4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf403f030 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6bf4003a80 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81a2db0 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4035070 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.480+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81a3250 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.481+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf8105fe0 con 0x7f6bf8105b80 2024-09-10T22:44:24.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.483+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f6bf403bb90 con 0x7f6bf8105b80 2024-09-10T22:44:24.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.483+0000 7f6beaffd640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 0x7f6be40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.484+0000 7f6bfdc64640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 0x7f6be40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.484+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6bf4104ca0 con 0x7f6bf8105b80 2024-09-10T22:44:24.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.484+0000 7f6bfdc64640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 0x7f6be40b7790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005fd0 tx=0x7f6bec005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.487+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6bf40cd810 con 0x7f6bf8105b80 2024-09-10T22:44:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:24 smithi176 ceph-mon[34982]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:24.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.675+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f6bf80fecd0 con 0x7f6bf8105b80 2024-09-10T22:44:24.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.679+0000 7f6beaffd640 1 -- 172.21.15.120:0/2891857194 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f6bf80fecd0 con 0x7f6bf8105b80 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 msgr2=0x7f6be40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 0x7f6be40b7790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005fd0 tx=0x7f6bec005e20 comp rx=0 tx=0).stop 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 msgr2=0x7f6bf8101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009230 tx=0x7f6bf4004250 comp rx=0 tx=0).stop 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 shutdown_connections 2024-09-10T22:44:24.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6be40b52d0 0x7f6be40b7790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bf8108730 0x7f6bf8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bf8105b80 0x7f6bf8101590 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 --2- 172.21.15.120:0/2891857194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf806b6e0 0x7f6bf8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 >> 172.21.15.120:0/2891857194 conn(0x7f6bf80fd1e0 msgr2=0x7f6bf8103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 shutdown_connections 2024-09-10T22:44:24.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.681+0000 7f6bffec8640 1 -- 172.21.15.120:0/2891857194 wait complete. 2024-09-10T22:44:24.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- 172.21.15.120:0/745491759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 msgr2=0x7f74dc105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 --2- 172.21.15.120:0/745491759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc105620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009900 tx=0x7f74cc02a410 comp rx=0 tx=0).stop 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- 172.21.15.120:0/745491759 shutdown_connections 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 --2- 172.21.15.120:0/745491759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 0x7f74dc10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 --2- 172.21.15.120:0/745491759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 0x7f74dc108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 --2- 172.21.15.120:0/745491759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc105620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- 172.21.15.120:0/745491759 >> 172.21.15.120:0/745491759 conn(0x7f74dc0fd210 msgr2=0x7f74dc0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- 172.21.15.120:0/745491759 shutdown_connections 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- 172.21.15.120:0/745491759 wait complete. 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 Processor -- start 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.720+0000 7f74e23a1640 1 -- start start 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 0x7f74dc1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 0x7f74dc101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10e5f0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10e470 con 0x7f74dc105bb0 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e23a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc10e770 con 0x7f74dc108760 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53108/0 (socket says 172.21.15.120:53108) 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 -- 172.21.15.120:0/2522139137 learned_addr learned my addr 172.21.15.120:0/2522139137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74e093e640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 0x7f74dc101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74db7fe640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 0x7f74dc1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 -- 172.21.15.120:0/2522139137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 msgr2=0x7f74dc101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 0x7f74dc101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 -- 172.21.15.120:0/2522139137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 msgr2=0x7f74dc1015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 0x7f74dc1015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 -- 172.21.15.120:0/2522139137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc1023c0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.721+0000 7f74dbfff640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74cc031440 tx=0x7f74cc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.722+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc03f070 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.722+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc1a2de0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.722+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f74cc031d80 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.722+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc035030 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.722+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc1a31e0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.723+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc106010 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.724+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f74cc03b6a0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.724+0000 7f74d97fa640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 0x7f74c40b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.725+0000 7f74db7fe640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 0x7f74c40b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.725+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f74cc104fe0 con 0x7f74dc06c0c0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.725+0000 7f74db7fe640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 0x7f74c40b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.727+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74cc0cd520 con 0x7f74dc06c0c0 2024-09-10T22:44:24.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.909+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f74dc0fed00 con 0x7f74dc06c0c0 2024-09-10T22:44:24.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.913+0000 7f74d97fa640 1 -- 172.21.15.120:0/2522139137 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v26)=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f74dc0fed00 con 0x7f74dc06c0c0 2024-09-10T22:44:24.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 msgr2=0x7f74c40b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 0x7f74c40b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058c0 tx=0x7f74d000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 msgr2=0x7f74dc102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74cc031440 tx=0x7f74cc003d50 comp rx=0 tx=0).stop 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 shutdown_connections 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74c40b5250 0x7f74c40b7710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74dc108760 0x7f74dc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74dc105bb0 0x7f74dc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 --2- 172.21.15.120:0/2522139137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74dc06c0c0 0x7f74dc102d50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 >> 172.21.15.120:0/2522139137 conn(0x7f74dc0fd210 msgr2=0x7f74dc103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 shutdown_connections 2024-09-10T22:44:24.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:24.915+0000 7f74e23a1640 1 -- 172.21.15.120:0/2522139137 wait complete. 2024-09-10T22:44:25.323 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:25.552 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:25.562 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3180324811' entity='client.admin' 2024-09-10T22:44:25.562 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2891857194' entity='client.admin' 2024-09-10T22:44:25.562 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2522139137' entity='client.admin' 2024-09-10T22:44:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3180324811' entity='client.admin' 2024-09-10T22:44:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2891857194' entity='client.admin' 2024-09-10T22:44:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2522139137' entity='client.admin' 2024-09-10T22:44:25.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3180324811' entity='client.admin' 2024-09-10T22:44:25.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2891857194' entity='client.admin' 2024-09-10T22:44:25.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2522139137' entity='client.admin' 2024-09-10T22:44:26.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 -- 172.21.15.120:0/1821035688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 msgr2=0x7f670c10aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 --2- 172.21.15.120:0/1821035688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c10aed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6704009cd0 tx=0x7f670402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 -- 172.21.15.120:0/1821035688 shutdown_connections 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 --2- 172.21.15.120:0/1821035688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c10aed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 --2- 172.21.15.120:0/1821035688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c103b90 0x7f670c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 --2- 172.21.15.120:0/1821035688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c102a30 0x7f670c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 -- 172.21.15.120:0/1821035688 >> 172.21.15.120:0/1821035688 conn(0x7f670c0fe320 msgr2=0x7f670c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 -- 172.21.15.120:0/1821035688 shutdown_connections 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 -- 172.21.15.120:0/1821035688 wait complete. 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.300+0000 7f6711c55640 1 Processor -- start 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 -- start start 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c102a30 0x7f670c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 0x7f670c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10e570 con 0x7f670c104550 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10e3f0 con 0x7f670c102a30 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f6711c55640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10e6f0 con 0x7f670c103b90 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53134/0 (socket says 172.21.15.120:53134) 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 -- 172.21.15.120:0/534043451 learned_addr learned my addr 172.21.15.120:0/534043451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670affd640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 0x7f670c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670b7fe640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c102a30 0x7f670c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 -- 172.21.15.120:0/534043451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 msgr2=0x7f670c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 0x7f670c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 -- 172.21.15.120:0/534043451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c102a30 msgr2=0x7f670c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c102a30 0x7f670c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670bfff640 1 -- 172.21.15.120:0/534043451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c19a820 con 0x7f670c104550 2024-09-10T22:44:26.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.301+0000 7f670affd640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 0x7f670c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f670bfff640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67040097b0 tx=0x7f6704003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704004240 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f67040305a0 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c19aab0 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670403b660 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.302+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1a11f0 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.303+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c102e30 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.304+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f670404d020 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.305+0000 7f6708ff9640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 0x7f66f40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.305+0000 7f670b7fe640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 0x7f66f40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.305+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f670402e030 con 0x7f670c104550 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.305+0000 7f670b7fe640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 0x7f66f40b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009bd0 tx=0x7f66fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:26.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.309+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f67040cd800 con 0x7f670c104550 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.495+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f670c063d50 con 0x7f670c104550 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.506+0000 7f6708ff9640 1 -- 172.21.15.120:0/534043451 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f67040cd100 con 0x7f670c104550 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 msgr2=0x7f66f40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 0x7f66f40b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009bd0 tx=0x7f66fc005f90 comp rx=0 tx=0).stop 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 msgr2=0x7f670c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67040097b0 tx=0x7f6704003ba0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 shutdown_connections 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66f40b52d0 0x7f66f40b7790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f670c104550 0x7f670c199f80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f670c103b90 0x7f670c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 --2- 172.21.15.120:0/534043451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f670c102a30 0x7f670c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 >> 172.21.15.120:0/534043451 conn(0x7f670c0fe320 msgr2=0x7f670c106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:26.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 shutdown_connections 2024-09-10T22:44:26.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:26.508+0000 7f6711c55640 1 -- 172.21.15.120:0/534043451 wait complete. 2024-09-10T22:44:26.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:26 smithi120 ceph-mon[31247]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:26.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:26 smithi176 ceph-mon[34982]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:26.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:26 smithi153 ceph-mon[34951]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 -- 172.21.15.120:0/2415373472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 msgr2=0x7f414c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 --2- 172.21.15.120:0/2415373472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c101d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f413c009930 tx=0x7f413c02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 -- 172.21.15.120:0/2415373472 shutdown_connections 2024-09-10T22:44:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 --2- 172.21.15.120:0/2415373472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 --2- 172.21.15.120:0/2415373472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 0x7f414c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 --2- 172.21.15.120:0/2415373472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c101d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 -- 172.21.15.120:0/2415373472 >> 172.21.15.120:0/2415373472 conn(0x7f414c0fd1e0 msgr2=0x7f414c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 -- 172.21.15.120:0/2415373472 shutdown_connections 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.172+0000 7f4150b71640 1 -- 172.21.15.120:0/2415373472 wait complete. 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.173+0000 7f4150b71640 1 Processor -- start 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.173+0000 7f4150b71640 1 -- start start 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 0x7f414c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10d4a0 con 0x7f414c101910 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10d320 con 0x7f414c102a70 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10d620 con 0x7f414c103430 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53160/0 (socket says 172.21.15.120:53160) 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 -- 172.21.15.120:0/569255183 learned_addr learned my addr 172.21.15.120:0/569255183 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414ad9d640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4149d9b640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 0x7f414c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 -- 172.21.15.120:0/569255183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 msgr2=0x7f414c19ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c19ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 -- 172.21.15.120:0/569255183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 msgr2=0x7f414c19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 0x7f414c19d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 -- 172.21.15.120:0/569255183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c1a0870 con 0x7f414c101910 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414ad9d640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c19ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f414a59c640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f413c005ab0 tx=0x7f413c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c03f070 con 0x7f414c101910 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f413c031b00 con 0x7f414c101910 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c1a0b00 con 0x7f414c101910 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c035030 con 0x7f414c101910 2024-09-10T22:44:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.174+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1a1030 con 0x7f414c101910 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.175+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414c101d10 con 0x7f414c101910 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.177+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f413c031670 con 0x7f414c101910 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.177+0000 7f413b7fe640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 0x7f41340b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.177+0000 7f4149d9b640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 0x7f41340b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.178+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f413c104cb0 con 0x7f414c101910 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.178+0000 7f4149d9b640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 0x7f41340b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f414000abe0 tx=0x7f414000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.180+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f413c0cd840 con 0x7f414c101910 2024-09-10T22:44:27.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.368+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f414c02e5c0 con 0x7f414c101910 2024-09-10T22:44:27.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.371+0000 7f413b7fe640 1 -- 172.21.15.120:0/569255183 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f413c0cd140 con 0x7f414c101910 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 msgr2=0x7f41340b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 0x7f41340b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f414000abe0 tx=0x7f414000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 msgr2=0x7f414c06b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f413c005ab0 tx=0x7f413c003b90 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 shutdown_connections 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.373+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41340b52d0 0x7f41340b7790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f414c103430 0x7f414c19ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f414c102a70 0x7f414c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 --2- 172.21.15.120:0/569255183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414c101910 0x7f414c06b3f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 >> 172.21.15.120:0/569255183 conn(0x7f414c0fd1e0 msgr2=0x7f414c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:27.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 shutdown_connections 2024-09-10T22:44:27.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:27.374+0000 7f4150b71640 1 -- 172.21.15.120:0/569255183 wait complete. 2024-09-10T22:44:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/534043451' entity='client.admin' 2024-09-10T22:44:27.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/569255183' entity='client.admin' 2024-09-10T22:44:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:27 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/534043451' entity='client.admin' 2024-09-10T22:44:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:27 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/569255183' entity='client.admin' 2024-09-10T22:44:27.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:27 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/534043451' entity='client.admin' 2024-09-10T22:44:27.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:27 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/569255183' entity='client.admin' 2024-09-10T22:44:28.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:28 smithi120 ceph-mon[31247]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:28.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.735 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:28.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:28 smithi176 ceph-mon[34982]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:28.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:28.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:28 smithi153 ceph-mon[34951]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:28.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:28.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:29.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/3104738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0102a30 msgr2=0x7f1fe0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/3104738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0102a30 0x7f1fe0102e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009930 tx=0x7f1fdc02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/3104738595 shutdown_connections 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/3104738595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0104550 0x7f1fe010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/3104738595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0103b90 0x7f1fe0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/3104738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0102a30 0x7f1fe0102e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/3104738595 >> 172.21.15.120:0/3104738595 conn(0x7f1fe00fe320 msgr2=0x7f1fe0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/3104738595 shutdown_connections 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.640+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/3104738595 wait complete. 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 Processor -- start 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 -- start start 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0102a30 0x7f1fe0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0103b90 0x7f1fe0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe010e550 con 0x7f1fe0104550 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe010e3d0 con 0x7f1fe0102a30 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe7dc7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe010e6d0 con 0x7f1fe0103b90 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe5362640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0103b90 0x7f1fe0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53182/0 (socket says 172.21.15.120:53182) 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 -- 172.21.15.120:0/251454279 learned_addr learned my addr 172.21.15.120:0/251454279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe5b63640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0102a30 0x7f1fe0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 -- 172.21.15.120:0/251454279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0103b90 msgr2=0x7f1fe0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0103b90 0x7f1fe0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 -- 172.21.15.120:0/251454279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0102a30 msgr2=0x7f1fe0198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0102a30 0x7f1fe0198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.641+0000 7f1fe6364640 1 -- 172.21.15.120:0/251454279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe019a870 con 0x7f1fe0104550 2024-09-10T22:44:29.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fe6364640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800eaa0 tx=0x7f1fd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:29.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800cdb0 con 0x7f1fe0104550 2024-09-10T22:44:29.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1fd8005020 con 0x7f1fe0104550 2024-09-10T22:44:29.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe019ab60 con 0x7f1fe0104550 2024-09-10T22:44:29.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80116d0 con 0x7f1fe0104550 2024-09-10T22:44:29.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.642+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01a1240 con 0x7f1fe0104550 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.643+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe0102e30 con 0x7f1fe0104550 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.644+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f1fd800f040 con 0x7f1fe0104550 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.645+0000 7f1fd6ffd640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 0x7f1fcc0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.645+0000 7f1fe5b63640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 0x7f1fcc0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.645+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1fd80e3ff0 con 0x7f1fe0104550 2024-09-10T22:44:29.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.645+0000 7f1fe5b63640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 0x7f1fcc0b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:29.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.648+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1fd8016030 con 0x7f1fe0104550 2024-09-10T22:44:29.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.829+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f1fe0109e30 con 0x7f1fe0104550 2024-09-10T22:44:29.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.833+0000 7f1fd6ffd640 1 -- 172.21.15.120:0/251454279 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v29)=0 v29) ==== 119+0+0 (secure 0 0 0) 0x7f1fe0109e30 con 0x7f1fe0104550 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 msgr2=0x7f1fcc0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 0x7f1fcc0b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc005720 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 msgr2=0x7f1fe0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800eaa0 tx=0x7f1fd800ef60 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 shutdown_connections 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1fcc0b52d0 0x7f1fcc0b7790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1fe0104550 0x7f1fe0199fd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe0103b90 0x7f1fe0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 --2- 172.21.15.120:0/251454279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe0102a30 0x7f1fe0198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.836+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 >> 172.21.15.120:0/251454279 conn(0x7f1fe00fe320 msgr2=0x7f1fe0107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.837+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 shutdown_connections 2024-09-10T22:44:29.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:29.837+0000 7f1fe7dc7640 1 -- 172.21.15.120:0/251454279 wait complete. 2024-09-10T22:44:30.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 -- 172.21.15.120:0/2186020137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac101910 msgr2=0x7f1cac101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 --2- 172.21.15.120:0/2186020137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac101910 0x7f1cac101d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009930 tx=0x7f1c9c02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 -- 172.21.15.120:0/2186020137 shutdown_connections 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 --2- 172.21.15.120:0/2186020137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac103430 0x7f1cac109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 --2- 172.21.15.120:0/2186020137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac102a70 0x7f1cac102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 --2- 172.21.15.120:0/2186020137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac101910 0x7f1cac101d10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 -- 172.21.15.120:0/2186020137 >> 172.21.15.120:0/2186020137 conn(0x7f1cac0fd1e0 msgr2=0x7f1cac0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 -- 172.21.15.120:0/2186020137 shutdown_connections 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.047+0000 7f1cb2880640 1 -- 172.21.15.120:0/2186020137 wait complete. 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 Processor -- start 2024-09-10T22:44:30.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 -- start start 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac101910 0x7f1cac19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac103430 0x7f1cac19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac10d430 con 0x7f1cac102a70 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac10d2b0 con 0x7f1cac101910 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cb2880640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac10d5b0 con 0x7f1cac103430 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53204/0 (socket says 172.21.15.120:53204) 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.048+0000 7f1cab7fe640 1 -- 172.21.15.120:0/878500654 learned_addr learned my addr 172.21.15.120:0/878500654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cabfff640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac101910 0x7f1cac19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:30.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cb0e1d640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac103430 0x7f1cac19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 -- 172.21.15.120:0/878500654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac103430 msgr2=0x7f1cac19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac103430 0x7f1cac19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 -- 172.21.15.120:0/878500654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac101910 msgr2=0x7f1cac19af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac101910 0x7f1cac19af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 -- 172.21.15.120:0/878500654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac19ca40 con 0x7f1cac102a70 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cab7fe640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d950 tx=0x7f1ca000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0017070 con 0x7f1cac102a70 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac19cd30 con 0x7f1cac102a70 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ca0009c30 con 0x7f1cac102a70 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000f620 con 0x7f1cac102a70 2024-09-10T22:44:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.049+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac1a3410 con 0x7f1cac102a70 2024-09-10T22:44:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.050+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac101d10 con 0x7f1cac102a70 2024-09-10T22:44:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.052+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f1ca0010460 con 0x7f1cac102a70 2024-09-10T22:44:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.053+0000 7f1ca97fa640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 0x7f1c940b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.053+0000 7f1cabfff640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 0x7f1c940b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.053+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ca00e3b60 con 0x7f1cac102a70 2024-09-10T22:44:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.053+0000 7f1cabfff640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 0x7f1c940b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009900 tx=0x7f1c9c005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.056+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1ca00ac650 con 0x7f1cac102a70 2024-09-10T22:44:30.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.236+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f1cac1a35e0 con 0x7f1cac102a70 2024-09-10T22:44:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.240+0000 7f1ca97fa640 1 -- 172.21.15.120:0/878500654 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v30)=0 v30) ==== 115+0+0 (secure 0 0 0) 0x7f1ca0009de0 con 0x7f1cac102a70 2024-09-10T22:44:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 msgr2=0x7f1c940b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:30.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 0x7f1c940b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009900 tx=0x7f1c9c005720 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 msgr2=0x7f1cac19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d950 tx=0x7f1ca000de10 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 shutdown_connections 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c940b5250 0x7f1c940b7710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cac103430 0x7f1cac19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cac102a70 0x7f1cac19b560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 --2- 172.21.15.120:0/878500654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cac101910 0x7f1cac19af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 >> 172.21.15.120:0/878500654 conn(0x7f1cac0fd1e0 msgr2=0x7f1cac06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 shutdown_connections 2024-09-10T22:44:30.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:30.242+0000 7f1cb2880640 1 -- 172.21.15.120:0/878500654 wait complete. 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.065+0000 7fb44a8b2640 1 -- 172.21.15.120:0/3529151150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 msgr2=0x7fb444101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.065+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/3529151150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb444101d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb434009930 tx=0x7fb43402a420 comp rx=0 tx=0).stop 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.065+0000 7fb44a8b2640 1 -- 172.21.15.120:0/3529151150 shutdown_connections 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.065+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/3529151150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 0x7fb444109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/3529151150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb444102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/3529151150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb444101d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 -- 172.21.15.120:0/3529151150 >> 172.21.15.120:0/3529151150 conn(0x7fb4440fd1e0 msgr2=0x7fb4440ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 -- 172.21.15.120:0/3529151150 shutdown_connections 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 -- 172.21.15.120:0/3529151150 wait complete. 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 Processor -- start 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.066+0000 7fb44a8b2640 1 -- start start 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb44419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 0x7fb4441a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410d4b0 con 0x7fb444101910 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410d330 con 0x7fb444102a70 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410d630 con 0x7fb444103430 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53228/0 (socket says 172.21.15.120:53228) 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 -- 172.21.15.120:0/1033980699 learned_addr learned my addr 172.21.15.120:0/1033980699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:31.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb448e4f640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 0x7fb4441a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 -- 172.21.15.120:0/1033980699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 msgr2=0x7fb4441a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb4437fe640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb44419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 0x7fb4441a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 -- 172.21.15.120:0/1033980699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 msgr2=0x7fb44419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb44419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 -- 172.21.15.120:0/1033980699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4441a0950 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb4437fe640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb44419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb443fff640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434009a50 tx=0x7fb434032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43403f030 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb4340044b0 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.067+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4441a0be0 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.068+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434035070 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.068+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4441a10c0 con 0x7fb444101910 2024-09-10T22:44:31.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.068+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444101d10 con 0x7fb444101910 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.071+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fb43403baf0 con 0x7fb444101910 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.071+0000 7fb4417fa640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 0x7fb42c0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.071+0000 7fb4437fe640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 0x7fb42c0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.071+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb434105090 con 0x7fb444101910 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.072+0000 7fb4437fe640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 0x7fb42c0b7710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4380058c0 tx=0x7fb438005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:31.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.074+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb4340cdc20 con 0x7fb444101910 2024-09-10T22:44:31.088 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/251454279' entity='client.admin' 2024-09-10T22:44:31.088 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:31.088 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:31.088 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:31.089 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:31.089 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:30 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/878500654' entity='client.admin' 2024-09-10T22:44:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/251454279' entity='client.admin' 2024-09-10T22:44:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:31.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:31.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:30 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/878500654' entity='client.admin' 2024-09-10T22:44:31.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.253+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fb4441a1350 con 0x7fb444101910 2024-09-10T22:44:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.257+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v31)=0 v31) ==== 123+0+0 (secure 0 0 0) 0x7fb4340d63d0 con 0x7fb444101910 2024-09-10T22:44:31.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.258+0000 7fb4417fa640 1 -- 172.21.15.120:0/1033980699 <== mon.0 v2:172.21.15.120:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43403b460 con 0x7fb444101910 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 msgr2=0x7fb42c0b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 0x7fb42c0b7710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4380058c0 tx=0x7fb438005d30 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 msgr2=0x7fb44406b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434009a50 tx=0x7fb434032f90 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 shutdown_connections 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb42c0b5250 0x7fb42c0b7710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444103430 0x7fb4441a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb444102a70 0x7fb44419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 --2- 172.21.15.120:0/1033980699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb444101910 0x7fb44406b450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 >> 172.21.15.120:0/1033980699 conn(0x7fb4440fd1e0 msgr2=0x7fb444103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 shutdown_connections 2024-09-10T22:44:31.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:31.259+0000 7fb44a8b2640 1 -- 172.21.15.120:0/1033980699 wait complete. 2024-09-10T22:44:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/251454279' entity='client.admin' 2024-09-10T22:44:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:31.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:30 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/878500654' entity='client.admin' 2024-09-10T22:44:32.544 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:32 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1033980699' entity='client.admin' 2024-09-10T22:44:32.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:32 smithi120 ceph-mon[31247]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:32 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1033980699' entity='client.admin' 2024-09-10T22:44:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:32 smithi176 ceph-mon[34982]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:32.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:32 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1033980699' entity='client.admin' 2024-09-10T22:44:32.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:32 smithi153 ceph-mon[34951]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:32.771 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.774 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd2838936-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi120') 2024-09-10T22:44:32.774 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T22:44:32.774 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:44:32.774 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:44:32.792 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:44:32.792 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-09-10T22:44:32.849 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:44:32.849 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:32.849 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T22:44:32.849 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 22:43:28.324396226 +0000 2024-09-10T22:44:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 22:38:49.431762705 +0000 2024-09-10T22:44:32.850 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 22:38:49.431762705 +0000 2024-09-10T22:44:32.850 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 22:38:49.431762705 +0000 2024-09-10T22:44:32.850 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:44:32.917 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T22:44:32.917 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T22:44:32.917 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000199377 s, 2.6 MB/s 2024-09-10T22:44:32.919 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:44:32.977 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 22:43:28.357395486 +0000 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 22:38:49.746755401 +0000 2024-09-10T22:44:33.033 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 22:38:49.746755401 +0000 2024-09-10T22:44:33.034 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 22:38:49.746755401 +0000 2024-09-10T22:44:33.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:44:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T22:44:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T22:44:33.097 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000220448 s, 2.3 MB/s 2024-09-10T22:44:33.099 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:44:33.155 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-09-10T22:44:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:44:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.211 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T22:44:33.211 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.212 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.212 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 22:43:28.389394769 +0000 2024-09-10T22:44:33.212 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 22:38:50.077747727 +0000 2024-09-10T22:44:33.212 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 22:38:50.077747727 +0000 2024-09-10T22:44:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 22:38:50.077747727 +0000 2024-09-10T22:44:33.212 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:44:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T22:44:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T22:44:33.275 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000217078 s, 2.4 MB/s 2024-09-10T22:44:33.277 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:44:33.333 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-09-10T22:44:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:44:33.388 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.388 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T22:44:33.388 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.388 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.389 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 22:43:28.423394007 +0000 2024-09-10T22:44:33.389 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 22:38:50.389740493 +0000 2024-09-10T22:44:33.389 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 22:38:50.389740493 +0000 2024-09-10T22:44:33.389 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 22:38:50.389740493 +0000 2024-09-10T22:44:33.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:44:33.451 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-09-10T22:44:33.451 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-09-10T22:44:33.452 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000241041 s, 2.1 MB/s 2024-09-10T22:44:33.453 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:44:33.509 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-09-10T22:44:33.509 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:44:33.529 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:44:33.529 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-09-10T22:44:33.587 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:44:33.587 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.587 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-10 22:44:13.905767277 +0000 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-10 22:39:04.212422026 +0000 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-10 22:39:04.212422026 +0000 2024-09-10T22:44:33.588 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-10 22:39:04.212422026 +0000 2024-09-10T22:44:33.588 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:44:33.657 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-10T22:44:33.657 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-10T22:44:33.657 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000293118 s, 1.7 MB/s 2024-09-10T22:44:33.659 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:44:33.717 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-09-10T22:44:33.774 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-10 22:44:13.905767277 +0000 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-10 22:39:04.542415727 +0000 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-10 22:39:04.542415727 +0000 2024-09-10T22:44:33.775 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-10 22:39:04.542415727 +0000 2024-09-10T22:44:33.775 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:44:33.840 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-10T22:44:33.840 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-10T22:44:33.840 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000210995 s, 2.4 MB/s 2024-09-10T22:44:33.842 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:44:33.901 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-10 22:44:13.906767259 +0000 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-10 22:39:04.865409561 +0000 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-10 22:39:04.865409561 +0000 2024-09-10T22:44:33.958 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-10 22:39:04.865409561 +0000 2024-09-10T22:44:33.959 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:44:34.024 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-10T22:44:34.024 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-10T22:44:34.024 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000205299 s, 2.5 MB/s 2024-09-10T22:44:34.025 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:44:34.082 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-09-10T22:44:34.138 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-09-10 22:44:13.906767259 +0000 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-09-10 22:39:05.204403090 +0000 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-09-10 22:39:05.204403090 +0000 2024-09-10T22:44:34.139 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-09-10 22:39:05.204403090 +0000 2024-09-10T22:44:34.139 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:44:34.204 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-09-10T22:44:34.204 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-09-10T22:44:34.204 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.00020612 s, 2.5 MB/s 2024-09-10T22:44:34.205 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:44:34.262 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-10T22:44:34.262 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:44:34.282 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:44:34.282 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-09-10T22:44:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:44:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:34.340 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-10T22:44:34.340 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:34.341 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:34.341 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-10 22:44:11.608328156 +0000 2024-09-10T22:44:34.341 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-10 22:39:06.752845508 +0000 2024-09-10T22:44:34.341 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-10 22:39:06.752845508 +0000 2024-09-10T22:44:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-10 22:39:06.752845508 +0000 2024-09-10T22:44:34.341 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:44:34.409 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-10T22:44:34.409 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-10T22:44:34.409 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000260247 s, 2.0 MB/s 2024-09-10T22:44:34.410 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:44:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:34 smithi120 ceph-mon[31247]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:34.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:34 smithi153 ceph-mon[34951]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:34.468 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-10 22:44:11.608328156 +0000 2024-09-10T22:44:34.526 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-10 22:39:07.082842325 +0000 2024-09-10T22:44:34.526 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-10 22:39:07.082842325 +0000 2024-09-10T22:44:34.526 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-10 22:39:07.082842325 +0000 2024-09-10T22:44:34.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:44:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:34 smithi176 ceph-mon[34982]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:34.590 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-10T22:44:34.590 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-10T22:44:34.591 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000221779 s, 2.3 MB/s 2024-09-10T22:44:34.592 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:44:34.649 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-10 22:44:11.609328147 +0000 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-10 22:39:07.412839142 +0000 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-10 22:39:07.412839142 +0000 2024-09-10T22:44:34.706 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-10 22:39:07.412839142 +0000 2024-09-10T22:44:34.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:44:34.771 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-10T22:44:34.771 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-10T22:44:34.771 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.00028277 s, 1.8 MB/s 2024-09-10T22:44:34.773 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:44:34.830 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-09-10T22:44:34.886 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:44:34.886 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:44:34.886 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-10 22:44:11.609328147 +0000 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-10 22:39:07.737836008 +0000 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-10 22:39:07.737836008 +0000 2024-09-10T22:44:34.887 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-09-10 22:39:07.737836008 +0000 2024-09-10T22:44:34.887 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:44:34.950 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-10T22:44:34.950 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-10T22:44:34.951 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000201793 s, 2.5 MB/s 2024-09-10T22:44:34.952 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:44:35.009 INFO:tasks.cephadm:Deploying osd.0 on smithi120 with /dev/vg_nvme/lv_4... 2024-09-10T22:44:35.009 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:44:35.268 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:36 smithi120 ceph-mon[31247]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:36 smithi153 ceph-mon[34951]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:36.553 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:36 smithi176 ceph-mon[34982]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:37.706 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:44:37.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-09-10T22:44:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:38.365 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:38 smithi120 ceph-mon[31247]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:38 smithi153 ceph-mon[34951]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:38.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 -- 172.21.15.120:0/3060389168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350101910 msgr2=0x7f0350101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:38.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 --2- 172.21.15.120:0/3060389168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350101910 0x7f0350101d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f034c009930 tx=0x7f034c02a420 comp rx=0 tx=0).stop 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 -- 172.21.15.120:0/3060389168 shutdown_connections 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 --2- 172.21.15.120:0/3060389168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350103430 0x7f0350109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 --2- 172.21.15.120:0/3060389168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350102a70 0x7f0350102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 --2- 172.21.15.120:0/3060389168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350101910 0x7f0350101d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 -- 172.21.15.120:0/3060389168 >> 172.21.15.120:0/3060389168 conn(0x7f03500fd1e0 msgr2=0x7f03500ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 -- 172.21.15.120:0/3060389168 shutdown_connections 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 -- 172.21.15.120:0/3060389168 wait complete. 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.458+0000 7f0357b7a640 1 Processor -- start 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 -- start start 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350101910 0x7f0350198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:38.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350103430 0x7f035019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f035010e700 con 0x7f0350102a70 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f035010e580 con 0x7f0350103430 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0357b7a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f035010e880 con 0x7f0350101910 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35738/0 (socket says 172.21.15.120:35738) 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 -- 172.21.15.120:0/561708292 learned_addr learned my addr 172.21.15.120:0/561708292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355916640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350101910 0x7f0350198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0356117640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350103430 0x7f035019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 -- 172.21.15.120:0/561708292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350101910 msgr2=0x7f0350198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350101910 0x7f0350198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 -- 172.21.15.120:0/561708292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350103430 msgr2=0x7f035019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350103430 0x7f035019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:38.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 -- 172.21.15.120:0/561708292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035019a960 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0355115640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.459+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017030 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.460+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0340004020 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.460+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035019ac50 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.460+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400047a0 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.460+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a1330 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.461+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350108c90 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.462+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f0340009a30 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.462+0000 7f0346ffd640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 0x7f033c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.462+0000 7f0355916640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 0x7f033c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.463+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f03400e3870 con 0x7f0350102a70 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.463+0000 7f0355916640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 0x7f033c0b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f034c009900 tx=0x7f034c031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:38.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.466+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03400ac400 con 0x7f0350102a70 2024-09-10T22:44:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:38 smithi176 ceph-mon[34982]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:38.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:44:38.649+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0350063590 con 0x7f033c0b52d0 2024-09-10T22:44:39.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:39 smithi120 ceph-mon[31247]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:39.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:39.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:39.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:39 smithi153 ceph-mon[34951]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:39 smithi176 ceph-mon[34982]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:40.433 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:40 smithi120 ceph-mon[31247]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:40 smithi153 ceph-mon[34951]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:40 smithi176 ceph-mon[34982]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:41.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:41 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"} : dispatch 2024-09-10T22:44:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:41 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"}]': finished 2024-09-10T22:44:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:41 smithi120 ceph-mon[31247]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:44:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:44:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:41 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"} : dispatch 2024-09-10T22:44:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:41 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"}]': finished 2024-09-10T22:44:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:41 smithi153 ceph-mon[34951]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:44:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:44:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:41 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"} : dispatch 2024-09-10T22:44:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:41 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/900876497' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1918891-2bc5-4422-b3ff-eaf029ff3161"}]': finished 2024-09-10T22:44:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:41 smithi176 ceph-mon[34982]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:44:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:44:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:42 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2481613367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:42 smithi120 ceph-mon[31247]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:42 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2481613367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:42 smithi153 ceph-mon[34951]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:42 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2481613367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:42 smithi176 ceph-mon[34982]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:44 smithi120 ceph-mon[31247]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:44.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:44 smithi153 ceph-mon[34951]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:44 smithi176 ceph-mon[34982]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:46 smithi120 ceph-mon[31247]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:46.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:46 smithi153 ceph-mon[34951]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:46 smithi176 ceph-mon[34982]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:48.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:48 smithi120 ceph-mon[31247]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:48 smithi153 ceph-mon[34951]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:48 smithi176 ceph-mon[34982]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:44:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:49.895 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:44:49.896 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:49.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:44:49.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:50 smithi176 ceph-mon[34982]: Deploying daemon osd.0 on smithi120 2024-09-10T22:44:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:50 smithi176 ceph-mon[34982]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:50.835 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:50 smithi120 ceph-mon[31247]: Deploying daemon osd.0 on smithi120 2024-09-10T22:44:50.835 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:50 smithi120 ceph-mon[31247]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:50 smithi153 ceph-mon[34951]: Deploying daemon osd.0 on smithi120 2024-09-10T22:44:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:50 smithi153 ceph-mon[34951]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:52.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:52 smithi120 ceph-mon[31247]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:52 smithi153 ceph-mon[34951]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:52 smithi176 ceph-mon[34982]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:54.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:54 smithi120 ceph-mon[31247]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:54 smithi153 ceph-mon[34951]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:54 smithi176 ceph-mon[34982]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:56.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:56.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:57.369 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:57 smithi120 ceph-mon[31247]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:57.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:57 smithi153 ceph-mon[34951]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:57 smithi176 ceph-mon[34982]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:58 smithi120 ceph-mon[31247]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:58.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:58 smithi153 ceph-mon[34951]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:58 smithi176 ceph-mon[34982]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:44:59.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:59.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:59.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:44:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:44:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:44:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:44:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.025+0000 7f0346ffd640 1 -- 172.21.15.120:0/561708292 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0350063590 con 0x7f033c0b52d0 2024-09-10T22:45:00.027 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 0 on host 'smithi120' 2024-09-10T22:45:00.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 msgr2=0x7f033c0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 0x7f033c0b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f034c009900 tx=0x7f034c031450 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 msgr2=0x7f0350199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 shutdown_connections 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f033c0b52d0 0x7f033c0b7790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0350103430 0x7f035019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0350102a70 0x7f0350199480 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 --2- 172.21.15.120:0/561708292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0350101910 0x7f0350198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 >> 172.21.15.120:0/561708292 conn(0x7f03500fd1e0 msgr2=0x7f0350105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 shutdown_connections 2024-09-10T22:45:00.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:00.028+0000 7f0357b7a640 1 -- 172.21.15.120:0/561708292 wait complete. 2024-09-10T22:45:00.640 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:00 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:00.641 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:00 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.641 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:00 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.641 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:00 smithi120 ceph-mon[31247]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:00.664 DEBUG:teuthology.orchestra.run.smithi120:osd.0> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T22:45:00.666 INFO:tasks.cephadm:Deploying osd.1 on smithi153 with /dev/vg_nvme/lv_4... 2024-09-10T22:45:00.666 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:45:00.692 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:00 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:00.693 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:00 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.693 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:00 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.693 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:00 smithi153 ceph-mon[34951]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:00 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:00 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:00 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:00 smithi176 ceph-mon[34982]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:00.915 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:45:02.378 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:02 smithi153 ceph-mon[34951]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:45:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:02 smithi120 ceph-mon[31247]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:45:02.447 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:02 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T22:45:02.046+0000 7f2f4f79e640 -1 osd.0 0 log_to_monitors true 2024-09-10T22:45:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:02 smithi176 ceph-mon[34982]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:45:03.256 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:45:03.280 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:45:03.344 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:45:03.385 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.564 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:45:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:45:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:45:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:45:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:45:03.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:45:03.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:03.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- 172.21.15.153:0/1802822396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 msgr2=0x7f6838102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 --2- 172.21.15.153:0/1802822396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838102f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6828009930 tx=0x7f682802a420 comp rx=0 tx=0).stop 2024-09-10T22:45:03.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- 172.21.15.153:0/1802822396 shutdown_connections 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 --2- 172.21.15.153:0/1802822396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838103470 0x7f6838109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 --2- 172.21.15.153:0/1802822396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838102f30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 --2- 172.21.15.153:0/1802822396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838101950 0x7f6838101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- 172.21.15.153:0/1802822396 >> 172.21.15.153:0/1802822396 conn(0x7f68380fd200 msgr2=0x7f68380ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- 172.21.15.153:0/1802822396 shutdown_connections 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- 172.21.15.153:0/1802822396 wait complete. 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 Processor -- start 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.950+0000 7f683f529640 1 -- start start 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 0x7f6838198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838103470 0x7f683819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f683810e700 con 0x7f6838101950 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f683810e580 con 0x7f6838102ab0 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f683810e880 con 0x7f6838103470 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56720/0 (socket says 172.21.15.153:56720) 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 -- 172.21.15.153:0/3890225202 learned_addr learned my addr 172.21.15.153:0/3890225202 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683d2c5640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 0x7f6838198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683dac6640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838103470 0x7f683819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 -- 172.21.15.153:0/3890225202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838103470 msgr2=0x7f683819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838103470 0x7f683819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 -- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 msgr2=0x7f6838198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 0x7f6838198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 -- 172.21.15.153:0/3890225202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683819a980 con 0x7f6838102ab0 2024-09-10T22:45:03.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683d2c5640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 0x7f6838198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683cac4640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6828005740 tx=0x7f68280042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682803f030 con 0x7f6838102ab0 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683819ac10 con 0x7f6838102ab0 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.951+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6828003cb0 con 0x7f6838102ab0 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.952+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828035070 con 0x7f6838102ab0 2024-09-10T22:45:03.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.952+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68381a1180 con 0x7f6838102ab0 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.953+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6838108cd0 con 0x7f6838102ab0 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.954+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f6828003800 con 0x7f6838102ab0 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.955+0000 7f682e7fc640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 0x7f68240b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.955+0000 7f683d2c5640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 0x7f68240b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.955+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f6828104c00 con 0x7f6838102ab0 2024-09-10T22:45:03.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.956+0000 7f683d2c5640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 0x7f68240b7710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f683400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:03.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:03.959+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f68280ccfe0 con 0x7f6838102ab0 2024-09-10T22:45:04.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:04.144+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6838063590 con 0x7f68240b5250 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:04.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' 2024-09-10T22:45:04.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:04.447 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T22:45:04.009+0000 7f2f4b721640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T22:45:04.447 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T22:45:04.152+0000 7f2f43540640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:45:04.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:04.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:04.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:04.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:04.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' 2024-09-10T22:45:04.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006]' entity='osd.0' 2024-09-10T22:45:04.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi120 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi120 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi120 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 17088M 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:05.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:05.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:06 smithi120 ceph-mon[31247]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:06 smithi120 ceph-mon[31247]: osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] boot 2024-09-10T22:45:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:06 smithi120 ceph-mon[31247]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:45:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:06 smithi120 ceph-mon[31247]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:06.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:06 smithi153 ceph-mon[34951]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:06.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:06 smithi153 ceph-mon[34951]: osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] boot 2024-09-10T22:45:06.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:06 smithi153 ceph-mon[34951]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:45:06.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:06.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:06 smithi153 ceph-mon[34951]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:06 smithi176 ceph-mon[34982]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:06 smithi176 ceph-mon[34982]: osd.0 [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] boot 2024-09-10T22:45:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:06 smithi176 ceph-mon[34982]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:45:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:45:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:06 smithi176 ceph-mon[34982]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/4209774443' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"}]': finished 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:45:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:07.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:07 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/3473226573' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/4209774443' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"}]': finished 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:07 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/3473226573' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/4209774443' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"} : dispatch 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb0107b-f7e1-48bd-8916-bd98f0075444"}]': finished 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:45:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:07.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:07 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/3473226573' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:08 smithi120 ceph-mon[31247]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:08 smithi153 ceph-mon[34951]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:08 smithi176 ceph-mon[34982]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:10 smithi120 ceph-mon[31247]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:10 smithi153 ceph-mon[34951]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:10 smithi176 ceph-mon[34982]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:12.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:12 smithi120 ceph-mon[31247]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:12 smithi153 ceph-mon[34951]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:12 smithi176 ceph-mon[34982]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:14.268 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:14 smithi153 ceph-mon[34951]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:14 smithi120 ceph-mon[31247]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:14 smithi176 ceph-mon[34982]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:15.273 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:45:15.273 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:15.273 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:15 smithi153 ceph-mon[34951]: Deploying daemon osd.1 on smithi153 2024-09-10T22:45:15.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:45:15.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:15.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:15 smithi120 ceph-mon[31247]: Deploying daemon osd.1 on smithi153 2024-09-10T22:45:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:45:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:15 smithi176 ceph-mon[34982]: Deploying daemon osd.1 on smithi153 2024-09-10T22:45:16.442 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:16 smithi153 ceph-mon[34951]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:16 smithi120 ceph-mon[31247]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:16 smithi176 ceph-mon[34982]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:18.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:18 smithi120 ceph-mon[31247]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:18.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:18 smithi153 ceph-mon[34951]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:18 smithi176 ceph-mon[34982]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:20 smithi153 ceph-mon[34951]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:20 smithi120 ceph-mon[31247]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:20 smithi176 ceph-mon[34982]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:21.430 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:21 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:21.431 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:21 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:21.431 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:21 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:21.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:21.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:21.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:21 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:21 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:21 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:22.289 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:22 smithi153 ceph-mon[34951]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:22 smithi120 ceph-mon[31247]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:22 smithi176 ceph-mon[34982]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:24.325 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.325 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.325 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:24.325 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:24.326 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:24.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:24.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.629+0000 7f682e7fc640 1 -- 172.21.15.153:0/3890225202 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6838063590 con 0x7f68240b5250 2024-09-10T22:45:24.631 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 1 on host 'smithi153' 2024-09-10T22:45:24.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 msgr2=0x7f68240b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 0x7f68240b7710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f683400b390 comp rx=0 tx=0).stop 2024-09-10T22:45:24.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 msgr2=0x7f6838199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6828005740 tx=0x7f68280042f0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 shutdown_connections 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68240b5250 0x7f68240b7710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6838103470 0x7f683819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.631+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6838102ab0 0x7f6838199510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.632+0000 7f683f529640 1 --2- 172.21.15.153:0/3890225202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6838101950 0x7f6838198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.632+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 >> 172.21.15.153:0/3890225202 conn(0x7f68380fd200 msgr2=0x7f6838105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.632+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 shutdown_connections 2024-09-10T22:45:24.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:45:24.632+0000 7f683f529640 1 -- 172.21.15.153:0/3890225202 wait complete. 2024-09-10T22:45:25.212 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:25 smithi153 ceph-mon[34951]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:25.212 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:25.212 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:25.213 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:25.249 DEBUG:teuthology.orchestra.run.smithi153:osd.1> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T22:45:25.291 INFO:tasks.cephadm:Deploying osd.2 on smithi176 with /dev/vg_nvme/lv_4... 2024-09-10T22:45:25.291 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:45:25.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:25 smithi120 ceph-mon[31247]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:25.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:25.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:25.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:25.540 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:45:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:25 smithi176 ceph-mon[34982]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:26.529 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T22:45:26.250+0000 7f2b7903e640 -1 osd.1 0 log_to_monitors true 2024-09-10T22:45:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:27 smithi120 ceph-mon[31247]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:27 smithi120 ceph-mon[31247]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:27 smithi120 ceph-mon[31247]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:27 smithi153 ceph-mon[34951]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:27 smithi153 ceph-mon[34951]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:27 smithi153 ceph-mon[34951]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:27 smithi176 ceph-mon[34982]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:27 smithi176 ceph-mon[34982]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:27 smithi176 ceph-mon[34982]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:45:27.853 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:45:27.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2024-09-10T22:45:28.132 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.371 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:45:28.372 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:45:28.450 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:45:28.451 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:28.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:28.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:28.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:45:28.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:45:28.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:28.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 -- 172.21.15.176:0/3028079487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858102a20 msgr2=0x7fb858102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/3028079487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858102a20 0x7fb858102e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009960 tx=0x7fb84c02a460 comp rx=0 tx=0).stop 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 -- 172.21.15.176:0/3028079487 shutdown_connections 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/3028079487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858104540 0x7fb85810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/3028079487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb858104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/3028079487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858102a20 0x7fb858102e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 -- 172.21.15.176:0/3028079487 >> 172.21.15.176:0/3028079487 conn(0x7fb8580fe2f0 msgr2=0x7fb858100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 -- 172.21.15.176:0/3028079487 shutdown_connections 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.612+0000 7fb85cf7e640 1 -- 172.21.15.176:0/3028079487 wait complete. 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 Processor -- start 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 -- start start 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858102a20 0x7fb858198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb8581994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb85810e6d0 con 0x7fb858103b80 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb85810e550 con 0x7fb858102a20 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85cf7e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb85810e850 con 0x7fb858104540 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb856d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb856d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56040/0 (socket says 172.21.15.176:56040) 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb856d9d640 1 -- 172.21.15.176:0/2065750465 learned_addr learned my addr 172.21.15.176:0/2065750465 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.613+0000 7fb85659c640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858102a20 0x7fb858198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb855d9b640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb8581994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 -- 172.21.15.176:0/2065750465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858102a20 msgr2=0x7fb858198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858102a20 0x7fb858198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 -- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 msgr2=0x7fb8581994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb8581994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 -- 172.21.15.176:0/2065750465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85819a9b0 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb855d9b640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb8581994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb856d9d640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb85000eaa0 tx=0x7fb85000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000cdb0 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85819aca0 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb850005020 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500116d0 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.614+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8581a1380 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.615+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb858109da0 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.617+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fb85000f040 con 0x7fb858104540 2024-09-10T22:45:28.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.617+0000 7fb8477fe640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 0x7fb8400b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.617+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fb8500e44b0 con 0x7fb858104540 2024-09-10T22:45:28.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.617+0000 7fb85659c640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 0x7fb8400b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.618+0000 7fb85659c640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 0x7fb8400b7710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb84c031460 tx=0x7fb84c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:28.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.621+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb850016030 con 0x7fb858104540 2024-09-10T22:45:28.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:28.805+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb858063590 con 0x7fb8400b5250 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi153 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='osd.1 ' entity='osd.1' 2024-09-10T22:45:29.415 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi153 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:29.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:29.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='osd.1 ' entity='osd.1' 2024-09-10T22:45:29.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:29.449 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T22:45:29.003+0000 7f2b74fc1640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T22:45:29.449 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T22:45:29.085+0000 7f2b6cde0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:45:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi153 2024-09-10T22:45:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 17088M 2024-09-10T22:45:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='osd.1 ' entity='osd.1' 2024-09-10T22:45:29.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:30.397 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:45:30.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:45:30.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:30.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] boot 2024-09-10T22:45:30.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:45:30.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] boot 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:45:30.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: osd.1 [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] boot 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:45:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:45:31.369 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:31.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/2006095950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"}]': finished 2024-09-10T22:45:31.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T22:45:31.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/2006095950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"}]': finished 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T22:45:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/2006095950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"} : dispatch 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcf71dbb-0b3e-4778-9735-44deb5142b73"}]': finished 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T22:45:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:32 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/2251153159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:32.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:32 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/2251153159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:32 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/2251153159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:33.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:33 smithi120 ceph-mon[31247]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:33.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:33 smithi153 ceph-mon[34951]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:33.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:33 smithi176 ceph-mon[34982]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:34 smithi120 ceph-mon[31247]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:34.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:34 smithi153 ceph-mon[34951]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:34 smithi176 ceph-mon[34982]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:36 smithi120 ceph-mon[31247]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:36 smithi153 ceph-mon[34951]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:36 smithi176 ceph-mon[34982]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:38 smithi120 ceph-mon[31247]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:38 smithi153 ceph-mon[34951]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:38.454 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:38 smithi176 ceph-mon[34982]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:39.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:45:39.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:45:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:39.736 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:45:39.737 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:40.668 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:40 smithi176 ceph-mon[34982]: Deploying daemon osd.2 on smithi176 2024-09-10T22:45:40.669 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:40 smithi176 ceph-mon[34982]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:40 smithi120 ceph-mon[31247]: Deploying daemon osd.2 on smithi176 2024-09-10T22:45:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:40 smithi120 ceph-mon[31247]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:40 smithi153 ceph-mon[34951]: Deploying daemon osd.2 on smithi176 2024-09-10T22:45:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:40 smithi153 ceph-mon[34951]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:42 smithi120 ceph-mon[31247]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:42 smithi153 ceph-mon[34951]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:42.534 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:42 smithi176 ceph-mon[34982]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:44 smithi120 ceph-mon[31247]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:44.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:44 smithi153 ceph-mon[34951]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:44.459 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:44 smithi176 ceph-mon[34982]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:46.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:46.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:46.989 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:46 smithi176 ceph-mon[34982]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:47.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:46 smithi120 ceph-mon[31247]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:47.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:46 smithi153 ceph-mon[34951]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:49.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:49.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:49.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:48 smithi176 ceph-mon[34982]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:48 smithi120 ceph-mon[31247]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:49.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:48 smithi153 ceph-mon[34951]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:49.826 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 2 on host 'smithi176' 2024-09-10T22:45:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.824+0000 7fb8477fe640 1 -- 172.21.15.176:0/2065750465 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb858063590 con 0x7fb8400b5250 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.826+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 msgr2=0x7fb8400b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.826+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 0x7fb8400b7710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb84c031460 tx=0x7fb84c005040 comp rx=0 tx=0).stop 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.826+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 msgr2=0x7fb85819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.826+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb85000eaa0 tx=0x7fb85000ef60 comp rx=0 tx=0).stop 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 shutdown_connections 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb8400b5250 0x7fb8400b7710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb858104540 0x7fb85819a110 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb858103b80 0x7fb8581994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 --2- 172.21.15.176:0/2065750465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb858102a20 0x7fb858198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 >> 172.21.15.176:0/2065750465 conn(0x7fb8580fe2f0 msgr2=0x7fb858106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:49.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 shutdown_connections 2024-09-10T22:45:49.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:45:49.827+0000 7fb85cf7e640 1 -- 172.21.15.176:0/2065750465 wait complete. 2024-09-10T22:45:50.137 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:50.138 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.138 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:50.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:50.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:50.465 DEBUG:teuthology.orchestra.run.smithi176:osd.2> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T22:45:50.469 INFO:tasks.cephadm:Deploying osd.3 on smithi120 with /dev/vg_nvme/lv_3... 2024-09-10T22:45:50.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:45:50.721 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:51.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:50 smithi176 ceph-mon[34982]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:51.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:50 smithi120 ceph-mon[31247]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:51.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:50 smithi153 ceph-mon[34951]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:51.882 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:51 smithi176 ceph-mon[34982]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:51.883 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:51 smithi176 ceph-mon[34982]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:51.883 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T22:45:51.703+0000 7f4ae31e3640 -1 osd.2 0 log_to_monitors true 2024-09-10T22:45:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:51 smithi120 ceph-mon[31247]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:52.104 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:51 smithi120 ceph-mon[31247]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:52.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:51 smithi153 ceph-mon[34951]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:52.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:51 smithi153 ceph-mon[34951]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:45:53.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:45:53.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.076 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:45:53.077 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:53.077 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.077 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:52 smithi120 ceph-mon[31247]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:52 smithi153 ceph-mon[34951]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: from='osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:45:53.209 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:52 smithi176 ceph-mon[34982]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:53.405 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:45:53.432 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-09-10T22:45:53.685 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:54.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:45:54.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:45:54.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.014 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.015 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:53 smithi120 ceph-mon[31247]: from='osd.2 ' entity='osd.2' 2024-09-10T22:45:54.073 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T22:45:53.791+0000 7f4adf166640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T22:45:54.073 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T22:45:54.021+0000 7f4ad6f85640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:54.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.075 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:53 smithi176 ceph-mon[34982]: from='osd.2 ' entity='osd.2' 2024-09-10T22:45:54.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 -- 172.21.15.120:0/3438951437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 msgr2=0x7ff5bc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/3438951437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc101d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009930 tx=0x7ff5ac02a420 comp rx=0 tx=0).stop 2024-09-10T22:45:54.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 -- 172.21.15.120:0/3438951437 shutdown_connections 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/3438951437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5bc103430 0x7ff5bc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/3438951437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5bc102a70 0x7ff5bc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/3438951437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc101d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.096+0000 7ff5c1f68640 1 -- 172.21.15.120:0/3438951437 >> 172.21.15.120:0/3438951437 conn(0x7ff5bc0fd1e0 msgr2=0x7ff5bc0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- 172.21.15.120:0/3438951437 shutdown_connections 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- 172.21.15.120:0/3438951437 wait complete. 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 Processor -- start 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- start start 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5bc102a70 0x7ff5bc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5bc103430 0x7ff5bc1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc10d670 con 0x7ff5bc101910 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc10d4f0 con 0x7ff5bc102a70 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.097+0000 7ff5c1f68640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc10d7f0 con 0x7ff5bc103430 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41776/0 (socket says 172.21.15.120:41776) 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 -- 172.21.15.120:0/1026008323 learned_addr learned my addr 172.21.15.120:0/1026008323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 -- 172.21.15.120:0/1026008323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5bc103430 msgr2=0x7ff5bc1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5bc103430 0x7ff5bc1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 -- 172.21.15.120:0/1026008323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5bc102a70 msgr2=0x7ff5bc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5bc102a70 0x7ff5bc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 -- 172.21.15.120:0/1026008323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1a0a90 con 0x7ff5bc101910 2024-09-10T22:45:54.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5bb7fe640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005ec0 tx=0x7ff5ac004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac03f030 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ac003cb0 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc1a0d20 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac035070 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.098+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1a1270 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.099+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc108c30 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.100+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7ff5ac03b460 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.101+0000 7ff5b8ff9640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 0x7ff5a40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.101+0000 7ff5baffd640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 0x7ff5a40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.101+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7ff5ac104f40 con 0x7ff5bc101910 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.102+0000 7ff5baffd640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 0x7ff5a40b7790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009710 tx=0x7ff5b0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:54.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.105+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff5ac0cd530 con 0x7ff5bc101910 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:54.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:53 smithi153 ceph-mon[34951]: from='osd.2 ' entity='osd.2' 2024-09-10T22:45:54.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:54.296+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff5bc063590 con 0x7ff5a40b52d0 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi176 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:55.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:55.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi176 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:55.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:55.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi176 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 17088M 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:55.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: pgmap v86: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:45:55.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:45:55.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:45:55.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:55.996 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:55 smithi120 ceph-mon[31247]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:55.997 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:55.997 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:55 smithi120 ceph-mon[31247]: osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] boot 2024-09-10T22:45:55.997 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:55 smithi120 ceph-mon[31247]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T22:45:55.997 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:56.114 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:55 smithi153 ceph-mon[34951]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:56.115 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:56.115 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:55 smithi153 ceph-mon[34951]: osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] boot 2024-09-10T22:45:56.115 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:55 smithi153 ceph-mon[34951]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T22:45:56.115 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:56.224 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:55 smithi176 ceph-mon[34982]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:56.225 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:56.225 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:55 smithi176 ceph-mon[34982]: osd.2 [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] boot 2024-09-10T22:45:56.225 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:55 smithi176 ceph-mon[34982]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T22:45:56.225 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: pgmap v89: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"} : dispatch 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"}]': finished 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:57.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:56 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: pgmap v89: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"} : dispatch 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"}]': finished 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:57.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:56 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:57.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:45:57.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: pgmap v89: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"} : dispatch 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3821760202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "422694c8-3c8d-4cae-987f-829b757f91eb"}]': finished 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:57.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:56 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:45:58.153 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/763981006' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:58.153 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:58.153 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:57 smithi176 ceph-mon[34982]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T22:45:58.153 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/763981006' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:57 smithi120 ceph-mon[31247]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T22:45:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:58.197 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.197 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.197 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52806]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.197 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52806]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.198 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.198 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.198 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52809]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.198 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52809]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/763981006' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:45:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:45:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:57 smithi153 ceph-mon[34951]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T22:45:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43387]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43387]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43390]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43390]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43867]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43867]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43867]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43867]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43870]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43870]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43870]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43870]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43861]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43861]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43861]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43861]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43864]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43864]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43864]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.574 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 22:45:58 smithi176 sudo[43864]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52812]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52812]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52815]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:58 smithi120 sudo[52815]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43393]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43396]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:45:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:58 smithi153 sudo[43396]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:45:59.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:45:59.078+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff5ac0c90a0 con 0x7ff5bc101910 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:45:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:45:59.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:45:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:45:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:45:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:45:59.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:45:59.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:45:59.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:45:59.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:45:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:00 smithi120 ceph-mon[31247]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:46:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:00 smithi153 ceph-mon[34951]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:46:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:00 smithi176 ceph-mon[34982]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:46:01.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:01 smithi120 ceph-mon[31247]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:01 smithi153 ceph-mon[34951]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:01 smithi176 ceph-mon[34982]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:02 smithi120 ceph-mon[31247]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:02 smithi153 ceph-mon[34951]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:02 smithi176 ceph-mon[34982]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:04.417 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:04 smithi120 ceph-mon[31247]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:04 smithi153 ceph-mon[34951]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:04 smithi176 ceph-mon[34982]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:05.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:46:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:46:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:46:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:06.737 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:06 smithi120 ceph-mon[31247]: Deploying daemon osd.3 on smithi120 2024-09-10T22:46:06.737 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:06 smithi120 ceph-mon[31247]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:06.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:06 smithi176 ceph-mon[34982]: Deploying daemon osd.3 on smithi120 2024-09-10T22:46:06.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:06 smithi176 ceph-mon[34982]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:06.832 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:06 smithi153 ceph-mon[34951]: Deploying daemon osd.3 on smithi120 2024-09-10T22:46:06.832 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:06 smithi153 ceph-mon[34951]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:08.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:08 smithi120 ceph-mon[31247]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:08 smithi153 ceph-mon[34951]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:08 smithi176 ceph-mon[34982]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:10.408 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:10 smithi120 ceph-mon[31247]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:10 smithi153 ceph-mon[34951]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:10 smithi176 ceph-mon[34982]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:11.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:11.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:11.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:12.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:12.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:12.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:13.034 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:12 smithi120 ceph-mon[31247]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:13.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:12 smithi176 ceph-mon[34982]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:13.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:12 smithi153 ceph-mon[34951]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.053 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:14 smithi120 ceph-mon[31247]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:14 smithi176 ceph-mon[34982]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:14 smithi153 ceph-mon[34951]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:15.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.678+0000 7ff5b8ff9640 1 -- 172.21.15.120:0/1026008323 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5bc063590 con 0x7ff5a40b52d0 2024-09-10T22:46:15.681 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 3 on host 'smithi120' 2024-09-10T22:46:15.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 msgr2=0x7ff5a40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 0x7ff5a40b7790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009710 tx=0x7ff5b0006c90 comp rx=0 tx=0).stop 2024-09-10T22:46:15.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 msgr2=0x7ff5bc06b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005ec0 tx=0x7ff5ac004360 comp rx=0 tx=0).stop 2024-09-10T22:46:15.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 shutdown_connections 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5a40b52d0 0x7ff5a40b7790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5bc103430 0x7ff5bc1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5bc102a70 0x7ff5bc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 --2- 172.21.15.120:0/1026008323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5bc101910 0x7ff5bc06b3f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 >> 172.21.15.120:0/1026008323 conn(0x7ff5bc0fd1e0 msgr2=0x7ff5bc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 shutdown_connections 2024-09-10T22:46:15.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:46:15.681+0000 7ff5c1f68640 1 -- 172.21.15.120:0/1026008323 wait complete. 2024-09-10T22:46:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:15.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:16.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:16.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:16.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:16.341 DEBUG:teuthology.orchestra.run.smithi120:osd.3> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T22:46:16.343 INFO:tasks.cephadm:Deploying osd.4 on smithi153 with /dev/vg_nvme/lv_3... 2024-09-10T22:46:16.344 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:46:16.597 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:46:16.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:16 smithi120 ceph-mon[31247]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:17.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:16 smithi176 ceph-mon[34982]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:17.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:16 smithi153 ceph-mon[34951]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:18.063 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 22:46:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T22:46:17.997+0000 7f1eedcb3640 -1 osd.3 0 log_to_monitors true 2024-09-10T22:46:18.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:18 smithi153 ceph-mon[34951]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:46:18.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:18 smithi176 ceph-mon[34982]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:46:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:18 smithi120 ceph-mon[31247]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:46:18.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:46:19.012 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-09-10T22:46:19.266 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:46:19.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:46:19.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:46:19.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:46:19.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:19.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:46:19.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:46:19.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:46:19.375 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:46:19.376 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.447 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T22:46:19.062+0000 7f1ee9c36640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T22:46:19.447 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T22:46:19.070+0000 7f1ee5a5d640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:46:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:46:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:46:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:46:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:46:19.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: from='osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:46:19.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:46:19.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 -- 172.21.15.153:0/1915598595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8102a40 msgr2=0x7f27c8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 --2- 172.21.15.153:0/1915598595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8102a40 0x7f27c8102e40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009890 tx=0x7f27b802a380 comp rx=0 tx=0).stop 2024-09-10T22:46:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 -- 172.21.15.153:0/1915598595 shutdown_connections 2024-09-10T22:46:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 --2- 172.21.15.153:0/1915598595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8104560 0x7f27c810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 --2- 172.21.15.153:0/1915598595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8103ba0 0x7f27c8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 --2- 172.21.15.153:0/1915598595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8102a40 0x7f27c8102e40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 -- 172.21.15.153:0/1915598595 >> 172.21.15.153:0/1915598595 conn(0x7f27c80fe2f0 msgr2=0x7f27c8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 -- 172.21.15.153:0/1915598595 shutdown_connections 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.727+0000 7f27cc809640 1 -- 172.21.15.153:0/1915598595 wait complete. 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 Processor -- start 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 -- start start 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8102a40 0x7f27c8198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 0x7f27c819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810e6c0 con 0x7f27c8104560 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810e540 con 0x7f27c8103ba0 2024-09-10T22:46:19.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27cc809640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27c810e840 con 0x7f27c8102a40 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c5d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c5d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56812/0 (socket says 172.21.15.153:56812) 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c5d9b640 1 -- 172.21.15.153:0/32802269 learned_addr learned my addr 172.21.15.153:0/32802269 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c6d9d640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 0x7f27c819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c659c640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8102a40 0x7f27c8198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c5d9b640 1 -- 172.21.15.153:0/32802269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8102a40 msgr2=0x7f27c8198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.728+0000 7f27c5d9b640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8102a40 0x7f27c8198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27c5d9b640 1 -- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 msgr2=0x7f27c819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27c5d9b640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 0x7f27c819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27c5d9b640 1 -- 172.21.15.153:0/32802269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c819a970 con 0x7f27c8103ba0 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27c6d9d640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 0x7f27c819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27c5d9b640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e9c0 tx=0x7f27bc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00cde0 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27bc0035f0 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c819ac60 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc010690 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.729+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a1340 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.730+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c8109dc0 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.731+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f27bc004000 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.732+0000 7f27b77fe640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 0x7f27b00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.732+0000 7f27c659c640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 0x7f27b00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.732+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f27bc0e4c20 con 0x7f27c8103ba0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.732+0000 7f27c659c640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 0x7f27b00b77a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27b8000c00 tx=0x7f27b8030480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.735+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f27bc0aceb0 con 0x7f27c8103ba0 2024-09-10T22:46:19.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:19.919+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f27c8063590 con 0x7f27b00b52e0 2024-09-10T22:46:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] boot 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:46:20.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] boot 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:46:20.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: osd.3 [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] boot 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:46:20.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi120 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:21.234 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:21.235 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:21 smithi153 ceph-mon[34951]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:46:21.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi120 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:21.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:21 smithi120 ceph-mon[31247]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi120 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 8544M 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:21.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:21 smithi176 ceph-mon[34982]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:46:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:46:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/3163751647' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"}]': finished 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:23 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/2817913732' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/3163751647' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"}]': finished 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/2817913732' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/3163751647' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"} : dispatch 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5b35613-0bf7-4f22-8640-e73ac586d2a8"}]': finished 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:23 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/2817913732' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:24 smithi120 ceph-mon[31247]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:24 smithi153 ceph-mon[34951]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:24 smithi176 ceph-mon[34982]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:26 smithi120 ceph-mon[31247]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:46:26.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:26 smithi153 ceph-mon[34951]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:46:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:26 smithi176 ceph-mon[34982]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:46:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:28 smithi120 ceph-mon[31247]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T22:46:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:28 smithi153 ceph-mon[34951]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T22:46:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:28 smithi176 ceph-mon[34982]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T22:46:30.245 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:30 smithi153 ceph-mon[34951]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:46:30.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:30 smithi120 ceph-mon[31247]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:46:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:30 smithi176 ceph-mon[34982]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:46:31.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:46:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:31.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:31 smithi120 ceph-mon[31247]: Deploying daemon osd.4 on smithi153 2024-09-10T22:46:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:46:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:31 smithi153 ceph-mon[34951]: Deploying daemon osd.4 on smithi153 2024-09-10T22:46:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:46:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:31 smithi176 ceph-mon[34982]: Deploying daemon osd.4 on smithi153 2024-09-10T22:46:32.298 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:32 smithi153 ceph-mon[34951]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:46:32.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:32 smithi120 ceph-mon[31247]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:46:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:32 smithi176 ceph-mon[34982]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:46:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:34 smithi120 ceph-mon[31247]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:34.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:34 smithi153 ceph-mon[34951]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:34 smithi176 ceph-mon[34982]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:36.202 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:36.203 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.203 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.203 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:36 smithi153 ceph-mon[34951]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:36 smithi120 ceph-mon[31247]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:36.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:36 smithi176 ceph-mon[34982]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:46:38.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:38 smithi153 ceph-mon[34951]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:38 smithi120 ceph-mon[31247]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:38 smithi176 ceph-mon[34982]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:39.824 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:39.825 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:39.825 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:39.825 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:39.825 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:39.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.863+0000 7f27b77fe640 1 -- 172.21.15.153:0/32802269 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f27c8063590 con 0x7f27b00b52e0 2024-09-10T22:46:39.866 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 4 on host 'smithi153' 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 msgr2=0x7f27b00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 0x7f27b00b77a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27b8000c00 tx=0x7f27b8030480 comp rx=0 tx=0).stop 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 msgr2=0x7f27c8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e9c0 tx=0x7f27bc00ee80 comp rx=0 tx=0).stop 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 shutdown_connections 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b52e0 0x7f27b00b77a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c8104560 0x7f27c819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c8103ba0 0x7f27c8199490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 --2- 172.21.15.153:0/32802269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c8102a40 0x7f27c8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 >> 172.21.15.153:0/32802269 conn(0x7f27c80fe2f0 msgr2=0x7f27c8106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:39.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 shutdown_connections 2024-09-10T22:46:39.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:46:39.866+0000 7f27cc809640 1 -- 172.21.15.153:0/32802269 wait complete. 2024-09-10T22:46:39.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:39.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:39.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:39.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:39.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:40.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.507 DEBUG:teuthology.orchestra.run.smithi153:osd.4> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T22:46:40.509 INFO:tasks.cephadm:Deploying osd.5 on smithi176 with /dev/vg_nvme/lv_3... 2024-09-10T22:46:40.509 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:46:40.737 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:40.737 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.737 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.737 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:40 smithi153 ceph-mon[34951]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:40.768 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:46:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:40 smithi176 ceph-mon[34982]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:40.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:40.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:40.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:40 smithi120 ceph-mon[31247]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:42.389 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:42 smithi153 ceph-mon[34951]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:42 smithi120 ceph-mon[31247]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:42.485 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:42 smithi176 ceph-mon[34982]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:42.699 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 22:46:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T22:46:42.385+0000 7f7f11ab8640 -1 osd.4 0 log_to_monitors true 2024-09-10T22:46:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:46:43.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:43 smithi153 ceph-mon[34951]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:43 smithi153 ceph-mon[34951]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.224 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2024-09-10T22:46:43.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:43 smithi120 ceph-mon[31247]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:43 smithi120 ceph-mon[31247]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.476 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:46:43.492 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:43 smithi176 ceph-mon[34982]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.492 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:43 smithi176 ceph-mon[34982]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:46:43.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 -- 172.21.15.176:0/2037263240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 msgr2=0x7ff040109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 --2- 172.21.15.176:0/2037263240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff040109db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff038009d00 tx=0x7ff03802a910 comp rx=0 tx=0).stop 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 -- 172.21.15.176:0/2037263240 shutdown_connections 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 --2- 172.21.15.176:0/2037263240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff040109db0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 --2- 172.21.15.176:0/2037263240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 0x7ff040102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 --2- 172.21.15.176:0/2037263240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 -- 172.21.15.176:0/2037263240 >> 172.21.15.176:0/2037263240 conn(0x7ff0400fd1e0 msgr2=0x7ff0400ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 -- 172.21.15.176:0/2037263240 shutdown_connections 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.937+0000 7ff046449640 1 -- 172.21.15.176:0/2037263240 wait complete. 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 Processor -- start 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 -- start start 2024-09-10T22:46:43.943 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 0x7ff040199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff04010d570 con 0x7ff040102a70 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff04010d3f0 con 0x7ff040101910 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff046449640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff04010d6f0 con 0x7ff040103430 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff0449e6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff0449e6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52530/0 (socket says 172.21.15.176:52530) 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff0449e6640 1 -- 172.21.15.176:0/644769271 learned_addr learned my addr 172.21.15.176:0/644769271 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.938+0000 7ff03ffff640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff03f7fe640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 0x7ff040199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 -- 172.21.15.176:0/644769271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 msgr2=0x7ff040198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 -- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 msgr2=0x7ff040199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 0x7ff040199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 -- 172.21.15.176:0/644769271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04019a960 con 0x7ff040103430 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff03ffff640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:46:43.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff0449e6640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098d0 tx=0x7ff0380317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038003cb0 con 0x7ff040103430 2024-09-10T22:46:43.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff038003e50 con 0x7ff040103430 2024-09-10T22:46:43.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04019abf0 con 0x7ff040103430 2024-09-10T22:46:43.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03802fbb0 con 0x7ff040103430 2024-09-10T22:46:43.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.939+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401a1330 con 0x7ff040103430 2024-09-10T22:46:43.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.941+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0401a1560 con 0x7ff040103430 2024-09-10T22:46:43.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.942+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff03802fd50 con 0x7ff040103430 2024-09-10T22:46:43.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.943+0000 7ff03d7fa640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 0x7ff0280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.943+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3805+0+0 (secure 0 0 0) 0x7ff038105b60 con 0x7ff040103430 2024-09-10T22:46:43.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.943+0000 7ff03ffff640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 0x7ff0280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.943+0000 7ff03ffff640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 0x7ff0280b7820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff030009fc0 tx=0x7ff030005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:43.946+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff0380cdc30 con 0x7ff040103430 2024-09-10T22:46:44.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:46:44.147+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff0401a1730 con 0x7ff0280b5360 2024-09-10T22:46:44.502 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi153 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.503 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:44.504 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:44 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi153 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:44.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:44 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:44.699 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T22:46:44.203+0000 7f7f0da3b640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T22:46:44.700 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T22:46:44.213+0000 7f7f09862640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi153 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 8544M 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:44.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:46:44.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:44.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:46:44.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:46:44.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:44 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:45 smithi176 ceph-mon[34982]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:45 smithi176 ceph-mon[34982]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:46:45.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:45 smithi176 ceph-mon[34982]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:46:45.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:45.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:45 smithi120 ceph-mon[31247]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:45 smithi120 ceph-mon[31247]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:46:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:45 smithi120 ceph-mon[31247]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:46:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:45 smithi153 ceph-mon[34951]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:45 smithi153 ceph-mon[34951]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:46:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:45 smithi153 ceph-mon[34951]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:46:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:46:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:46:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] boot 2024-09-10T22:46:46.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:46:46.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:46 smithi176 ceph-mon[34982]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] boot 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:46 smithi120 ceph-mon[31247]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: osd.4 [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] boot 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:46:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:46 smithi153 ceph-mon[34951]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:46:47.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:46:47.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/2413562469' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"}]': finished 2024-09-10T22:46:47.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:46:47.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:46:47.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:47 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/1514018600' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/2413562469' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"}]': finished 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:46:47.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:46:47.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:47 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/1514018600' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/2413562469' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"} : dispatch 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3110d3e-1def-43cf-afae-30ca5fb29647"}]': finished 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:46:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:47 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/1514018600' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:46:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:48 smithi176 ceph-mon[34982]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:48.598 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:48 smithi120 ceph-mon[31247]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:48.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:48 smithi153 ceph-mon[34951]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:50 smithi120 ceph-mon[31247]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:50.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:50 smithi153 ceph-mon[34951]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:50 smithi176 ceph-mon[34982]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:52.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:52 smithi120 ceph-mon[31247]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:52 smithi153 ceph-mon[34951]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:52.458 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:52 smithi176 ceph-mon[34982]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:54.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:54 smithi120 ceph-mon[31247]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:54.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:54 smithi153 ceph-mon[34951]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:54.478 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:54 smithi176 ceph-mon[34982]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:55.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:46:55.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:55.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:55 smithi120 ceph-mon[31247]: Deploying daemon osd.5 on smithi176 2024-09-10T22:46:55.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:46:55.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:55.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:55 smithi153 ceph-mon[34951]: Deploying daemon osd.5 on smithi176 2024-09-10T22:46:55.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:46:55.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:55.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:55 smithi176 ceph-mon[34982]: Deploying daemon osd.5 on smithi176 2024-09-10T22:46:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:56 smithi120 ceph-mon[31247]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:56 smithi153 ceph-mon[34951]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:56.492 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:56 smithi176 ceph-mon[34982]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:58.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:46:58 smithi176 ceph-mon[34982]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:46:58 smithi120 ceph-mon[31247]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:46:58.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:46:58 smithi153 ceph-mon[34951]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:00.398 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:00 smithi176 ceph-mon[34982]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:00 smithi120 ceph-mon[31247]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:00.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:00 smithi153 ceph-mon[34951]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:01.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:02 smithi176 ceph-mon[34982]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:02 smithi120 ceph-mon[31247]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:02.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:02 smithi153 ceph-mon[34951]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:05.083 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.084 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.084 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:05.084 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:05.084 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.084 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:04 smithi176 ceph-mon[34982]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:05.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.124+0000 7ff03d7fa640 1 -- 172.21.15.176:0/644769271 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0401a1730 con 0x7ff0280b5360 2024-09-10T22:47:05.127 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 5 on host 'smithi176' 2024-09-10T22:47:05.129 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 msgr2=0x7ff0280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.129 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 0x7ff0280b7820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff030009fc0 tx=0x7ff030005f90 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 msgr2=0x7ff04019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098d0 tx=0x7ff0380317d0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 shutdown_connections 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0280b5360 0x7ff0280b7820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff040103430 0x7ff04019a0c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff040102a70 0x7ff040199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 --2- 172.21.15.176:0/644769271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff040101910 0x7ff040198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.126+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 >> 172.21.15.176:0/644769271 conn(0x7ff0400fd1e0 msgr2=0x7ff0400fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.127+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 shutdown_connections 2024-09-10T22:47:05.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:05.127+0000 7ff046449640 1 -- 172.21.15.176:0/644769271 wait complete. 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:04 smithi120 ceph-mon[31247]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:05.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:04 smithi153 ceph-mon[34951]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:05.798 DEBUG:teuthology.orchestra.run.smithi176:osd.5> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T22:47:05.801 INFO:tasks.cephadm:Deploying osd.6 on smithi120 with /dev/vg_nvme/lv_2... 2024-09-10T22:47:05.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:47:06.060 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:06.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:06.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:05 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:06.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.181 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:05 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:05 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:06.823 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 22:47:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T22:47:06.559+0000 7f4ad17fc640 -1 osd.5 0 log_to_monitors true 2024-09-10T22:47:07.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:06 smithi120 ceph-mon[31247]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:07.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:06 smithi120 ceph-mon[31247]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:06 smithi120 ceph-mon[31247]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:06 smithi153 ceph-mon[34951]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:06 smithi153 ceph-mon[34951]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:06 smithi153 ceph-mon[34951]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.205 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:06 smithi176 ceph-mon[34982]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:07.205 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:06 smithi176 ceph-mon[34982]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.205 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:06 smithi176 ceph-mon[34982]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:47:07.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:07 smithi120 ceph-mon[31247]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:47:07.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:07 smithi120 ceph-mon[31247]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:07.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:07 smithi120 ceph-mon[31247]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:47:07.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:07 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:07.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:07 smithi120 ceph-mon[31247]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:07 smithi153 ceph-mon[34951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:47:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:07 smithi153 ceph-mon[34951]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:07 smithi153 ceph-mon[34951]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:47:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:07 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:07 smithi153 ceph-mon[34951]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:08.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T22:47:07.879+0000 7f4acd77f640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T22:47:08.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T22:47:07.890+0000 7f4ac95a6640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:47:08.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-mon[34982]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:47:08.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-mon[34982]: from='osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:08.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-mon[34982]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:47:08.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:08.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:07 smithi176 ceph-mon[34982]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:47:08.890 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:47:08.916 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-09-10T22:47:09.165 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] boot 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:09.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:09.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] boot 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:09.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:47:09.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:47:09.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: osd.5 [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] boot 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:09.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 -- 172.21.15.120:0/3827123876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880103430 msgr2=0x7fb880109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 --2- 172.21.15.120:0/3827123876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880103430 0x7fb880109db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb870009cd0 tx=0x7fb87002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 -- 172.21.15.120:0/3827123876 shutdown_connections 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 --2- 172.21.15.120:0/3827123876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880103430 0x7fb880109db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 --2- 172.21.15.120:0/3827123876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880102a70 0x7fb880102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 --2- 172.21.15.120:0/3827123876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880101910 0x7fb880101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.663+0000 7fb88504c640 1 -- 172.21.15.120:0/3827123876 >> 172.21.15.120:0/3827123876 conn(0x7fb8800fd1e0 msgr2=0x7fb8800ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:09.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.664+0000 7fb88504c640 1 -- 172.21.15.120:0/3827123876 shutdown_connections 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.664+0000 7fb88504c640 1 -- 172.21.15.120:0/3827123876 wait complete. 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.664+0000 7fb88504c640 1 Processor -- start 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.664+0000 7fb88504c640 1 -- start start 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 0x7fb8801993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880103430 0x7fb880199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d640 con 0x7fb880101910 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d4c0 con 0x7fb880102a70 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb88010d7c0 con 0x7fb880103430 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44394/0 (socket says 172.21.15.120:44394) 2024-09-10T22:47:09.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 -- 172.21.15.120:0/201059979 learned_addr learned my addr 172.21.15.120:0/201059979 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87f59e640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880103430 0x7fb880199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87e59c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 0x7fb8801993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 -- 172.21.15.120:0/201059979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880103430 msgr2=0x7fb880199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880103430 0x7fb880199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 -- 172.21.15.120:0/201059979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 msgr2=0x7fb8801993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 0x7fb8801993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 -- 172.21.15.120:0/201059979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019a890 con 0x7fb880101910 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87e59c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 0x7fb8801993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb87ed9d640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b4a0 tx=0x7fb87800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878014070 con 0x7fb880101910 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88019ab80 con 0x7fb880101910 2024-09-10T22:47:09.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb878010ac0 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878017c30 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.665+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a1260 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.666+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8801a14f0 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.668+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fb878010610 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.668+0000 7fb877fff640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 0x7fb86c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.668+0000 7fb87e59c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 0x7fb86c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.669+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4533+0+0 (secure 0 0 0) 0x7fb8780e4670 con 0x7fb880101910 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.669+0000 7fb87e59c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 0x7fb86c0b77a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb86800baa0 tx=0x7fb868005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:09.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.673+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8780ac350 con 0x7fb880101910 2024-09-10T22:47:09.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:09.857+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb88006b1c0 con 0x7fb86c0b52e0 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi176 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:10.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:09 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi176 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:09 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:47:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:47:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi176 2024-09-10T22:47:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 8544M 2024-09-10T22:47:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:47:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:10.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:09 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:10 smithi120 ceph-mon[31247]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:10 smithi120 ceph-mon[31247]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:10 smithi120 ceph-mon[31247]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:47:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:10 smithi153 ceph-mon[34951]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:10 smithi153 ceph-mon[34951]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:10 smithi153 ceph-mon[34951]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:47:11.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:10 smithi176 ceph-mon[34982]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:11.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:10 smithi176 ceph-mon[34982]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:11.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:10 smithi176 ceph-mon[34982]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:47:12.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:12 smithi120 ceph-mon[31247]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:12 smithi153 ceph-mon[34951]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:12 smithi176 ceph-mon[34982]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:13.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"} : dispatch 2024-09-10T22:47:13.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"}]': finished 2024-09-10T22:47:13.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:13 smithi120 ceph-mon[31247]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:47:13.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:13 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:13.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3961358793' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"} : dispatch 2024-09-10T22:47:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"}]': finished 2024-09-10T22:47:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:13 smithi153 ceph-mon[34951]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:47:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:13 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3961358793' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"} : dispatch 2024-09-10T22:47:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4013604164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8062f8c3-d12f-4df5-8ef5-4286a273c161"}]': finished 2024-09-10T22:47:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:13 smithi176 ceph-mon[34982]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:47:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:13 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:13.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3961358793' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:14 smithi176 ceph-mon[34982]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:14 smithi120 ceph-mon[31247]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:14 smithi153 ceph-mon[34951]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:16 smithi120 ceph-mon[31247]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T22:47:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:16 smithi153 ceph-mon[34951]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T22:47:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:16 smithi176 ceph-mon[34982]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T22:47:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:18 smithi120 ceph-mon[31247]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:47:18.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:18 smithi153 ceph-mon[34951]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:47:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:18 smithi176 ceph-mon[34982]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:47:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:20 smithi120 ceph-mon[31247]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:20 smithi153 ceph-mon[34951]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:20 smithi176 ceph-mon[34982]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:21.406 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:47:21.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:21 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:21.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:21 smithi120 ceph-mon[31247]: Deploying daemon osd.6 on smithi120 2024-09-10T22:47:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:21 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:47:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:21 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:21 smithi153 ceph-mon[34951]: Deploying daemon osd.6 on smithi120 2024-09-10T22:47:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:21 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:47:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:21 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:21 smithi176 ceph-mon[34982]: Deploying daemon osd.6 on smithi120 2024-09-10T22:47:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:22 smithi120 ceph-mon[31247]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:22 smithi176 ceph-mon[34982]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:22 smithi153 ceph-mon[34951]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:47:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:24 smithi120 ceph-mon[31247]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T22:47:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:24 smithi153 ceph-mon[34951]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T22:47:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:24 smithi176 ceph-mon[34982]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T22:47:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:26 smithi120 ceph-mon[31247]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:47:26.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:26 smithi153 ceph-mon[34951]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:47:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:26 smithi176 ceph-mon[34982]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:47:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:27.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:27.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:28.328 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:28 smithi120 ceph-mon[31247]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:28 smithi176 ceph-mon[34982]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:28 smithi153 ceph-mon[34951]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:31.011 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.012 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:30 smithi120 ceph-mon[31247]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:31.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.050+0000 7fb877fff640 1 -- 172.21.15.120:0/201059979 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb88006b1c0 con 0x7fb86c0b52e0 2024-09-10T22:47:31.053 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 6 on host 'smithi120' 2024-09-10T22:47:31.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 msgr2=0x7fb86c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 0x7fb86c0b77a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb86800baa0 tx=0x7fb868005d90 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 msgr2=0x7fb880198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b4a0 tx=0x7fb87800b960 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 shutdown_connections 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb86c0b52e0 0x7fb86c0b77a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb880103430 0x7fb880199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb880102a70 0x7fb8801993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 --2- 172.21.15.120:0/201059979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880101910 0x7fb880198d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 >> 172.21.15.120:0/201059979 conn(0x7fb8800fd1e0 msgr2=0x7fb88006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 shutdown_connections 2024-09-10T22:47:31.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:47:31.052+0000 7fb88504c640 1 -- 172.21.15.120:0/201059979 wait complete. 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:30 smithi176 ceph-mon[34982]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:30 smithi153 ceph-mon[34951]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:31.863 DEBUG:teuthology.orchestra.run.smithi120:osd.6> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T22:47:31.866 INFO:tasks.cephadm:Deploying osd.7 on smithi153 with /dev/vg_nvme/lv_2... 2024-09-10T22:47:31.866 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:47:31.987 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:31.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:31.988 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:32.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:32.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:32.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:47:32.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:32.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:32.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:32.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:32 smithi120 ceph-mon[31247]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:32.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:32 smithi120 ceph-mon[31247]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:47:32.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 22:47:32 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T22:47:32.641+0000 7fa8635cc640 -1 osd.6 0 log_to_monitors true 2024-09-10T22:47:32.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:32 smithi153 ceph-mon[34951]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:32.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:32 smithi153 ceph-mon[34951]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:47:33.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:32 smithi176 ceph-mon[34982]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:33.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:32 smithi176 ceph-mon[34982]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:47:33.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:33 smithi153 ceph-mon[34951]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:47:33.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:33 smithi153 ceph-mon[34951]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:47:33.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:33 smithi153 ceph-mon[34951]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:47:33.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:33 smithi176 ceph-mon[34982]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:47:34.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:33 smithi176 ceph-mon[34982]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:47:34.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:33 smithi176 ceph-mon[34982]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:47:34.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:33 smithi120 ceph-mon[31247]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:47:34.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:33 smithi120 ceph-mon[31247]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:47:34.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:33 smithi120 ceph-mon[31247]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:47:34.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.688 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:47:34.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-09-10T22:47:34.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T22:47:34.619+0000 7fa85fd63640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T22:47:34.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T22:47:34.735+0000 7fa85736e640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:47:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:47:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:47:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:34.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' 2024-09-10T22:47:34.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' 2024-09-10T22:47:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:34.972 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:47:35.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:47:35.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:47:35.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:35.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:35.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:47:35.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: from='osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812]' entity='osd.6' 2024-09-10T22:47:35.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:35.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 -- 172.21.15.153:0/3714786638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db006c100 msgr2=0x7f7db0105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/3714786638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db006c100 0x7f7db0105610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009ca0 tx=0x7f7da002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 -- 172.21.15.153:0/3714786638 shutdown_connections 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/3714786638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0108750 0x7f7db010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/3714786638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db0105ba0 0x7f7db0108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/3714786638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db006c100 0x7f7db0105610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 -- 172.21.15.153:0/3714786638 >> 172.21.15.153:0/3714786638 conn(0x7f7db00fd200 msgr2=0x7f7db00ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 -- 172.21.15.153:0/3714786638 shutdown_connections 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.427+0000 7f7db5ca9640 1 -- 172.21.15.153:0/3714786638 wait complete. 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 Processor -- start 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 -- start start 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db006c100 0x7f7db0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0105ba0 0x7f7db0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010e6a0 con 0x7f7db0105ba0 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010e520 con 0x7f7db0108750 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7db5ca9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010e820 con 0x7f7db006c100 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.428+0000 7f7daffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44596/0 (socket says 172.21.15.153:44596) 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 -- 172.21.15.153:0/1308508981 learned_addr learned my addr 172.21.15.153:0/1308508981 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daf7fe640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db006c100 0x7f7db0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daeffd640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0105ba0 0x7f7db0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 -- 172.21.15.153:0/1308508981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db006c100 msgr2=0x7f7db0198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db006c100 0x7f7db0198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 -- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0105ba0 msgr2=0x7f7db0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0105ba0 0x7f7db0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 -- 172.21.15.153:0/1308508981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db019a960 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7daffff640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c970 tx=0x7f7da800cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800ae60 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da80054d0 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db019ac50 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004d60 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.429+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01a1330 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.430+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db00fecf0 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.432+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f7da8004f20 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.433+0000 7f7dacff9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 0x7f7d980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.433+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f7da80e4ea0 con 0x7f7db0108750 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.433+0000 7f7daf7fe640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 0x7f7d980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.434+0000 7f7daf7fe640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 0x7f7d980b77a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000f80 tx=0x7f7da0005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.437+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7da8014030 con 0x7f7db0108750 2024-09-10T22:47:35.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:35.621+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7db0063590 con 0x7f7d980b52e0 2024-09-10T22:47:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:47:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:47:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi120 2024-09-10T22:47:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] boot 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:36.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:36.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi120 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T22:47:36.347 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] boot 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:36.348 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi120 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 5696M 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: osd.6 [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] boot 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:47:36.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:37.287 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:37 smithi153 ceph-mon[34951]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:37.287 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:37 smithi153 ceph-mon[34951]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:37.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:37 smithi176 ceph-mon[34982]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:37.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:37 smithi176 ceph-mon[34982]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:37 smithi120 ceph-mon[31247]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:37 smithi120 ceph-mon[31247]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/1153974137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"}]': finished 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:47:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:47:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/1153974137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"}]': finished 2024-09-10T22:47:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:47:38.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/1153974137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"} : dispatch 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44778377-474e-4621-bd97-1cb74a97869f"}]': finished 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:47:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:39 smithi176 ceph-mon[34982]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:39 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/511627055' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:39.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:39 smithi120 ceph-mon[31247]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:39 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/511627055' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:39 smithi153 ceph-mon[34951]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:39 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/511627055' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:47:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:40 smithi120 ceph-mon[31247]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:40 smithi153 ceph-mon[34951]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:40 smithi176 ceph-mon[34982]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:42 smithi120 ceph-mon[31247]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:42 smithi153 ceph-mon[34951]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:42 smithi176 ceph-mon[34982]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:44.398 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:44 smithi153 ceph-mon[34951]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:44.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:44 smithi120 ceph-mon[31247]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:44 smithi176 ceph-mon[34982]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:47:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:47:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:47:46.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:47.311 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:46 smithi153 ceph-mon[34951]: Deploying daemon osd.7 on smithi153 2024-09-10T22:47:47.311 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:46 smithi153 ceph-mon[34951]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:47.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:46 smithi176 ceph-mon[34982]: Deploying daemon osd.7 on smithi153 2024-09-10T22:47:47.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:46 smithi176 ceph-mon[34982]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:47.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:46 smithi120 ceph-mon[31247]: Deploying daemon osd.7 on smithi153 2024-09-10T22:47:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:46 smithi120 ceph-mon[31247]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:48 smithi120 ceph-mon[31247]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:48 smithi153 ceph-mon[34951]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:48 smithi176 ceph-mon[34982]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:50.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:50 smithi120 ceph-mon[31247]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:50.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:50 smithi153 ceph-mon[34951]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:50 smithi176 ceph-mon[34982]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:52.196 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:52.196 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:52.196 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:52.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:53.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:52 smithi176 ceph-mon[34982]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:53.110 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:52 smithi153 ceph-mon[34951]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:53.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:52 smithi120 ceph-mon[31247]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:54.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:54 smithi153 ceph-mon[34951]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:54 smithi176 ceph-mon[34982]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:54 smithi120 ceph-mon[31247]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:55.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.267+0000 7f7dacff9640 1 -- 172.21.15.153:0/1308508981 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7db0063590 con 0x7f7d980b52e0 2024-09-10T22:47:55.269 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 7 on host 'smithi153' 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 msgr2=0x7f7d980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 0x7f7d980b77a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000f80 tx=0x7f7da0005f30 comp rx=0 tx=0).stop 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 msgr2=0x7f7db019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c970 tx=0x7f7da800cd70 comp rx=0 tx=0).stop 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 shutdown_connections 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d980b52e0 0x7f7d980b77a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0108750 0x7f7db019a0c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:55.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7db0105ba0 0x7f7db0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:55.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 --2- 172.21.15.153:0/1308508981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7db006c100 0x7f7db0198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:55.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.269+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 >> 172.21.15.153:0/1308508981 conn(0x7f7db00fd200 msgr2=0x7f7db0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:55.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.270+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 shutdown_connections 2024-09-10T22:47:55.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:47:55.270+0000 7f7db5ca9640 1 -- 172.21.15.153:0/1308508981 wait complete. 2024-09-10T22:47:55.614 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.614 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.614 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:55.614 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:55.614 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.615 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:55.615 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.615 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:55.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:55.934 DEBUG:teuthology.orchestra.run.smithi153:osd.7> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T22:47:55.936 INFO:tasks.cephadm:Deploying osd.8 on smithi176 with /dev/vg_nvme/lv_2... 2024-09-10T22:47:55.936 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:47:56.189 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:47:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:56 smithi176 ceph-mon[34982]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:56.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:56 smithi120 ceph-mon[31247]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:56.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:56 smithi153 ceph-mon[34951]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:58.066 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:57 smithi153 ceph-mon[34951]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.067 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:57 smithi153 ceph-mon[34951]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.067 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 22:47:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T22:47:57.764+0000 7fe14659b640 -1 osd.7 0 log_to_monitors true 2024-09-10T22:47:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:57 smithi120 ceph-mon[31247]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:57 smithi120 ceph-mon[31247]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:57 smithi176 ceph-mon[34982]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:57 smithi176 ceph-mon[34982]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:47:58.627 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:47:58.655 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2024-09-10T22:47:58.903 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:47:58.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T22:47:58.834+0000 7fe14251e640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T22:47:58.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T22:47:58.843+0000 7fe13e345640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:47:58.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:58 smithi153 ceph-mon[34951]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:47:59.000 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:47:59.000 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:47:59.000 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.001 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:58 smithi176 ceph-mon[34982]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:47:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:58 smithi120 ceph-mon[31247]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:47:59.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 -- 172.21.15.176:0/4235391862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 msgr2=0x7f32f4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/4235391862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4107fe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802a410 comp rx=0 tx=0).stop 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 -- 172.21.15.176:0/4235391862 shutdown_connections 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/4235391862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/4235391862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4107fe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/4235391862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 -- 172.21.15.176:0/4235391862 >> 172.21.15.176:0/4235391862 conn(0x7f32f40fd1e0 msgr2=0x7f32f40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.309+0000 7f32fbb3f640 1 -- 172.21.15.176:0/4235391862 shutdown_connections 2024-09-10T22:47:59.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.310+0000 7f32fbb3f640 1 -- 172.21.15.176:0/4235391862 wait complete. 2024-09-10T22:47:59.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 Processor -- start 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 -- start start 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32f410e670 con 0x7f32f4108720 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32f410e4f0 con 0x7f32f406b640 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fbb3f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f32f410e7f0 con 0x7f32f4105bd0 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49152/0 (socket says 172.21.15.176:49152) 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 -- 172.21.15.176:0/2618844409 learned_addr learned my addr 172.21.15.176:0/2618844409 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-10T22:47:59.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f98db640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32fa0dc640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 -- 172.21.15.176:0/2618844409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 msgr2=0x7f32f4198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 -- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 msgr2=0x7f32f419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f90da640 1 -- 172.21.15.176:0/2618844409 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f419a920 con 0x7f32f4105bd0 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.311+0000 7f32f98db640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32fa0dc640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32f90da640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005940 tx=0x7f32e8031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e803f030 con 0x7f32f4105bd0 2024-09-10T22:47:59.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f419abb0 con 0x7f32f4105bd0 2024-09-10T22:47:59.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e8003aa0 con 0x7f32f4105bd0 2024-09-10T22:47:59.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8035070 con 0x7f32f4105bd0 2024-09-10T22:47:59.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.312+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41a12f0 con 0x7f32f4105bd0 2024-09-10T22:47:59.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.313+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f40fecd0 con 0x7f32f4105bd0 2024-09-10T22:47:59.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.314+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f32e803bbb0 con 0x7f32f4105bd0 2024-09-10T22:47:59.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.315+0000 7f32f2ffd640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 0x7f32dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.315+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7f32e8105930 con 0x7f32f4105bd0 2024-09-10T22:47:59.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.315+0000 7f32f98db640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 0x7f32dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.316+0000 7f32f98db640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 0x7f32dc0b77a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:59.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.319+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f32e80cd470 con 0x7f32f4105bd0 2024-09-10T22:47:59.505 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:47:59.504+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f32f4063590 con 0x7f32dc0b52e0 2024-09-10T22:48:00.096 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi153 2024-09-10T22:48:00.096 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] boot 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:48:00.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:47:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.121 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi153 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] boot 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:48:00.122 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:47:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi153 2024-09-10T22:48:00.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 5696M 2024-09-10T22:48:00.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:48:00.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: osd.7 [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] boot 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:48:00.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:47:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:48:00.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:00 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:48:00.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:00 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:48:00.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:00 smithi120 ceph-mon[31247]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:00.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:00 smithi120 ceph-mon[31247]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:00.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:00 smithi120 ceph-mon[31247]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:48:01.040 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:00 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:48:01.040 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:00 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:48:01.040 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:00 smithi176 ceph-mon[34982]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:01.041 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:00 smithi176 ceph-mon[34982]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:01.041 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:00 smithi176 ceph-mon[34982]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:48:01.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:00 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:48:01.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:00 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:48:01.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:00 smithi153 ceph-mon[34951]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:01.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:00 smithi153 ceph-mon[34951]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:01.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:00 smithi153 ceph-mon[34951]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:48:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:01 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/3076646917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:01 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:01 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"}]': finished 2024-09-10T22:48:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:01 smithi120 ceph-mon[31247]: osdmap e48: 9 total, 8 up, 9 in 2024-09-10T22:48:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:02.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:01 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/3076646917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:01 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:01 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"}]': finished 2024-09-10T22:48:02.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:01 smithi153 ceph-mon[34951]: osdmap e48: 9 total, 8 up, 9 in 2024-09-10T22:48:02.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:02.321 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:01 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/3076646917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.321 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:01 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"} : dispatch 2024-09-10T22:48:02.321 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:01 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9699f84b-810d-46f9-b062-ea48812dafea"}]': finished 2024-09-10T22:48:02.321 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:01 smithi176 ceph-mon[34982]: osdmap e48: 9 total, 8 up, 9 in 2024-09-10T22:48:02.321 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:02 smithi120 ceph-mon[31247]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:02 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/1533435653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:02 smithi120 ceph-mon[31247]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:48:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:02 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:03.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:02 smithi153 ceph-mon[34951]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:03.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:02 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/1533435653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:03.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:02 smithi153 ceph-mon[34951]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:48:03.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:02 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:02 smithi176 ceph-mon[34982]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:02 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/1533435653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:02 smithi176 ceph-mon[34982]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:48:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:02 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:04 smithi153 ceph-mon[34951]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:04 smithi176 ceph-mon[34982]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:04.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:04 smithi120 ceph-mon[31247]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:06.565 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:06 smithi176 ceph-mon[34982]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:48:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:06 smithi120 ceph-mon[31247]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:48:06.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:06 smithi153 ceph-mon[34951]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-09-10T22:48:08.319 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:08 smithi176 ceph-mon[34982]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-10T22:48:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:08 smithi153 ceph-mon[34951]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-10T22:48:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:08 smithi120 ceph-mon[31247]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-10T22:48:10.460 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:10 smithi120 ceph-mon[31247]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-10T22:48:10.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:48:10.461 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:10.546 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:10 smithi176 ceph-mon[34982]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-10T22:48:10.546 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:48:10.546 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:10.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:10 smithi153 ceph-mon[34951]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-10T22:48:10.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:48:10.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:11.509 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:11 smithi176 ceph-mon[34982]: Deploying daemon osd.8 on smithi176 2024-09-10T22:48:11.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:11 smithi120 ceph-mon[31247]: Deploying daemon osd.8 on smithi176 2024-09-10T22:48:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:11 smithi153 ceph-mon[34951]: Deploying daemon osd.8 on smithi176 2024-09-10T22:48:12.548 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:12 smithi176 ceph-mon[34982]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:48:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:12 smithi120 ceph-mon[31247]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:48:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:12 smithi153 ceph-mon[34951]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T22:48:14.490 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:14 smithi176 ceph-mon[34982]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T22:48:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:14 smithi120 ceph-mon[31247]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T22:48:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:14 smithi153 ceph-mon[34951]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T22:48:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:16.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:16.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:17.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:16 smithi176 ceph-mon[34982]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:48:17.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:16 smithi120 ceph-mon[31247]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:48:17.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:16 smithi153 ceph-mon[34951]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:48:18.457 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:18 smithi176 ceph-mon[34982]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:18.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:18 smithi120 ceph-mon[31247]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:18.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:18 smithi153 ceph-mon[34951]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:19.774 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:19.775 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:19.775 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:19.775 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:19.775 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:19 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:19.819 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 8 on host 'smithi176' 2024-09-10T22:48:19.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.816+0000 7f32f2ffd640 1 -- 172.21.15.176:0/2618844409 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32f4063590 con 0x7f32dc0b52e0 2024-09-10T22:48:19.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 msgr2=0x7f32dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:19.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 0x7f32dc0b77a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4009500 comp rx=0 tx=0).stop 2024-09-10T22:48:19.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 msgr2=0x7f32f4199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:19.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005940 tx=0x7f32e8031a50 comp rx=0 tx=0).stop 2024-09-10T22:48:19.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 shutdown_connections 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32dc0b52e0 0x7f32dc0b77a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f4108720 0x7f32f419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4105bd0 0x7f32f4199440 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 --2- 172.21.15.176:0/2618844409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f406b640 0x7f32f4198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 >> 172.21.15.176:0/2618844409 conn(0x7f32f40fd1e0 msgr2=0x7f32f4103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 shutdown_connections 2024-09-10T22:48:19.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:48:19.819+0000 7f32fbb3f640 1 -- 172.21.15.176:0/2618844409 wait complete. 2024-09-10T22:48:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:20.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:19 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:20.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:19 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.505 DEBUG:teuthology.orchestra.run.smithi176:osd.8> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T22:48:20.508 INFO:tasks.cephadm:Deploying osd.9 on smithi120 with /dev/vg_nvme/lv_1... 2024-09-10T22:48:20.508 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:48:20.767 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:20.887 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:20.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.888 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:20 smithi120 ceph-mon[31247]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:20.998 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:20.998 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.998 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:20.998 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:20 smithi176 ceph-mon[34982]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:21.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:21.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:21.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:21.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:20 smithi153 ceph-mon[34951]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:22 smithi120 ceph-mon[31247]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:22 smithi120 ceph-mon[31247]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:22 smithi176 ceph-mon[34982]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:22 smithi176 ceph-mon[34982]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:22.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 22:48:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T22:48:21.996+0000 7f27f4e53640 -1 osd.8 0 log_to_monitors true 2024-09-10T22:48:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:22 smithi153 ceph-mon[34951]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:22 smithi153 ceph-mon[34951]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:23 smithi120 ceph-mon[31247]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:48:23.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:48:23.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.283 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:48:23.283 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:23.283 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.283 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:23.283 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-mon[34982]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: from='osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:23 smithi153 ceph-mon[34951]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:48:23.716 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:48:23.741 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-09-10T22:48:23.986 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:24.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 22:48:23 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T22:48:23.954+0000 7f27f0dd6640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T22:48:24.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T22:48:24.070+0000 7f27e8bf5640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='osd.8 ' entity='osd.8' 2024-09-10T22:48:24.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:24.358 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.359 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='osd.8 ' entity='osd.8' 2024-09-10T22:48:24.359 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='osd.8 ' entity='osd.8' 2024-09-10T22:48:24.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 -- 172.21.15.120:0/686450679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768102aa0 msgr2=0x7f0768102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 --2- 172.21.15.120:0/686450679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768102aa0 0x7f0768102f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0758009930 tx=0x7f075802a420 comp rx=0 tx=0).stop 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 -- 172.21.15.120:0/686450679 shutdown_connections 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 --2- 172.21.15.120:0/686450679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768103460 0x7f0768109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 --2- 172.21.15.120:0/686450679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768102aa0 0x7f0768102f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.438+0000 7f076fd75640 1 --2- 172.21.15.120:0/686450679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768101940 0x7f0768101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:24.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- 172.21.15.120:0/686450679 >> 172.21.15.120:0/686450679 conn(0x7f07680fd1f0 msgr2=0x7f07680ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- 172.21.15.120:0/686450679 shutdown_connections 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- 172.21.15.120:0/686450679 wait complete. 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 Processor -- start 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- start start 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 0x7f0768199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 0x7f076819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f076810e6f0 con 0x7f0768101940 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f076810e570 con 0x7f0768103460 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076fd75640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f076810e870 con 0x7f0768102aa0 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076db11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:24.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076db11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35408/0 (socket says 172.21.15.120:35408) 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.439+0000 7f076db11640 1 -- 172.21.15.120:0/303154058 learned_addr learned my addr 172.21.15.120:0/303154058 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076e312640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 0x7f076819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 -- 172.21.15.120:0/303154058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 msgr2=0x7f0768199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076d310640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 0x7f0768199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 0x7f0768199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 -- 172.21.15.120:0/303154058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 msgr2=0x7f076819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 0x7f076819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 -- 172.21.15.120:0/303154058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076819a8e0 con 0x7f0768101940 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076e312640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 0x7f076819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076d310640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 0x7f0768199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076db11640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f076400cce0 tx=0x7f076400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400f650 con 0x7f0768101940 2024-09-10T22:48:24.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076819abd0 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076400fca0 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764018b40 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.440+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07681a12b0 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.441+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0768108c60 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.442+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f076400f7f0 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.443+0000 7f075effd640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 0x7f07540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.443+0000 7f076d310640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 0x7f07540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.444+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5862+0+0 (secure 0 0 0) 0x7f07640e57f0 con 0x7f0768101940 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.444+0000 7f076d310640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 0x7f07540b77a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07580096c0 tx=0x7f0758000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:24.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.448+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0764014030 con 0x7f0768101940 2024-09-10T22:48:24.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:24.633+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0768063590 con 0x7f07540b52e0 2024-09-10T22:48:25.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:48:25.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:48:25.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi176 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] boot 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:25.303 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi176 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] boot 2024-09-10T22:48:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:48:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi176 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 5696M 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: osd.8 [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] boot 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:25.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:26.340 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:26 smithi120 ceph-mon[31247]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:26.341 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:26 smithi120 ceph-mon[31247]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:48:26.350 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:26 smithi153 ceph-mon[34951]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:26.350 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:26 smithi153 ceph-mon[34951]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:48:26.474 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:26 smithi176 ceph-mon[34982]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:26.474 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:26 smithi176 ceph-mon[34982]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:48:27.406 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:27 smithi120 ceph-mon[31247]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:27.406 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:27 smithi120 ceph-mon[31247]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T22:48:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:27 smithi153 ceph-mon[34951]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:27 smithi153 ceph-mon[34951]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T22:48:27.487 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:27 smithi176 ceph-mon[34982]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:27.487 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:27 smithi176 ceph-mon[34982]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T22:48:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"} : dispatch 2024-09-10T22:48:28.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"}]': finished 2024-09-10T22:48:28.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:28 smithi120 ceph-mon[31247]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:48:28.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:28 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:28.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2382709052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"} : dispatch 2024-09-10T22:48:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"}]': finished 2024-09-10T22:48:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:28 smithi153 ceph-mon[34951]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:48:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:28 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2382709052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"} : dispatch 2024-09-10T22:48:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/947286690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f143c9a-4955-4d2d-9670-42a73378a865"}]': finished 2024-09-10T22:48:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:28 smithi176 ceph-mon[34982]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:48:28.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:28 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:28.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2382709052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:29.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:29 smithi120 ceph-mon[31247]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:29 smithi153 ceph-mon[34951]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:29 smithi176 ceph-mon[34982]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:31.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:31 smithi120 ceph-mon[31247]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:48:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:31 smithi153 ceph-mon[34951]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:48:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:31 smithi176 ceph-mon[34982]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T22:48:33.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:33 smithi120 ceph-mon[31247]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:48:33.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:33 smithi153 ceph-mon[34951]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:48:33.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:33 smithi176 ceph-mon[34982]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T22:48:35.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:35 smithi120 ceph-mon[31247]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:48:35.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:35 smithi153 ceph-mon[34951]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:48:35.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:35 smithi176 ceph-mon[34982]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:48:36.400 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:48:36.400 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:48:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:48:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:37.289 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:37 smithi120 ceph-mon[31247]: Deploying daemon osd.9 on smithi120 2024-09-10T22:48:37.289 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:37 smithi120 ceph-mon[31247]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:48:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:37 smithi153 ceph-mon[34951]: Deploying daemon osd.9 on smithi120 2024-09-10T22:48:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:37 smithi153 ceph-mon[34951]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:48:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:37 smithi176 ceph-mon[34982]: Deploying daemon osd.9 on smithi120 2024-09-10T22:48:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:37 smithi176 ceph-mon[34982]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T22:48:39.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:39 smithi120 ceph-mon[31247]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-10T22:48:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:39 smithi153 ceph-mon[34951]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-10T22:48:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:39 smithi176 ceph-mon[34982]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-10T22:48:41.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:41 smithi120 ceph-mon[31247]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:48:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:41 smithi153 ceph-mon[34951]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:48:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:41 smithi176 ceph-mon[34982]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:48:42.431 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:42 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:42.431 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:42 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.431 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:42 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.431 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:42 smithi120 ceph-mon[31247]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:42 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:42 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:42 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:42 smithi153 ceph-mon[34951]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:42 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:42 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:42 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:42 smithi176 ceph-mon[34982]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:44.312 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:44 smithi120 ceph-mon[31247]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:44 smithi176 ceph-mon[34982]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:44.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:44 smithi153 ceph-mon[34951]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:45.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.425+0000 7f075effd640 1 -- 172.21.15.120:0/303154058 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0768063590 con 0x7f07540b52e0 2024-09-10T22:48:45.426 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 9 on host 'smithi120' 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 msgr2=0x7f07540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 0x7f07540b77a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07580096c0 tx=0x7f0758000bc0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 msgr2=0x7f0768198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f076400cce0 tx=0x7f076400a620 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 shutdown_connections 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07540b52e0 0x7f07540b77a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0768103460 0x7f076819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0768102aa0 0x7f0768199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 --2- 172.21.15.120:0/303154058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0768101940 0x7f0768198db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 >> 172.21.15.120:0/303154058 conn(0x7f07680fd1f0 msgr2=0x7f0768105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 shutdown_connections 2024-09-10T22:48:45.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:48:45.427+0000 7f076fd75640 1 -- 172.21.15.120:0/303154058 wait complete. 2024-09-10T22:48:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.036 DEBUG:teuthology.orchestra.run.smithi120:osd.9> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T22:48:46.039 INFO:tasks.cephadm:Deploying osd.10 on smithi153 with /dev/vg_nvme/lv_1... 2024-09-10T22:48:46.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:48:46.296 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:48:46.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.538 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.539 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:46 smithi120 ceph-mon[31247]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:46.637 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.638 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.638 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.638 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:46 smithi153 ceph-mon[34951]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:46 smithi176 ceph-mon[34982]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:47.609 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:47 smithi153 ceph-mon[34951]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:48:47.615 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:47 smithi120 ceph-mon[31247]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:48:47.615 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 22:48:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T22:48:47.297+0000 7f7fe5778640 -1 osd.9 0 log_to_monitors true 2024-09-10T22:48:47.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:47 smithi176 ceph-mon[34982]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:48:48.632 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:48 smithi120 ceph-mon[31247]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:48:48.633 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:48 smithi120 ceph-mon[31247]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:48:48.633 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:48 smithi120 ceph-mon[31247]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:48:48.633 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:48.633 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:48 smithi120 ceph-mon[31247]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:48.682 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:48 smithi153 ceph-mon[34951]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:48:48.682 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:48 smithi153 ceph-mon[34951]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:48:48.682 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:48 smithi153 ceph-mon[34951]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:48:48.682 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:48.682 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:48 smithi153 ceph-mon[34951]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:48.684 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-09-10T22:48:48.711 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2024-09-10T22:48:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:48 smithi176 ceph-mon[34982]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:48:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:48 smithi176 ceph-mon[34982]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:48:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:48 smithi176 ceph-mon[34982]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]} : dispatch 2024-09-10T22:48:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:48 smithi176 ceph-mon[34982]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:48.963 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:48:49.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 -- 172.21.15.153:0/1977368483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 msgr2=0x7f6028105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 --2- 172.21.15.153:0/1977368483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f6028105e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6018009900 tx=0x7f601802a410 comp rx=0 tx=0).stop 2024-09-10T22:48:49.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 -- 172.21.15.153:0/1977368483 shutdown_connections 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 --2- 172.21.15.153:0/1977368483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 0x7f6028106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 --2- 172.21.15.153:0/1977368483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f6028105e40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 --2- 172.21.15.153:0/1977368483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f6028103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 -- 172.21.15.153:0/1977368483 >> 172.21.15.153:0/1977368483 conn(0x7f60280fd200 msgr2=0x7f60280ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 -- 172.21.15.153:0/1977368483 shutdown_connections 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.411+0000 7f602f4b1640 1 -- 172.21.15.153:0/1977368483 wait complete. 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 Processor -- start 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 -- start start 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f602819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 0x7f60281a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:49.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6028114ce0 con 0x7f602806c0c0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6028114b60 con 0x7f60281039b0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602f4b1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6028114e60 con 0x7f6028106560 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602ca4c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f602819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602ca4c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47330/0 (socket says 172.21.15.153:47330) 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602ca4c640 1 -- 172.21.15.153:0/920632933 learned_addr learned my addr 172.21.15.153:0/920632933 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602da4e640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 0x7f60281a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 -- 172.21.15.153:0/920632933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 msgr2=0x7f60281a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 0x7f60281a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 -- 172.21.15.153:0/920632933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 msgr2=0x7f602819fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.412+0000 7f602d24d640 1 -- 172.21.15.153:0/920632933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a0f90 con 0x7f602806c0c0 2024-09-10T22:48:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f602ca4c640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f602d24d640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f602819f460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9f0 tx=0x7f602400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400ccc0 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60240108e0 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a1280 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024018880 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.413+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a7960 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.414+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60280fecf0 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.415+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f6024010430 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.416+0000 7f601e7fc640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 0x7f60140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.416+0000 7f602ca4c640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 0x7f60140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.416+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5981+0+0 (secure 0 0 0) 0x7f60240e5a50 con 0x7f602806c0c0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.416+0000 7f602ca4c640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 0x7f60140b7820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6018032fd0 tx=0x7f60180314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.419+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f60240acb40 con 0x7f602806c0c0 2024-09-10T22:48:49.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:48:49.607+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6028063590 con 0x7f60140b5360 2024-09-10T22:48:49.610 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:48:49.610 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T22:48:49.610 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.610 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi120 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.611 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T22:48:49.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:49.612 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:49.613 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.613 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' 2024-09-10T22:48:49.613 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.697 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T22:48:49.317+0000 7f7fe1f0f640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T22:48:49.697 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T22:48:49.497+0000 7f7fd951a640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:48:49.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:48:49.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T22:48:49.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi120 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.699 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' 2024-09-10T22:48:49.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi120 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 4272M 2024-09-10T22:48:49.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:49.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:49.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:48:49.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809]' entity='osd.9' 2024-09-10T22:48:49.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:48:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:50.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:50.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:50.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:50.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:48:50.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:51.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:51 smithi120 ceph-mon[31247]: osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] boot 2024-09-10T22:48:51.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:51 smithi120 ceph-mon[31247]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:48:51.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:51 smithi176 ceph-mon[34982]: osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] boot 2024-09-10T22:48:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:51 smithi176 ceph-mon[34982]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:48:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:51 smithi153 ceph-mon[34951]: osd.9 [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] boot 2024-09-10T22:48:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:51 smithi153 ceph-mon[34951]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:48:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:48:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T22:48:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/2358598728' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"}]': finished 2024-09-10T22:48:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:48:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:48:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:52 smithi176 ceph-mon[34982]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:52.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/2358598728' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"}]': finished 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:48:52.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:52 smithi120 ceph-mon[31247]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/2358598728' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"} : dispatch 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7801ec3e-95f4-4dcb-9134-5d9dbf72145f"}]': finished 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:48:52.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:52 smithi153 ceph-mon[34951]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:53 smithi176 ceph-mon[34982]: from='client.? 172.21.15.153:0/4235427571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:53 smithi120 ceph-mon[31247]: from='client.? 172.21.15.153:0/4235427571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:53.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:53 smithi153 ceph-mon[34951]: from='client.? 172.21.15.153:0/4235427571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:48:54.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:54 smithi176 ceph-mon[34982]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:54.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:54 smithi120 ceph-mon[31247]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:54.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:54 smithi153 ceph-mon[34951]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:56 smithi176 ceph-mon[34982]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:56.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:56 smithi120 ceph-mon[31247]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:56.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:56 smithi153 ceph-mon[34951]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:58.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:48:58 smithi153 ceph-mon[34951]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:48:58 smithi176 ceph-mon[34982]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:48:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:48:58 smithi120 ceph-mon[31247]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:00.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:00 smithi153 ceph-mon[34951]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:00 smithi176 ceph-mon[34982]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:00 smithi120 ceph-mon[31247]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:01.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:49:01.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:01.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:01 smithi153 ceph-mon[34951]: Deploying daemon osd.10 on smithi153 2024-09-10T22:49:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:49:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:01 smithi176 ceph-mon[34982]: Deploying daemon osd.10 on smithi153 2024-09-10T22:49:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:49:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:01 smithi120 ceph-mon[31247]: Deploying daemon osd.10 on smithi153 2024-09-10T22:49:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:02 smithi153 ceph-mon[34951]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:02 smithi176 ceph-mon[34982]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:02 smithi120 ceph-mon[31247]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:04.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:04 smithi153 ceph-mon[34951]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:04 smithi176 ceph-mon[34982]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:04 smithi120 ceph-mon[31247]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:06.513 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:06 smithi153 ceph-mon[34951]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:06 smithi176 ceph-mon[34982]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:06 smithi120 ceph-mon[31247]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:07.308 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:07 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:07.309 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:07 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:07.309 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:07 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:07 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:07 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:07 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:07 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:07.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:07 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:07.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:07 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:08.306 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:08 smithi153 ceph-mon[34951]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:08 smithi176 ceph-mon[34982]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:08 smithi120 ceph-mon[31247]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:10.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.035+0000 7f601e7fc640 1 -- 172.21.15.153:0/920632933 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6028063590 con 0x7f60140b5360 2024-09-10T22:49:10.038 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 10 on host 'smithi153' 2024-09-10T22:49:10.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 msgr2=0x7f60140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:10.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 0x7f60140b7820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6018032fd0 tx=0x7f60180314a0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 msgr2=0x7f602819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:10.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f602819f460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9f0 tx=0x7f602400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 shutdown_connections 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60140b5360 0x7f60140b7820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6028106560 0x7f60281a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60281039b0 0x7f602819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 --2- 172.21.15.153:0/920632933 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f602806c0c0 0x7f602819f460 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 >> 172.21.15.153:0/920632933 conn(0x7f60280fd200 msgr2=0x7f6028104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 shutdown_connections 2024-09-10T22:49:10.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-09-10T22:49:10.038+0000 7f602f4b1640 1 -- 172.21.15.153:0/920632933 wait complete. 2024-09-10T22:49:10.342 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.342 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.342 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.342 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.342 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.343 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.343 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.343 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.343 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:10 smithi153 ceph-mon[34951]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:10 smithi176 ceph-mon[34982]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:10.670 DEBUG:teuthology.orchestra.run.smithi153:osd.10> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T22:49:10.673 INFO:tasks.cephadm:Deploying osd.11 on smithi176 with /dev/vg_nvme/lv_1... 2024-09-10T22:49:10.673 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:10.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:10 smithi120 ceph-mon[31247]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:10.929 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:49:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:12 smithi153 ceph-mon[34951]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:12.450 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 22:49:12 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T22:49:12.387+0000 7fedc026d640 -1 osd.10 0 log_to_monitors true 2024-09-10T22:49:12.521 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:12 smithi176 ceph-mon[34982]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:12 smithi120 ceph-mon[31247]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:13.480 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:13 smithi176 ceph-mon[34982]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.480 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:13 smithi176 ceph-mon[34982]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.595 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-10T22:49:13.622 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_1 2024-09-10T22:49:13.673 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:13 smithi153 ceph-mon[34951]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.673 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:13 smithi153 ceph-mon[34951]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:13 smithi120 ceph-mon[31247]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:13 smithi120 ceph-mon[31247]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:49:13.881 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:49:14.270 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:49:14.270 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:49:14.270 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:14.270 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.270 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi153 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.271 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:14.272 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:14.272 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.272 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:14 smithi176 ceph-mon[34982]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:14.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 -- 172.21.15.176:0/1521441875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52101028f0 msgr2=0x7f5210102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 --2- 172.21.15.176:0/1521441875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52101028f0 0x7f5210102d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009930 tx=0x7f51fc02a420 comp rx=0 tx=0).stop 2024-09-10T22:49:14.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 -- 172.21.15.176:0/1521441875 shutdown_connections 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 --2- 172.21.15.176:0/1521441875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210103460 0x7f5210109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 --2- 172.21.15.176:0/1521441875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52101028f0 0x7f5210102d70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 --2- 172.21.15.176:0/1521441875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5210101fb0 0x7f52101023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 -- 172.21.15.176:0/1521441875 >> 172.21.15.176:0/1521441875 conn(0x7f52100fc990 msgr2=0x7f52100fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 -- 172.21.15.176:0/1521441875 shutdown_connections 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.315+0000 7f5217099640 1 -- 172.21.15.176:0/1521441875 wait complete. 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 Processor -- start 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 -- start start 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210101fb0 0x7f52101015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 0x7f521019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f521010d5c0 con 0x7f5210101fb0 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f521010d440 con 0x7f52101028f0 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5217099640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f521010d740 con 0x7f5210103460 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55784/0 (socket says 172.21.15.176:55784) 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 -- 172.21.15.176:0/2920454434 learned_addr learned my addr 172.21.15.176:0/2920454434 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5214e35640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210101fb0 0x7f52101015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f520ffff640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 0x7f521019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 -- 172.21.15.176:0/2920454434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 msgr2=0x7f521019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 0x7f521019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 -- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210101fb0 msgr2=0x7f52101015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.316+0000 7f5215636640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210101fb0 0x7f52101015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f5215636640 1 -- 172.21.15.176:0/2920454434 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52101a0a30 con 0x7f5210103460 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f520ffff640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 0x7f521019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f5215636640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fd0 tx=0x7f520800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208017070 con 0x7f5210103460 2024-09-10T22:49:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208003e20 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52101a0d20 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208010460 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.317+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52101a1200 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.318+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210108e90 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.319+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f5208003970 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.320+0000 7f520dffb640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 0x7f52000b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.320+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6414+0+0 (secure 0 0 0) 0x7f52080e5210 con 0x7f5210103460 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.320+0000 7f5214e35640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 0x7f52000b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.321+0000 7f5214e35640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 0x7f52000b78a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5204009bd0 tx=0x7f5204005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.324+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52080ac7a0 con 0x7f5210103460 2024-09-10T22:49:14.512 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:14.511+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5210063590 con 0x7f52000b53e0 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi153 2024-09-10T22:49:14.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:14 smithi120 ceph-mon[31247]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:14.699 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T22:49:14.226+0000 7fedbc1f0640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T22:49:14.699 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T22:49:14.238+0000 7fedb8017640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]} : dispatch 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi153 2024-09-10T22:49:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 4272M 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:14.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:14 smithi153 ceph-mon[34951]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:49:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T22:49:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:49:15.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:15.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:49:15.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:16.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:49:16.525 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:49:16.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] boot 2024-09-10T22:49:16.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:49:16.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:16.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:16 smithi176 ceph-mon[34982]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] boot 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:16.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:16 smithi120 ceph-mon[31247]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: osd.10 [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] boot 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:49:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:16 smithi153 ceph-mon[34951]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/2937972691' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"}]': finished 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:49:17.425 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/2937972691' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"}]': finished 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:49:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/2937972691' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"} : dispatch 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbced0d0-788b-48aa-9271-9cc721683b01"}]': finished 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:49:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:18 smithi176 ceph-mon[34982]: from='client.? 172.21.15.176:0/3723007915' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:18 smithi176 ceph-mon[34982]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:18.597 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:18 smithi120 ceph-mon[31247]: from='client.? 172.21.15.176:0/3723007915' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:18.597 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:18 smithi120 ceph-mon[31247]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:18.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:18 smithi153 ceph-mon[34951]: from='client.? 172.21.15.176:0/3723007915' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:49:18.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:18 smithi153 ceph-mon[34951]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:20 smithi176 ceph-mon[34982]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T22:49:20.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:20 smithi120 ceph-mon[31247]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T22:49:20.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:20 smithi153 ceph-mon[34951]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T22:49:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:22 smithi176 ceph-mon[34982]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T22:49:22.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:22 smithi120 ceph-mon[31247]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T22:49:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:22 smithi153 ceph-mon[34951]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T22:49:24.370 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:24 smithi176 ceph-mon[34982]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:49:24.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:24 smithi120 ceph-mon[31247]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:49:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:24 smithi153 ceph-mon[34951]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:49:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:49:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:25 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:25.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:49:25.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:25 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:25.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:49:25.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:25 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:26 smithi176 ceph-mon[34982]: Deploying daemon osd.11 on smithi176 2024-09-10T22:49:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:26 smithi176 ceph-mon[34982]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:49:26.600 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:26 smithi153 ceph-mon[34951]: Deploying daemon osd.11 on smithi176 2024-09-10T22:49:26.600 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:26 smithi153 ceph-mon[34951]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:49:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:26 smithi120 ceph-mon[31247]: Deploying daemon osd.11 on smithi176 2024-09-10T22:49:26.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:26 smithi120 ceph-mon[31247]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:49:28.461 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:28 smithi176 ceph-mon[34982]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T22:49:28.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:28 smithi120 ceph-mon[31247]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T22:49:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:28 smithi153 ceph-mon[34951]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T22:49:30.419 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:30 smithi176 ceph-mon[34982]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:49:30.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:30 smithi120 ceph-mon[31247]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:49:30.717 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:30 smithi153 ceph-mon[34951]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:49:31.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:31.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:31.526 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:31 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:31.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:31.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:31.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:31 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:31.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:31.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:31 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:32.491 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:32 smithi176 ceph-mon[34982]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:32 smithi120 ceph-mon[31247]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:32.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:32 smithi153 ceph-mon[34951]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:34.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.513+0000 7f520dffb640 1 -- 172.21.15.176:0/2920454434 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5210063590 con 0x7f52000b53e0 2024-09-10T22:49:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 11 on host 'smithi176' 2024-09-10T22:49:34.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 msgr2=0x7f52000b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 0x7f52000b78a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5204009bd0 tx=0x7f5204005f90 comp rx=0 tx=0).stop 2024-09-10T22:49:34.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 msgr2=0x7f52101a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5208009fd0 tx=0x7f520800b260 comp rx=0 tx=0).stop 2024-09-10T22:49:34.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 shutdown_connections 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52000b53e0 0x7f52000b78a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5210103460 0x7f52101a0170 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52101028f0 0x7f521019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 --2- 172.21.15.176:0/2920454434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5210101fb0 0x7f52101015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 >> 172.21.15.176:0/2920454434 conn(0x7f52100fc990 msgr2=0x7f5210106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 shutdown_connections 2024-09-10T22:49:34.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:34.516+0000 7f5217099640 1 -- 172.21.15.176:0/2920454434 wait complete. 2024-09-10T22:49:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:34.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:34.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:34.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:34 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:34 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:34.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:34 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:35.203 DEBUG:teuthology.orchestra.run.smithi176:osd.11> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T22:49:35.206 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T22:49:35.206 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:49:35.462 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:35.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.907+0000 7faff31be640 1 -- 172.21.15.120:0/326761020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec101940 msgr2=0x7fafec101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:35.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.907+0000 7faff31be640 1 --2- 172.21.15.120:0/326761020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec101940 0x7fafec101d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009930 tx=0x7fafdc02a420 comp rx=0 tx=0).stop 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- 172.21.15.120:0/326761020 shutdown_connections 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- 172.21.15.120:0/326761020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec103460 0x7fafec109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- 172.21.15.120:0/326761020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec102aa0 0x7fafec102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- 172.21.15.120:0/326761020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec101940 0x7fafec101d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- 172.21.15.120:0/326761020 >> 172.21.15.120:0/326761020 conn(0x7fafec0fd1f0 msgr2=0x7fafec0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- 172.21.15.120:0/326761020 shutdown_connections 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- 172.21.15.120:0/326761020 wait complete. 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 Processor -- start 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- start start 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec101940 0x7fafec198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:35.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec103460 0x7fafec19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafec10e730 con 0x7fafec102aa0 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafec10e5b0 con 0x7fafec103460 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7faff31be640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafec10e8b0 con 0x7fafec101940 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7fafebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7fafebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60122/0 (socket says 172.21.15.120:60122) 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.908+0000 7fafebfff640 1 -- 172.21.15.120:0/2093814833 learned_addr learned my addr 172.21.15.120:0/2093814833 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7faff0f5a640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec101940 0x7fafec198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 -- 172.21.15.120:0/2093814833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec101940 msgr2=0x7fafec198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec101940 0x7fafec198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 -- 172.21.15.120:0/2093814833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec103460 msgr2=0x7fafec19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec103460 0x7fafec19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 -- 172.21.15.120:0/2093814833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec19a930 con 0x7fafec102aa0 2024-09-10T22:49:35.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafebfff640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c920 tx=0x7fafe000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:35.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0007c70 con 0x7fafec102aa0 2024-09-10T22:49:35.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe0007e10 con 0x7fafec102aa0 2024-09-10T22:49:35.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0018820 con 0x7fafec102aa0 2024-09-10T22:49:35.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafec19ac20 con 0x7fafec102aa0 2024-09-10T22:49:35.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.909+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafec10aa40 con 0x7fafec102aa0 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.910+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafec101d40 con 0x7fafec102aa0 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.912+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fafe000f450 con 0x7fafec102aa0 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.912+0000 7fafe9ffb640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 0x7fafd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.913+0000 7faff0f5a640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 0x7fafd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.913+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7fafe00e5820 con 0x7fafec102aa0 2024-09-10T22:49:35.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.913+0000 7faff0f5a640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 0x7fafd40b77a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009230 tx=0x7fafdc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:35.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:35.917+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fafe0014030 con 0x7fafec102aa0 2024-09-10T22:49:36.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.091+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fafec02e5c0 con 0x7fafec102aa0 2024-09-10T22:49:36.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.091+0000 7fafe9ffb640 1 -- 172.21.15.120:0/2093814833 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fafe00accc0 con 0x7fafec102aa0 2024-09-10T22:49:36.093 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 msgr2=0x7fafd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 0x7fafd40b77a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009230 tx=0x7fafdc000bc0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 msgr2=0x7fafec199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c920 tx=0x7fafe000cde0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 shutdown_connections 2024-09-10T22:49:36.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafd40b52e0 0x7fafd40b77a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafec103460 0x7fafec19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafec102aa0 0x7fafec199450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 --2- 172.21.15.120:0/2093814833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafec101940 0x7fafec198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 >> 172.21.15.120:0/2093814833 conn(0x7fafec0fd1f0 msgr2=0x7fafec105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 shutdown_connections 2024-09-10T22:49:36.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:36.094+0000 7faff31be640 1 -- 172.21.15.120:0/2093814833 wait complete. 2024-09-10T22:49:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:36 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2093814833' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:36.441 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:36 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2093814833' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:36 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2093814833' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:36.859 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726008555,"num_in_osds":12,"osd_in_since":1726008556,"num_remapped_pgs":0} 2024-09-10T22:49:37.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 22:49:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T22:49:36.830+0000 7f083e02b640 -1 osd.11 0 log_to_monitors true 2024-09-10T22:49:37.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:37 smithi120 ceph-mon[31247]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:37 smithi120 ceph-mon[31247]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:37 smithi120 ceph-mon[31247]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:37 smithi153 ceph-mon[34951]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:37 smithi153 ceph-mon[34951]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:37 smithi153 ceph-mon[34951]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:37 smithi176 ceph-mon[34982]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:37 smithi176 ceph-mon[34982]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:37 smithi176 ceph-mon[34982]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:49:37.860 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:49:38.115 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:38.522 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T22:49:38.208+0000 7f0839fae640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T22:49:38.522 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T22:49:38.220+0000 7f0835dd5640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.522 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: Detected new or changed devices on smithi176 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.523 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:38 smithi176 ceph-mon[34982]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:38.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:49:38.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:49:38.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: Detected new or changed devices on smithi176 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.546 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:38 smithi120 ceph-mon[31247]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 -- 172.21.15.120:0/3971846130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8100e30 msgr2=0x7febe8101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 --2- 172.21.15.120:0/3971846130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8100e30 0x7febe8101210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7febd8009900 tx=0x7febd802a410 comp rx=0 tx=0).stop 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 -- 172.21.15.120:0/3971846130 shutdown_connections 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 --2- 172.21.15.120:0/3971846130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8108830 0x7febe810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 --2- 172.21.15.120:0/3971846130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8105f00 0x7febe81082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 --2- 172.21.15.120:0/3971846130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8100e30 0x7febe8101210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 -- 172.21.15.120:0/3971846130 >> 172.21.15.120:0/3971846130 conn(0x7febe80fc9a0 msgr2=0x7febe80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 -- 172.21.15.120:0/3971846130 shutdown_connections 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.624+0000 7febee96d640 1 -- 172.21.15.120:0/3971846130 wait complete. 2024-09-10T22:49:38.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 Processor -- start 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 -- start start 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8100e30 0x7febe8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8105f00 0x7febe81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e6f0 con 0x7febe8108830 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e570 con 0x7febe8105f00 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febee96d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e870 con 0x7febe8100e30 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60138/0 (socket says 172.21.15.120:60138) 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 -- 172.21.15.120:0/3078774283 learned_addr learned my addr 172.21.15.120:0/3078774283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febe7fff640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8100e30 0x7febe8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febdffff640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8105f00 0x7febe81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 -- 172.21.15.120:0/3078774283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8100e30 msgr2=0x7febe8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8100e30 0x7febe8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 -- 172.21.15.120:0/3078774283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8105f00 msgr2=0x7febe81994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8105f00 0x7febe81994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 -- 172.21.15.120:0/3078774283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe819a9d0 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febecf0a640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febe000eaa0 tx=0x7febe000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.625+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0004d20 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.626+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe0005020 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.626+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe819acc0 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.626+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00116b0 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.626+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a13a0 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.626+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe80fe490 con 0x7febe8108830 2024-09-10T22:49:38.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.628+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7febe000f040 con 0x7febe8108830 2024-09-10T22:49:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.629+0000 7febe5ffb640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 0x7febd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.629+0000 7febe7fff640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 0x7febd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.629+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6847+0+0 (secure 0 0 0) 0x7febe00e5860 con 0x7febe8108830 2024-09-10T22:49:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.630+0000 7febe7fff640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 0x7febd40b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febd8000c00 tx=0x7febd80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:38.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.632+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7febe0016030 con 0x7febe8108830 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]} : dispatch 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: Detected new or changed devices on smithi176 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 4272M 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:38.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:38 smithi153 ceph-mon[34951]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:38.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.807+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7febe802e810 con 0x7febe8108830 2024-09-10T22:49:38.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.807+0000 7febe5ffb640 1 -- 172.21.15.120:0/3078774283 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7febe00acce0 con 0x7febe8108830 2024-09-10T22:49:38.808 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:38.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 msgr2=0x7febd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 0x7febd40b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febd8000c00 tx=0x7febd80314a0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 msgr2=0x7febe819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febe000eaa0 tx=0x7febe000eea0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 shutdown_connections 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febd40b52e0 0x7febd40b77a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febe8108830 0x7febe819a130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febe8105f00 0x7febe81994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 --2- 172.21.15.120:0/3078774283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe8100e30 0x7febe8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 >> 172.21.15.120:0/3078774283 conn(0x7febe80fc9a0 msgr2=0x7febe8106b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 shutdown_connections 2024-09-10T22:49:38.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:38.810+0000 7febee96d640 1 -- 172.21.15.120:0/3078774283 wait complete. 2024-09-10T22:49:39.500 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:39 smithi120 ceph-mon[31247]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:49:39.500 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:39 smithi120 ceph-mon[31247]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:49:39.501 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.501 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.501 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:39 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3078774283' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:39.529 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726008555,"num_in_osds":12,"osd_in_since":1726008556,"num_remapped_pgs":0} 2024-09-10T22:49:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:39 smithi176 ceph-mon[34982]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:49:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:39 smithi176 ceph-mon[34982]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:49:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:39 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3078774283' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:39 smithi153 ceph-mon[34951]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-09-10T22:49:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:39 smithi153 ceph-mon[34951]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:49:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:39 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3078774283' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:40.530 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: purged_snaps scrub starts 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: purged_snaps scrub ok 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] boot 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:40.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:40 smithi120 ceph-mon[31247]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: purged_snaps scrub starts 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: purged_snaps scrub ok 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] boot 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:40 smithi176 ceph-mon[34982]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: purged_snaps scrub starts 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: purged_snaps scrub ok 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: osd.11 [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] boot 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:49:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:40 smithi153 ceph-mon[34951]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:49:40.788 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:41.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 -- 172.21.15.120:0/4057397021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868101940 msgr2=0x7ff868101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:41.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 --2- 172.21.15.120:0/4057397021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868101940 0x7ff868101d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff864009930 tx=0x7ff86402a420 comp rx=0 tx=0).stop 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 -- 172.21.15.120:0/4057397021 shutdown_connections 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 --2- 172.21.15.120:0/4057397021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 0x7ff868109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 --2- 172.21.15.120:0/4057397021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868102aa0 0x7ff868102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 --2- 172.21.15.120:0/4057397021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868101940 0x7ff868101d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 -- 172.21.15.120:0/4057397021 >> 172.21.15.120:0/4057397021 conn(0x7ff8680fd210 msgr2=0x7ff8680ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 -- 172.21.15.120:0/4057397021 shutdown_connections 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.275+0000 7ff8706ec640 1 -- 172.21.15.120:0/4057397021 wait complete. 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 Processor -- start 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 -- start start 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868101940 0x7ff86810c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 0x7ff86810af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:41.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810e730 con 0x7ff868102aa0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810e5b0 con 0x7ff868101940 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff8706ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff86810e8b0 con 0x7ff868103460 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60146/0 (socket says 172.21.15.120:60146) 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 -- 172.21.15.120:0/3800659041 learned_addr learned my addr 172.21.15.120:0/3800659041 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86ec89640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 0x7ff86810af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86e488640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868101940 0x7ff86810c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 -- 172.21.15.120:0/3800659041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 msgr2=0x7ff86810af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 0x7ff86810af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 -- 172.21.15.120:0/3800659041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868101940 msgr2=0x7ff86810c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868101940 0x7ff86810c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 -- 172.21.15.120:0/3800659041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86810b830 con 0x7ff868102aa0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.276+0000 7ff86dc87640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff85800b4a0 tx=0x7ff85800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:41.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858014070 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff858010ac0 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681a2fa0 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858017c30 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a3450 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.277+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff868108cc0 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.279+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff858010610 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.279+0000 7ff85f7fe640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 0x7ff8540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.280+0000 7ff86e488640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 0x7ff8540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.280+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff8580e5090 con 0x7ff868102aa0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.280+0000 7ff86e488640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 0x7ff8540b77a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff864006010 tx=0x7ff8640095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:41.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.282+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8580ac3f0 con 0x7ff868102aa0 2024-09-10T22:49:41.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:41 smithi120 ceph-mon[31247]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:49:41.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.461+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff86802e7c0 con 0x7ff868102aa0 2024-09-10T22:49:41.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.461+0000 7ff85f7fe640 1 -- 172.21.15.120:0/3800659041 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7ff8580abcf0 con 0x7ff868102aa0 2024-09-10T22:49:41.463 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 msgr2=0x7ff8540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 0x7ff8540b77a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff864006010 tx=0x7ff8640095f0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 msgr2=0x7ff86810aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff85800b4a0 tx=0x7ff85800b960 comp rx=0 tx=0).stop 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 shutdown_connections 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8540b52e0 0x7ff8540b77a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff868103460 0x7ff86810af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff868102aa0 0x7ff86810aa30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 --2- 172.21.15.120:0/3800659041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff868101940 0x7ff86810c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 >> 172.21.15.120:0/3800659041 conn(0x7ff8680fd210 msgr2=0x7ff868105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 shutdown_connections 2024-09-10T22:49:41.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:41.464+0000 7ff8706ec640 1 -- 172.21.15.120:0/3800659041 wait complete. 2024-09-10T22:49:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:41 smithi176 ceph-mon[34982]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:49:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:41 smithi153 ceph-mon[34951]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:49:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726008579,"num_in_osds":12,"osd_in_since":1726008556,"num_remapped_pgs":0} 2024-09-10T22:49:42.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:49:42.357 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:42.478 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:42 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3800659041' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:42.478 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:42 smithi120 ceph-mon[31247]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:42 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3800659041' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:42 smithi176 ceph-mon[34982]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:42.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:42 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3800659041' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:49:42.701 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:42 smithi153 ceph-mon[34951]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:42.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.846+0000 7fb1821c7640 1 -- 172.21.15.120:0/1273319976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 msgr2=0x7fb17c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 --2- 172.21.15.120:0/1273319976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c10e820 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb174009ca0 tx=0x7fb17402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 -- 172.21.15.120:0/1273319976 shutdown_connections 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 --2- 172.21.15.120:0/1273319976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c10e820 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 --2- 172.21.15.120:0/1273319976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 0x7fb17c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 --2- 172.21.15.120:0/1273319976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 0x7fb17c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 -- 172.21.15.120:0/1273319976 >> 172.21.15.120:0/1273319976 conn(0x7fb17c0fd700 msgr2=0x7fb17c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 -- 172.21.15.120:0/1273319976 shutdown_connections 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 -- 172.21.15.120:0/1273319976 wait complete. 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 Processor -- start 2024-09-10T22:49:42.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.847+0000 7fb1821c7640 1 -- start start 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 0x7fb17c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 0x7fb17c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c111ca0 con 0x7fb17c108020 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c111b20 con 0x7fb17c101c90 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c111e20 con 0x7fb17c1076a0 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60166/0 (socket says 172.21.15.120:60166) 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 -- 172.21.15.120:0/454496300 learned_addr learned my addr 172.21.15.120:0/454496300 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17b7fe640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 0x7fb17c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17affd640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 0x7fb17c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 -- 172.21.15.120:0/454496300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 msgr2=0x7fb17c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 0x7fb17c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 -- 172.21.15.120:0/454496300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 msgr2=0x7fb17c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 0x7fb17c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 -- 172.21.15.120:0/454496300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c19e1e0 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb17bfff640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb174005cc0 tx=0x7fb17402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174003cf0 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1740044f0 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c19e470 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.848+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17402fd10 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.849+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c06aab0 con 0x7fb17c108020 2024-09-10T22:49:42.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.850+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c102070 con 0x7fb17c108020 2024-09-10T22:49:42.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.851+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fb174004040 con 0x7fb17c108020 2024-09-10T22:49:42.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.851+0000 7fb178ff9640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 0x7fb1680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.852+0000 7fb17b7fe640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 0x7fb1680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.852+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb174106ce0 con 0x7fb17c108020 2024-09-10T22:49:42.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.852+0000 7fb17b7fe640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 0x7fb1680b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0058c0 tx=0x7fb16c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:42.855+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1740ce040 con 0x7fb17c108020 2024-09-10T22:49:43.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.036+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb17c06b590 con 0x7fb17c108020 2024-09-10T22:49:43.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.037+0000 7fb178ff9640 1 -- 172.21.15.120:0/454496300 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18549 (secure 0 0 0) 0x7fb1740cd940 con 0x7fb17c108020 2024-09-10T22:49:43.039 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:43.040 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":70,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:49:40.212094+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 msgr2=0x7fb1680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 0x7fb1680b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0058c0 tx=0x7fb16c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 msgr2=0x7fb17c19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb174005cc0 tx=0x7fb17402f470 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 shutdown_connections 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1680b52e0 0x7fb1680b77a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb17c108020 0x7fb17c19da10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb17c1076a0 0x7fb17c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 --2- 172.21.15.120:0/454496300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17c101c90 0x7fb17c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 >> 172.21.15.120:0/454496300 conn(0x7fb17c0fd700 msgr2=0x7fb17c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 shutdown_connections 2024-09-10T22:49:43.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:43.039+0000 7fb1821c7640 1 -- 172.21.15.120:0/454496300 wait complete. 2024-09-10T22:49:43.390 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:43 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/454496300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:49:43.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:43 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/454496300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:49:43.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:43 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/454496300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:49:43.721 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:45:56.375659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:49:43.721 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T22:49:43.965 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:44.345 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:44 smithi120 ceph-mon[31247]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:44.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 -- 172.21.15.120:0/3067538732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc103b30 msgr2=0x7f87bc103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 --2- 172.21.15.120:0/3067538732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc103b30 0x7f87bc103f10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009ca0 tx=0x7f87b402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 -- 172.21.15.120:0/3067538732 shutdown_connections 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 --2- 172.21.15.120:0/3067538732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc10d0e0 0x7f87bc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 --2- 172.21.15.120:0/3067538732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 0x7f87bc10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 --2- 172.21.15.120:0/3067538732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc103b30 0x7f87bc103f10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 -- 172.21.15.120:0/3067538732 >> 172.21.15.120:0/3067538732 conn(0x7f87bc0fd700 msgr2=0x7f87bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 -- 172.21.15.120:0/3067538732 shutdown_connections 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.425+0000 7f87c17d6640 1 -- 172.21.15.120:0/3067538732 wait complete. 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 Processor -- start 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 -- start start 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 0x7f87bc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 0x7f87bc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc112e20 con 0x7f87bc10d0e0 2024-09-10T22:49:44.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc112ca0 con 0x7f87bc104530 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87c17d6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc112fa0 con 0x7f87bc103b30 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58948/0 (socket says 172.21.15.120:58948) 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87bb7fe640 1 -- 172.21.15.120:0/1871188008 learned_addr learned my addr 172.21.15.120:0/1871188008 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87ba7fc640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 0x7f87bc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.426+0000 7f87baffd640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 0x7f87bc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 -- 172.21.15.120:0/1871188008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 msgr2=0x7f87bc199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 0x7f87bc199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 -- 172.21.15.120:0/1871188008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 msgr2=0x7f87bc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 0x7f87bc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 -- 172.21.15.120:0/1871188008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc19f4c0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87baffd640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 0x7f87bc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:44.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87bb7fe640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009fd0 tx=0x7f87ac00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac015020 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ac011ce0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc19f7b0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac010dc0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.427+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc1a5ef0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.428+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc103f90 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.429+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f87ac011830 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.430+0000 7f87b3fff640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 0x7f87a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.430+0000 7f87baffd640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 0x7f87a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.430+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f87ac0e56c0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.430+0000 7f87baffd640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 0x7f87a80b77a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000f80 tx=0x7f87b402f480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.433+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f87ac01f020 con 0x7f87bc10d0e0 2024-09-10T22:49:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:44 smithi176 ceph-mon[34982]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:44.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.607+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87bc02e5c0 con 0x7f87bc10d0e0 2024-09-10T22:49:44.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.607+0000 7f87b3fff640 1 -- 172.21.15.120:0/1871188008 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f87ac0aca20 con 0x7f87bc10d0e0 2024-09-10T22:49:44.610 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 msgr2=0x7f87a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 0x7f87a80b77a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000f80 tx=0x7f87b402f480 comp rx=0 tx=0).stop 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 msgr2=0x7f87bc19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009fd0 tx=0x7f87ac00b820 comp rx=0 tx=0).stop 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 shutdown_connections 2024-09-10T22:49:44.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f87a80b52e0 0x7f87a80b77a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87bc10d0e0 0x7f87bc19ecf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87bc104530 0x7f87bc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 --2- 172.21.15.120:0/1871188008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87bc103b30 0x7f87bc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 >> 172.21.15.120:0/1871188008 conn(0x7f87bc0fd700 msgr2=0x7f87bc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 shutdown_connections 2024-09-10T22:49:44.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:44.610+0000 7f87c17d6640 1 -- 172.21.15.120:0/1871188008 wait complete. 2024-09-10T22:49:44.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:44 smithi153 ceph-mon[34951]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.a on smithi120 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.d on smithi120 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.g on smithi120 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.j on smithi120 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.b on smithi153 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.e on smithi153 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.h on smithi153 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.k on smithi153 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.c on smithi176 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.f on smithi176 2024-09-10T22:49:45.218 INFO:tasks.cephadm:Adding mds.i on smithi176 2024-09-10T22:49:45.219 INFO:tasks.cephadm:Adding mds.l on smithi176 2024-09-10T22:49:45.219 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l' 2024-09-10T22:49:45.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:45 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1871188008' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:49:45.478 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:49:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:45 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1871188008' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:49:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:45 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1871188008' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:49:45.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 -- 172.21.15.176:0/3129520235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f826406c0c0 msgr2=0x7f8264105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:45.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 --2- 172.21.15.176:0/3129520235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f826406c0c0 0x7f8264105620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8254009900 tx=0x7f825402a410 comp rx=0 tx=0).stop 2024-09-10T22:49:45.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 -- 172.21.15.176:0/3129520235 shutdown_connections 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 --2- 172.21.15.176:0/3129520235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 0x7f826410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 --2- 172.21.15.176:0/3129520235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8264105bb0 0x7f8264108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 --2- 172.21.15.176:0/3129520235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f826406c0c0 0x7f8264105620 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 -- 172.21.15.176:0/3129520235 >> 172.21.15.176:0/3129520235 conn(0x7f82640fd210 msgr2=0x7f82640ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 -- 172.21.15.176:0/3129520235 shutdown_connections 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 -- 172.21.15.176:0/3129520235 wait complete. 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.891+0000 7f826957b640 1 Processor -- start 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 -- start start 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826406c0c0 0x7f8264198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 0x7f826419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f826410e7b0 con 0x7f8264108760 2024-09-10T22:49:45.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f826410e630 con 0x7f826406c0c0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f826957b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f826410e930 con 0x7f8264105bb0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36220/0 (socket says 172.21.15.176:36220) 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f8262ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826406c0c0 0x7f8264198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 -- 172.21.15.176:0/2563695722 learned_addr learned my addr 172.21.15.176:0/2563695722 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82637fe640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 0x7f826419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 -- 172.21.15.176:0/2563695722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826406c0c0 msgr2=0x7f8264198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826406c0c0 0x7f8264198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 -- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 msgr2=0x7f826419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 0x7f826419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 -- 172.21.15.176:0/2563695722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826419aaf0 con 0x7f8264105bb0 2024-09-10T22:49:45.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.892+0000 7f82627fc640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f825800b450 tx=0x7f825800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:45.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.893+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258015020 con 0x7f8264105bb0 2024-09-10T22:49:45.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.893+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826419ade0 con 0x7f8264105bb0 2024-09-10T22:49:45.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.893+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8258011d40 con 0x7f8264105bb0 2024-09-10T22:49:45.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.893+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825800f340 con 0x7f8264105bb0 2024-09-10T22:49:45.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.893+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82641a14c0 con 0x7f8264105bb0 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.894+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82640fed00 con 0x7f8264105bb0 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.896+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f8258010500 con 0x7f8264105bb0 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.896+0000 7f824bfff640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 0x7f824c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.897+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f82580e59a0 con 0x7f8264105bb0 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.897+0000 7f8262ffd640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 0x7f824c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:45.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.897+0000 7f8262ffd640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 0x7f824c0b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8254000c00 tx=0x7f82540314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:45.900 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:45.899+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f825801f020 con 0x7f8264105bb0 2024-09-10T22:49:46.085 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.084+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}) -- 0x7f8264063690 con 0x7f824c0b52e0 2024-09-10T22:49:46.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.091+0000 7f824bfff640 1 -- 172.21.15.176:0/2563695722 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8264063690 con 0x7f824c0b52e0 2024-09-10T22:49:46.092 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mds.all update... 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 msgr2=0x7f824c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 0x7f824c0b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8254000c00 tx=0x7f82540314a0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 msgr2=0x7f8264199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f825800b450 tx=0x7f825800b910 comp rx=0 tx=0).stop 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 shutdown_connections 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f824c0b52e0 0x7f824c0b77a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8264108760 0x7f826419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8264105bb0 0x7f8264199610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 --2- 172.21.15.176:0/2563695722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f826406c0c0 0x7f8264198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.093+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 >> 172.21.15.176:0/2563695722 conn(0x7f82640fd210 msgr2=0x7f8264103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.094+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 shutdown_connections 2024-09-10T22:49:46.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-10T22:49:46.094+0000 7f826957b640 1 -- 172.21.15.176:0/2563695722 wait complete. 2024-09-10T22:49:46.816 DEBUG:teuthology.orchestra.run.smithi120:mds.a> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T22:49:46.818 DEBUG:teuthology.orchestra.run.smithi120:mds.d> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T22:49:46.861 DEBUG:teuthology.orchestra.run.smithi120:mds.g> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T22:49:46.904 DEBUG:teuthology.orchestra.run.smithi120:mds.j> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T22:49:46.947 DEBUG:teuthology.orchestra.run.smithi153:mds.b> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T22:49:46.949 DEBUG:teuthology.orchestra.run.smithi153:mds.e> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T22:49:46.992 DEBUG:teuthology.orchestra.run.smithi153:mds.h> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T22:49:47.035 DEBUG:teuthology.orchestra.run.smithi153:mds.k> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T22:49:47.079 DEBUG:teuthology.orchestra.run.smithi176:mds.c> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T22:49:47.081 DEBUG:teuthology.orchestra.run.smithi176:mds.f> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T22:49:47.124 DEBUG:teuthology.orchestra.run.smithi176:mds.i> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T22:49:47.167 DEBUG:teuthology.orchestra.run.smithi176:mds.l> sudo journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T22:49:47.211 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T22:49:47.212 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:47.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:47 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:47.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:47 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: Saving service mds.all spec with placement smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:47.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:47 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:47.456 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:47.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- 172.21.15.120:0/3620302981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a0103cb0 msgr2=0x7f41a0104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/3620302981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a0103cb0 0x7f41a0104090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4190009900 tx=0x7f419002a410 comp rx=0 tx=0).stop 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- 172.21.15.120:0/3620302981 shutdown_connections 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/3620302981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a010d110 0x7f41a010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/3620302981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a01045d0 0x7f41a010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/3620302981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a0103cb0 0x7f41a0104090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- 172.21.15.120:0/3620302981 >> 172.21.15.120:0/3620302981 conn(0x7f41a00fd6d0 msgr2=0x7f41a00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- 172.21.15.120:0/3620302981 shutdown_connections 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- 172.21.15.120:0/3620302981 wait complete. 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 Processor -- start 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.955+0000 7f41a6e8d640 1 -- start start 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 0x7f41a01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a01045d0 0x7f41a0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41a0112d80 con 0x7f41a010d110 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41a0112c00 con 0x7f41a01045d0 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f41a0112f00 con 0x7f41a0103cb0 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58966/0 (socket says 172.21.15.120:58966) 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 -- 172.21.15.120:0/815448876 learned_addr learned my addr 172.21.15.120:0/815448876 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a4c29640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 0x7f41a01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 -- 172.21.15.120:0/815448876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 msgr2=0x7f41a01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 0x7f41a01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 -- 172.21.15.120:0/815448876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a01045d0 msgr2=0x7f41a0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a01045d0 0x7f41a0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 -- 172.21.15.120:0/815448876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a019f3a0 con 0x7f41a010d110 2024-09-10T22:49:47.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a4c29640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 0x7f41a01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a542a640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f419800cce0 tx=0x7f419800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419800f450 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419800faa0 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a019f690 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198018b70 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.956+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a01a5d50 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.958+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a0104090 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.959+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f419800f5f0 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.959+0000 7f419dffb640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 0x7f41880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.960+0000 7f41a4c29640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 0x7f41880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.960+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f41980e5ef0 con 0x7f41a010d110 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.960+0000 7f41a4c29640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 0x7f41880b77a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4190000c00 tx=0x7f4190031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:47.964+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4198014030 con 0x7f41a010d110 2024-09-10T22:49:48.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.175+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41a01081e0 con 0x7f41a010d110 2024-09-10T22:49:48.178 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:48 smithi120 ceph-mon[31247]: Deploying daemon mds.l on smithi176 2024-09-10T22:49:48.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.176+0000 7f419dffb640 1 -- 172.21.15.120:0/815448876 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f41980ad250 con 0x7f41a010d110 2024-09-10T22:49:48.179 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:48.179 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":1,"btime":"2024-09-10T22:42:26:976507+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T22:49:48.179 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 1 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 msgr2=0x7f41880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 0x7f41880b77a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4190000c00 tx=0x7f4190031400 comp rx=0 tx=0).stop 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 msgr2=0x7f41a019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f419800cce0 tx=0x7f419800a620 comp rx=0 tx=0).stop 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 shutdown_connections 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f41880b52e0 0x7f41880b77a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41a010d110 0x7f41a019ebd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41a01045d0 0x7f41a0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 --2- 172.21.15.120:0/815448876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41a0103cb0 0x7f41a01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.178+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 >> 172.21.15.120:0/815448876 conn(0x7f41a00fd6d0 msgr2=0x7f41a00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.179+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 shutdown_connections 2024-09-10T22:49:48.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:48.179+0000 7f41a6e8d640 1 -- 172.21.15.120:0/815448876 wait complete. 2024-09-10T22:49:48.215 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:48 smithi176 ceph-mon[34982]: Deploying daemon mds.l on smithi176 2024-09-10T22:49:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:48 smithi153 ceph-mon[34951]: Deploying daemon mds.l on smithi176 2024-09-10T22:49:48.573 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 systemd[1]: Starting Ceph mds.l for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:49:48.880 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 podman[64215]: 2024-09-10 22:49:48.601971277 +0000 UTC m=+0.110506934 container create c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T22:49:48.880 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 podman[64215]: 2024-09-10 22:49:48.507219267 +0000 UTC m=+0.015754932 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:49:48.881 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 podman[64215]: 2024-09-10 22:49:48.795925273 +0000 UTC m=+0.304460935 container init c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T22:49:48.881 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 podman[64215]: 2024-09-10 22:49:48.798846616 +0000 UTC m=+0.307382271 container start c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T22:49:48.881 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 ceph-mds[64235]: -- 172.21.15.176:0/1542319254 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f0a52f860 con 0x555f0a50d400 2024-09-10T22:49:48.881 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l[64231]: starting mds.l at 2024-09-10T22:49:48.881 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 bash[64215]: c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/815448876' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6832/1099927074,v1:172.21.15.176:6833/1099927074] up:boot 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: fsmap 1 up:standby 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:49.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:49.323 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 22:49:48 smithi176 systemd[1]: Started Ceph mds.l for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:49:49.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:49.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/815448876' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6832/1099927074,v1:172.21.15.176:6833/1099927074] up:boot 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: fsmap 1 up:standby 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:49.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:49.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:49.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/815448876' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:49.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6832/1099927074,v1:172.21.15.176:6833/1099927074] up:boot 2024-09-10T22:49:49.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: fsmap 1 up:standby 2024-09-10T22:49:49.371 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:49.372 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:50.377 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:50 smithi153 ceph-mon[34951]: Deploying daemon mds.k on smithi153 2024-09-10T22:49:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:50 smithi120 ceph-mon[31247]: Deploying daemon mds.k on smithi153 2024-09-10T22:49:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:50 smithi176 ceph-mon[34982]: Deploying daemon mds.k on smithi153 2024-09-10T22:49:50.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:49:51.109 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:50 smithi153 systemd[1]: Starting Ceph mds.k for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:49:51.211 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:51.333 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:51 smithi120 ceph-mon[31247]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:51.361 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:51 smithi153 ceph-mon[34951]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:51.361 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 podman[64117]: 2024-09-10 22:49:51.027372773 +0000 UTC m=+0.015691173 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:49:51.361 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 podman[64117]: 2024-09-10 22:49:51.128410813 +0000 UTC m=+0.116729200 container create 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:49:51.361 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 podman[64117]: 2024-09-10 22:49:51.331328437 +0000 UTC m=+0.319646824 container init 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T22:49:51.361 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 podman[64117]: 2024-09-10 22:49:51.33417835 +0000 UTC m=+0.322496746 container start 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T22:49:51.362 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 ceph-mds[64136]: -- 172.21.15.153:0/465019689 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc2e733860 con 0x55dc2e711000 2024-09-10T22:49:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:51 smithi176 ceph-mon[34982]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:51.699 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k[64132]: starting mds.k at 2024-09-10T22:49:51.699 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 bash[64117]: 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d 2024-09-10T22:49:51.699 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 22:49:51 smithi153 systemd[1]: Started Ceph mds.k for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 -- 172.21.15.120:0/3095049068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c1076c0 msgr2=0x7f003c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 --2- 172.21.15.120:0/3095049068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c1076c0 0x7f003c107b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0024009900 tx=0x7f002402a410 comp rx=0 tx=0).stop 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 -- 172.21.15.120:0/3095049068 shutdown_connections 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 --2- 172.21.15.120:0/3095049068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c108040 0x7f003c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 --2- 172.21.15.120:0/3095049068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c1076c0 0x7f003c107b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 --2- 172.21.15.120:0/3095049068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c101cb0 0x7f003c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 -- 172.21.15.120:0/3095049068 >> 172.21.15.120:0/3095049068 conn(0x7f003c0fd700 msgr2=0x7f003c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.727+0000 7f0042461640 1 -- 172.21.15.120:0/3095049068 shutdown_connections 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 -- 172.21.15.120:0/3095049068 wait complete. 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 Processor -- start 2024-09-10T22:49:51.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 -- start start 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 0x7f003c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c108040 0x7f003c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111d00 con 0x7f003c101cb0 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111b80 con 0x7f003c108040 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f0042461640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111e80 con 0x7f003c1076c0 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f003bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f003bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58982/0 (socket says 172.21.15.120:58982) 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f003bfff640 1 -- 172.21.15.120:0/1459266861 learned_addr learned my addr 172.21.15.120:0/1459266861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.728+0000 7f00409fe640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c108040 0x7f003c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0033fff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 0x7f003c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 -- 172.21.15.120:0/1459266861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 msgr2=0x7f003c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 0x7f003c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 -- 172.21.15.120:0/1459266861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c108040 msgr2=0x7f003c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c108040 0x7f003c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 -- 172.21.15.120:0/1459266861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c1a5410 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0033fff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 0x7f003c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f003bfff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f002c00e9f0 tx=0x7f002c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00cd80 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c1a5700 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002c0108e0 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c018980 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.729+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1a5c60 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.730+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c102090 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.731+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f002c010430 con 0x7f003c101cb0 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.732+0000 7f0039ffb640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 0x7f00280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.732+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f002c0e5fe0 con 0x7f003c101cb0 2024-09-10T22:49:51.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.732+0000 7f0033fff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 0x7f00280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.733+0000 7f0033fff640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 0x7f00280b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0024031440 tx=0x7f0024005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.735+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f002c0ad340 con 0x7f003c101cb0 2024-09-10T22:49:51.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.956+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f003c1a6030 con 0x7f003c101cb0 2024-09-10T22:49:51.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.957+0000 7f0039ffb640 1 -- 172.21.15.120:0/1459266861 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f002c0acc40 con 0x7f003c101cb0 2024-09-10T22:49:51.959 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:51.960 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":2,"btime":"2024-09-10T22:49:48:826391+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:49:51.960 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 2 2024-09-10T22:49:51.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 msgr2=0x7f00280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 0x7f00280b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0024031440 tx=0x7f0024005a20 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 msgr2=0x7f003c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f002c00e9f0 tx=0x7f002c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 shutdown_connections 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00280b52e0 0x7f00280b77a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f003c108040 0x7f003c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f003c1076c0 0x7f003c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 --2- 172.21.15.120:0/1459266861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f003c101cb0 0x7f003c19a030 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 >> 172.21.15.120:0/1459266861 conn(0x7f003c0fd700 msgr2=0x7f003c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 shutdown_connections 2024-09-10T22:49:51.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:51.959+0000 7f0042461640 1 -- 172.21.15.120:0/1459266861 wait complete. 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: Deploying daemon mds.j on smithi120 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1459266861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.153:6832/811375010,v1:172.21.15.153:6833/811375010] up:boot 2024-09-10T22:49:52.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: fsmap 2 up:standby 2024-09-10T22:49:52.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:49:52.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:52 smithi120 ceph-mon[31247]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:52.793 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: Deploying daemon mds.j on smithi120 2024-09-10T22:49:52.794 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1459266861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:52.794 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.153:6832/811375010,v1:172.21.15.153:6833/811375010] up:boot 2024-09-10T22:49:52.794 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: fsmap 2 up:standby 2024-09-10T22:49:52.794 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:49:52.794 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:52 smithi153 ceph-mon[34951]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:52.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: Deploying daemon mds.j on smithi120 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1459266861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.153:6832/811375010,v1:172.21.15.153:6833/811375010] up:boot 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: fsmap 2 up:standby 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:49:52.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:52 smithi176 ceph-mon[34982]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:53.947 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:53 smithi120 systemd[1]: Starting Ceph mds.j for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:49:54.214 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:53 smithi120 podman[73901]: 2024-09-10 22:49:53.861167796 +0000 UTC m=+0.015684932 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:49:54.214 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:53 smithi120 podman[73901]: 2024-09-10 22:49:53.994056772 +0000 UTC m=+0.148573901 container create 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T22:49:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:54 smithi176 ceph-mon[34982]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:54 smithi120 ceph-mon[31247]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 podman[73901]: 2024-09-10 22:49:54.246686779 +0000 UTC m=+0.401203909 container init 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 podman[73901]: 2024-09-10 22:49:54.249660182 +0000 UTC m=+0.404177317 container start 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 ceph-mds[73920]: -- 172.21.15.120:0/2656733849 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6d05e9860 con 0x55e6d05c7000 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j[73916]: starting mds.j at 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 bash[73901]: 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 2024-09-10T22:49:54.697 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 22:49:54 smithi120 systemd[1]: Started Ceph mds.j for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:49:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:54 smithi153 ceph-mon[34951]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:55.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:55.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: Deploying daemon mds.d on smithi120 2024-09-10T22:49:55.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.120:6834/1392508310,v1:172.21.15.120:6835/1392508310] up:boot 2024-09-10T22:49:55.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: fsmap 3 up:standby 2024-09-10T22:49:55.698 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: Deploying daemon mds.d on smithi120 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.120:6834/1392508310,v1:172.21.15.120:6835/1392508310] up:boot 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: fsmap 3 up:standby 2024-09-10T22:49:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:49:55.740 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:49:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: Deploying daemon mds.d on smithi120 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.120:6834/1392508310,v1:172.21.15.120:6835/1392508310] up:boot 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: fsmap 3 up:standby 2024-09-10T22:49:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:49:55.996 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:56.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.558+0000 7fc63cd13640 1 -- 172.21.15.120:0/3884594941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638102da0 msgr2=0x7fc638103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.558+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3884594941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638102da0 0x7fc638103180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc628009ca0 tx=0x7fc62802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 -- 172.21.15.120:0/3884594941 shutdown_connections 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3884594941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638109130 0x7fc63810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3884594941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3884594941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638102da0 0x7fc638103180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 -- 172.21.15.120:0/3884594941 >> 172.21.15.120:0/3884594941 conn(0x7fc6380fe810 msgr2=0x7fc638100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 -- 172.21.15.120:0/3884594941 shutdown_connections 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 -- 172.21.15.120:0/3884594941 wait complete. 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 Processor -- start 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.559+0000 7fc63cd13640 1 -- start start 2024-09-10T22:49:56.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 0x7fc6381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc638112e30 con 0x7fc638109130 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc638112cb0 con 0x7fc638102da0 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc638112fb0 con 0x7fc6381087b0 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45198/0 (socket says 172.21.15.120:45198) 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 -- 172.21.15.120:0/3700365930 learned_addr learned my addr 172.21.15.120:0/3700365930 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc635d9b640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63659c640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 0x7fc6381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 -- 172.21.15.120:0/3700365930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 msgr2=0x7fc638199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 -- 172.21.15.120:0/3700365930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 msgr2=0x7fc6381993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 0x7fc6381993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 -- 172.21.15.120:0/3700365930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63819f470 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc635d9b640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63659c640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 0x7fc6381993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc636d9d640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc630009fd0 tx=0x7fc63000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630011830 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc630016070 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63819f760 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630010de0 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.560+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6381a5ea0 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.561+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc638103200 con 0x7fc638109130 2024-09-10T22:49:56.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.563+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fc6300119d0 con 0x7fc638109130 2024-09-10T22:49:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.564+0000 7fc6277fe640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 0x7fc6200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.564+0000 7fc63659c640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 0x7fc6200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.564+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc6300e4d20 con 0x7fc638109130 2024-09-10T22:49:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.564+0000 7fc63659c640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 0x7fc6200b7820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc628000f80 tx=0x7fc6280098e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:56.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.568+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6300ac080 con 0x7fc638109130 2024-09-10T22:49:56.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:56 smithi120 ceph-mon[31247]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:56.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:56 smithi153 ceph-mon[34951]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:56.724 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:56 smithi176 ceph-mon[34982]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.792+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc63802e5c0 con 0x7fc638109130 2024-09-10T22:49:56.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.792+0000 7fc6277fe640 1 -- 172.21.15.120:0/3700365930 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fc6300ab980 con 0x7fc638109130 2024-09-10T22:49:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:49:56.795 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":4,"btime":"2024-09-10T22:49:55:213831+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:49:56.795 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 4 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 msgr2=0x7fc6200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 0x7fc6200b7820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc628000f80 tx=0x7fc6280098e0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 msgr2=0x7fc63819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc630009fd0 tx=0x7fc63000b8e0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 shutdown_connections 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6200b5360 0x7fc6200b7820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc638109130 0x7fc63819eca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6381087b0 0x7fc638199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 --2- 172.21.15.120:0/3700365930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc638102da0 0x7fc6381993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 >> 172.21.15.120:0/3700365930 conn(0x7fc6380fe810 msgr2=0x7fc638100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 shutdown_connections 2024-09-10T22:49:56.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:49:56.795+0000 7fc63cd13640 1 -- 172.21.15.120:0/3700365930 wait complete. 2024-09-10T22:49:57.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3700365930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3700365930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3700365930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:57.879 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 22:49:57 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d[74332]: starting mds.d at 2024-09-10T22:49:57.880 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 22:49:57 smithi120 bash[74304]: 928e6f5136890a98c211f4afd58b643007d430fd007960f12c2e37e18aca5b8a 2024-09-10T22:49:58.196 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 22:49:57 smithi120 systemd[1]: Started Ceph mds.d for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:49:58.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:58.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: Deploying daemon mds.e on smithi153 2024-09-10T22:49:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.120:6836/3319509867,v1:172.21.15.120:6837/3319509867] up:boot 2024-09-10T22:49:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: fsmap 4 up:standby 2024-09-10T22:49:58.951 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:49:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: Deploying daemon mds.e on smithi153 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:59.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.120:6836/3319509867,v1:172.21.15.120:6837/3319509867] up:boot 2024-09-10T22:49:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: fsmap 4 up:standby 2024-09-10T22:49:59.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:49:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:49:59.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:49:59.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: Deploying daemon mds.e on smithi153 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.120:6836/3319509867,v1:172.21.15.120:6837/3319509867] up:boot 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: fsmap 4 up:standby 2024-09-10T22:49:59.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:49:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:49:59.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:49:59.885 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:59.946 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:49:59 smithi153 systemd[1]: Starting Ceph mds.e for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:49:59.946 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:49:59 smithi153 podman[64427]: 2024-09-10 22:49:59.847311928 +0000 UTC m=+0.015654718 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:00 smithi153 ceph-mon[34951]: overall HEALTH_OK 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:49:59 smithi153 podman[64427]: 2024-09-10 22:49:59.948091401 +0000 UTC m=+0.116434229 container create 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 podman[64427]: 2024-09-10 22:50:00.167204097 +0000 UTC m=+0.335546875 container init 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 podman[64427]: 2024-09-10 22:50:00.170018796 +0000 UTC m=+0.338361581 container start 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 ceph-mds[64447]: -- 172.21.15.153:0/3619562381 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611c5f31860 con 0x5611c5f0f000 2024-09-10T22:50:00.199 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e[64442]: starting mds.e at 2024-09-10T22:50:00.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:00 smithi120 ceph-mon[31247]: overall HEALTH_OK 2024-09-10T22:50:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:00 smithi176 ceph-mon[34982]: overall HEALTH_OK 2024-09-10T22:50:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 -- 172.21.15.120:0/849725816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a81045e0 msgr2=0x7f66a810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 --2- 172.21.15.120:0/849725816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a81045e0 0x7f66a810cad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f669802a410 comp rx=0 tx=0).stop 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 -- 172.21.15.120:0/849725816 shutdown_connections 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 --2- 172.21.15.120:0/849725816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a810d120 0x7f66a810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 --2- 172.21.15.120:0/849725816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a81045e0 0x7f66a810cad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 --2- 172.21.15.120:0/849725816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a8103cc0 0x7f66a81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 -- 172.21.15.120:0/849725816 >> 172.21.15.120:0/849725816 conn(0x7f66a80fd700 msgr2=0x7f66a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 -- 172.21.15.120:0/849725816 shutdown_connections 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.386+0000 7f66b07ee640 1 -- 172.21.15.120:0/849725816 wait complete. 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 Processor -- start 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 -- start start 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 0x7f66a819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 0x7f66a819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112d90 con 0x7f66a810d120 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112c10 con 0x7f66a8103cc0 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66b07ee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112f10 con 0x7f66a81045e0 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66aed8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66aed8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45232/0 (socket says 172.21.15.120:45232) 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66aed8b640 1 -- 172.21.15.120:0/998379915 learned_addr learned my addr 172.21.15.120:0/998379915 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66add89640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 0x7f66a819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.387+0000 7f66ae58a640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 0x7f66a819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 -- 172.21.15.120:0/998379915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 msgr2=0x7f66a819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 0x7f66a819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 -- 172.21.15.120:0/998379915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 msgr2=0x7f66a819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 0x7f66a819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 -- 172.21.15.120:0/998379915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a81a15d0 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66add89640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 0x7f66a819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66ae58a640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 0x7f66a819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66aed8b640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cad0 tx=0x7f66a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000ae20 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0005020 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81a18c0 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000f6b0 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.388+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81a7f80 con 0x7f66a810d120 2024-09-10T22:50:00.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.389+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a81040a0 con 0x7f66a810d120 2024-09-10T22:50:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.391+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f66a00054f0 con 0x7f66a810d120 2024-09-10T22:50:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.391+0000 7f669f7fe640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 0x7f66940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.391+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f66a00e5c10 con 0x7f66a810d120 2024-09-10T22:50:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.391+0000 7f66ae58a640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 0x7f66940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.392+0000 7f66ae58a640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 0x7f66940b77a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66a40058c0 tx=0x7f66a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.394+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66a0014030 con 0x7f66a810d120 2024-09-10T22:50:00.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.602+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a81a82d0 con 0x7f66a810d120 2024-09-10T22:50:00.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.603+0000 7f669f7fe640 1 -- 172.21.15.120:0/998379915 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f66a00acf70 con 0x7f66a810d120 2024-09-10T22:50:00.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:00.606 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":5,"btime":"2024-09-10T22:49:58:412284+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:50:00.606 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 5 2024-09-10T22:50:00.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 msgr2=0x7f66940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 0x7f66940b77a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66a40058c0 tx=0x7f66a4005850 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 msgr2=0x7f66a81a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cad0 tx=0x7f66a000cf90 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 shutdown_connections 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66940b52e0 0x7f66940b77a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a810d120 0x7f66a81a0e00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66a81045e0 0x7f66a819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 --2- 172.21.15.120:0/998379915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66a8103cc0 0x7f66a819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 >> 172.21.15.120:0/998379915 conn(0x7f66a80fd700 msgr2=0x7f66a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 shutdown_connections 2024-09-10T22:50:00.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:00.605+0000 7f66b07ee640 1 -- 172.21.15.120:0/998379915 wait complete. 2024-09-10T22:50:00.699 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 bash[64427]: 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f 2024-09-10T22:50:00.699 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 22:50:00 smithi153 systemd[1]: Started Ceph mds.e for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:01.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:01.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:01.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:01.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:01.074 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:01 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/998379915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:01.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:01.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:01.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:01.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:01.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:01 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/998379915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:01 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/998379915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:02.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-mon[34982]: Deploying daemon mds.f on smithi176 2024-09-10T22:50:02.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.153:6834/1790379992,v1:172.21.15.153:6835/1790379992] up:boot 2024-09-10T22:50:02.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-mon[34982]: fsmap 5 up:standby 2024-09-10T22:50:02.282 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:50:02.282 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 systemd[1]: Starting Ceph mds.f for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:50:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:02 smithi120 ceph-mon[31247]: Deploying daemon mds.f on smithi176 2024-09-10T22:50:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:02 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.153:6834/1790379992,v1:172.21.15.153:6835/1790379992] up:boot 2024-09-10T22:50:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:02 smithi120 ceph-mon[31247]: fsmap 5 up:standby 2024-09-10T22:50:02.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:02 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:50:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:02 smithi153 ceph-mon[34951]: Deploying daemon mds.f on smithi176 2024-09-10T22:50:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:02 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.153:6834/1790379992,v1:172.21.15.153:6835/1790379992] up:boot 2024-09-10T22:50:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:02 smithi153 ceph-mon[34951]: fsmap 5 up:standby 2024-09-10T22:50:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:02 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:50:02.550 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 podman[64516]: 2024-09-10 22:50:02.183270542 +0000 UTC m=+0.015557785 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:02.550 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 podman[64516]: 2024-09-10 22:50:02.286405725 +0000 UTC m=+0.118692961 container create a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T22:50:02.550 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 podman[64516]: 2024-09-10 22:50:02.52230511 +0000 UTC m=+0.354592357 container init a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T22:50:02.550 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 podman[64516]: 2024-09-10 22:50:02.525162655 +0000 UTC m=+0.357449892 container start a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:50:02.550 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-mds[64535]: -- 172.21.15.176:0/3325072916 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557aebe31c20 con 0x557aebe0f400 2024-09-10T22:50:02.823 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f[64531]: starting mds.f at 2024-09-10T22:50:02.823 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 bash[64516]: a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 2024-09-10T22:50:02.823 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 22:50:02 smithi176 systemd[1]: Started Ceph mds.f for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:03.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:03.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:03.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:03 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:03.426 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:03.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:03 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:03.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:03 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:03.687 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 -- 172.21.15.120:0/3919038990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828109150 msgr2=0x7ff82810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 --2- 172.21.15.120:0/3919038990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828109150 0x7ff82810f950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff820009ca0 tx=0x7ff82002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 -- 172.21.15.120:0/3919038990 shutdown_connections 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 --2- 172.21.15.120:0/3919038990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828109150 0x7ff82810f950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 --2- 172.21.15.120:0/3919038990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 0x7ff828108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 --2- 172.21.15.120:0/3919038990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828102dc0 0x7ff8281031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 -- 172.21.15.120:0/3919038990 >> 172.21.15.120:0/3919038990 conn(0x7ff8280fe870 msgr2=0x7ff828100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:04.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 -- 172.21.15.120:0/3919038990 shutdown_connections 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.195+0000 7ff82d037640 1 -- 172.21.15.120:0/3919038990 wait complete. 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 Processor -- start 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 -- start start 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 0x7ff8280759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828109150 0x7ff828075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112e60 con 0x7ff828102dc0 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112ce0 con 0x7ff828109150 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82d037640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112fe0 con 0x7ff8281087d0 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55506/0 (socket says 172.21.15.120:55506) 2024-09-10T22:50:04.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 -- 172.21.15.120:0/3759685485 learned_addr learned my addr 172.21.15.120:0/3759685485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82659c640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 0x7ff8280759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff82759e640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828109150 0x7ff828075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 -- 172.21.15.120:0/3759685485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 msgr2=0x7ff8280759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 0x7ff8280759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 -- 172.21.15.120:0/3759685485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828109150 msgr2=0x7ff828075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828109150 0x7ff828075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.196+0000 7ff826d9d640 1 -- 172.21.15.120:0/3759685485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a7690 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff826d9d640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c890 tx=0x7ff81800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818004220 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8180043c0 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818004a00 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a7980 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.197+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a7f40 con 0x7ff828102dc0 2024-09-10T22:50:04.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.198+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8281031a0 con 0x7ff828102dc0 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.200+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff8180079b0 con 0x7ff828102dc0 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.201+0000 7ff81ffff640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 0x7ff8140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.201+0000 7ff82659c640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 0x7ff8140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.201+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff8180e5820 con 0x7ff828102dc0 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.201+0000 7ff82659c640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 0x7ff8140b7820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff828076f90 tx=0x7ff810009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:04.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.204+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff818014030 con 0x7ff828102dc0 2024-09-10T22:50:04.313 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-mon[34982]: Deploying daemon mds.c on smithi176 2024-09-10T22:50:04.313 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6834/4233369048,v1:172.21.15.176:6835/4233369048] up:boot 2024-09-10T22:50:04.313 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-mon[34982]: fsmap 6 up:standby 2024-09-10T22:50:04.313 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:50:04.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.414+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8280767a0 con 0x7ff828102dc0 2024-09-10T22:50:04.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:04 smithi120 ceph-mon[31247]: Deploying daemon mds.c on smithi176 2024-09-10T22:50:04.416 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:04 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6834/4233369048,v1:172.21.15.176:6835/4233369048] up:boot 2024-09-10T22:50:04.416 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:04 smithi120 ceph-mon[31247]: fsmap 6 up:standby 2024-09-10T22:50:04.416 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:04 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:50:04.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.415+0000 7ff81ffff640 1 -- 172.21.15.120:0/3759685485 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7ff8180acae0 con 0x7ff828102dc0 2024-09-10T22:50:04.416 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:04.417 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":7,"btime":"2024-09-10T22:50:03:058536+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-10T22:50:04.417 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 7 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 msgr2=0x7ff8140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 0x7ff8140b7820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff828076f90 tx=0x7ff810009110 comp rx=0 tx=0).stop 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 msgr2=0x7ff8280754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c890 tx=0x7ff81800cc90 comp rx=0 tx=0).stop 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 shutdown_connections 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff8140b5360 0x7ff8140b7820 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff828109150 0x7ff828075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281087d0 0x7ff8280759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 --2- 172.21.15.120:0/3759685485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff828102dc0 0x7ff8280754a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:04.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 >> 172.21.15.120:0/3759685485 conn(0x7ff8280fe870 msgr2=0x7ff828100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:04.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 shutdown_connections 2024-09-10T22:50:04.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:04.417+0000 7ff82d037640 1 -- 172.21.15.120:0/3759685485 wait complete. 2024-09-10T22:50:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:04 smithi153 ceph-mon[34951]: Deploying daemon mds.c on smithi176 2024-09-10T22:50:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:04 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6834/4233369048,v1:172.21.15.176:6835/4233369048] up:boot 2024-09-10T22:50:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:04 smithi153 ceph-mon[34951]: fsmap 6 up:standby 2024-09-10T22:50:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:04 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:50:04.573 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 systemd[1]: Starting Ceph mds.c for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:50:04.968 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 podman[64822]: 2024-09-10 22:50:04.609948647 +0000 UTC m=+0.015875576 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:04.968 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 podman[64822]: 2024-09-10 22:50:04.748088434 +0000 UTC m=+0.154015362 container create ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-10T22:50:04.968 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 podman[64822]: 2024-09-10 22:50:04.967252337 +0000 UTC m=+0.373179268 container init ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T22:50:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:05 smithi120 ceph-mon[31247]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:05.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:05 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3759685485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:05.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:05 smithi176 ceph-mon[34982]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:05.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:05 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3759685485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:05.323 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 podman[64822]: 2024-09-10 22:50:04.970178779 +0000 UTC m=+0.376105707 container start ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:50:05.324 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/1984586904 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625aebaf860 con 0x5625aeb8c800 2024-09-10T22:50:05.324 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T22:50:05.324 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:05 smithi176 bash[64822]: ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a 2024-09-10T22:50:05.324 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:50:05 smithi176 systemd[1]: Started Ceph mds.c for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:05 smithi153 ceph-mon[34951]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:05.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:05 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3759685485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/1046536521,v1:172.21.15.176:6837/1046536521] up:boot 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: fsmap 7 up:standby 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:06.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:06.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:06.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:06 smithi153 ceph-mon[34951]: Deploying daemon mds.h on smithi153 2024-09-10T22:50:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/1046536521,v1:172.21.15.176:6837/1046536521] up:boot 2024-09-10T22:50:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: fsmap 7 up:standby 2024-09-10T22:50:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:50:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:06.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:06 smithi120 ceph-mon[31247]: Deploying daemon mds.h on smithi153 2024-09-10T22:50:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/1046536521,v1:172.21.15.176:6837/1046536521] up:boot 2024-09-10T22:50:06.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: fsmap 7 up:standby 2024-09-10T22:50:06.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:50:06.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:06.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:06.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:06.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:06.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:06 smithi176 ceph-mon[34982]: Deploying daemon mds.h on smithi153 2024-09-10T22:50:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:07 smithi153 ceph-mon[34951]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:07.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:07.355 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:07 smithi120 ceph-mon[31247]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:07.548 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 22:50:07 smithi153 podman[64732]: 2024-09-10 22:50:07.523206075 +0000 UTC m=+0.361988638 container start 862be588a002a0ac3e91ca1a1086e6261381e7dd31541bd39e47c5edbb47e49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T22:50:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:07 smithi176 ceph-mon[34982]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:07.587 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:07.949 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 22:50:07 smithi153 ceph-mds[64751]: -- 172.21.15.153:0/1268363912 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560112ccf860 con 0x560112cad400 2024-09-10T22:50:07.949 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 22:50:07 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h[64747]: starting mds.h at 2024-09-10T22:50:07.949 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 22:50:07 smithi153 bash[64732]: 862be588a002a0ac3e91ca1a1086e6261381e7dd31541bd39e47c5edbb47e49f 2024-09-10T22:50:07.949 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 22:50:07 smithi153 systemd[1]: Started Ceph mds.h for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 -- 172.21.15.120:0/3040527291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84102da0 msgr2=0x7fbb84103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3040527291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84102da0 0x7fbb84103180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009900 tx=0x7fbb8002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 -- 172.21.15.120:0/3040527291 shutdown_connections 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3040527291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84109130 0x7fbb8410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3040527291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 0x7fbb84108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3040527291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84102da0 0x7fbb84103180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 -- 172.21.15.120:0/3040527291 >> 172.21.15.120:0/3040527291 conn(0x7fbb840fe810 msgr2=0x7fbb84100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 -- 172.21.15.120:0/3040527291 shutdown_connections 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.069+0000 7fbb8b832640 1 -- 172.21.15.120:0/3040527291 wait complete. 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 Processor -- start 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 -- start start 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84102da0 0x7fbb840754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 0x7fbb840759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112e30 con 0x7fbb84109130 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112cb0 con 0x7fbb84102da0 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb8b832640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112fb0 con 0x7fbb841087b0 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb89dcf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb89dcf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55532/0 (socket says 172.21.15.120:55532) 2024-09-10T22:50:08.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb89dcf640 1 -- 172.21.15.120:0/3871469203 learned_addr learned my addr 172.21.15.120:0/3871469203 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.070+0000 7fbb88dcd640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 0x7fbb840759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb895ce640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84102da0 0x7fbb840754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 -- 172.21.15.120:0/3871469203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 msgr2=0x7fbb840759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 0x7fbb840759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 -- 172.21.15.120:0/3871469203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84102da0 msgr2=0x7fbb840754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84102da0 0x7fbb840754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 -- 172.21.15.120:0/3871469203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841a77e0 con 0x7fbb84109130 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb89dcf640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00b970 tx=0x7fbb7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c019070 con 0x7fbb84109130 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb7c014440 con 0x7fbb84109130 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841a7ad0 con 0x7fbb84109130 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0075d0 con 0x7fbb84109130 2024-09-10T22:50:08.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.071+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841a8040 con 0x7fbb84109130 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.072+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84103200 con 0x7fbb84109130 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.073+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fbb7c0145f0 con 0x7fbb84109130 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.074+0000 7fbb7a7fc640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 0x7fbb700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.074+0000 7fbb895ce640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 0x7fbb700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.074+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fbb7c0e5380 con 0x7fbb84109130 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.074+0000 7fbb895ce640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 0x7fbb700b77a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb80031440 tx=0x7fbb800095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:08.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.077+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbb7c0ac6e0 con 0x7fbb84109130 2024-09-10T22:50:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.288+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb8402e5c0 con 0x7fbb84109130 2024-09-10T22:50:08.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.289+0000 7fbb7a7fc640 1 -- 172.21.15.120:0/3871469203 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7fbb7c0abfe0 con 0x7fbb84109130 2024-09-10T22:50:08.291 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:08.291 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":9,"btime":"2024-09-10T22:50:08:079338+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T22:50:08.291 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 9 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 msgr2=0x7fbb700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 0x7fbb700b77a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb80031440 tx=0x7fbb800095f0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 msgr2=0x7fbb84075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00b970 tx=0x7fbb7c00bd70 comp rx=0 tx=0).stop 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 shutdown_connections 2024-09-10T22:50:08.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb700b52e0 0x7fbb700b77a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb84109130 0x7fbb84075f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb841087b0 0x7fbb840759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 --2- 172.21.15.120:0/3871469203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb84102da0 0x7fbb840754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 >> 172.21.15.120:0/3871469203 conn(0x7fbb840fe810 msgr2=0x7fbb84100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 shutdown_connections 2024-09-10T22:50:08.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:08.292+0000 7fbb8b832640 1 -- 172.21.15.120:0/3871469203 wait complete. 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:08.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: Deploying daemon mds.i on smithi176 2024-09-10T22:50:08.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.153:6836/1833265650,v1:172.21.15.153:6837/1833265650] up:boot 2024-09-10T22:50:08.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: fsmap 8 up:standby 2024-09-10T22:50:08.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:50:08.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:08.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:08 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3871469203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: Deploying daemon mds.i on smithi176 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.153:6836/1833265650,v1:172.21.15.153:6837/1833265650] up:boot 2024-09-10T22:50:08.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: fsmap 8 up:standby 2024-09-10T22:50:08.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:50:08.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:08.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:08 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3871469203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:08.998 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: Deploying daemon mds.i on smithi176 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.153:6836/1833265650,v1:172.21.15.153:6837/1833265650] up:boot 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: fsmap 8 up:standby 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:08.999 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:08 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3871469203' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:09.573 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 systemd[1]: Starting Ceph mds.i for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:50:09.954 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 podman[65123]: 2024-09-10 22:50:09.691874427 +0000 UTC m=+0.112913449 container create b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T22:50:09.955 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 podman[65123]: 2024-09-10 22:50:09.595296842 +0000 UTC m=+0.016335834 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:09.955 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 podman[65123]: 2024-09-10 22:50:09.92750143 +0000 UTC m=+0.348540409 container init b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T22:50:09.955 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 podman[65123]: 2024-09-10 22:50:09.930146985 +0000 UTC m=+0.351185965 container start b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T22:50:10.323 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 ceph-mds[65142]: -- 172.21.15.176:0/2670199900 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfb6bffc20 con 0x55bfb6bdd400 2024-09-10T22:50:10.323 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: starting mds.i at 2024-09-10T22:50:10.323 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:10 smithi176 bash[65123]: b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 2024-09-10T22:50:10.323 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:10 smithi176 systemd[1]: Started Ceph mds.i for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:11.020 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:boot 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: fsmap 9 up:standby 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:11.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:11.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:11.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: Deploying daemon mds.g on smithi120 2024-09-10T22:50:11.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:11 smithi120 ceph-mon[31247]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:boot 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: fsmap 9 up:standby 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: Deploying daemon mds.g on smithi120 2024-09-10T22:50:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:11 smithi153 ceph-mon[34951]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:11.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:boot 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: fsmap 9 up:standby 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:11.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:11.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:11.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: Deploying daemon mds.g on smithi120 2024-09-10T22:50:11.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:11 smithi176 ceph-mon[34982]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:11.711 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/530246895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 msgr2=0x7ffa6410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/530246895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6410f960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ca0 tx=0x7ffa5402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/530246895 shutdown_connections 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/530246895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6410f960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/530246895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 0x7ffa64108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/530246895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/530246895 >> 172.21.15.120:0/530246895 conn(0x7ffa64078140 msgr2=0x7ffa640ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/530246895 shutdown_connections 2024-09-10T22:50:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.273+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/530246895 wait complete. 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 Processor -- start 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 -- start start 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 0x7ffa641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64112dd0 con 0x7ffa641090f0 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64112c50 con 0x7ffa64101270 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa6b2c6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64112f50 con 0x7ffa64108750 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55556/0 (socket says 172.21.15.120:55556) 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 -- 172.21.15.120:0/250228593 learned_addr learned my addr 172.21.15.120:0/250228593 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:12.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa68861640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 0x7ffa641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69062640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 -- 172.21.15.120:0/250228593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 msgr2=0x7ffa641997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 0x7ffa641997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 -- 172.21.15.120:0/250228593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 msgr2=0x7ffa64199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 -- 172.21.15.120:0/250228593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6419f310 con 0x7ffa641090f0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69062640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.274+0000 7ffa69863640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005cc0 tx=0x7ffa54032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.275+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54003c70 con 0x7ffa641090f0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.275+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6419f5a0 con 0x7ffa641090f0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.275+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa54003e10 con 0x7ffa641090f0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.275+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5402fc20 con 0x7ffa641090f0 2024-09-10T22:50:12.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.275+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa641a5d40 con 0x7ffa641090f0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.276+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa64101650 con 0x7ffa641090f0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.277+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ffa540046a0 con 0x7ffa641090f0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.278+0000 7ffa627fc640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.278+0000 7ffa69062640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.278+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ffa54106c90 con 0x7ffa641090f0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.278+0000 7ffa69062640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00baa0 tx=0x7ffa5c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:12.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.281+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffa540cdff0 con 0x7ffa641090f0 2024-09-10T22:50:12.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:12 smithi120 ceph-mon[31247]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:12.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.507+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa6410afa0 con 0x7ffa641090f0 2024-09-10T22:50:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.507+0000 7ffa627fc640 1 -- 172.21.15.120:0/250228593 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8061 (secure 0 0 0) 0x7ffa540cd8f0 con 0x7ffa641090f0 2024-09-10T22:50:12.509 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:12.509 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":10,"btime":"2024-09-10T22:50:10:081984+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T22:50:12.509 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 10 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 msgr2=0x7ffa4c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00baa0 tx=0x7ffa5c005d90 comp rx=0 tx=0).stop 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 msgr2=0x7ffa6419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005cc0 tx=0x7ffa54032fd0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 shutdown_connections 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa641090f0 0x7ffa6419eb40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa64108750 0x7ffa641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 --2- 172.21.15.120:0/250228593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa64101270 0x7ffa64199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:12.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 >> 172.21.15.120:0/250228593 conn(0x7ffa64078140 msgr2=0x7ffa64105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:12.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 shutdown_connections 2024-09-10T22:50:12.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:12.510+0000 7ffa6b2c6640 1 -- 172.21.15.120:0/250228593 wait complete. 2024-09-10T22:50:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:12 smithi176 ceph-mon[34982]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:12 smithi153 ceph-mon[34951]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:13.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/250228593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:13.510 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 22:50:13 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g[75099]: starting mds.g at 2024-09-10T22:50:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/250228593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:13.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/250228593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:13.946 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 22:50:13 smithi120 bash[75071]: 0c133ff96da924ecd8c0783cae8cb0b7b712f6dfeecf9b42cbbc57abed6934b7 2024-09-10T22:50:13.947 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 22:50:13 smithi120 systemd[1]: Started Ceph mds.g for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:14.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:14.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: Deploying daemon mds.a on smithi120 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.120:6838/987205566,v1:172.21.15.120:6839/987205566] up:boot 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: fsmap 10 up:standby 2024-09-10T22:50:14.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: Deploying daemon mds.a on smithi120 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:14.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.120:6838/987205566,v1:172.21.15.120:6839/987205566] up:boot 2024-09-10T22:50:14.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: fsmap 10 up:standby 2024-09-10T22:50:14.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: Deploying daemon mds.a on smithi120 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.120:6838/987205566,v1:172.21.15.120:6839/987205566] up:boot 2024-09-10T22:50:14.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: fsmap 10 up:standby 2024-09-10T22:50:14.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:50:15.246 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:15.540 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:15.662 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:15 smithi120 systemd[1]: Starting Ceph mds.a for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:50:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.010+0000 7f896bfa3640 1 -- 172.21.15.120:0/1774308961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 msgr2=0x7f89641020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.010+0000 7f896bfa3640 1 --2- 172.21.15.120:0/1774308961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641020b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8960009900 tx=0x7f896002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:16.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 -- 172.21.15.120:0/1774308961 shutdown_connections 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 --2- 172.21.15.120:0/1774308961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8964108060 0x7f896410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 --2- 172.21.15.120:0/1774308961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89641076e0 0x7f8964107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 --2- 172.21.15.120:0/1774308961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641020b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 -- 172.21.15.120:0/1774308961 >> 172.21.15.120:0/1774308961 conn(0x7f89640fd700 msgr2=0x7f89640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 -- 172.21.15.120:0/1774308961 shutdown_connections 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 -- 172.21.15.120:0/1774308961 wait complete. 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 Processor -- start 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.011+0000 7f896bfa3640 1 -- start start 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89641076e0 0x7f8964199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8964108060 0x7f896419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:16.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111d60 con 0x7f8964101cd0 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111be0 con 0x7f89641076e0 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111ee0 con 0x7f8964108060 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59722/0 (socket says 172.21.15.120:59722) 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 -- 172.21.15.120:0/3451407130 learned_addr learned my addr 172.21.15.120:0/3451407130 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896a540640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8964108060 0x7f896419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896953e640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89641076e0 0x7f8964199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 -- 172.21.15.120:0/3451407130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8964108060 msgr2=0x7f896419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:16.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8964108060 0x7f896419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 -- 172.21.15.120:0/3451407130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89641076e0 msgr2=0x7f8964199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89641076e0 0x7f8964199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 -- 172.21.15.120:0/3451407130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896419f490 con 0x7f8964101cd0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f8969d3f640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8960006010 tx=0x7f89600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896003f030 con 0x7f8964101cd0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8960003800 con 0x7f8964101cd0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896419f720 con 0x7f8964101cd0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960035070 con 0x7f8964101cd0 2024-09-10T22:50:16.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.012+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a5e40 con 0x7f8964101cd0 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.014+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8964102130 con 0x7f8964101cd0 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.015+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f8960031b10 con 0x7f8964101cd0 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.016+0000 7f895affd640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 0x7f89500b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.016+0000 7f896953e640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 0x7f89500b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.016+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f8960106350 con 0x7f8964101cd0 2024-09-10T22:50:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.016+0000 7f896953e640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 0x7f89500b7820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f89641073c0 tx=0x7f8954008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:16.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.019+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f89600cd6b0 con 0x7f8964101cd0 2024-09-10T22:50:16.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.240+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f896419a510 con 0x7f8964101cd0 2024-09-10T22:50:16.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.240+0000 7f895affd640 1 -- 172.21.15.120:0/3451407130 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8887 (secure 0 0 0) 0x7f89600ccfb0 con 0x7f8964101cd0 2024-09-10T22:50:16.242 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:16.242 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":11,"btime":"2024-09-10T22:50:14:218995+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T22:50:16.242 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 11 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 msgr2=0x7f89500b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 0x7f89500b7820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f89641073c0 tx=0x7f8954008040 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 msgr2=0x7f89641993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8960006010 tx=0x7f89600042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 shutdown_connections 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89500b5360 0x7f89500b7820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8964108060 0x7f896419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89641076e0 0x7f8964199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 --2- 172.21.15.120:0/3451407130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8964101cd0 0x7f89641993e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:16.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.243+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 >> 172.21.15.120:0/3451407130 conn(0x7f89640fd700 msgr2=0x7f896406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:16.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.244+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 shutdown_connections 2024-09-10T22:50:16.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:16.244+0000 7f896bfa3640 1 -- 172.21.15.120:0/3451407130 wait complete. 2024-09-10T22:50:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:16 smithi176 ceph-mon[34982]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:16.638 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:16 smithi120 ceph-mon[31247]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:16.639 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 podman[75486]: 2024-09-10 22:50:16.109280267 +0000 UTC m=+0.015918331 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:16.639 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 podman[75486]: 2024-09-10 22:50:16.241585555 +0000 UTC m=+0.148223619 container create a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:50:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:16 smithi153 ceph-mon[34951]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 podman[75486]: 2024-09-10 22:50:16.922502529 +0000 UTC m=+0.829140593 container init a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 podman[75486]: 2024-09-10 22:50:16.925752025 +0000 UTC m=+0.832390090 container start a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 ceph-mds[75522]: -- 172.21.15.120:0/3473386067 <== mon.0 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563392415860 con 0x5633923f3400 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a[75513]: starting mds.a at 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:17 smithi120 bash[75486]: a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb 2024-09-10T22:50:17.184 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 22:50:17 smithi120 systemd[1]: Started Ceph mds.a for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3451407130' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:17 smithi120 ceph-mon[31247]: Deploying daemon mds.b on smithi153 2024-09-10T22:50:17.516 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3451407130' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:17.517 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:17 smithi153 ceph-mon[34951]: Deploying daemon mds.b on smithi153 2024-09-10T22:50:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3451407130' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:50:17.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:17.575 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:17 smithi176 ceph-mon[34982]: Deploying daemon mds.b on smithi153 2024-09-10T22:50:18.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:18 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.120:6840/420929141,v1:172.21.15.120:6841/420929141] up:boot 2024-09-10T22:50:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:18 smithi153 ceph-mon[34951]: fsmap 11 up:standby 2024-09-10T22:50:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:18 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:50:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:18 smithi153 ceph-mon[34951]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:18 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.120:6840/420929141,v1:172.21.15.120:6841/420929141] up:boot 2024-09-10T22:50:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:18 smithi176 ceph-mon[34982]: fsmap 11 up:standby 2024-09-10T22:50:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:18 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:50:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:18 smithi176 ceph-mon[34982]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:18.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:18 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.120:6840/420929141,v1:172.21.15.120:6841/420929141] up:boot 2024-09-10T22:50:18.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:18 smithi120 ceph-mon[31247]: fsmap 11 up:standby 2024-09-10T22:50:18.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:18 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:50:18.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:18 smithi120 ceph-mon[31247]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:19.186 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:19.199 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 systemd[1]: Starting Ceph mds.b for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:50:19.451 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:19.499 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 podman[65032]: 2024-09-10 22:50:19.116393356 +0000 UTC m=+0.016393784 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:50:19.499 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 podman[65032]: 2024-09-10 22:50:19.25435352 +0000 UTC m=+0.154353940 container create fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T22:50:19.499 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 podman[65032]: 2024-09-10 22:50:19.49787584 +0000 UTC m=+0.397876262 container init fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.908+0000 7f8d402e2640 1 -- 172.21.15.120:0/3961081417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d381076a0 msgr2=0x7f8d38107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.908+0000 7f8d402e2640 1 --2- 172.21.15.120:0/3961081417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d381076a0 0x7f8d38107ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009900 tx=0x7f8d2802a410 comp rx=0 tx=0).stop 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 -- 172.21.15.120:0/3961081417 shutdown_connections 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 --2- 172.21.15.120:0/3961081417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 0x7f8d3810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 --2- 172.21.15.120:0/3961081417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d381076a0 0x7f8d38107ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 --2- 172.21.15.120:0/3961081417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d38101c90 0x7f8d38102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:19.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 -- 172.21.15.120:0/3961081417 >> 172.21.15.120:0/3961081417 conn(0x7f8d380fd700 msgr2=0x7f8d380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 -- 172.21.15.120:0/3961081417 shutdown_connections 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 -- 172.21.15.120:0/3961081417 wait complete. 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 Processor -- start 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.909+0000 7f8d402e2640 1 -- start start 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d381076a0 0x7f8d38199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 0x7f8d3819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38111ca0 con 0x7f8d38101c90 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38111b20 con 0x7f8d381076a0 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38111e20 con 0x7f8d38108020 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59750/0 (socket says 172.21.15.120:59750) 2024-09-10T22:50:19.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 -- 172.21.15.120:0/94073766 learned_addr learned my addr 172.21.15.120:0/94073766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3d87d640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d381076a0 0x7f8d38199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e87f640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 0x7f8d3819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 -- 172.21.15.120:0/94073766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 msgr2=0x7f8d3819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 0x7f8d3819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 -- 172.21.15.120:0/94073766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d381076a0 msgr2=0x7f8d38199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d381076a0 0x7f8d38199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 -- 172.21.15.120:0/94073766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3819f3f0 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d3e07e640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34017030 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3819f6e0 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34004020 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340047a0 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.910+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d381a5f70 con 0x7f8d38101c90 2024-09-10T22:50:19.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.912+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d38102070 con 0x7f8d38101c90 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.913+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f8d34009a30 con 0x7f8d38101c90 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.914+0000 7f8d2f7fe640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 0x7f8d240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.914+0000 7f8d3d87d640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 0x7f8d240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.914+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f8d340e5740 con 0x7f8d38101c90 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.914+0000 7f8d3d87d640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 0x7f8d240b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d28031440 tx=0x7f8d28005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:19.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:19.917+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d340acaa0 con 0x7f8d38101c90 2024-09-10T22:50:19.949 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 podman[65032]: 2024-09-10 22:50:19.501824848 +0000 UTC m=+0.401825267 container start fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T22:50:19.949 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 ceph-mds[65051]: -- 172.21.15.153:0/3272204843 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a0d8f5860 con 0x556a0d8d2800 2024-09-10T22:50:19.949 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b[65047]: starting mds.b at 2024-09-10T22:50:19.949 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 bash[65032]: fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d 2024-09-10T22:50:19.949 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 22:50:19 smithi153 systemd[1]: Started Ceph mds.b for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:50:20.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.141+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d381a6140 con 0x7f8d38101c90 2024-09-10T22:50:20.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.142+0000 7f8d2f7fe640 1 -- 172.21.15.120:0/94073766 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7f8d340ac3a0 con 0x7f8d38101c90 2024-09-10T22:50:20.143 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:20.143 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":13,"btime":"2024-09-10T22:50:19:580750+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T22:50:20.144 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 13 2024-09-10T22:50:20.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 msgr2=0x7f8d240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 0x7f8d240b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d28031440 tx=0x7f8d28005b50 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 msgr2=0x7f8d38199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400de10 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 shutdown_connections 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d240b52e0 0x7f8d240b77a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d38108020 0x7f8d3819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d381076a0 0x7f8d38199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 --2- 172.21.15.120:0/94073766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d38101c90 0x7f8d38199340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 >> 172.21.15.120:0/94073766 conn(0x7f8d380fd700 msgr2=0x7f8d38104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 shutdown_connections 2024-09-10T22:50:20.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:20.145+0000 7f8d402e2640 1 -- 172.21.15.120:0/94073766 wait complete. 2024-09-10T22:50:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.153:6838/2945283653,v1:172.21.15.153:6839/2945283653] up:boot 2024-09-10T22:50:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: fsmap 12 up:standby 2024-09-10T22:50:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/94073766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:20.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:20 smithi153 ceph-mon[34951]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:20.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.153:6838/2945283653,v1:172.21.15.153:6839/2945283653] up:boot 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: fsmap 12 up:standby 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:20.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/94073766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:20.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:20 smithi176 ceph-mon[34982]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:20.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.153:6838/2945283653,v1:172.21.15.153:6839/2945283653] up:boot 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: fsmap 12 up:standby 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/94073766' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:20.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:20 smithi120 ceph-mon[31247]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:21.549 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:21.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 -- 172.21.15.120:0/726978860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 msgr2=0x7feee8107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 --2- 172.21.15.120:0/726978860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee8107b60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feed0009900 tx=0x7feed002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 -- 172.21.15.120:0/726978860 shutdown_connections 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 --2- 172.21.15.120:0/726978860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee81080a0 0x7feee810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 --2- 172.21.15.120:0/726978860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee8107b60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.968+0000 7feeed269640 1 --2- 172.21.15.120:0/726978860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee8101d10 0x7feee81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- 172.21.15.120:0/726978860 >> 172.21.15.120:0/726978860 conn(0x7feee80fd7c0 msgr2=0x7feee80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- 172.21.15.120:0/726978860 shutdown_connections 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- 172.21.15.120:0/726978860 wait complete. 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 Processor -- start 2024-09-10T22:50:21.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- start start 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 0x7feee8199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee81080a0 0x7feee819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feee8111d20 con 0x7feee8107720 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feee8111ba0 con 0x7feee8101d10 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feeed269640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feee8111ea0 con 0x7feee81080a0 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feedffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feedffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59766/0 (socket says 172.21.15.120:59766) 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.969+0000 7feedffff640 1 -- 172.21.15.120:0/2492806416 learned_addr learned my addr 172.21.15.120:0/2492806416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee77fe640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee81080a0 0x7feee819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:21.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 -- 172.21.15.120:0/2492806416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee81080a0 msgr2=0x7feee819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee81080a0 0x7feee819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee6ffd640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 0x7feee8199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 -- 172.21.15.120:0/2492806416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 msgr2=0x7feee8199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 0x7feee8199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 -- 172.21.15.120:0/2492806416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee819f410 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee6ffd640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 0x7feee8199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feedffff640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed0006010 tx=0x7feed00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed003f030 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed0003800 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee819f6a0 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0035070 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.970+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a5dc0 con 0x7feee8107720 2024-09-10T22:50:21.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.971+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee81020f0 con 0x7feee8107720 2024-09-10T22:50:21.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.973+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7feed0031b10 con 0x7feee8107720 2024-09-10T22:50:21.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.973+0000 7feee4ff9640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 0x7feed40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:21.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.973+0000 7feee6ffd640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 0x7feed40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:21.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.974+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7feed0106300 con 0x7feee8107720 2024-09-10T22:50:21.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.974+0000 7feee6ffd640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 0x7feed40b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feed80058c0 tx=0x7feed8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:21.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:21.976+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7feed00cd660 con 0x7feee8107720 2024-09-10T22:50:22.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.186+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feee81a5ff0 con 0x7feee8107720 2024-09-10T22:50:22.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.186+0000 7feee4ff9640 1 -- 172.21.15.120:0/2492806416 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7feed00ccf60 con 0x7feee8107720 2024-09-10T22:50:22.188 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout:[ 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout: "poolnum": 1, 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout: "poolname": ".mgr" 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:50:22.189 INFO:teuthology.orchestra.run.smithi120.stdout:] 2024-09-10T22:50:22.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 msgr2=0x7feed40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 0x7feed40b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feed80058c0 tx=0x7feed8005d30 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 msgr2=0x7feee81998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed0006010 tx=0x7feed00042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 shutdown_connections 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed40b52e0 0x7feed40b77a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee81080a0 0x7feee819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee8107720 0x7feee81998a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 --2- 172.21.15.120:0/2492806416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee8101d10 0x7feee8199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 >> 172.21.15.120:0/2492806416 conn(0x7feee80fd7c0 msgr2=0x7feee81006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 shutdown_connections 2024-09-10T22:50:22.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:22.190+0000 7feeed269640 1 -- 172.21.15.120:0/2492806416 wait complete. 2024-09-10T22:50:22.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:22.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2492806416' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:50:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:22.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2492806416' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2492806416' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:50:22.880 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:50:22.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:50:23.156 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.696+0000 7f4912c8d640 1 -- 172.21.15.120:0/1553582500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c1080a0 msgr2=0x7f490c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.696+0000 7f4912c8d640 1 --2- 172.21.15.120:0/1553582500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c1080a0 0x7f490c10e8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4904009ca0 tx=0x7f490402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- 172.21.15.120:0/1553582500 shutdown_connections 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- 172.21.15.120:0/1553582500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c1080a0 0x7f490c10e8a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- 172.21.15.120:0/1553582500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 0x7f490c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- 172.21.15.120:0/1553582500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c101d10 0x7f490c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- 172.21.15.120:0/1553582500 >> 172.21.15.120:0/1553582500 conn(0x7f490c0fd7c0 msgr2=0x7f490c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- 172.21.15.120:0/1553582500 shutdown_connections 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- 172.21.15.120:0/1553582500 wait complete. 2024-09-10T22:50:23.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 Processor -- start 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- start start 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 0x7f490c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 0x7f490c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111d20 con 0x7f490c101d10 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111ba0 con 0x7f490c1080a0 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.697+0000 7f4912c8d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f490c111ea0 con 0x7f490c107720 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41154/0 (socket says 172.21.15.120:41154) 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 -- 172.21.15.120:0/4204460672 learned_addr learned my addr 172.21.15.120:0/4204460672 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f490bfff640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 0x7f490c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:23.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f491122a640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 0x7f490c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 -- 172.21.15.120:0/4204460672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 msgr2=0x7f490c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 0x7f490c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 -- 172.21.15.120:0/4204460672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 msgr2=0x7f490c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 0x7f490c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 -- 172.21.15.120:0/4204460672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a5410 con 0x7f490c101d10 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f491122a640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 0x7f490c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4910a29640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f490000c920 tx=0x7f490000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900007c70 con 0x7f490c101d10 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900007e10 con 0x7f490c101d10 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a5700 con 0x7f490c101d10 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.698+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900018820 con 0x7f490c101d10 2024-09-10T22:50:23.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.699+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a5c90 con 0x7f490c101d10 2024-09-10T22:50:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.699+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c1020f0 con 0x7f490c101d10 2024-09-10T22:50:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.701+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f490000f450 con 0x7f490c101d10 2024-09-10T22:50:23.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.701+0000 7f4909ffb640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 0x7f48fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.701+0000 7f490bfff640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 0x7f48fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.701+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f49000e57f0 con 0x7f490c101d10 2024-09-10T22:50:23.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.702+0000 7f490bfff640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 0x7f48fc0b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f490c06bef0 tx=0x7f48f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:23.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.705+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4900014030 con 0x7f490c101d10 2024-09-10T22:50:23.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.896+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f490c06b700 con 0x7f490c101d10 2024-09-10T22:50:23.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.980+0000 7f4909ffb640 1 -- 172.21.15.120:0/4204460672 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f49000acb50 con 0x7f490c101d10 2024-09-10T22:50:23.981 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' created 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.982+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 msgr2=0x7f48fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.982+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 0x7f48fc0b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f490c06bef0 tx=0x7f48f800b410 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.982+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 msgr2=0x7f490c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.982+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f490000c920 tx=0x7f490000cde0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 shutdown_connections 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b5360 0x7f48fc0b7820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f490c1080a0 0x7f490c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f490c107720 0x7f490c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 --2- 172.21.15.120:0/4204460672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f490c101d10 0x7f490c19a010 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:23.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 >> 172.21.15.120:0/4204460672 conn(0x7f490c0fd7c0 msgr2=0x7f490c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:23.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 shutdown_connections 2024-09-10T22:50:23.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:23.983+0000 7f4912c8d640 1 -- 172.21.15.120:0/4204460672 wait complete. 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:24.175 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:23 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:23 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:50:24.825 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:50:25.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.217 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:25.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:50:25.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:50:25.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:25.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4204460672' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:50:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:50:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 -- 172.21.15.120:0/1963152431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 msgr2=0x7f99a81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 --2- 172.21.15.120:0/1963152431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a81031a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c02a410 comp rx=0 tx=0).stop 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 -- 172.21.15.120:0/1963152431 shutdown_connections 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 --2- 172.21.15.120:0/1963152431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a8109150 0x7f99a810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 --2- 172.21.15.120:0/1963152431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a81087d0 0x7f99a8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 --2- 172.21.15.120:0/1963152431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a81031a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 -- 172.21.15.120:0/1963152431 >> 172.21.15.120:0/1963152431 conn(0x7f99a80fe830 msgr2=0x7f99a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 -- 172.21.15.120:0/1963152431 shutdown_connections 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.722+0000 7f99aece3640 1 -- 172.21.15.120:0/1963152431 wait complete. 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 Processor -- start 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 -- start start 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 0x7f99a819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 0x7f99a81a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112dd0 con 0x7f99a8102dc0 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112c50 con 0x7f99a81087d0 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aece3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112f50 con 0x7f99a8109150 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41178/0 (socket says 172.21.15.120:41178) 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 -- 172.21.15.120:0/3409483596 learned_addr learned my addr 172.21.15.120:0/3409483596 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99ad280640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 0x7f99a81a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99a7fff640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 0x7f99a819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 -- 172.21.15.120:0/3409483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 msgr2=0x7f99a81a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 0x7f99a81a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 -- 172.21.15.120:0/3409483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 msgr2=0x7f99a819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 0x7f99a819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99aca7f640 1 -- 172.21.15.120:0/3409483596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a1520 con 0x7f99a8102dc0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99ad280640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 0x7f99a81a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.723+0000 7f99a7fff640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 0x7f99a819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99aca7f640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c03f030 con 0x7f99a8102dc0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f999c003800 con 0x7f99a8102dc0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a17b0 con 0x7f99a8102dc0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c035070 con 0x7f99a8102dc0 2024-09-10T22:50:25.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.724+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a7ed0 con 0x7f99a8102dc0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.725+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a81031a0 con 0x7f99a8102dc0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.726+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f999c031b10 con 0x7f99a8102dc0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.727+0000 7f99a5ffb640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 0x7f99980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.727+0000 7f99a7fff640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 0x7f99980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.728+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f999c1062f0 con 0x7f99a8102dc0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.728+0000 7f99a7fff640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 0x7f99980b7820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99a8108060 tx=0x7f9994005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:25.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.732+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f999c0cd440 con 0x7f99a8102dc0 2024-09-10T22:50:25.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.932+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f99a802e5c0 con 0x7f99a8102dc0 2024-09-10T22:50:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.987+0000 7f99a5ffb640 1 -- 172.21.15.120:0/3409483596 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f999c0ccd40 con 0x7f99a8102dc0 2024-09-10T22:50:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' created 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.989+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 msgr2=0x7f99980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.989+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 0x7f99980b7820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99a8108060 tx=0x7f9994005d90 comp rx=0 tx=0).stop 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.989+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 msgr2=0x7f99a819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.989+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 shutdown_connections 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99980b5360 0x7f99980b7820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99a8109150 0x7f99a81a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99a81087d0 0x7f99a819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 --2- 172.21.15.120:0/3409483596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99a8102dc0 0x7f99a819b470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:25.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.990+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 >> 172.21.15.120:0/3409483596 conn(0x7f99a80fe830 msgr2=0x7f99a8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:25.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.991+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 shutdown_connections 2024-09-10T22:50:25.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:25.991+0000 7f99aece3640 1 -- 172.21.15.120:0/3409483596 wait complete. 2024-09-10T22:50:26.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:25 smithi120 ceph-mon[31247]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:50:26.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:25 smithi120 ceph-mon[31247]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:50:26.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:50:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:25 smithi176 ceph-mon[34982]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:50:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:25 smithi176 ceph-mon[34982]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:50:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:50:26.350 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:25 smithi153 ceph-mon[34951]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:50:26.350 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:25 smithi153 ceph-mon[34951]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:50:26.350 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:50:26.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:50:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: pgmap v279: 97 pgs: 4 creating+activating, 11 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:27.152 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.153 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:27 smithi176 ceph-mon[34982]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: pgmap v279: 97 pgs: 4 creating+activating, 11 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:27.322 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:27.323 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.323 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:27 smithi120 ceph-mon[31247]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:50:27.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 -- 172.21.15.120:0/1708900257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61701076c0 msgr2=0x7f6170107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 --2- 172.21.15.120:0/1708900257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61701076c0 0x7f6170107b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6164009900 tx=0x7f616402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:27.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 -- 172.21.15.120:0/1708900257 shutdown_connections 2024-09-10T22:50:27.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 --2- 172.21.15.120:0/1708900257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6170108040 0x7f617010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 --2- 172.21.15.120:0/1708900257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61701076c0 0x7f6170107b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 --2- 172.21.15.120:0/1708900257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170101cb0 0x7f6170102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 -- 172.21.15.120:0/1708900257 >> 172.21.15.120:0/1708900257 conn(0x7f61700fd700 msgr2=0x7f61700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 -- 172.21.15.120:0/1708900257 shutdown_connections 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.401+0000 7f617800f640 1 -- 172.21.15.120:0/1708900257 wait complete. 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 Processor -- start 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 -- start start 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61701076c0 0x7f61701997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170108040 0x7f617019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6170111d00 con 0x7f6170101cb0 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6170111b80 con 0x7f61701076c0 2024-09-10T22:50:27.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f617800f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6170111e80 con 0x7f6170108040 2024-09-10T22:50:27.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f6175dab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f6175dab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41190/0 (socket says 172.21.15.120:41190) 2024-09-10T22:50:27.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f6175dab640 1 -- 172.21.15.120:0/3722892403 learned_addr learned my addr 172.21.15.120:0/3722892403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:27.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f61755aa640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61701076c0 0x7f61701997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.402+0000 7f61765ac640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170108040 0x7f617019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 -- 172.21.15.120:0/3722892403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170108040 msgr2=0x7f617019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170108040 0x7f617019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 -- 172.21.15.120:0/3722892403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61701076c0 msgr2=0x7f61701997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61701076c0 0x7f61701997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 -- 172.21.15.120:0/3722892403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617019f330 con 0x7f6170101cb0 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6175dab640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f616c00d770 tx=0x7f616c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c017070 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616c0054d0 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617019f620 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c004c30 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.403+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a5ce0 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.404+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6170102090 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.405+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f616c004dd0 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.406+0000 7f6162ffd640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 0x7f615c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.406+0000 7f61755aa640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 0x7f615c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.406+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f616c0e5850 con 0x7f6170101cb0 2024-09-10T22:50:27.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.407+0000 7f61755aa640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 0x7f615c0b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6164031440 tx=0x7f6164005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.409+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f616c0ac270 con 0x7f6170101cb0 2024-09-10T22:50:27.462 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3409483596' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:50:27.462 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: pgmap v279: 97 pgs: 4 creating+activating, 11 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:50:27.463 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:27 smithi153 ceph-mon[34951]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:50:27.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:27.619+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f617019a3b0 con 0x7f6170101cb0 2024-09-10T22:50:28.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.014+0000 7f6162ffd640 1 -- 172.21.15.120:0/3722892403 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f616c0b0ca0 con 0x7f6170101cb0 2024-09-10T22:50:28.016 INFO:teuthology.orchestra.run.smithi120.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 msgr2=0x7f615c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 0x7f615c0b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6164031440 tx=0x7f6164005b50 comp rx=0 tx=0).stop 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 msgr2=0x7f6170199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f616c00d770 tx=0x7f616c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 shutdown_connections 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f615c0b52e0 0x7f615c0b77a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6170108040 0x7f617019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61701076c0 0x7f61701997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 --2- 172.21.15.120:0/3722892403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6170101cb0 0x7f6170199280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 >> 172.21.15.120:0/3722892403 conn(0x7f61700fd700 msgr2=0x7f617006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 shutdown_connections 2024-09-10T22:50:28.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:28.016+0000 7f617800f640 1 -- 172.21.15.120:0/3722892403 wait complete. 2024-09-10T22:50:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:50:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:28 smithi176 ceph-mon[34982]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T22:50:28.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:50:28.357 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:28 smithi120 ceph-mon[31247]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T22:50:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:50:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:28 smithi153 ceph-mon[34951]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T22:50:28.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:50:28.948 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:29.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:50:29.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:50:29.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T22:50:29.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T22:50:29.265 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: pgmap v282: 97 pgs: 4 creating+activating, 19 creating+peering, 21 active+clean, 53 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:29.266 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:active 2024-09-10T22:50:29.266 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:29 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:29.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:50:29.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:50:29.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T22:50:29.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T22:50:29.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: pgmap v282: 97 pgs: 4 creating+activating, 19 creating+peering, 21 active+clean, 53 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:29.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:active 2024-09-10T22:50:29.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:29 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3722892403' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: pgmap v282: 97 pgs: 4 creating+activating, 19 creating+peering, 21 active+clean, 53 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] up:active 2024-09-10T22:50:29.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:29 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:29.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 -- 172.21.15.120:0/1798214089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93810d100 msgr2=0x7fe93810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 --2- 172.21.15.120:0/1798214089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93810d100 0x7fe93810f560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 -- 172.21.15.120:0/1798214089 shutdown_connections 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 --2- 172.21.15.120:0/1798214089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93810d100 0x7fe93810f560 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 --2- 172.21.15.120:0/1798214089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9381045c0 0x7fe93810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.464+0000 7fe93f47e640 1 --2- 172.21.15.120:0/1798214089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe938103ca0 0x7fe938104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- 172.21.15.120:0/1798214089 >> 172.21.15.120:0/1798214089 conn(0x7fe9380fd700 msgr2=0x7fe9380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- 172.21.15.120:0/1798214089 shutdown_connections 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- 172.21.15.120:0/1798214089 wait complete. 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 Processor -- start 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- start start 2024-09-10T22:50:29.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9381045c0 0x7fe938199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 0x7fe93819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112e50 con 0x7fe938103ca0 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112cd0 con 0x7fe93810d100 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93f47e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112fd0 con 0x7fe9381045c0 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.465+0000 7fe93d21a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41206/0 (socket says 172.21.15.120:41206) 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 -- 172.21.15.120:0/631457382 learned_addr learned my addr 172.21.15.120:0/631457382 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93da1b640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 0x7fe93819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 -- 172.21.15.120:0/631457382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9381045c0 msgr2=0x7fe938199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9381045c0 0x7fe938199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 -- 172.21.15.120:0/631457382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 msgr2=0x7fe93819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:29.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 0x7fe93819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 -- 172.21.15.120:0/631457382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93819f470 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93da1b640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 0x7fe93819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93d21a640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe92800ba50 tx=0x7fe92800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928019070 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93819f760 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92800cbe0 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9280075d0 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.466+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9381a5ea0 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.467+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938104100 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.468+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fe928007830 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.469+0000 7fe9367fc640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.469+0000 7fe93ca19640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.469+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fe9280e57c0 con 0x7fe938103ca0 2024-09-10T22:50:29.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.470+0000 7fe93ca19640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe93810f840 tx=0x7fe92c006d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:29.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.472+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe9280ac7d0 con 0x7fe938103ca0 2024-09-10T22:50:29.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.648+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe93819a4f0 con 0x7fe938103ca0 2024-09-10T22:50:29.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.650+0000 7fe9367fc640 1 -- 172.21.15.120:0/631457382 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22360 (secure 0 0 0) 0x7fe9280ac0d0 con 0x7fe938103ca0 2024-09-10T22:50:29.652 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:29.653 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":75,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:50:27.621440+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:50:29.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 msgr2=0x7fe9200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:29.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe93810f840 tx=0x7fe92c006d30 comp rx=0 tx=0).stop 2024-09-10T22:50:29.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 msgr2=0x7fe9381993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe92800ba50 tx=0x7fe92800bf10 comp rx=0 tx=0).stop 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 shutdown_connections 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe93810d100 0x7fe93819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9381045c0 0x7fe938199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 --2- 172.21.15.120:0/631457382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe938103ca0 0x7fe9381993c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 >> 172.21.15.120:0/631457382 conn(0x7fe9380fd700 msgr2=0x7fe9380fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 shutdown_connections 2024-09-10T22:50:29.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:29.652+0000 7fe93f47e640 1 -- 172.21.15.120:0/631457382 wait complete. 2024-09-10T22:50:30.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:30 smithi120 ceph-mon[31247]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:50:30.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:30 smithi120 ceph-mon[31247]: Cluster is now healthy 2024-09-10T22:50:30.301 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:30 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/631457382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:30 smithi176 ceph-mon[34982]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:50:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:30 smithi176 ceph-mon[34982]: Cluster is now healthy 2024-09-10T22:50:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:30 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/631457382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:30.323 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 22:50:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: 2024-09-10T22:50:30.092+0000 7fe871a3e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:50:30.328 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:50:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:30 smithi153 ceph-mon[34951]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:50:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:30 smithi153 ceph-mon[34951]: Cluster is now healthy 2024-09-10T22:50:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:30 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/631457382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:31.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- 172.21.15.120:0/3396817336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c104570 msgr2=0x7f392c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 --2- 172.21.15.120:0/3396817336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c104570 0x7f392c10cb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3914009900 tx=0x7f391402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- 172.21.15.120:0/3396817336 shutdown_connections 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 --2- 172.21.15.120:0/3396817336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c10d120 0x7f392c10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 --2- 172.21.15.120:0/3396817336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c104570 0x7f392c10cb50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 --2- 172.21.15.120:0/3396817336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c103b70 0x7f392c103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- 172.21.15.120:0/3396817336 >> 172.21.15.120:0/3396817336 conn(0x7f392c0fd700 msgr2=0x7f392c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- 172.21.15.120:0/3396817336 shutdown_connections 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- 172.21.15.120:0/3396817336 wait complete. 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 Processor -- start 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.068+0000 7f393266b640 1 -- start start 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c103b70 0x7f392c1019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 0x7f392c101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f392c112d80 con 0x7f392c10d120 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f392c112c00 con 0x7f392c103b70 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f392c112f00 con 0x7f392c104570 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41228/0 (socket says 172.21.15.120:41228) 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 -- 172.21.15.120:0/2731329926 learned_addr learned my addr 172.21.15.120:0/2731329926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3923fff640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 0x7f392c101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f392bfff640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c103b70 0x7f392c1019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 -- 172.21.15.120:0/2731329926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 msgr2=0x7f392c101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 0x7f392c101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 -- 172.21.15.120:0/2731329926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c103b70 msgr2=0x7f392c1019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c103b70 0x7f392c1019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 -- 172.21.15.120:0/2731329926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c1a7650 con 0x7f392c10d120 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3923fff640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 0x7f392c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:31.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3930c08640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f392400cce0 tx=0x7f392400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392400f450 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392400faa0 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c1a7940 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924018b70 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.069+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c1a7e50 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.070+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392c103f50 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.072+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f392400f5f0 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.073+0000 7f3929ffb640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 0x7f39180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.073+0000 7f392bfff640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 0x7f39180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.073+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f39240e62c0 con 0x7f392c10d120 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.073+0000 7f392bfff640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 0x7f39180b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f391c005fd0 tx=0x7f391c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.076+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3924014030 con 0x7f392c10d120 2024-09-10T22:50:31.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.253+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f392c02e5c0 con 0x7f392c10d120 2024-09-10T22:50:31.255 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:31 smithi120 ceph-mon[31247]: pgmap v283: 97 pgs: 16 creating+peering, 64 active+clean, 17 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s wr, 4 op/s 2024-09-10T22:50:31.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.254+0000 7f3929ffb640 1 -- 172.21.15.120:0/2731329926 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22360 (secure 0 0 0) 0x7f39240ad3c0 con 0x7f392c10d120 2024-09-10T22:50:31.257 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:31.257 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":75,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:50:27.621440+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 msgr2=0x7f39180b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 0x7f39180b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f391c005fd0 tx=0x7f391c005850 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 msgr2=0x7f392c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f392400cce0 tx=0x7f392400a620 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 shutdown_connections 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f39180b5360 0x7f39180b7820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f392c10d120 0x7f392c102470 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f392c104570 0x7f392c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 --2- 172.21.15.120:0/2731329926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f392c103b70 0x7f392c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.256+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 >> 172.21.15.120:0/2731329926 conn(0x7f392c0fd700 msgr2=0x7f392c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.257+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 shutdown_connections 2024-09-10T22:50:31.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:31.257+0000 7f393266b640 1 -- 172.21.15.120:0/2731329926 wait complete. 2024-09-10T22:50:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:31 smithi176 ceph-mon[34982]: pgmap v283: 97 pgs: 16 creating+peering, 64 active+clean, 17 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s wr, 4 op/s 2024-09-10T22:50:31.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:31 smithi153 ceph-mon[34951]: pgmap v283: 97 pgs: 16 creating+peering, 64 active+clean, 17 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s wr, 4 op/s 2024-09-10T22:50:31.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:50:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:32.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:32 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2731329926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:32.340 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:32 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2731329926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:32.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:32 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2731329926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- 172.21.15.120:0/3625754323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 msgr2=0x7f3c1010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:32.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/3625754323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1010f950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009ca0 tx=0x7f3c0802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- 172.21.15.120:0/3625754323 shutdown_connections 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/3625754323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1010f950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/3625754323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 0x7f3c10108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/3625754323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 0x7f3c101031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- 172.21.15.120:0/3625754323 >> 172.21.15.120:0/3625754323 conn(0x7f3c100fe830 msgr2=0x7f3c10100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- 172.21.15.120:0/3625754323 shutdown_connections 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- 172.21.15.120:0/3625754323 wait complete. 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 Processor -- start 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- start start 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 0x7f3c101992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 0x7f3c10199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:32.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112e10 con 0x7f3c10109150 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.676+0000 7f3c16bdc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112c90 con 0x7f3c10102dc0 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c16bdc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112f90 con 0x7f3c101087d0 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41250/0 (socket says 172.21.15.120:41250) 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 -- 172.21.15.120:0/2354436089 learned_addr learned my addr 172.21.15.120:0/2354436089 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 0x7f3c10199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c14978640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 0x7f3c101992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 -- 172.21.15.120:0/2354436089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 msgr2=0x7f3c10199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 0x7f3c10199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 -- 172.21.15.120:0/2354436089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 msgr2=0x7f3c101992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:32.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 0x7f3c101992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 -- 172.21.15.120:0/2354436089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1019f370 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c15179640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005cc0 tx=0x7f3c0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08003bf0 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1019f600 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08003d90 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.677+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802fb20 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.678+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a5da0 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.679+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c101031a0 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.680+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f3c08004580 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.681+0000 7f3c0dffb640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 0x7f3c000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.681+0000 7f3c14978640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 0x7f3c000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.681+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3c08106ff0 con 0x7f3c10109150 2024-09-10T22:50:32.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.681+0000 7f3c14978640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 0x7f3c000b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005fd0 tx=0x7f3c04005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:32.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.684+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3c080cdf80 con 0x7f3c10109150 2024-09-10T22:50:32.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:32.895+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3c1002e5c0 con 0x7f3c10109150 2024-09-10T22:50:33.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.022+0000 7f3c0dffb640 1 -- 172.21.15.120:0/2354436089 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f3c080cd880 con 0x7f3c10109150 2024-09-10T22:50:33.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 msgr2=0x7f3c000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 0x7f3c000b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005fd0 tx=0x7f3c04005850 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 msgr2=0x7f3c1019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005cc0 tx=0x7f3c0802f380 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 shutdown_connections 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c000b52e0 0x7f3c000b77a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c10109150 0x7f3c1019eba0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c101087d0 0x7f3c10199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 --2- 172.21.15.120:0/2354436089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c10102dc0 0x7f3c101992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 >> 172.21.15.120:0/2354436089 conn(0x7f3c100fe830 msgr2=0x7f3c10100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 shutdown_connections 2024-09-10T22:50:33.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:33.024+0000 7f3c16bdc640 1 -- 172.21.15.120:0/2354436089 wait complete. 2024-09-10T22:50:33.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:33 smithi120 ceph-mon[31247]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:50:33.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:33 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:50:33.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:33 smithi176 ceph-mon[34982]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:50:33.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:33 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:50:33.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:33 smithi153 ceph-mon[34951]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:50:33.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:33 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:50:33.686 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T22:50:33.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T22:50:33.930 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:34.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:34 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:50:34.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:34 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:34.345 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:34 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:50:34.346 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:34 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:34.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.425+0000 7f58a8463640 1 -- 172.21.15.120:0/1674373682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 msgr2=0x7f58a0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:34.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.425+0000 7f58a8463640 1 --2- 172.21.15.120:0/1674373682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a0107ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5890009900 tx=0x7f589002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:34.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- 172.21.15.120:0/1674373682 shutdown_connections 2024-09-10T22:50:34.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- 172.21.15.120:0/1674373682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:34.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- 172.21.15.120:0/1674373682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a0107ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- 172.21.15.120:0/1674373682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- 172.21.15.120:0/1674373682 >> 172.21.15.120:0/1674373682 conn(0x7f58a00fd700 msgr2=0x7f58a00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- 172.21.15.120:0/1674373682 shutdown_connections 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- 172.21.15.120:0/1674373682 wait complete. 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 Processor -- start 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- start start 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0111d30 con 0x7f58a01076a0 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0111bb0 con 0x7f58a0108020 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.426+0000 7f58a8463640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0111eb0 con 0x7f58a0101c90 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:34.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33886/0 (socket says 172.21.15.120:33886) 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 -- 172.21.15.120:0/2471400566 learned_addr learned my addr 172.21.15.120:0/2471400566 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a61ff640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 -- 172.21.15.120:0/2471400566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 msgr2=0x7f58a019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 -- 172.21.15.120:0/2471400566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 msgr2=0x7f58a019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a6a00640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 -- 172.21.15.120:0/2471400566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a01a54a0 con 0x7f58a01076a0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a61ff640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a6a00640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a59fe640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5890005ab0 tx=0x7f5890003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589003f070 con 0x7f58a01076a0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a01a5730 con 0x7f58a01076a0 2024-09-10T22:50:34.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5890031d60 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890035030 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.427+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a01a5c10 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.428+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0102070 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.429+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f589003b560 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.430+0000 7f58977fe640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 0x7f588c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.430+0000 7f58a61ff640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 0x7f588c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.430+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5890106cd0 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.431+0000 7f58a61ff640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 0x7f588c0b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f589c0058c0 tx=0x7f589c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:34.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.433+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58900cdce0 con 0x7f58a01076a0 2024-09-10T22:50:34.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:34 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2354436089' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:50:34.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:34 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:34.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:34.647+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f58a002e5c0 con 0x7f58a01076a0 2024-09-10T22:50:35.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.028+0000 7f58977fe640 1 -- 172.21.15.120:0/2471400566 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f58900cd5e0 con 0x7f58a01076a0 2024-09-10T22:50:35.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 msgr2=0x7f588c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:35.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 0x7f588c0b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f589c0058c0 tx=0x7f589c005850 comp rx=0 tx=0).stop 2024-09-10T22:50:35.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 msgr2=0x7f58a019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:35.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5890005ab0 tx=0x7f5890003cd0 comp rx=0 tx=0).stop 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 shutdown_connections 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f588c0b52e0 0x7f588c0b77a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58a0108020 0x7f58a019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58a01076a0 0x7f58a019a600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 --2- 172.21.15.120:0/2471400566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58a0101c90 0x7f58a019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 >> 172.21.15.120:0/2471400566 conn(0x7f58a00fd700 msgr2=0x7f58a00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 shutdown_connections 2024-09-10T22:50:35.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:35.031+0000 7f58a8463640 1 -- 172.21.15.120:0/2471400566 wait complete. 2024-09-10T22:50:35.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:35 smithi176 ceph-mon[34982]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:50:35.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:35 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:50:35.399 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:35 smithi120 ceph-mon[31247]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:50:35.400 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:35 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:50:35.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:35 smithi153 ceph-mon[34951]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:50:35.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:35 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:50:35.756 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T22:50:36.003 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:36.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:36 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:50:36.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:36 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:36.293 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:36 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:36 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:50:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:36 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:36.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:36 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:36 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2471400566' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:50:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:36 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:50:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:36 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 -- 172.21.15.120:0/2460034018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110108020 msgr2=0x7fd11010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 --2- 172.21.15.120:0/2460034018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110108020 0x7fd11010e820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd108009ca0 tx=0x7fd10802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 -- 172.21.15.120:0/2460034018 shutdown_connections 2024-09-10T22:50:36.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 --2- 172.21.15.120:0/2460034018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110108020 0x7fd11010e820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 --2- 172.21.15.120:0/2460034018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1101076a0 0x7fd110107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 --2- 172.21.15.120:0/2460034018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd110101c90 0x7fd110102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 -- 172.21.15.120:0/2460034018 >> 172.21.15.120:0/2460034018 conn(0x7fd1100fd700 msgr2=0x7fd1100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 -- 172.21.15.120:0/2460034018 shutdown_connections 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.518+0000 7fd11778c640 1 -- 172.21.15.120:0/2460034018 wait complete. 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 Processor -- start 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 -- start start 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1101076a0 0x7fd1101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd110108020 0x7fd11019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111ce0 con 0x7fd110101c90 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111b60 con 0x7fd110108020 2024-09-10T22:50:36.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd11778c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111e60 con 0x7fd1101076a0 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33910/0 (socket says 172.21.15.120:33910) 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 -- 172.21.15.120:0/1485944905 learned_addr learned my addr 172.21.15.120:0/1485944905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115d29640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd110108020 0x7fd11019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd114d27640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1101076a0 0x7fd1101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 -- 172.21.15.120:0/1485944905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1101076a0 msgr2=0x7fd1101986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1101076a0 0x7fd1101986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 -- 172.21.15.120:0/1485944905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd110108020 msgr2=0x7fd11019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd110108020 0x7fd11019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 -- 172.21.15.120:0/1485944905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11019e240 con 0x7fd110101c90 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.519+0000 7fd115528640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b4a0 tx=0x7fd10c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.520+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c014070 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.520+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c010ac0 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.520+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11019e530 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.520+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c017c30 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.520+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1101a4c70 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.521+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd110102070 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.522+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fd10c010610 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.523+0000 7fd1067fc640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 0x7fd0fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.523+0000 7fd114d27640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 0x7fd0fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.523+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd10c0e5420 con 0x7fd110101c90 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.523+0000 7fd114d27640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 0x7fd0fc0b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd100005fd0 tx=0x7fd100005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.527+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd10c0ac430 con 0x7fd110101c90 2024-09-10T22:50:36.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:36.739+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd11002e5c0 con 0x7fd110101c90 2024-09-10T22:50:37.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.042+0000 7fd1067fc640 1 -- 172.21.15.120:0/1485944905 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7fd10c0abd30 con 0x7fd110101c90 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.044+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 msgr2=0x7fd0fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.044+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 0x7fd0fc0b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd100005fd0 tx=0x7fd100005850 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.044+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 msgr2=0x7fd110198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.044+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b4a0 tx=0x7fd10c00b960 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 shutdown_connections 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0fc0b52e0 0x7fd0fc0b77a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd110108020 0x7fd11019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1101076a0 0x7fd1101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 --2- 172.21.15.120:0/1485944905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd110101c90 0x7fd110198190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 >> 172.21.15.120:0/1485944905 conn(0x7fd1100fd700 msgr2=0x7fd1100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 shutdown_connections 2024-09-10T22:50:37.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:37.045+0000 7fd11778c640 1 -- 172.21.15.120:0/1485944905 wait complete. 2024-09-10T22:50:37.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:37 smithi176 ceph-mon[34982]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 447 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T22:50:37.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:37 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:50:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:37 smithi120 ceph-mon[31247]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 447 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T22:50:37.325 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:37 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:50:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:37 smithi153 ceph-mon[34951]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 447 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T22:50:37.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:37 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:50:37.665 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-10T22:50:37.666 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T22:50:37.912 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:38.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:50:38.302 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:38 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:50:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:38 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:38.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/3226833043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8101cd0 msgr2=0x7fc6b81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/3226833043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8101cd0 0x7fc6b81020b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009900 tx=0x7fc6b402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:38.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/3226833043 shutdown_connections 2024-09-10T22:50:38.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/3226833043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8108060 0x7fc6b810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/3226833043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 0x7fc6b8107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/3226833043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8101cd0 0x7fc6b81020b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/3226833043 >> 172.21.15.120:0/3226833043 conn(0x7fc6b80fd720 msgr2=0x7fc6b80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/3226833043 shutdown_connections 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/3226833043 wait complete. 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.382+0000 7fc6bf3b9640 1 Processor -- start 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 -- start start 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 0x7fc6b8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 0x7fc6b8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8111db0 con 0x7fc6b8108060 2024-09-10T22:50:38.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8111c30 con 0x7fc6b81076e0 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8111f30 con 0x7fc6b8101cd0 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33930/0 (socket says 172.21.15.120:33930) 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 -- 172.21.15.120:0/2320897111 learned_addr learned my addr 172.21.15.120:0/2320897111 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bc954640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 0x7fc6b8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd155640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 0x7fc6b8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 -- 172.21.15.120:0/2320897111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 msgr2=0x7fc6b8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 0x7fc6b8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 -- 172.21.15.120:0/2320897111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 msgr2=0x7fc6b8198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 0x7fc6b8198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 -- 172.21.15.120:0/2320897111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b81a77f0 con 0x7fc6b8108060 2024-09-10T22:50:38.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd155640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 0x7fc6b8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bd956640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000b970 tx=0x7fc6b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0019070 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b81a7ae0 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b0014440 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.383+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00075d0 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.384+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b81a80a0 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.384+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b8102130 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.386+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fc6b000cb40 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.386+0000 7fc6ae7fc640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 0x7fc6a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.387+0000 7fc6bd155640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 0x7fc6a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.387+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc6b00e5740 con 0x7fc6b8108060 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.387+0000 7fc6bd155640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 0x7fc6a40b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4005ab0 tx=0x7fc6b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.389+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6b00ac750 con 0x7fc6b8108060 2024-09-10T22:50:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:38 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1485944905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:50:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:38 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:50:38.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.592+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc6b8063590 con 0x7fc6a40b52e0 2024-09-10T22:50:38.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.611+0000 7fc6ae7fc640 1 -- 172.21.15.120:0/2320897111 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6b8063590 con 0x7fc6a40b52e0 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 msgr2=0x7fc6a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 0x7fc6a40b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4005ab0 tx=0x7fc6b4000bc0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 msgr2=0x7fc6b8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000b970 tx=0x7fc6b000be30 comp rx=0 tx=0).stop 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 shutdown_connections 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6a40b52e0 0x7fc6a40b77a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6b8108060 0x7fc6b8198a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6b81076e0 0x7fc6b8198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 --2- 172.21.15.120:0/2320897111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6b8101cd0 0x7fc6b8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 >> 172.21.15.120:0/2320897111 conn(0x7fc6b80fd720 msgr2=0x7fc6b80ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 shutdown_connections 2024-09-10T22:50:38.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:38.614+0000 7fc6bf3b9640 1 -- 172.21.15.120:0/2320897111 wait complete. 2024-09-10T22:50:38.946 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:50:38.596+0000 7f3f74334640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:50:38.947 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:50:38.596+0000 7f3f74334640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:50:38.947 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:50:38.596+0000 7f3f74334640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:50:38.947 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:50:38.596+0000 7f3f74334640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:50:38.947 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 22:50:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x[31475]: 2024-09-10T22:50:38.596+0000 7f3f74334640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:50:39.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T22:50:39.306 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:39 smithi120 ceph-mon[31247]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T22:50:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:39 smithi176 ceph-mon[34982]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T22:50:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:39 smithi153 ceph-mon[34951]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T22:50:39.541 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 -- 172.21.15.120:0/3696222908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688101c90 msgr2=0x7f0688102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 --2- 172.21.15.120:0/3696222908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688101c90 0x7f0688102070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0684009900 tx=0x7f068402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 -- 172.21.15.120:0/3696222908 shutdown_connections 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 --2- 172.21.15.120:0/3696222908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0688108020 0x7f068810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 --2- 172.21.15.120:0/3696222908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06881076a0 0x7f0688107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 --2- 172.21.15.120:0/3696222908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688101c90 0x7f0688102070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.036+0000 7f068f78b640 1 -- 172.21.15.120:0/3696222908 >> 172.21.15.120:0/3696222908 conn(0x7f06880fd700 msgr2=0x7f06880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- 172.21.15.120:0/3696222908 shutdown_connections 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- 172.21.15.120:0/3696222908 wait complete. 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 Processor -- start 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- start start 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 0x7f0688199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 0x7f06881998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0688111ce0 con 0x7f0688108020 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0688111b60 con 0x7f0688101c90 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.037+0000 7f068f78b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0688111e60 con 0x7f06881076a0 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33970/0 (socket says 172.21.15.120:33970) 2024-09-10T22:50:40.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 -- 172.21.15.120:0/604691393 learned_addr learned my addr 172.21.15.120:0/604691393 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068d527640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 0x7f0688199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068cd26640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 0x7f06881998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 -- 172.21.15.120:0/604691393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 msgr2=0x7f06881998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 0x7f06881998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 -- 172.21.15.120:0/604691393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 msgr2=0x7f0688199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 0x7f0688199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 -- 172.21.15.120:0/604691393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068819f420 con 0x7f0688108020 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068d527640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 0x7f0688199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068cd26640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 0x7f06881998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068dd28640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0680009f60 tx=0x7f0680009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068000cd80 con 0x7f0688108020 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068819f710 con 0x7f0688108020 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680013770 con 0x7f0688108020 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680005380 con 0x7f0688108020 2024-09-10T22:50:40.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.038+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06880fdae0 con 0x7f0688108020 2024-09-10T22:50:40.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.040+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688102070 con 0x7f0688108020 2024-09-10T22:50:40.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.041+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f068000f040 con 0x7f0688108020 2024-09-10T22:50:40.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.042+0000 7f067e7fc640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 0x7f06740b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.042+0000 7f068d527640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 0x7f06740b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.042+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f06800e9940 con 0x7f0688108020 2024-09-10T22:50:40.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.043+0000 7f068d527640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 0x7f06740b7820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.046+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06800b0950 con 0x7f0688108020 2024-09-10T22:50:40.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:40 smithi120 ceph-mon[31247]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:40.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.258+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f068810ae40 con 0x7f06740b5360 2024-09-10T22:50:40.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.284+0000 7f067e7fc640 1 -- 172.21.15.120:0/604691393 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f068810ae40 con 0x7f06740b5360 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 msgr2=0x7f06740b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 0x7f06740b7820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684031400 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 msgr2=0x7f068819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0680009f60 tx=0x7f0680009f90 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 shutdown_connections 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f06740b5360 0x7f06740b7820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0688108020 0x7f068819ec50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06881076a0 0x7f06881998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 --2- 172.21.15.120:0/604691393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0688101c90 0x7f0688199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 >> 172.21.15.120:0/604691393 conn(0x7f06880fd700 msgr2=0x7f068806ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 shutdown_connections 2024-09-10T22:50:40.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:40.286+0000 7f068f78b640 1 -- 172.21.15.120:0/604691393 wait complete. 2024-09-10T22:50:40.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:40 smithi176 ceph-mon[34982]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:40 smithi153 ceph-mon[34951]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:40.976 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T22:50:41.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:41 smithi120 ceph-mon[31247]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T22:50:41.272 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:41.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:41 smithi176 ceph-mon[34982]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T22:50:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:41 smithi153 ceph-mon[34951]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- 172.21.15.120:0/3885679516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 msgr2=0x7f21c810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3885679516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c810f950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009ca0 tx=0x7f21c002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- 172.21.15.120:0/3885679516 shutdown_connections 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3885679516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c810f950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3885679516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c81087d0 0x7f21c8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3885679516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c8102dc0 0x7f21c81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- 172.21.15.120:0/3885679516 >> 172.21.15.120:0/3885679516 conn(0x7f21c80fe830 msgr2=0x7f21c8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- 172.21.15.120:0/3885679516 shutdown_connections 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- 172.21.15.120:0/3885679516 wait complete. 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 Processor -- start 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.726+0000 7f21ce15b640 1 -- start start 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 0x7f21c8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:41.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c81087d0 0x7f21c81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8112e10 con 0x7f21c8109150 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8112c90 con 0x7f21c8102dc0 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8112f90 con 0x7f21c81087d0 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33998/0 (socket says 172.21.15.120:33998) 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 -- 172.21.15.120:0/3364155331 learned_addr learned my addr 172.21.15.120:0/3364155331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c6ffd640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c81087d0 0x7f21c81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c77fe640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 0x7f21c8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 -- 172.21.15.120:0/3364155331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c81087d0 msgr2=0x7f21c81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c81087d0 0x7f21c81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 -- 172.21.15.120:0/3364155331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 msgr2=0x7f21c8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 0x7f21c8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 -- 172.21.15.120:0/3364155331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c819f330 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c77fe640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 0x7f21c8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c7fff640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21c0005cc0 tx=0x7f21c002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0003bf0 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c819f5c0 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21c0003d90 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.727+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c002fbc0 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.728+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c81a5ce0 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.728+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c81031a0 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.730+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f21c0004610 con 0x7f21c8109150 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.731+0000 7f21c4ff9640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 0x7f21b00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:41.745 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.731+0000 7f21c77fe640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 0x7f21b00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.731+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f21c0107220 con 0x7f21c8109150 2024-09-10T22:50:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.731+0000 7f21c77fe640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 0x7f21b00b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009930 tx=0x7f21b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.734+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f21c00ce130 con 0x7f21c8109150 2024-09-10T22:50:41.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.936+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f21c8063590 con 0x7f21b00b5370 2024-09-10T22:50:41.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.941+0000 7f21c4ff9640 1 -- 172.21.15.120:0/3364155331 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f21c8063590 con 0x7f21b00b5370 2024-09-10T22:50:41.943 INFO:teuthology.orchestra.run.smithi120.stdout:/volumes/qa/sv_0/aa6222d4-e523-4584-883a-9948e984610f 2024-09-10T22:50:41.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 msgr2=0x7f21b00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:41.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 0x7f21b00b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009930 tx=0x7f21b8008040 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 msgr2=0x7f21c819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21c0005cc0 tx=0x7f21c002f380 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 shutdown_connections 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21b00b5370 0x7f21b00b7830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21c8109150 0x7f21c819eb60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c81087d0 0x7f21c81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 --2- 172.21.15.120:0/3364155331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21c8102dc0 0x7f21c8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 >> 172.21.15.120:0/3364155331 conn(0x7f21c80fe830 msgr2=0x7f21c8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 shutdown_connections 2024-09-10T22:50:41.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:41.943+0000 7f21ce15b640 1 -- 172.21.15.120:0/3364155331 wait complete. 2024-09-10T22:50:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:42 smithi120 ceph-mon[31247]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:42 smithi120 ceph-mon[31247]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:50:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:42 smithi176 ceph-mon[34982]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:42 smithi176 ceph-mon[34982]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:50:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:42 smithi153 ceph-mon[34951]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:42 smithi153 ceph-mon[34951]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:50:42.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T22:50:42.879 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:43.285 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:43 smithi120 ceph-mon[31247]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:43.285 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:43 smithi120 ceph-mon[31247]: pgmap v289: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-09-10T22:50:43.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:43 smithi176 ceph-mon[34982]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:43.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:43 smithi176 ceph-mon[34982]: pgmap v289: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 -- 172.21.15.120:0/2331110625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce041087d0 msgr2=0x7fce04108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 --2- 172.21.15.120:0/2331110625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce041087d0 0x7fce04108c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009900 tx=0x7fcdec02a410 comp rx=0 tx=0).stop 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 -- 172.21.15.120:0/2331110625 shutdown_connections 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 --2- 172.21.15.120:0/2331110625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce04109150 0x7fce0410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 --2- 172.21.15.120:0/2331110625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce041087d0 0x7fce04108c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 --2- 172.21.15.120:0/2331110625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04102dc0 0x7fce041031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 -- 172.21.15.120:0/2331110625 >> 172.21.15.120:0/2331110625 conn(0x7fce040fe830 msgr2=0x7fce04100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 -- 172.21.15.120:0/2331110625 shutdown_connections 2024-09-10T22:50:43.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.368+0000 7fce096e1640 1 -- 172.21.15.120:0/2331110625 wait complete. 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 Processor -- start 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 -- start start 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce041087d0 0x7fce04199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04109150 0x7fce0419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112e10 con 0x7fce04102dc0 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112c90 con 0x7fce04109150 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce096e1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112f90 con 0x7fce041087d0 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55598/0 (socket says 172.21.15.120:55598) 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce02ffd640 1 -- 172.21.15.120:0/2785105409 learned_addr learned my addr 172.21.15.120:0/2785105409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.369+0000 7fce037fe640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04109150 0x7fce0419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 -- 172.21.15.120:0/2785105409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce041087d0 msgr2=0x7fce04199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce041087d0 0x7fce04199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 -- 172.21.15.120:0/2785105409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04109150 msgr2=0x7fce0419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04109150 0x7fce0419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 -- 172.21.15.120:0/2785105409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0419f390 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce02ffd640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d950 tx=0x7fcdf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4017070 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0419f680 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf4009c30 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400f620 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.370+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a5f10 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.371+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce041031a0 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.372+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fcdf4010460 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.373+0000 7fce00ff9640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 0x7fcdf00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.373+0000 7fcdfa7fc640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 0x7fcdf00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.373+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fcdf40e58b0 con 0x7fce04102dc0 2024-09-10T22:50:43.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.374+0000 7fcdfa7fc640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 0x7fcdf00b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0096c0 tx=0x7fcdec031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.377+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcdf40ac7c0 con 0x7fce04102dc0 2024-09-10T22:50:43.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:43 smithi153 ceph-mon[34951]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:43.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:43 smithi153 ceph-mon[34951]: pgmap v289: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-09-10T22:50:43.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.577+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fce04063590 con 0x7fcdf00b5370 2024-09-10T22:50:43.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.602+0000 7fce00ff9640 1 -- 172.21.15.120:0/2785105409 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce04063590 con 0x7fcdf00b5370 2024-09-10T22:50:43.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 msgr2=0x7fcdf00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 0x7fcdf00b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0096c0 tx=0x7fcdec031400 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 msgr2=0x7fce041992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d950 tx=0x7fcdf400de10 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 shutdown_connections 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdf00b5370 0x7fcdf00b7830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce04109150 0x7fce0419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce041087d0 0x7fce04199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 --2- 172.21.15.120:0/2785105409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce04102dc0 0x7fce041992e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 >> 172.21.15.120:0/2785105409 conn(0x7fce040fe830 msgr2=0x7fce04100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 shutdown_connections 2024-09-10T22:50:43.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:43.604+0000 7fce096e1640 1 -- 172.21.15.120:0/2785105409 wait complete. 2024-09-10T22:50:44.307 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:44 smithi120 ceph-mon[31247]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:44.307 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:44 smithi120 ceph-mon[31247]: pgmap v290: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-10T22:50:44.336 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T22:50:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:44 smithi176 ceph-mon[34982]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:44 smithi176 ceph-mon[34982]: pgmap v290: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-10T22:50:44.582 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:44.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:44 smithi153 ceph-mon[34951]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:44.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:44 smithi153 ceph-mon[34951]: pgmap v290: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-10T22:50:45.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 -- 172.21.15.120:0/1075233362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40101cd0 msgr2=0x7fac401020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 --2- 172.21.15.120:0/1075233362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40101cd0 0x7fac401020b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009900 tx=0x7fac3c02a410 comp rx=0 tx=0).stop 2024-09-10T22:50:45.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 -- 172.21.15.120:0/1075233362 shutdown_connections 2024-09-10T22:50:45.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 --2- 172.21.15.120:0/1075233362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40108060 0x7fac4010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 --2- 172.21.15.120:0/1075233362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac401076e0 0x7fac40107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 --2- 172.21.15.120:0/1075233362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40101cd0 0x7fac401020b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 -- 172.21.15.120:0/1075233362 >> 172.21.15.120:0/1075233362 conn(0x7fac400fd700 msgr2=0x7fac400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.035+0000 7fac47b55640 1 -- 172.21.15.120:0/1075233362 shutdown_connections 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 -- 172.21.15.120:0/1075233362 wait complete. 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 Processor -- start 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 -- start start 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 0x7fac401992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac401076e0 0x7fac40199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac40111d20 con 0x7fac40108060 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac40111ba0 con 0x7fac40101cd0 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac47b55640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fac40111ea0 con 0x7fac401076e0 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac460f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac460f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55612/0 (socket says 172.21.15.120:55612) 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.036+0000 7fac460f2640 1 -- 172.21.15.120:0/3424761748 learned_addr learned my addr 172.21.15.120:0/3424761748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac458f1640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 0x7fac401992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 -- 172.21.15.120:0/3424761748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac401076e0 msgr2=0x7fac40199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac401076e0 0x7fac40199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 -- 172.21.15.120:0/3424761748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 msgr2=0x7fac401992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 0x7fac401992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 -- 172.21.15.120:0/3424761748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4019f380 con 0x7fac40108060 2024-09-10T22:50:45.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac458f1640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 0x7fac401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac460f2640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cce0 tx=0x7fac3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800f450 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4019f670 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3800faa0 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38018b70 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.037+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac401a5d30 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.038+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac401020b0 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.039+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fac3800f5f0 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.040+0000 7fac36ffd640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 0x7fac2c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.040+0000 7fac458f1640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 0x7fac2c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.040+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fac380e6490 con 0x7fac40108060 2024-09-10T22:50:45.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.041+0000 7fac458f1640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 0x7fac2c0b7830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac3c031440 tx=0x7fac3c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.043+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac38014030 con 0x7fac40108060 2024-09-10T22:50:45.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.249+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fac40063590 con 0x7fac2c0b5370 2024-09-10T22:50:45.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.253+0000 7fac36ffd640 1 -- 172.21.15.120:0/3424761748 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fac40063590 con 0x7fac2c0b5370 2024-09-10T22:50:45.256 INFO:teuthology.orchestra.run.smithi120.stdout:/volumes/qa/sv_1/64d96756-0a18-479f-8469-4787856ad6c3 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 msgr2=0x7fac2c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 0x7fac2c0b7830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac3c031440 tx=0x7fac3c0095f0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 msgr2=0x7fac4019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cce0 tx=0x7fac3800a620 comp rx=0 tx=0).stop 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 shutdown_connections 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac2c0b5370 0x7fac2c0b7830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac40108060 0x7fac4019ebb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac401076e0 0x7fac40199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 --2- 172.21.15.120:0/3424761748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac40101cd0 0x7fac401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 >> 172.21.15.120:0/3424761748 conn(0x7fac400fd700 msgr2=0x7fac4006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 shutdown_connections 2024-09-10T22:50:45.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:45.256+0000 7fac47b55640 1 -- 172.21.15.120:0/3424761748 wait complete. 2024-09-10T22:50:45.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:46.200 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:46.484 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:46 smithi120 ceph-mon[31247]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:46.484 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:46 smithi120 ceph-mon[31247]: pgmap v291: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T22:50:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:46 smithi176 ceph-mon[34982]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:46 smithi176 ceph-mon[34982]: pgmap v291: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- 172.21.15.120:0/3937448435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d41076c0 msgr2=0x7f35d4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 --2- 172.21.15.120:0/3937448435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d41076c0 0x7f35d4107b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009ca0 tx=0x7f35c802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- 172.21.15.120:0/3937448435 shutdown_connections 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 --2- 172.21.15.120:0/3937448435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d4108040 0x7f35d410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 --2- 172.21.15.120:0/3937448435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d41076c0 0x7f35d4107b00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 --2- 172.21.15.120:0/3937448435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4101cb0 0x7f35d4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- 172.21.15.120:0/3937448435 >> 172.21.15.120:0/3937448435 conn(0x7f35d40fd700 msgr2=0x7f35d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:46.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- 172.21.15.120:0/3937448435 shutdown_connections 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- 172.21.15.120:0/3937448435 wait complete. 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 Processor -- start 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.664+0000 7f35db82d640 1 -- start start 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d41076c0 0x7f35d4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 0x7f35d4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111d40 con 0x7f35d4101cb0 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111bc0 con 0x7f35d41076c0 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111ec0 con 0x7f35d4108040 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55626/0 (socket says 172.21.15.120:55626) 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 -- 172.21.15.120:0/2860626591 learned_addr learned my addr 172.21.15.120:0/2860626591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:46.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 -- 172.21.15.120:0/2860626591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 msgr2=0x7f35d4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d9dca640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 0x7f35d4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 0x7f35d4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 -- 172.21.15.120:0/2860626591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d41076c0 msgr2=0x7f35d4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d8dc8640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d41076c0 0x7f35d4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d41076c0 0x7f35d4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 -- 172.21.15.120:0/2860626591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41a7800 con 0x7f35d4101cb0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d9dca640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 0x7f35d4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d95c9640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b4a0 tx=0x7f35cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc014070 con 0x7f35d4101cb0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35cc010ac0 con 0x7f35d4101cb0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d41a7af0 con 0x7f35d4101cb0 2024-09-10T22:50:46.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc017c30 con 0x7f35d4101cb0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.665+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a7fe0 con 0x7f35d4101cb0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.666+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4102110 con 0x7f35d4101cb0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.668+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f35cc010610 con 0x7f35d4101cb0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.669+0000 7f35d27fc640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 0x7f35bc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.669+0000 7f35d8dc8640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 0x7f35bc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.669+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f35cc0e55c0 con 0x7f35d4101cb0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.669+0000 7f35d8dc8640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 0x7f35bc0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006010 tx=0x7f35c8000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:46.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.672+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f35cc0ac4d0 con 0x7f35d4101cb0 2024-09-10T22:50:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:46 smithi153 ceph-mon[34951]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:50:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:46 smithi153 ceph-mon[34951]: pgmap v291: 97 pgs: 97 active+clean; 599 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T22:50:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.886+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35d406b840 con 0x7f35d4101cb0 2024-09-10T22:50:46.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.887+0000 7f35d27fc640 1 -- 172.21.15.120:0/2860626591 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f35d406b840 con 0x7f35d4101cb0 2024-09-10T22:50:46.889 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:46.889 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:50:46.889 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.889+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 msgr2=0x7f35bc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.889+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 0x7f35bc0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006010 tx=0x7f35c8000f40 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.889+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 msgr2=0x7f35d4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.889+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b4a0 tx=0x7f35cc00b960 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 shutdown_connections 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35bc0b5370 0x7f35bc0b7830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35d4108040 0x7f35d4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d41076c0 0x7f35d4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 --2- 172.21.15.120:0/2860626591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35d4101cb0 0x7f35d4198020 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:46.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 >> 172.21.15.120:0/2860626591 conn(0x7f35d40fd700 msgr2=0x7f35d40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 shutdown_connections 2024-09-10T22:50:46.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:46.890+0000 7f35db82d640 1 -- 172.21.15.120:0/2860626591 wait complete. 2024-09-10T22:50:47.518 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:47 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2860626591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:47.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:50:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:47 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2860626591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:47 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2860626591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:47.790 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:48.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- 172.21.15.120:0/2728739354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 msgr2=0x7f5250102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- 172.21.15.120:0/2728739354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f5250102070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5248009900 tx=0x7f524802a410 comp rx=0 tx=0).stop 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- 172.21.15.120:0/2728739354 shutdown_connections 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- 172.21.15.120:0/2728739354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- 172.21.15.120:0/2728739354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f5250107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- 172.21.15.120:0/2728739354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f5250102070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- 172.21.15.120:0/2728739354 >> 172.21.15.120:0/2728739354 conn(0x7f52500fd700 msgr2=0x7f52500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- 172.21.15.120:0/2728739354 shutdown_connections 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- 172.21.15.120:0/2728739354 wait complete. 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 Processor -- start 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- start start 2024-09-10T22:50:48.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f525019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5250111d20 con 0x7f5250101c90 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5250111ba0 con 0x7f52501076a0 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.286+0000 7f5258447640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5250111ea0 con 0x7f5250108020 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55642/0 (socket says 172.21.15.120:55642) 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 -- 172.21.15.120:0/2597131524 learned_addr learned my addr 172.21.15.120:0/2597131524 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52569e4640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52559e2640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f525019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 -- 172.21.15.120:0/2597131524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 msgr2=0x7f525019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 -- 172.21.15.120:0/2597131524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 msgr2=0x7f525019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f525019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 -- 172.21.15.120:0/2597131524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52501a55d0 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52569e4640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52559e2640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f525019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f52561e3640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524803f030 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5248003800 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501a5860 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248035070 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.287+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501a5dc0 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.288+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52501020f0 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.289+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f5248031b10 con 0x7f5250101c90 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.290+0000 7f524f7fe640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 0x7f52380b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:48.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.290+0000 7f52559e2640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 0x7f52380b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:48.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.290+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5248106830 con 0x7f5250101c90 2024-09-10T22:50:48.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.291+0000 7f52559e2640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 0x7f52380b7830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f525006b6b0 tx=0x7f52440073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:48.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.293+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52480cd740 con 0x7f5250101c90 2024-09-10T22:50:48.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.469+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f525002e5c0 con 0x7f5250101c90 2024-09-10T22:50:48.471 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:48 smithi120 ceph-mon[31247]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:48.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.470+0000 7f524f7fe640 1 -- 172.21.15.120:0/2597131524 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22360 (secure 0 0 0) 0x7f52480cd040 con 0x7f5250101c90 2024-09-10T22:50:48.472 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:48.472 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":75,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:50:27.621440+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 msgr2=0x7f52380b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 0x7f52380b7830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f525006b6b0 tx=0x7f52440073c0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 msgr2=0x7f525019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 shutdown_connections 2024-09-10T22:50:48.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52380b5370 0x7f52380b7830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5250108020 0x7f525019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52501076a0 0x7f525019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 --2- 172.21.15.120:0/2597131524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5250101c90 0x7f525019a1f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 >> 172.21.15.120:0/2597131524 conn(0x7f52500fd700 msgr2=0x7f52500ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 shutdown_connections 2024-09-10T22:50:48.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:48.473+0000 7f5258447640 1 -- 172.21.15.120:0/2597131524 wait complete. 2024-09-10T22:50:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:48 smithi176 ceph-mon[34982]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:48.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:48 smithi153 ceph-mon[34951]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:49.278 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:50:49.279 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:50:49.531 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:49 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2597131524' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:49.652 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:49 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2597131524' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:49.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:49 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2597131524' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:49.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 -- 172.21.15.120:0/2587524548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 msgr2=0x7f0ddc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:49.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 --2- 172.21.15.120:0/2587524548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc102070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 -- 172.21.15.120:0/2587524548 shutdown_connections 2024-09-10T22:50:49.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 --2- 172.21.15.120:0/2587524548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ddc108020 0x7f0ddc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 --2- 172.21.15.120:0/2587524548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ddc1076a0 0x7f0ddc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 --2- 172.21.15.120:0/2587524548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc102070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 -- 172.21.15.120:0/2587524548 >> 172.21.15.120:0/2587524548 conn(0x7f0ddc0fd700 msgr2=0x7f0ddc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 -- 172.21.15.120:0/2587524548 shutdown_connections 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.984+0000 7f0de4430640 1 -- 172.21.15.120:0/2587524548 wait complete. 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 Processor -- start 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 -- start start 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ddc1076a0 0x7f0ddc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ddc108020 0x7f0ddc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:49.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc111d20 con 0x7f0ddc101c90 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc111ba0 con 0x7f0ddc1076a0 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de4430640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc111ea0 con 0x7f0ddc108020 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de21cc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de21cc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55662/0 (socket says 172.21.15.120:55662) 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.985+0000 7f0de21cc640 1 -- 172.21.15.120:0/3247600344 learned_addr learned my addr 172.21.15.120:0/3247600344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 -- 172.21.15.120:0/3247600344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ddc108020 msgr2=0x7f0ddc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ddc108020 0x7f0ddc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 -- 172.21.15.120:0/3247600344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ddc1076a0 msgr2=0x7f0ddc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ddc1076a0 0x7f0ddc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 -- 172.21.15.120:0/3247600344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc19f510 con 0x7f0ddc101c90 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de21cc640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc02ade0 tx=0x7f0dcc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003d60 con 0x7f0ddc101c90 2024-09-10T22:50:49.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ddc19f7a0 con 0x7f0ddc101c90 2024-09-10T22:50:49.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dcc0044f0 con 0x7f0ddc101c90 2024-09-10T22:50:49.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc02fd10 con 0x7f0ddc101c90 2024-09-10T22:50:49.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.986+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ddc1a5ec0 con 0x7f0ddc101c90 2024-09-10T22:50:49.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.987+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ddc1020f0 con 0x7f0ddc101c90 2024-09-10T22:50:49.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.989+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0dcc004040 con 0x7f0ddc101c90 2024-09-10T22:50:49.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.990+0000 7f0ddb7fe640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 0x7f0dc40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:49.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.990+0000 7f0de19cb640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 0x7f0dc40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:49.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.990+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f0dcc1072c0 con 0x7f0ddc101c90 2024-09-10T22:50:49.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.990+0000 7f0de19cb640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 0x7f0dc40b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc1073f0 tx=0x7f0dd0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:49.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:49.993+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0dcc03b410 con 0x7f0ddc101c90 2024-09-10T22:50:50.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.166+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ddc063590 con 0x7f0dc40b5370 2024-09-10T22:50:50.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.171+0000 7f0ddb7fe640 1 -- 172.21.15.120:0/3247600344 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258622 (secure 0 0 0) 0x7f0ddc063590 con 0x7f0dc40b5370 2024-09-10T22:50:50.175 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:50.176 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 msgr2=0x7f0dc40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 0x7f0dc40b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc1073f0 tx=0x7f0dd0006c90 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 msgr2=0x7f0ddc199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc02ade0 tx=0x7f0dcc003970 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 shutdown_connections 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0dc40b5370 0x7f0dc40b7830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ddc108020 0x7f0ddc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ddc1076a0 0x7f0ddc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 --2- 172.21.15.120:0/3247600344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ddc101c90 0x7f0ddc199460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 >> 172.21.15.120:0/3247600344 conn(0x7f0ddc0fd700 msgr2=0x7f0ddc06ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 shutdown_connections 2024-09-10T22:50:50.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:50.174+0000 7f0de4430640 1 -- 172.21.15.120:0/3247600344 wait complete. 2024-09-10T22:50:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:50 smithi120 ceph-mon[31247]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:50.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:50 smithi120 ceph-mon[31247]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:50 smithi176 ceph-mon[34982]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:50 smithi176 ceph-mon[34982]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:50 smithi153 ceph-mon[34951]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-09-10T22:50:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:50 smithi153 ceph-mon[34951]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:51.043 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-10T22:50:50.152355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638200,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":112,"num_read_kb":130,"num_write":194,"num_write_kb":2219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377312,"kb_used_data":11408,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564512,"statfs":{"total":1151940427776,"available":1151554060288,"internally_reserved":0,"allocated":11681792,"data_stored":6576624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45422,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":57,"num_write":26,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010885+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:50:28.010885+0000","last_peered":"2024-09-10T22:50:28.010885+0000","last_clean":"2024-09-10T22:50:28.010885+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:50:28.010885+0000","last_undegraded":"2024-09-10T22:50:28.010885+0000","last_fullsized":"2024-09-10T22:50:28.010885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026244+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:50:28.026244+0000","last_peered":"2024-09-10T22:50:28.026244+0000","last_clean":"2024-09-10T22:50:28.026244+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:50:28.026244+0000","last_undegraded":"2024-09-10T22:50:28.026244+0000","last_fullsized":"2024-09-10T22:50:28.026244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025805+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:50:28.025805+0000","last_peered":"2024-09-10T22:50:28.025805+0000","last_clean":"2024-09-10T22:50:28.025805+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:50:28.025805+0000","last_undegraded":"2024-09-10T22:50:28.025805+0000","last_fullsized":"2024-09-10T22:50:28.025805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023167+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:50:28.023167+0000","last_peered":"2024-09-10T22:50:28.023167+0000","last_clean":"2024-09-10T22:50:28.023167+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:50:28.023167+0000","last_undegraded":"2024-09-10T22:50:28.023167+0000","last_fullsized":"2024-09-10T22:50:28.023167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022490+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:50:28.022490+0000","last_peered":"2024-09-10T22:50:28.022490+0000","last_clean":"2024-09-10T22:50:28.022490+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:50:28.022490+0000","last_undegraded":"2024-09-10T22:50:28.022490+0000","last_fullsized":"2024-09-10T22:50:28.022490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010866+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:50:28.010866+0000","last_peered":"2024-09-10T22:50:28.010866+0000","last_clean":"2024-09-10T22:50:28.010866+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:50:28.010866+0000","last_undegraded":"2024-09-10T22:50:28.010866+0000","last_fullsized":"2024-09-10T22:50:28.010866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023282+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:50:28.023282+0000","last_peered":"2024-09-10T22:50:28.023282+0000","last_clean":"2024-09-10T22:50:28.023282+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:50:28.023282+0000","last_undegraded":"2024-09-10T22:50:28.023282+0000","last_fullsized":"2024-09-10T22:50:28.023282+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024418+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:50:28.024418+0000","last_peered":"2024-09-10T22:50:28.024418+0000","last_clean":"2024-09-10T22:50:28.024418+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:50:28.024418+0000","last_undegraded":"2024-09-10T22:50:28.024418+0000","last_fullsized":"2024-09-10T22:50:28.024418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025789+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:50:28.025789+0000","last_peered":"2024-09-10T22:50:28.025789+0000","last_clean":"2024-09-10T22:50:28.025789+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:50:28.025789+0000","last_undegraded":"2024-09-10T22:50:28.025789+0000","last_fullsized":"2024-09-10T22:50:28.025789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011429+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:50:28.011429+0000","last_peered":"2024-09-10T22:50:28.011429+0000","last_clean":"2024-09-10T22:50:28.011429+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:50:28.011429+0000","last_undegraded":"2024-09-10T22:50:28.011429+0000","last_fullsized":"2024-09-10T22:50:28.011429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023673+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:50:28.023673+0000","last_peered":"2024-09-10T22:50:28.023673+0000","last_clean":"2024-09-10T22:50:28.023673+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:50:28.023673+0000","last_undegraded":"2024-09-10T22:50:28.023673+0000","last_fullsized":"2024-09-10T22:50:28.023673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023207+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:50:28.023207+0000","last_peered":"2024-09-10T22:50:28.023207+0000","last_clean":"2024-09-10T22:50:28.023207+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:50:28.023207+0000","last_undegraded":"2024-09-10T22:50:28.023207+0000","last_fullsized":"2024-09-10T22:50:28.023207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022980+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:50:28.022980+0000","last_peered":"2024-09-10T22:50:28.022980+0000","last_clean":"2024-09-10T22:50:28.022980+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:50:28.022980+0000","last_undegraded":"2024-09-10T22:50:28.022980+0000","last_fullsized":"2024-09-10T22:50:28.022980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025604+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:50:28.025604+0000","last_peered":"2024-09-10T22:50:28.025604+0000","last_clean":"2024-09-10T22:50:28.025604+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:50:28.025604+0000","last_undegraded":"2024-09-10T22:50:28.025604+0000","last_fullsized":"2024-09-10T22:50:28.025604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025461+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:50:28.025461+0000","last_peered":"2024-09-10T22:50:28.025461+0000","last_clean":"2024-09-10T22:50:28.025461+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:50:28.025461+0000","last_undegraded":"2024-09-10T22:50:28.025461+0000","last_fullsized":"2024-09-10T22:50:28.025461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025180+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:50:28.025180+0000","last_peered":"2024-09-10T22:50:28.025180+0000","last_clean":"2024-09-10T22:50:28.025180+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:50:28.025180+0000","last_undegraded":"2024-09-10T22:50:28.025180+0000","last_fullsized":"2024-09-10T22:50:28.025180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010897+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:50:28.010897+0000","last_peered":"2024-09-10T22:50:28.010897+0000","last_clean":"2024-09-10T22:50:28.010897+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:50:28.010897+0000","last_undegraded":"2024-09-10T22:50:28.010897+0000","last_fullsized":"2024-09-10T22:50:28.010897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010551+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:50:28.010551+0000","last_peered":"2024-09-10T22:50:28.010551+0000","last_clean":"2024-09-10T22:50:28.010551+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:50:28.010551+0000","last_undegraded":"2024-09-10T22:50:28.010551+0000","last_fullsized":"2024-09-10T22:50:28.010551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027910+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:50:28.027910+0000","last_peered":"2024-09-10T22:50:28.027910+0000","last_clean":"2024-09-10T22:50:28.027910+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:50:28.027910+0000","last_undegraded":"2024-09-10T22:50:28.027910+0000","last_fullsized":"2024-09-10T22:50:28.027910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010883+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:50:28.010883+0000","last_peered":"2024-09-10T22:50:28.010883+0000","last_clean":"2024-09-10T22:50:28.010883+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:50:28.010883+0000","last_undegraded":"2024-09-10T22:50:28.010883+0000","last_fullsized":"2024-09-10T22:50:28.010883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022987+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:50:28.022987+0000","last_peered":"2024-09-10T22:50:28.022987+0000","last_clean":"2024-09-10T22:50:28.022987+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:50:28.022987+0000","last_undegraded":"2024-09-10T22:50:28.022987+0000","last_fullsized":"2024-09-10T22:50:28.022987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025815+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:50:28.025815+0000","last_peered":"2024-09-10T22:50:28.025815+0000","last_clean":"2024-09-10T22:50:28.025815+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:50:28.025815+0000","last_undegraded":"2024-09-10T22:50:28.025815+0000","last_fullsized":"2024-09-10T22:50:28.025815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023606+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:50:28.023606+0000","last_peered":"2024-09-10T22:50:28.023606+0000","last_clean":"2024-09-10T22:50:28.023606+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:50:28.023606+0000","last_undegraded":"2024-09-10T22:50:28.023606+0000","last_fullsized":"2024-09-10T22:50:28.023606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023962+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:50:28.023962+0000","last_peered":"2024-09-10T22:50:28.023962+0000","last_clean":"2024-09-10T22:50:28.023962+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:50:28.023962+0000","last_undegraded":"2024-09-10T22:50:28.023962+0000","last_fullsized":"2024-09-10T22:50:28.023962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023986+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:50:28.023986+0000","last_peered":"2024-09-10T22:50:28.023986+0000","last_clean":"2024-09-10T22:50:28.023986+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:50:28.023986+0000","last_undegraded":"2024-09-10T22:50:28.023986+0000","last_fullsized":"2024-09-10T22:50:28.023986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.028214+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:50:28.028214+0000","last_peered":"2024-09-10T22:50:28.028214+0000","last_clean":"2024-09-10T22:50:28.028214+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:50:28.028214+0000","last_undegraded":"2024-09-10T22:50:28.028214+0000","last_fullsized":"2024-09-10T22:50:28.028214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011135+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:50:28.011135+0000","last_peered":"2024-09-10T22:50:28.011135+0000","last_clean":"2024-09-10T22:50:28.011135+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:50:28.011135+0000","last_undegraded":"2024-09-10T22:50:28.011135+0000","last_fullsized":"2024-09-10T22:50:28.011135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023455+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:50:28.023455+0000","last_peered":"2024-09-10T22:50:28.023455+0000","last_clean":"2024-09-10T22:50:28.023455+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:50:28.023455+0000","last_undegraded":"2024-09-10T22:50:28.023455+0000","last_fullsized":"2024-09-10T22:50:28.023455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026170+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:50:28.026170+0000","last_peered":"2024-09-10T22:50:28.026170+0000","last_clean":"2024-09-10T22:50:28.026170+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:50:28.026170+0000","last_undegraded":"2024-09-10T22:50:28.026170+0000","last_fullsized":"2024-09-10T22:50:28.026170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022672+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:50:28.022672+0000","last_peered":"2024-09-10T22:50:28.022672+0000","last_clean":"2024-09-10T22:50:28.022672+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:50:28.022672+0000","last_undegraded":"2024-09-10T22:50:28.022672+0000","last_fullsized":"2024-09-10T22:50:28.022672+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027308+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:50:28.027308+0000","last_peered":"2024-09-10T22:50:28.027308+0000","last_clean":"2024-09-10T22:50:28.027308+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:50:28.027308+0000","last_undegraded":"2024-09-10T22:50:28.027308+0000","last_fullsized":"2024-09-10T22:50:28.027308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010521+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:50:28.010521+0000","last_peered":"2024-09-10T22:50:28.010521+0000","last_clean":"2024-09-10T22:50:28.010521+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:50:28.010521+0000","last_undegraded":"2024-09-10T22:50:28.010521+0000","last_fullsized":"2024-09-10T22:50:28.010521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022560+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:50:28.022560+0000","last_peered":"2024-09-10T22:50:28.022560+0000","last_clean":"2024-09-10T22:50:28.022560+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:50:28.022560+0000","last_undegraded":"2024-09-10T22:50:28.022560+0000","last_fullsized":"2024-09-10T22:50:28.022560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031747+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:50:28.031747+0000","last_peered":"2024-09-10T22:50:28.031747+0000","last_clean":"2024-09-10T22:50:28.031747+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:50:28.031747+0000","last_undegraded":"2024-09-10T22:50:28.031747+0000","last_fullsized":"2024-09-10T22:50:28.031747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027131+0000","last_change":"2024-09-10T22:50:27.014963+0000","last_active":"2024-09-10T22:50:28.027131+0000","last_peered":"2024-09-10T22:50:28.027131+0000","last_clean":"2024-09-10T22:50:28.027131+0000","last_became_active":"2024-09-10T22:50:27.013791+0000","last_became_peered":"2024-09-10T22:50:27.013791+0000","last_unstale":"2024-09-10T22:50:28.027131+0000","last_undegraded":"2024-09-10T22:50:28.027131+0000","last_fullsized":"2024-09-10T22:50:28.027131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:51.048840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010994+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:50:28.010994+0000","last_peered":"2024-09-10T22:50:28.010994+0000","last_clean":"2024-09-10T22:50:28.010994+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:50:28.010994+0000","last_undegraded":"2024-09-10T22:50:28.010994+0000","last_fullsized":"2024-09-10T22:50:28.010994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022610+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:50:28.022610+0000","last_peered":"2024-09-10T22:50:28.022610+0000","last_clean":"2024-09-10T22:50:28.022610+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:50:28.022610+0000","last_undegraded":"2024-09-10T22:50:28.022610+0000","last_fullsized":"2024-09-10T22:50:28.022610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024259+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:50:28.024259+0000","last_peered":"2024-09-10T22:50:28.024259+0000","last_clean":"2024-09-10T22:50:28.024259+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:50:28.024259+0000","last_undegraded":"2024-09-10T22:50:28.024259+0000","last_fullsized":"2024-09-10T22:50:28.024259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024159+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:50:28.024159+0000","last_peered":"2024-09-10T22:50:28.024159+0000","last_clean":"2024-09-10T22:50:28.024159+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:50:28.024159+0000","last_undegraded":"2024-09-10T22:50:28.024159+0000","last_fullsized":"2024-09-10T22:50:28.024159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010350+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:50:28.010350+0000","last_peered":"2024-09-10T22:50:28.010350+0000","last_clean":"2024-09-10T22:50:28.010350+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:50:28.010350+0000","last_undegraded":"2024-09-10T22:50:28.010350+0000","last_fullsized":"2024-09-10T22:50:28.010350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011351+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:50:28.011351+0000","last_peered":"2024-09-10T22:50:28.011351+0000","last_clean":"2024-09-10T22:50:28.011351+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:50:28.011351+0000","last_undegraded":"2024-09-10T22:50:28.011351+0000","last_fullsized":"2024-09-10T22:50:28.011351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025694+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:50:28.025694+0000","last_peered":"2024-09-10T22:50:28.025694+0000","last_clean":"2024-09-10T22:50:28.025694+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:50:28.025694+0000","last_undegraded":"2024-09-10T22:50:28.025694+0000","last_fullsized":"2024-09-10T22:50:28.025694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023547+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:50:28.023547+0000","last_peered":"2024-09-10T22:50:28.023547+0000","last_clean":"2024-09-10T22:50:28.023547+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:50:28.023547+0000","last_undegraded":"2024-09-10T22:50:28.023547+0000","last_fullsized":"2024-09-10T22:50:28.023547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026226+0000","last_change":"2024-09-10T22:50:27.014115+0000","last_active":"2024-09-10T22:50:28.026226+0000","last_peered":"2024-09-10T22:50:28.026226+0000","last_clean":"2024-09-10T22:50:28.026226+0000","last_became_active":"2024-09-10T22:50:27.013782+0000","last_became_peered":"2024-09-10T22:50:27.013782+0000","last_unstale":"2024-09-10T22:50:28.026226+0000","last_undegraded":"2024-09-10T22:50:28.026226+0000","last_fullsized":"2024-09-10T22:50:28.026226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:22.655177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.029675+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:50:28.029675+0000","last_peered":"2024-09-10T22:50:28.029675+0000","last_clean":"2024-09-10T22:50:28.029675+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:50:28.029675+0000","last_undegraded":"2024-09-10T22:50:28.029675+0000","last_fullsized":"2024-09-10T22:50:28.029675+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025438+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:50:28.025438+0000","last_peered":"2024-09-10T22:50:28.025438+0000","last_clean":"2024-09-10T22:50:28.025438+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:50:28.025438+0000","last_undegraded":"2024-09-10T22:50:28.025438+0000","last_fullsized":"2024-09-10T22:50:28.025438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'20","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:46.066019+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:50:46.066019+0000","last_peered":"2024-09-10T22:50:46.066019+0000","last_clean":"2024-09-10T22:50:46.066019+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:50:46.066019+0000","last_undegraded":"2024-09-10T22:50:46.066019+0000","last_fullsized":"2024-09-10T22:50:46.066019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46800,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":60,"num_write":20,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022671+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:50:28.022671+0000","last_peered":"2024-09-10T22:50:28.022671+0000","last_clean":"2024-09-10T22:50:28.022671+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:50:28.022671+0000","last_undegraded":"2024-09-10T22:50:28.022671+0000","last_fullsized":"2024-09-10T22:50:28.022671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031247+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:50:28.031247+0000","last_peered":"2024-09-10T22:50:28.031247+0000","last_clean":"2024-09-10T22:50:28.031247+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:50:28.031247+0000","last_undegraded":"2024-09-10T22:50:28.031247+0000","last_fullsized":"2024-09-10T22:50:28.031247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022796+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:50:28.022796+0000","last_peered":"2024-09-10T22:50:28.022796+0000","last_clean":"2024-09-10T22:50:28.022796+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:50:28.022796+0000","last_undegraded":"2024-09-10T22:50:28.022796+0000","last_fullsized":"2024-09-10T22:50:28.022796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024305+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:50:28.024305+0000","last_peered":"2024-09-10T22:50:28.024305+0000","last_clean":"2024-09-10T22:50:28.024305+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:50:28.024305+0000","last_undegraded":"2024-09-10T22:50:28.024305+0000","last_fullsized":"2024-09-10T22:50:28.024305+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024615+0000","last_change":"2024-09-10T22:50:27.016226+0000","last_active":"2024-09-10T22:50:28.024615+0000","last_peered":"2024-09-10T22:50:28.024615+0000","last_clean":"2024-09-10T22:50:28.024615+0000","last_became_active":"2024-09-10T22:50:27.015904+0000","last_became_peered":"2024-09-10T22:50:27.015904+0000","last_unstale":"2024-09-10T22:50:28.024615+0000","last_undegraded":"2024-09-10T22:50:28.024615+0000","last_fullsized":"2024-09-10T22:50:28.024615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:47.344702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023587+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:50:28.023587+0000","last_peered":"2024-09-10T22:50:28.023587+0000","last_clean":"2024-09-10T22:50:28.023587+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:50:28.023587+0000","last_undegraded":"2024-09-10T22:50:28.023587+0000","last_fullsized":"2024-09-10T22:50:28.023587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024456+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:50:28.024456+0000","last_peered":"2024-09-10T22:50:28.024456+0000","last_clean":"2024-09-10T22:50:28.024456+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:50:28.024456+0000","last_undegraded":"2024-09-10T22:50:28.024456+0000","last_fullsized":"2024-09-10T22:50:28.024456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031628+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:50:28.031628+0000","last_peered":"2024-09-10T22:50:28.031628+0000","last_clean":"2024-09-10T22:50:28.031628+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:50:28.031628+0000","last_undegraded":"2024-09-10T22:50:28.031628+0000","last_fullsized":"2024-09-10T22:50:28.031628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024210+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:50:28.024210+0000","last_peered":"2024-09-10T22:50:28.024210+0000","last_clean":"2024-09-10T22:50:28.024210+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:50:28.024210+0000","last_undegraded":"2024-09-10T22:50:28.024210+0000","last_fullsized":"2024-09-10T22:50:28.024210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031262+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:50:28.031262+0000","last_peered":"2024-09-10T22:50:28.031262+0000","last_clean":"2024-09-10T22:50:28.031262+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:50:28.031262+0000","last_undegraded":"2024-09-10T22:50:28.031262+0000","last_fullsized":"2024-09-10T22:50:28.031262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010980+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:50:28.010980+0000","last_peered":"2024-09-10T22:50:28.010980+0000","last_clean":"2024-09-10T22:50:28.010980+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:50:28.010980+0000","last_undegraded":"2024-09-10T22:50:28.010980+0000","last_fullsized":"2024-09-10T22:50:28.010980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.029778+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:50:28.029778+0000","last_peered":"2024-09-10T22:50:28.029778+0000","last_clean":"2024-09-10T22:50:28.029778+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:50:28.029778+0000","last_undegraded":"2024-09-10T22:50:28.029778+0000","last_fullsized":"2024-09-10T22:50:28.029778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023713+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:50:28.023713+0000","last_peered":"2024-09-10T22:50:28.023713+0000","last_clean":"2024-09-10T22:50:28.023713+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:50:28.023713+0000","last_undegraded":"2024-09-10T22:50:28.023713+0000","last_fullsized":"2024-09-10T22:50:28.023713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.897796+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:50:28.897796+0000","last_peered":"2024-09-10T22:50:28.897796+0000","last_clean":"2024-09-10T22:50:28.897796+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:50:28.897796+0000","last_undegraded":"2024-09-10T22:50:28.897796+0000","last_fullsized":"2024-09-10T22:50:28.897796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:29.851657+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:50:29.851657+0000","last_peered":"2024-09-10T22:50:29.851657+0000","last_clean":"2024-09-10T22:50:29.851657+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:50:29.851657+0000","last_undegraded":"2024-09-10T22:50:29.851657+0000","last_fullsized":"2024-09-10T22:50:29.851657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027567+0000","last_change":"2024-09-10T22:50:25.292234+0000","last_active":"2024-09-10T22:50:28.027567+0000","last_peered":"2024-09-10T22:50:28.027567+0000","last_clean":"2024-09-10T22:50:28.027567+0000","last_became_active":"2024-09-10T22:50:25.291841+0000","last_became_peered":"2024-09-10T22:50:25.291841+0000","last_unstale":"2024-09-10T22:50:28.027567+0000","last_undegraded":"2024-09-10T22:50:28.027567+0000","last_fullsized":"2024-09-10T22:50:28.027567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:01.518495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023815+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:50:28.023815+0000","last_peered":"2024-09-10T22:50:28.023815+0000","last_clean":"2024-09-10T22:50:28.023815+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:50:28.023815+0000","last_undegraded":"2024-09-10T22:50:28.023815+0000","last_fullsized":"2024-09-10T22:50:28.023815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024516+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:50:28.024516+0000","last_peered":"2024-09-10T22:50:28.024516+0000","last_clean":"2024-09-10T22:50:28.024516+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:50:28.024516+0000","last_undegraded":"2024-09-10T22:50:28.024516+0000","last_fullsized":"2024-09-10T22:50:28.024516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023729+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:50:28.023729+0000","last_peered":"2024-09-10T22:50:28.023729+0000","last_clean":"2024-09-10T22:50:28.023729+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:50:28.023729+0000","last_undegraded":"2024-09-10T22:50:28.023729+0000","last_fullsized":"2024-09-10T22:50:28.023729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024439+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:50:28.024439+0000","last_peered":"2024-09-10T22:50:28.024439+0000","last_clean":"2024-09-10T22:50:28.024439+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:50:28.024439+0000","last_undegraded":"2024-09-10T22:50:28.024439+0000","last_fullsized":"2024-09-10T22:50:28.024439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024503+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:50:28.024503+0000","last_peered":"2024-09-10T22:50:28.024503+0000","last_clean":"2024-09-10T22:50:28.024503+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:50:28.024503+0000","last_undegraded":"2024-09-10T22:50:28.024503+0000","last_fullsized":"2024-09-10T22:50:28.024503+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023174+0000","last_change":"2024-09-10T22:50:27.013595+0000","last_active":"2024-09-10T22:50:28.023174+0000","last_peered":"2024-09-10T22:50:28.023174+0000","last_clean":"2024-09-10T22:50:28.023174+0000","last_became_active":"2024-09-10T22:50:27.013297+0000","last_became_peered":"2024-09-10T22:50:27.013297+0000","last_unstale":"2024-09-10T22:50:28.023174+0000","last_undegraded":"2024-09-10T22:50:28.023174+0000","last_fullsized":"2024-09-10T22:50:28.023174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:40.563283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022850+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:50:28.022850+0000","last_peered":"2024-09-10T22:50:28.022850+0000","last_clean":"2024-09-10T22:50:28.022850+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:50:28.022850+0000","last_undegraded":"2024-09-10T22:50:28.022850+0000","last_fullsized":"2024-09-10T22:50:28.022850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023959+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:50:28.023959+0000","last_peered":"2024-09-10T22:50:28.023959+0000","last_clean":"2024-09-10T22:50:28.023959+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:50:28.023959+0000","last_undegraded":"2024-09-10T22:50:28.023959+0000","last_fullsized":"2024-09-10T22:50:28.023959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024436+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:50:28.024436+0000","last_peered":"2024-09-10T22:50:28.024436+0000","last_clean":"2024-09-10T22:50:28.024436+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:50:28.024436+0000","last_undegraded":"2024-09-10T22:50:28.024436+0000","last_fullsized":"2024-09-10T22:50:28.024436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011224+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:50:28.011224+0000","last_peered":"2024-09-10T22:50:28.011224+0000","last_clean":"2024-09-10T22:50:28.011224+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:50:28.011224+0000","last_undegraded":"2024-09-10T22:50:28.011224+0000","last_fullsized":"2024-09-10T22:50:28.011224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010475+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:50:28.010475+0000","last_peered":"2024-09-10T22:50:28.010475+0000","last_clean":"2024-09-10T22:50:28.010475+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:50:28.010475+0000","last_undegraded":"2024-09-10T22:50:28.010475+0000","last_fullsized":"2024-09-10T22:50:28.010475+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011306+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:50:28.011306+0000","last_peered":"2024-09-10T22:50:28.011306+0000","last_clean":"2024-09-10T22:50:28.011306+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:50:28.011306+0000","last_undegraded":"2024-09-10T22:50:28.011306+0000","last_fullsized":"2024-09-10T22:50:28.011306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026449+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:50:28.026449+0000","last_peered":"2024-09-10T22:50:28.026449+0000","last_clean":"2024-09-10T22:50:28.026449+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:50:28.026449+0000","last_undegraded":"2024-09-10T22:50:28.026449+0000","last_fullsized":"2024-09-10T22:50:28.026449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023654+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:50:28.023654+0000","last_peered":"2024-09-10T22:50:28.023654+0000","last_clean":"2024-09-10T22:50:28.023654+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:50:28.023654+0000","last_undegraded":"2024-09-10T22:50:28.023654+0000","last_fullsized":"2024-09-10T22:50:28.023654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'6","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:45.023429+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:50:45.023429+0000","last_peered":"2024-09-10T22:50:45.023429+0000","last_clean":"2024-09-10T22:50:45.023429+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:50:45.023429+0000","last_undegraded":"2024-09-10T22:50:45.023429+0000","last_fullsized":"2024-09-10T22:50:45.023429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024201+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:50:28.024201+0000","last_peered":"2024-09-10T22:50:28.024201+0000","last_clean":"2024-09-10T22:50:28.024201+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:50:28.024201+0000","last_undegraded":"2024-09-10T22:50:28.024201+0000","last_fullsized":"2024-09-10T22:50:28.024201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025968+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:50:28.025968+0000","last_peered":"2024-09-10T22:50:28.025968+0000","last_clean":"2024-09-10T22:50:28.025968+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:50:28.025968+0000","last_undegraded":"2024-09-10T22:50:28.025968+0000","last_fullsized":"2024-09-10T22:50:28.025968+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026418+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:50:28.026418+0000","last_peered":"2024-09-10T22:50:28.026418+0000","last_clean":"2024-09-10T22:50:28.026418+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:50:28.026418+0000","last_undegraded":"2024-09-10T22:50:28.026418+0000","last_fullsized":"2024-09-10T22:50:28.026418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025236+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:50:28.025236+0000","last_peered":"2024-09-10T22:50:28.025236+0000","last_clean":"2024-09-10T22:50:28.025236+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:50:28.025236+0000","last_undegraded":"2024-09-10T22:50:28.025236+0000","last_fullsized":"2024-09-10T22:50:28.025236+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022462+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:50:28.022462+0000","last_peered":"2024-09-10T22:50:28.022462+0000","last_clean":"2024-09-10T22:50:28.022462+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:50:28.022462+0000","last_undegraded":"2024-09-10T22:50:28.022462+0000","last_fullsized":"2024-09-10T22:50:28.022462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023804+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:50:28.023804+0000","last_peered":"2024-09-10T22:50:28.023804+0000","last_clean":"2024-09-10T22:50:28.023804+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:50:28.023804+0000","last_undegraded":"2024-09-10T22:50:28.023804+0000","last_fullsized":"2024-09-10T22:50:28.023804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010588+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:50:28.010588+0000","last_peered":"2024-09-10T22:50:28.010588+0000","last_clean":"2024-09-10T22:50:28.010588+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:50:28.010588+0000","last_undegraded":"2024-09-10T22:50:28.010588+0000","last_fullsized":"2024-09-10T22:50:28.010588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022805+0000","last_change":"2024-09-10T22:50:25.001893+0000","last_active":"2024-09-10T22:50:28.022805+0000","last_peered":"2024-09-10T22:50:28.022805+0000","last_clean":"2024-09-10T22:50:28.022805+0000","last_became_active":"2024-09-10T22:50:25.001565+0000","last_became_peered":"2024-09-10T22:50:25.001565+0000","last_unstale":"2024-09-10T22:50:28.022805+0000","last_undegraded":"2024-09-10T22:50:28.022805+0000","last_fullsized":"2024-09-10T22:50:28.022805+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:59.673835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024751+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:50:28.024751+0000","last_peered":"2024-09-10T22:50:28.024751+0000","last_clean":"2024-09-10T22:50:28.024751+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:50:28.024751+0000","last_undegraded":"2024-09-10T22:50:28.024751+0000","last_fullsized":"2024-09-10T22:50:28.024751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031501+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:50:28.031501+0000","last_peered":"2024-09-10T22:50:28.031501+0000","last_clean":"2024-09-10T22:50:28.031501+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:50:28.031501+0000","last_undegraded":"2024-09-10T22:50:28.031501+0000","last_fullsized":"2024-09-10T22:50:28.031501+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024216+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:50:28.024216+0000","last_peered":"2024-09-10T22:50:28.024216+0000","last_clean":"2024-09-10T22:50:28.024216+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:50:28.024216+0000","last_undegraded":"2024-09-10T22:50:28.024216+0000","last_fullsized":"2024-09-10T22:50:28.024216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024470+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:50:28.024470+0000","last_peered":"2024-09-10T22:50:28.024470+0000","last_clean":"2024-09-10T22:50:28.024470+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:50:28.024470+0000","last_undegraded":"2024-09-10T22:50:28.024470+0000","last_fullsized":"2024-09-10T22:50:28.024470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.030844+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:50:28.030844+0000","last_peered":"2024-09-10T22:50:28.030844+0000","last_clean":"2024-09-10T22:50:28.030844+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:50:28.030844+0000","last_undegraded":"2024-09-10T22:50:28.030844+0000","last_fullsized":"2024-09-10T22:50:28.030844+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025330+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:50:28.025330+0000","last_peered":"2024-09-10T22:50:28.025330+0000","last_clean":"2024-09-10T22:50:28.025330+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:50:28.025330+0000","last_undegraded":"2024-09-10T22:50:28.025330+0000","last_fullsized":"2024-09-10T22:50:28.025330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031480+0000","last_change":"2024-09-10T22:50:24.998725+0000","last_active":"2024-09-10T22:50:28.031480+0000","last_peered":"2024-09-10T22:50:28.031480+0000","last_clean":"2024-09-10T22:50:28.031480+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:50:28.031480+0000","last_undegraded":"2024-09-10T22:50:28.031480+0000","last_fullsized":"2024-09-10T22:50:28.031480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:48.158115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026012+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:50:28.026012+0000","last_peered":"2024-09-10T22:50:28.026012+0000","last_clean":"2024-09-10T22:50:28.026012+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:50:28.026012+0000","last_undegraded":"2024-09-10T22:50:28.026012+0000","last_fullsized":"2024-09-10T22:50:28.026012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025227+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:50:28.025227+0000","last_peered":"2024-09-10T22:50:28.025227+0000","last_clean":"2024-09-10T22:50:28.025227+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:50:28.025227+0000","last_undegraded":"2024-09-10T22:50:28.025227+0000","last_fullsized":"2024-09-10T22:50:28.025227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024719+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:50:28.024719+0000","last_peered":"2024-09-10T22:50:28.024719+0000","last_clean":"2024-09-10T22:50:28.024719+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:50:28.024719+0000","last_undegraded":"2024-09-10T22:50:28.024719+0000","last_fullsized":"2024-09-10T22:50:28.024719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025918+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:50:28.025918+0000","last_peered":"2024-09-10T22:50:28.025918+0000","last_clean":"2024-09-10T22:50:28.025918+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:50:28.025918+0000","last_undegraded":"2024-09-10T22:50:28.025918+0000","last_fullsized":"2024-09-10T22:50:28.025918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47598,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":36,"num_read_kb":66,"num_write":69,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":152934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":802816,"data_stored":387657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1470464,"data_stored":1025136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070491,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":388239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":811008,"data_stored":388261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1404928,"data_stored":978724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":806912,"data_stored":387774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1413120,"data_stored":978324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":387679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":387995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":434491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":444597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":806912,"data_stored":387747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":46834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":46994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:50:51.047 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:50:51.295 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:51.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 -- 172.21.15.120:0/1871192939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 msgr2=0x7f2c9c10cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 --2- 172.21.15.120:0/1871192939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c10cb90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009900 tx=0x7f2c9002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 -- 172.21.15.120:0/1871192939 shutdown_connections 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 --2- 172.21.15.120:0/1871192939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 --2- 172.21.15.120:0/1871192939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c10cb90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 --2- 172.21.15.120:0/1871192939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 -- 172.21.15.120:0/1871192939 >> 172.21.15.120:0/1871192939 conn(0x7f2c9c0fd700 msgr2=0x7f2c9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.792+0000 7f2ca2218640 1 -- 172.21.15.120:0/1871192939 shutdown_connections 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 -- 172.21.15.120:0/1871192939 wait complete. 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 Processor -- start 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 -- start start 2024-09-10T22:50:51.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c112de0 con 0x7f2c9c1045b0 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c112c60 con 0x7f2c9c103bb0 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.793+0000 7f2ca2218640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c112f60 con 0x7f2c9c10d160 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55682/0 (socket says 172.21.15.120:55682) 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 -- 172.21.15.120:0/2733594000 learned_addr learned my addr 172.21.15.120:0/2733594000 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9b7fe640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9bfff640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 -- 172.21.15.120:0/2733594000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 msgr2=0x7f2c9c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:51.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 -- 172.21.15.120:0/2733594000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 msgr2=0x7f2c9c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 -- 172.21.15.120:0/2733594000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c19f360 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9b7fe640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9bfff640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c9affd640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9003f030 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c90003800 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c19f5f0 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90035070 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.794+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c1a5d10 con 0x7f2c9c1045b0 2024-09-10T22:50:51.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.795+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c103f90 con 0x7f2c9c1045b0 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.797+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f2c90031b10 con 0x7f2c9c1045b0 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.797+0000 7f2c98ff9640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 0x7f2c840b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.797+0000 7f2c9b7fe640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 0x7f2c840b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.797+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f2c901068d0 con 0x7f2c9c1045b0 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.798+0000 7f2c9b7fe640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 0x7f2c840b78b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0058c0 tx=0x7f2c8c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:51.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.800+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2c900cd6e0 con 0x7f2c9c1045b0 2024-09-10T22:50:51.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.973+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c9c0ffa20 con 0x7f2c840b53f0 2024-09-10T22:50:51.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.980+0000 7f2c98ff9640 1 -- 172.21.15.120:0/2733594000 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258622 (secure 0 0 0) 0x7f2c9c0ffa20 con 0x7f2c840b53f0 2024-09-10T22:50:51.983 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:51.984 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:50:51.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 msgr2=0x7f2c840b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 0x7f2c840b78b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0058c0 tx=0x7f2c8c005d30 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 msgr2=0x7f2c9c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c900042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 shutdown_connections 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c840b53f0 0x7f2c840b78b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c9c10d160 0x7f2c9c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c9c1045b0 0x7f2c9c1997f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 --2- 172.21.15.120:0/2733594000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c9c103bb0 0x7f2c9c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 >> 172.21.15.120:0/2733594000 conn(0x7f2c9c0fd700 msgr2=0x7f2c9c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 shutdown_connections 2024-09-10T22:50:51.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:51.983+0000 7f2ca2218640 1 -- 172.21.15.120:0/2733594000 wait complete. 2024-09-10T22:50:52.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:52 smithi120 ceph-mon[31247]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:52.335 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:52 smithi120 ceph-mon[31247]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2024-09-10T22:50:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:52 smithi176 ceph-mon[34982]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:52 smithi176 ceph-mon[34982]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2024-09-10T22:50:52.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:52 smithi153 ceph-mon[34951]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:50:52.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:52 smithi153 ceph-mon[34951]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2024-09-10T22:50:52.726 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-10T22:50:50.152355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638200,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":112,"num_read_kb":130,"num_write":194,"num_write_kb":2219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377312,"kb_used_data":11408,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564512,"statfs":{"total":1151940427776,"available":1151554060288,"internally_reserved":0,"allocated":11681792,"data_stored":6576624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45422,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":57,"num_write":26,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010885+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:50:28.010885+0000","last_peered":"2024-09-10T22:50:28.010885+0000","last_clean":"2024-09-10T22:50:28.010885+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:50:28.010885+0000","last_undegraded":"2024-09-10T22:50:28.010885+0000","last_fullsized":"2024-09-10T22:50:28.010885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026244+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:50:28.026244+0000","last_peered":"2024-09-10T22:50:28.026244+0000","last_clean":"2024-09-10T22:50:28.026244+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:50:28.026244+0000","last_undegraded":"2024-09-10T22:50:28.026244+0000","last_fullsized":"2024-09-10T22:50:28.026244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025805+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:50:28.025805+0000","last_peered":"2024-09-10T22:50:28.025805+0000","last_clean":"2024-09-10T22:50:28.025805+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:50:28.025805+0000","last_undegraded":"2024-09-10T22:50:28.025805+0000","last_fullsized":"2024-09-10T22:50:28.025805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023167+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:50:28.023167+0000","last_peered":"2024-09-10T22:50:28.023167+0000","last_clean":"2024-09-10T22:50:28.023167+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:50:28.023167+0000","last_undegraded":"2024-09-10T22:50:28.023167+0000","last_fullsized":"2024-09-10T22:50:28.023167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022490+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:50:28.022490+0000","last_peered":"2024-09-10T22:50:28.022490+0000","last_clean":"2024-09-10T22:50:28.022490+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:50:28.022490+0000","last_undegraded":"2024-09-10T22:50:28.022490+0000","last_fullsized":"2024-09-10T22:50:28.022490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010866+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:50:28.010866+0000","last_peered":"2024-09-10T22:50:28.010866+0000","last_clean":"2024-09-10T22:50:28.010866+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:50:28.010866+0000","last_undegraded":"2024-09-10T22:50:28.010866+0000","last_fullsized":"2024-09-10T22:50:28.010866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023282+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:50:28.023282+0000","last_peered":"2024-09-10T22:50:28.023282+0000","last_clean":"2024-09-10T22:50:28.023282+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:50:28.023282+0000","last_undegraded":"2024-09-10T22:50:28.023282+0000","last_fullsized":"2024-09-10T22:50:28.023282+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024418+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:50:28.024418+0000","last_peered":"2024-09-10T22:50:28.024418+0000","last_clean":"2024-09-10T22:50:28.024418+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:50:28.024418+0000","last_undegraded":"2024-09-10T22:50:28.024418+0000","last_fullsized":"2024-09-10T22:50:28.024418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025789+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:50:28.025789+0000","last_peered":"2024-09-10T22:50:28.025789+0000","last_clean":"2024-09-10T22:50:28.025789+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:50:28.025789+0000","last_undegraded":"2024-09-10T22:50:28.025789+0000","last_fullsized":"2024-09-10T22:50:28.025789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011429+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:50:28.011429+0000","last_peered":"2024-09-10T22:50:28.011429+0000","last_clean":"2024-09-10T22:50:28.011429+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:50:28.011429+0000","last_undegraded":"2024-09-10T22:50:28.011429+0000","last_fullsized":"2024-09-10T22:50:28.011429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023673+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:50:28.023673+0000","last_peered":"2024-09-10T22:50:28.023673+0000","last_clean":"2024-09-10T22:50:28.023673+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:50:28.023673+0000","last_undegraded":"2024-09-10T22:50:28.023673+0000","last_fullsized":"2024-09-10T22:50:28.023673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023207+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:50:28.023207+0000","last_peered":"2024-09-10T22:50:28.023207+0000","last_clean":"2024-09-10T22:50:28.023207+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:50:28.023207+0000","last_undegraded":"2024-09-10T22:50:28.023207+0000","last_fullsized":"2024-09-10T22:50:28.023207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022980+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:50:28.022980+0000","last_peered":"2024-09-10T22:50:28.022980+0000","last_clean":"2024-09-10T22:50:28.022980+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:50:28.022980+0000","last_undegraded":"2024-09-10T22:50:28.022980+0000","last_fullsized":"2024-09-10T22:50:28.022980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025604+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:50:28.025604+0000","last_peered":"2024-09-10T22:50:28.025604+0000","last_clean":"2024-09-10T22:50:28.025604+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:50:28.025604+0000","last_undegraded":"2024-09-10T22:50:28.025604+0000","last_fullsized":"2024-09-10T22:50:28.025604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025461+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:50:28.025461+0000","last_peered":"2024-09-10T22:50:28.025461+0000","last_clean":"2024-09-10T22:50:28.025461+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:50:28.025461+0000","last_undegraded":"2024-09-10T22:50:28.025461+0000","last_fullsized":"2024-09-10T22:50:28.025461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025180+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:50:28.025180+0000","last_peered":"2024-09-10T22:50:28.025180+0000","last_clean":"2024-09-10T22:50:28.025180+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:50:28.025180+0000","last_undegraded":"2024-09-10T22:50:28.025180+0000","last_fullsized":"2024-09-10T22:50:28.025180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010897+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:50:28.010897+0000","last_peered":"2024-09-10T22:50:28.010897+0000","last_clean":"2024-09-10T22:50:28.010897+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:50:28.010897+0000","last_undegraded":"2024-09-10T22:50:28.010897+0000","last_fullsized":"2024-09-10T22:50:28.010897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010551+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:50:28.010551+0000","last_peered":"2024-09-10T22:50:28.010551+0000","last_clean":"2024-09-10T22:50:28.010551+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:50:28.010551+0000","last_undegraded":"2024-09-10T22:50:28.010551+0000","last_fullsized":"2024-09-10T22:50:28.010551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027910+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:50:28.027910+0000","last_peered":"2024-09-10T22:50:28.027910+0000","last_clean":"2024-09-10T22:50:28.027910+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:50:28.027910+0000","last_undegraded":"2024-09-10T22:50:28.027910+0000","last_fullsized":"2024-09-10T22:50:28.027910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010883+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:50:28.010883+0000","last_peered":"2024-09-10T22:50:28.010883+0000","last_clean":"2024-09-10T22:50:28.010883+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:50:28.010883+0000","last_undegraded":"2024-09-10T22:50:28.010883+0000","last_fullsized":"2024-09-10T22:50:28.010883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022987+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:50:28.022987+0000","last_peered":"2024-09-10T22:50:28.022987+0000","last_clean":"2024-09-10T22:50:28.022987+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:50:28.022987+0000","last_undegraded":"2024-09-10T22:50:28.022987+0000","last_fullsized":"2024-09-10T22:50:28.022987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025815+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:50:28.025815+0000","last_peered":"2024-09-10T22:50:28.025815+0000","last_clean":"2024-09-10T22:50:28.025815+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:50:28.025815+0000","last_undegraded":"2024-09-10T22:50:28.025815+0000","last_fullsized":"2024-09-10T22:50:28.025815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023606+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:50:28.023606+0000","last_peered":"2024-09-10T22:50:28.023606+0000","last_clean":"2024-09-10T22:50:28.023606+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:50:28.023606+0000","last_undegraded":"2024-09-10T22:50:28.023606+0000","last_fullsized":"2024-09-10T22:50:28.023606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023962+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:50:28.023962+0000","last_peered":"2024-09-10T22:50:28.023962+0000","last_clean":"2024-09-10T22:50:28.023962+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:50:28.023962+0000","last_undegraded":"2024-09-10T22:50:28.023962+0000","last_fullsized":"2024-09-10T22:50:28.023962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023986+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:50:28.023986+0000","last_peered":"2024-09-10T22:50:28.023986+0000","last_clean":"2024-09-10T22:50:28.023986+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:50:28.023986+0000","last_undegraded":"2024-09-10T22:50:28.023986+0000","last_fullsized":"2024-09-10T22:50:28.023986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.028214+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:50:28.028214+0000","last_peered":"2024-09-10T22:50:28.028214+0000","last_clean":"2024-09-10T22:50:28.028214+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:50:28.028214+0000","last_undegraded":"2024-09-10T22:50:28.028214+0000","last_fullsized":"2024-09-10T22:50:28.028214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011135+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:50:28.011135+0000","last_peered":"2024-09-10T22:50:28.011135+0000","last_clean":"2024-09-10T22:50:28.011135+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:50:28.011135+0000","last_undegraded":"2024-09-10T22:50:28.011135+0000","last_fullsized":"2024-09-10T22:50:28.011135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023455+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:50:28.023455+0000","last_peered":"2024-09-10T22:50:28.023455+0000","last_clean":"2024-09-10T22:50:28.023455+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:50:28.023455+0000","last_undegraded":"2024-09-10T22:50:28.023455+0000","last_fullsized":"2024-09-10T22:50:28.023455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026170+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:50:28.026170+0000","last_peered":"2024-09-10T22:50:28.026170+0000","last_clean":"2024-09-10T22:50:28.026170+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:50:28.026170+0000","last_undegraded":"2024-09-10T22:50:28.026170+0000","last_fullsized":"2024-09-10T22:50:28.026170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022672+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:50:28.022672+0000","last_peered":"2024-09-10T22:50:28.022672+0000","last_clean":"2024-09-10T22:50:28.022672+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:50:28.022672+0000","last_undegraded":"2024-09-10T22:50:28.022672+0000","last_fullsized":"2024-09-10T22:50:28.022672+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027308+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:50:28.027308+0000","last_peered":"2024-09-10T22:50:28.027308+0000","last_clean":"2024-09-10T22:50:28.027308+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:50:28.027308+0000","last_undegraded":"2024-09-10T22:50:28.027308+0000","last_fullsized":"2024-09-10T22:50:28.027308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010521+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:50:28.010521+0000","last_peered":"2024-09-10T22:50:28.010521+0000","last_clean":"2024-09-10T22:50:28.010521+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:50:28.010521+0000","last_undegraded":"2024-09-10T22:50:28.010521+0000","last_fullsized":"2024-09-10T22:50:28.010521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022560+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:50:28.022560+0000","last_peered":"2024-09-10T22:50:28.022560+0000","last_clean":"2024-09-10T22:50:28.022560+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:50:28.022560+0000","last_undegraded":"2024-09-10T22:50:28.022560+0000","last_fullsized":"2024-09-10T22:50:28.022560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031747+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:50:28.031747+0000","last_peered":"2024-09-10T22:50:28.031747+0000","last_clean":"2024-09-10T22:50:28.031747+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:50:28.031747+0000","last_undegraded":"2024-09-10T22:50:28.031747+0000","last_fullsized":"2024-09-10T22:50:28.031747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027131+0000","last_change":"2024-09-10T22:50:27.014963+0000","last_active":"2024-09-10T22:50:28.027131+0000","last_peered":"2024-09-10T22:50:28.027131+0000","last_clean":"2024-09-10T22:50:28.027131+0000","last_became_active":"2024-09-10T22:50:27.013791+0000","last_became_peered":"2024-09-10T22:50:27.013791+0000","last_unstale":"2024-09-10T22:50:28.027131+0000","last_undegraded":"2024-09-10T22:50:28.027131+0000","last_fullsized":"2024-09-10T22:50:28.027131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:51.048840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010994+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:50:28.010994+0000","last_peered":"2024-09-10T22:50:28.010994+0000","last_clean":"2024-09-10T22:50:28.010994+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:50:28.010994+0000","last_undegraded":"2024-09-10T22:50:28.010994+0000","last_fullsized":"2024-09-10T22:50:28.010994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022610+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:50:28.022610+0000","last_peered":"2024-09-10T22:50:28.022610+0000","last_clean":"2024-09-10T22:50:28.022610+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:50:28.022610+0000","last_undegraded":"2024-09-10T22:50:28.022610+0000","last_fullsized":"2024-09-10T22:50:28.022610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024259+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:50:28.024259+0000","last_peered":"2024-09-10T22:50:28.024259+0000","last_clean":"2024-09-10T22:50:28.024259+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:50:28.024259+0000","last_undegraded":"2024-09-10T22:50:28.024259+0000","last_fullsized":"2024-09-10T22:50:28.024259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024159+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:50:28.024159+0000","last_peered":"2024-09-10T22:50:28.024159+0000","last_clean":"2024-09-10T22:50:28.024159+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:50:28.024159+0000","last_undegraded":"2024-09-10T22:50:28.024159+0000","last_fullsized":"2024-09-10T22:50:28.024159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010350+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:50:28.010350+0000","last_peered":"2024-09-10T22:50:28.010350+0000","last_clean":"2024-09-10T22:50:28.010350+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:50:28.010350+0000","last_undegraded":"2024-09-10T22:50:28.010350+0000","last_fullsized":"2024-09-10T22:50:28.010350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011351+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:50:28.011351+0000","last_peered":"2024-09-10T22:50:28.011351+0000","last_clean":"2024-09-10T22:50:28.011351+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:50:28.011351+0000","last_undegraded":"2024-09-10T22:50:28.011351+0000","last_fullsized":"2024-09-10T22:50:28.011351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025694+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:50:28.025694+0000","last_peered":"2024-09-10T22:50:28.025694+0000","last_clean":"2024-09-10T22:50:28.025694+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:50:28.025694+0000","last_undegraded":"2024-09-10T22:50:28.025694+0000","last_fullsized":"2024-09-10T22:50:28.025694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023547+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:50:28.023547+0000","last_peered":"2024-09-10T22:50:28.023547+0000","last_clean":"2024-09-10T22:50:28.023547+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:50:28.023547+0000","last_undegraded":"2024-09-10T22:50:28.023547+0000","last_fullsized":"2024-09-10T22:50:28.023547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026226+0000","last_change":"2024-09-10T22:50:27.014115+0000","last_active":"2024-09-10T22:50:28.026226+0000","last_peered":"2024-09-10T22:50:28.026226+0000","last_clean":"2024-09-10T22:50:28.026226+0000","last_became_active":"2024-09-10T22:50:27.013782+0000","last_became_peered":"2024-09-10T22:50:27.013782+0000","last_unstale":"2024-09-10T22:50:28.026226+0000","last_undegraded":"2024-09-10T22:50:28.026226+0000","last_fullsized":"2024-09-10T22:50:28.026226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:22.655177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.029675+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:50:28.029675+0000","last_peered":"2024-09-10T22:50:28.029675+0000","last_clean":"2024-09-10T22:50:28.029675+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:50:28.029675+0000","last_undegraded":"2024-09-10T22:50:28.029675+0000","last_fullsized":"2024-09-10T22:50:28.029675+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025438+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:50:28.025438+0000","last_peered":"2024-09-10T22:50:28.025438+0000","last_clean":"2024-09-10T22:50:28.025438+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:50:28.025438+0000","last_undegraded":"2024-09-10T22:50:28.025438+0000","last_fullsized":"2024-09-10T22:50:28.025438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'20","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:46.066019+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:50:46.066019+0000","last_peered":"2024-09-10T22:50:46.066019+0000","last_clean":"2024-09-10T22:50:46.066019+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:50:46.066019+0000","last_undegraded":"2024-09-10T22:50:46.066019+0000","last_fullsized":"2024-09-10T22:50:46.066019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46800,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":60,"num_write":20,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022671+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:50:28.022671+0000","last_peered":"2024-09-10T22:50:28.022671+0000","last_clean":"2024-09-10T22:50:28.022671+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:50:28.022671+0000","last_undegraded":"2024-09-10T22:50:28.022671+0000","last_fullsized":"2024-09-10T22:50:28.022671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031247+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:50:28.031247+0000","last_peered":"2024-09-10T22:50:28.031247+0000","last_clean":"2024-09-10T22:50:28.031247+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:50:28.031247+0000","last_undegraded":"2024-09-10T22:50:28.031247+0000","last_fullsized":"2024-09-10T22:50:28.031247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022796+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:50:28.022796+0000","last_peered":"2024-09-10T22:50:28.022796+0000","last_clean":"2024-09-10T22:50:28.022796+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:50:28.022796+0000","last_undegraded":"2024-09-10T22:50:28.022796+0000","last_fullsized":"2024-09-10T22:50:28.022796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024305+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:50:28.024305+0000","last_peered":"2024-09-10T22:50:28.024305+0000","last_clean":"2024-09-10T22:50:28.024305+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:50:28.024305+0000","last_undegraded":"2024-09-10T22:50:28.024305+0000","last_fullsized":"2024-09-10T22:50:28.024305+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024615+0000","last_change":"2024-09-10T22:50:27.016226+0000","last_active":"2024-09-10T22:50:28.024615+0000","last_peered":"2024-09-10T22:50:28.024615+0000","last_clean":"2024-09-10T22:50:28.024615+0000","last_became_active":"2024-09-10T22:50:27.015904+0000","last_became_peered":"2024-09-10T22:50:27.015904+0000","last_unstale":"2024-09-10T22:50:28.024615+0000","last_undegraded":"2024-09-10T22:50:28.024615+0000","last_fullsized":"2024-09-10T22:50:28.024615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:47.344702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023587+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:50:28.023587+0000","last_peered":"2024-09-10T22:50:28.023587+0000","last_clean":"2024-09-10T22:50:28.023587+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:50:28.023587+0000","last_undegraded":"2024-09-10T22:50:28.023587+0000","last_fullsized":"2024-09-10T22:50:28.023587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024456+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:50:28.024456+0000","last_peered":"2024-09-10T22:50:28.024456+0000","last_clean":"2024-09-10T22:50:28.024456+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:50:28.024456+0000","last_undegraded":"2024-09-10T22:50:28.024456+0000","last_fullsized":"2024-09-10T22:50:28.024456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031628+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:50:28.031628+0000","last_peered":"2024-09-10T22:50:28.031628+0000","last_clean":"2024-09-10T22:50:28.031628+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:50:28.031628+0000","last_undegraded":"2024-09-10T22:50:28.031628+0000","last_fullsized":"2024-09-10T22:50:28.031628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024210+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:50:28.024210+0000","last_peered":"2024-09-10T22:50:28.024210+0000","last_clean":"2024-09-10T22:50:28.024210+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:50:28.024210+0000","last_undegraded":"2024-09-10T22:50:28.024210+0000","last_fullsized":"2024-09-10T22:50:28.024210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031262+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:50:28.031262+0000","last_peered":"2024-09-10T22:50:28.031262+0000","last_clean":"2024-09-10T22:50:28.031262+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:50:28.031262+0000","last_undegraded":"2024-09-10T22:50:28.031262+0000","last_fullsized":"2024-09-10T22:50:28.031262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010980+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:50:28.010980+0000","last_peered":"2024-09-10T22:50:28.010980+0000","last_clean":"2024-09-10T22:50:28.010980+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:50:28.010980+0000","last_undegraded":"2024-09-10T22:50:28.010980+0000","last_fullsized":"2024-09-10T22:50:28.010980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.029778+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:50:28.029778+0000","last_peered":"2024-09-10T22:50:28.029778+0000","last_clean":"2024-09-10T22:50:28.029778+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:50:28.029778+0000","last_undegraded":"2024-09-10T22:50:28.029778+0000","last_fullsized":"2024-09-10T22:50:28.029778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023713+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:50:28.023713+0000","last_peered":"2024-09-10T22:50:28.023713+0000","last_clean":"2024-09-10T22:50:28.023713+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:50:28.023713+0000","last_undegraded":"2024-09-10T22:50:28.023713+0000","last_fullsized":"2024-09-10T22:50:28.023713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.897796+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:50:28.897796+0000","last_peered":"2024-09-10T22:50:28.897796+0000","last_clean":"2024-09-10T22:50:28.897796+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:50:28.897796+0000","last_undegraded":"2024-09-10T22:50:28.897796+0000","last_fullsized":"2024-09-10T22:50:28.897796+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:29.851657+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:50:29.851657+0000","last_peered":"2024-09-10T22:50:29.851657+0000","last_clean":"2024-09-10T22:50:29.851657+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:50:29.851657+0000","last_undegraded":"2024-09-10T22:50:29.851657+0000","last_fullsized":"2024-09-10T22:50:29.851657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.027567+0000","last_change":"2024-09-10T22:50:25.292234+0000","last_active":"2024-09-10T22:50:28.027567+0000","last_peered":"2024-09-10T22:50:28.027567+0000","last_clean":"2024-09-10T22:50:28.027567+0000","last_became_active":"2024-09-10T22:50:25.291841+0000","last_became_peered":"2024-09-10T22:50:25.291841+0000","last_unstale":"2024-09-10T22:50:28.027567+0000","last_undegraded":"2024-09-10T22:50:28.027567+0000","last_fullsized":"2024-09-10T22:50:28.027567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:01.518495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023815+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:50:28.023815+0000","last_peered":"2024-09-10T22:50:28.023815+0000","last_clean":"2024-09-10T22:50:28.023815+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:50:28.023815+0000","last_undegraded":"2024-09-10T22:50:28.023815+0000","last_fullsized":"2024-09-10T22:50:28.023815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024516+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:50:28.024516+0000","last_peered":"2024-09-10T22:50:28.024516+0000","last_clean":"2024-09-10T22:50:28.024516+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:50:28.024516+0000","last_undegraded":"2024-09-10T22:50:28.024516+0000","last_fullsized":"2024-09-10T22:50:28.024516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023729+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:50:28.023729+0000","last_peered":"2024-09-10T22:50:28.023729+0000","last_clean":"2024-09-10T22:50:28.023729+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:50:28.023729+0000","last_undegraded":"2024-09-10T22:50:28.023729+0000","last_fullsized":"2024-09-10T22:50:28.023729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024439+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:50:28.024439+0000","last_peered":"2024-09-10T22:50:28.024439+0000","last_clean":"2024-09-10T22:50:28.024439+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:50:28.024439+0000","last_undegraded":"2024-09-10T22:50:28.024439+0000","last_fullsized":"2024-09-10T22:50:28.024439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024503+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:50:28.024503+0000","last_peered":"2024-09-10T22:50:28.024503+0000","last_clean":"2024-09-10T22:50:28.024503+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:50:28.024503+0000","last_undegraded":"2024-09-10T22:50:28.024503+0000","last_fullsized":"2024-09-10T22:50:28.024503+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023174+0000","last_change":"2024-09-10T22:50:27.013595+0000","last_active":"2024-09-10T22:50:28.023174+0000","last_peered":"2024-09-10T22:50:28.023174+0000","last_clean":"2024-09-10T22:50:28.023174+0000","last_became_active":"2024-09-10T22:50:27.013297+0000","last_became_peered":"2024-09-10T22:50:27.013297+0000","last_unstale":"2024-09-10T22:50:28.023174+0000","last_undegraded":"2024-09-10T22:50:28.023174+0000","last_fullsized":"2024-09-10T22:50:28.023174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:40.563283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022850+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:50:28.022850+0000","last_peered":"2024-09-10T22:50:28.022850+0000","last_clean":"2024-09-10T22:50:28.022850+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:50:28.022850+0000","last_undegraded":"2024-09-10T22:50:28.022850+0000","last_fullsized":"2024-09-10T22:50:28.022850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023959+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:50:28.023959+0000","last_peered":"2024-09-10T22:50:28.023959+0000","last_clean":"2024-09-10T22:50:28.023959+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:50:28.023959+0000","last_undegraded":"2024-09-10T22:50:28.023959+0000","last_fullsized":"2024-09-10T22:50:28.023959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024436+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:50:28.024436+0000","last_peered":"2024-09-10T22:50:28.024436+0000","last_clean":"2024-09-10T22:50:28.024436+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:50:28.024436+0000","last_undegraded":"2024-09-10T22:50:28.024436+0000","last_fullsized":"2024-09-10T22:50:28.024436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011224+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:50:28.011224+0000","last_peered":"2024-09-10T22:50:28.011224+0000","last_clean":"2024-09-10T22:50:28.011224+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:50:28.011224+0000","last_undegraded":"2024-09-10T22:50:28.011224+0000","last_fullsized":"2024-09-10T22:50:28.011224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010475+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:50:28.010475+0000","last_peered":"2024-09-10T22:50:28.010475+0000","last_clean":"2024-09-10T22:50:28.010475+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:50:28.010475+0000","last_undegraded":"2024-09-10T22:50:28.010475+0000","last_fullsized":"2024-09-10T22:50:28.010475+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.011306+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:50:28.011306+0000","last_peered":"2024-09-10T22:50:28.011306+0000","last_clean":"2024-09-10T22:50:28.011306+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:50:28.011306+0000","last_undegraded":"2024-09-10T22:50:28.011306+0000","last_fullsized":"2024-09-10T22:50:28.011306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026449+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:50:28.026449+0000","last_peered":"2024-09-10T22:50:28.026449+0000","last_clean":"2024-09-10T22:50:28.026449+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:50:28.026449+0000","last_undegraded":"2024-09-10T22:50:28.026449+0000","last_fullsized":"2024-09-10T22:50:28.026449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023654+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:50:28.023654+0000","last_peered":"2024-09-10T22:50:28.023654+0000","last_clean":"2024-09-10T22:50:28.023654+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:50:28.023654+0000","last_undegraded":"2024-09-10T22:50:28.023654+0000","last_fullsized":"2024-09-10T22:50:28.023654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'6","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:45.023429+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:50:45.023429+0000","last_peered":"2024-09-10T22:50:45.023429+0000","last_clean":"2024-09-10T22:50:45.023429+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:50:45.023429+0000","last_undegraded":"2024-09-10T22:50:45.023429+0000","last_fullsized":"2024-09-10T22:50:45.023429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024201+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:50:28.024201+0000","last_peered":"2024-09-10T22:50:28.024201+0000","last_clean":"2024-09-10T22:50:28.024201+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:50:28.024201+0000","last_undegraded":"2024-09-10T22:50:28.024201+0000","last_fullsized":"2024-09-10T22:50:28.024201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025968+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:50:28.025968+0000","last_peered":"2024-09-10T22:50:28.025968+0000","last_clean":"2024-09-10T22:50:28.025968+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:50:28.025968+0000","last_undegraded":"2024-09-10T22:50:28.025968+0000","last_fullsized":"2024-09-10T22:50:28.025968+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026418+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:50:28.026418+0000","last_peered":"2024-09-10T22:50:28.026418+0000","last_clean":"2024-09-10T22:50:28.026418+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:50:28.026418+0000","last_undegraded":"2024-09-10T22:50:28.026418+0000","last_fullsized":"2024-09-10T22:50:28.026418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025236+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:50:28.025236+0000","last_peered":"2024-09-10T22:50:28.025236+0000","last_clean":"2024-09-10T22:50:28.025236+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:50:28.025236+0000","last_undegraded":"2024-09-10T22:50:28.025236+0000","last_fullsized":"2024-09-10T22:50:28.025236+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022462+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:50:28.022462+0000","last_peered":"2024-09-10T22:50:28.022462+0000","last_clean":"2024-09-10T22:50:28.022462+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:50:28.022462+0000","last_undegraded":"2024-09-10T22:50:28.022462+0000","last_fullsized":"2024-09-10T22:50:28.022462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.023804+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:50:28.023804+0000","last_peered":"2024-09-10T22:50:28.023804+0000","last_clean":"2024-09-10T22:50:28.023804+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:50:28.023804+0000","last_undegraded":"2024-09-10T22:50:28.023804+0000","last_fullsized":"2024-09-10T22:50:28.023804+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.010588+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:50:28.010588+0000","last_peered":"2024-09-10T22:50:28.010588+0000","last_clean":"2024-09-10T22:50:28.010588+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:50:28.010588+0000","last_undegraded":"2024-09-10T22:50:28.010588+0000","last_fullsized":"2024-09-10T22:50:28.010588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.022805+0000","last_change":"2024-09-10T22:50:25.001893+0000","last_active":"2024-09-10T22:50:28.022805+0000","last_peered":"2024-09-10T22:50:28.022805+0000","last_clean":"2024-09-10T22:50:28.022805+0000","last_became_active":"2024-09-10T22:50:25.001565+0000","last_became_peered":"2024-09-10T22:50:25.001565+0000","last_unstale":"2024-09-10T22:50:28.022805+0000","last_undegraded":"2024-09-10T22:50:28.022805+0000","last_fullsized":"2024-09-10T22:50:28.022805+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:59.673835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024751+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:50:28.024751+0000","last_peered":"2024-09-10T22:50:28.024751+0000","last_clean":"2024-09-10T22:50:28.024751+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:50:28.024751+0000","last_undegraded":"2024-09-10T22:50:28.024751+0000","last_fullsized":"2024-09-10T22:50:28.024751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031501+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:50:28.031501+0000","last_peered":"2024-09-10T22:50:28.031501+0000","last_clean":"2024-09-10T22:50:28.031501+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:50:28.031501+0000","last_undegraded":"2024-09-10T22:50:28.031501+0000","last_fullsized":"2024-09-10T22:50:28.031501+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024216+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:50:28.024216+0000","last_peered":"2024-09-10T22:50:28.024216+0000","last_clean":"2024-09-10T22:50:28.024216+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:50:28.024216+0000","last_undegraded":"2024-09-10T22:50:28.024216+0000","last_fullsized":"2024-09-10T22:50:28.024216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024470+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:50:28.024470+0000","last_peered":"2024-09-10T22:50:28.024470+0000","last_clean":"2024-09-10T22:50:28.024470+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:50:28.024470+0000","last_undegraded":"2024-09-10T22:50:28.024470+0000","last_fullsized":"2024-09-10T22:50:28.024470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.030844+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:50:28.030844+0000","last_peered":"2024-09-10T22:50:28.030844+0000","last_clean":"2024-09-10T22:50:28.030844+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:50:28.030844+0000","last_undegraded":"2024-09-10T22:50:28.030844+0000","last_fullsized":"2024-09-10T22:50:28.030844+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025330+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:50:28.025330+0000","last_peered":"2024-09-10T22:50:28.025330+0000","last_clean":"2024-09-10T22:50:28.025330+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:50:28.025330+0000","last_undegraded":"2024-09-10T22:50:28.025330+0000","last_fullsized":"2024-09-10T22:50:28.025330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.031480+0000","last_change":"2024-09-10T22:50:24.998725+0000","last_active":"2024-09-10T22:50:28.031480+0000","last_peered":"2024-09-10T22:50:28.031480+0000","last_clean":"2024-09-10T22:50:28.031480+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:50:28.031480+0000","last_undegraded":"2024-09-10T22:50:28.031480+0000","last_fullsized":"2024-09-10T22:50:28.031480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:48.158115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.026012+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:50:28.026012+0000","last_peered":"2024-09-10T22:50:28.026012+0000","last_clean":"2024-09-10T22:50:28.026012+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:50:28.026012+0000","last_undegraded":"2024-09-10T22:50:28.026012+0000","last_fullsized":"2024-09-10T22:50:28.026012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025227+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:50:28.025227+0000","last_peered":"2024-09-10T22:50:28.025227+0000","last_clean":"2024-09-10T22:50:28.025227+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:50:28.025227+0000","last_undegraded":"2024-09-10T22:50:28.025227+0000","last_fullsized":"2024-09-10T22:50:28.025227+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.024719+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:50:28.024719+0000","last_peered":"2024-09-10T22:50:28.024719+0000","last_clean":"2024-09-10T22:50:28.024719+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:50:28.024719+0000","last_undegraded":"2024-09-10T22:50:28.024719+0000","last_fullsized":"2024-09-10T22:50:28.024719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:50:28.025918+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:50:28.025918+0000","last_peered":"2024-09-10T22:50:28.025918+0000","last_clean":"2024-09-10T22:50:28.025918+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:50:28.025918+0000","last_undegraded":"2024-09-10T22:50:28.025918+0000","last_fullsized":"2024-09-10T22:50:28.025918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47598,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":36,"num_read_kb":66,"num_write":69,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":152934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":802816,"data_stored":387657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1470464,"data_stored":1025136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070491,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":388239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":811008,"data_stored":388261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1404928,"data_stored":978724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":806912,"data_stored":387774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1413120,"data_stored":978324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":387679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":387995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":434491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":444597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":806912,"data_stored":387747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":46834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":46994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:50:52.729 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:50:52.730 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T22:50:52.730 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T22:50:52.975 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:53.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 -- 172.21.15.120:0/644787827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990103450 msgr2=0x7fd990109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 --2- 172.21.15.120:0/644787827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990103450 0x7fd990109dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd988009cd0 tx=0x7fd98802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 -- 172.21.15.120:0/644787827 shutdown_connections 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 --2- 172.21.15.120:0/644787827 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990103450 0x7fd990109dd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 --2- 172.21.15.120:0/644787827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990102a90 0x7fd990102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 --2- 172.21.15.120:0/644787827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 0x7fd990101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 -- 172.21.15.120:0/644787827 >> 172.21.15.120:0/644787827 conn(0x7fd9900fd200 msgr2=0x7fd9900ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 -- 172.21.15.120:0/644787827 shutdown_connections 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.449+0000 7fd9981ac640 1 -- 172.21.15.120:0/644787827 wait complete. 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 Processor -- start 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 -- start start 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 0x7fd99006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 0x7fd9901a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d610 con 0x7fd990102a90 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d490 con 0x7fd990101930 2024-09-10T22:50:53.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd9981ac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d790 con 0x7fd990103450 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39964/0 (socket says 172.21.15.120:39964) 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 -- 172.21.15.120:0/3713005408 learned_addr learned my addr 172.21.15.120:0/3713005408 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995f48640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 0x7fd99006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd996749640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 0x7fd9901a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 -- 172.21.15.120:0/3713005408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 msgr2=0x7fd9901a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 0x7fd9901a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 -- 172.21.15.120:0/3713005408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 msgr2=0x7fd99006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 0x7fd99006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 -- 172.21.15.120:0/3713005408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9901a0ab0 con 0x7fd990102a90 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd996749640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 0x7fd9901a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.450+0000 7fd995747640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b4a0 tx=0x7fd98000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.451+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980014070 con 0x7fd990102a90 2024-09-10T22:50:53.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.451+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9901a0da0 con 0x7fd990102a90 2024-09-10T22:50:53.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.451+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980010ac0 con 0x7fd990102a90 2024-09-10T22:50:53.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.451+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980017c30 con 0x7fd990102a90 2024-09-10T22:50:53.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.451+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9901a1260 con 0x7fd990102a90 2024-09-10T22:50:53.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.452+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd990108cb0 con 0x7fd990102a90 2024-09-10T22:50:53.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.454+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fd980010610 con 0x7fd990102a90 2024-09-10T22:50:53.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.455+0000 7fd986ffd640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 0x7fd97c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.455+0000 7fd995f48640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 0x7fd97c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.455+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd9800e5520 con 0x7fd990102a90 2024-09-10T22:50:53.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.455+0000 7fd995f48640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 0x7fd97c0b7830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005d00 tx=0x7fd98c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.458+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd9800ac330 con 0x7fd990102a90 2024-09-10T22:50:53.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.703+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd99002e7c0 con 0x7fd990102a90 2024-09-10T22:50:53.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.705+0000 7fd986ffd640 1 -- 172.21.15.120:0/3713005408 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd9800abc30 con 0x7fd990102a90 2024-09-10T22:50:53.707 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-09-10T22:50:53.707 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBNzeBmm30CKhAAscxfO9bORR9zFiwWop3WfA== 2024-09-10T22:50:53.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 msgr2=0x7fd97c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 0x7fd97c0b7830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005d00 tx=0x7fd98c00b410 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 msgr2=0x7fd99019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b4a0 tx=0x7fd98000b960 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 shutdown_connections 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd97c0b5370 0x7fd97c0b7830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd990103450 0x7fd9901a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd990102a90 0x7fd99019dc80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 --2- 172.21.15.120:0/3713005408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd990101930 0x7fd99006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 >> 172.21.15.120:0/3713005408 conn(0x7fd9900fd200 msgr2=0x7fd990105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 shutdown_connections 2024-09-10T22:50:53.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:53.708+0000 7fd9981ac640 1 -- 172.21.15.120:0/3713005408 wait complete. 2024-09-10T22:50:54.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:53 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:50:54.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:53 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:50:54.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:53 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:50:54.112 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:53 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:50:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:53 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:50:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:53 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3713005408' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:50:54.309 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:50:54.309 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T22:50:54.309 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T22:50:54.346 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T22:50:54.632 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:55.018 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:54 smithi120 ceph-mon[31247]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:55.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:54 smithi176 ceph-mon[34982]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- 172.21.15.120:0/2112749385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 msgr2=0x7fcce810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 --2- 172.21.15.120:0/2112749385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce810f950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009ca0 tx=0x7fccd802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- 172.21.15.120:0/2112749385 shutdown_connections 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 --2- 172.21.15.120:0/2112749385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce810f950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 --2- 172.21.15.120:0/2112749385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce81087d0 0x7fcce8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 --2- 172.21.15.120:0/2112749385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce8102dc0 0x7fcce81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- 172.21.15.120:0/2112749385 >> 172.21.15.120:0/2112749385 conn(0x7fcce80fe830 msgr2=0x7fcce8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- 172.21.15.120:0/2112749385 shutdown_connections 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- 172.21.15.120:0/2112749385 wait complete. 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 Processor -- start 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.098+0000 7fccef52e640 1 -- start start 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 0x7fcce80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce81087d0 0x7fcce80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8112e50 con 0x7fcce8109150 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8112cd0 con 0x7fcce8102dc0 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8112fd0 con 0x7fcce81087d0 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39982/0 (socket says 172.21.15.120:39982) 2024-09-10T22:50:55.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 -- 172.21.15.120:0/3228671521 learned_addr learned my addr 172.21.15.120:0/3228671521 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccecac9640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce81087d0 0x7fcce80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fcced2ca640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 0x7fcce80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 -- 172.21.15.120:0/3228671521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce81087d0 msgr2=0x7fcce80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce81087d0 0x7fcce80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 -- 172.21.15.120:0/3228671521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 msgr2=0x7fcce80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 0x7fcce80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 -- 172.21.15.120:0/3228671521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81a78a0 con 0x7fcce8109150 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fcced2ca640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 0x7fcce80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccedacb640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005cc0 tx=0x7fccd802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8003cf0 con 0x7fcce8109150 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81a7b30 con 0x7fcce8109150 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd80044f0 con 0x7fcce8109150 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.099+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd802fc30 con 0x7fcce8109150 2024-09-10T22:50:55.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.100+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a8070 con 0x7fcce8109150 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.101+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8103220 con 0x7fcce8109150 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.102+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fccd8004040 con 0x7fcce8109150 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.103+0000 7fcce67fc640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 0x7fccd00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.103+0000 7fcced2ca640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 0x7fccd00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.103+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fccd8107210 con 0x7fcce8109150 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.103+0000 7fcced2ca640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 0x7fccd00b7830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009930 tx=0x7fcce0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:55.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.105+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fccd803b410 con 0x7fcce8109150 2024-09-10T22:50:55.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:54 smithi153 ceph-mon[34951]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:55.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.283+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fcce802e620 con 0x7fcce8109150 2024-09-10T22:50:55.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.283+0000 7fcce67fc640 1 -- 172.21.15.120:0/3228671521 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v50) ==== 86+0+133 (secure 0 0 0) 0x7fccd80c9030 con 0x7fcce8109150 2024-09-10T22:50:55.285 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:55.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 msgr2=0x7fccd00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:55.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 0x7fccd00b7830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009930 tx=0x7fcce0008040 comp rx=0 tx=0).stop 2024-09-10T22:50:55.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 msgr2=0x7fcce8075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:55.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005cc0 tx=0x7fccd802f470 comp rx=0 tx=0).stop 2024-09-10T22:50:55.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 shutdown_connections 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fccd00b5370 0x7fccd00b7830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcce8109150 0x7fcce8075f20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce81087d0 0x7fcce80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 --2- 172.21.15.120:0/3228671521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8102dc0 0x7fcce80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 >> 172.21.15.120:0/3228671521 conn(0x7fcce80fe830 msgr2=0x7fcce8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 shutdown_connections 2024-09-10T22:50:55.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:55.286+0000 7fccef52e640 1 -- 172.21.15.120:0/3228671521 wait complete. 2024-09-10T22:50:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:55 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3228671521' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:50:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:55 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3228671521' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:50:56.100 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:55 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3228671521' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:50:56.112 INFO:teuthology.orchestra.run.smithi120.stdout:[{"version":50,"timestamp":"2024-09-10T22:50:19.638920+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-10T22:50:56.112 INFO:tasks.ceph_manager:config epoch is 50 2024-09-10T22:50:56.113 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:50:56.113 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:50:56.113 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T22:50:56.364 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.837+0000 7f93349e7640 1 -- 172.21.15.120:0/125518573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 msgr2=0x7f9330103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.837+0000 7f93349e7640 1 --2- 172.21.15.120:0/125518573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f9330103f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9320009900 tx=0x7f932002a410 comp rx=0 tx=0).stop 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 -- 172.21.15.120:0/125518573 shutdown_connections 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 --2- 172.21.15.120:0/125518573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 0x7f933010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 --2- 172.21.15.120:0/125518573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 0x7f933010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 --2- 172.21.15.120:0/125518573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f9330103f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 -- 172.21.15.120:0/125518573 >> 172.21.15.120:0/125518573 conn(0x7f93300fd700 msgr2=0x7f93300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 -- 172.21.15.120:0/125518573 shutdown_connections 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 -- 172.21.15.120:0/125518573 wait complete. 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 Processor -- start 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.838+0000 7f93349e7640 1 -- start start 2024-09-10T22:50:56.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 0x7f93301996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 0x7f933019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112d60 con 0x7f9330103b70 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112be0 con 0x7f933010d120 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112ee0 con 0x7f9330104570 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40000/0 (socket says 172.21.15.120:40000) 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 -- 172.21.15.120:0/2608443788 learned_addr learned my addr 172.21.15.120:0/2608443788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:56.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932ed9d640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 0x7f933019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932dd9b640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 0x7f93301996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 -- 172.21.15.120:0/2608443788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 msgr2=0x7f93301996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 0x7f93301996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 -- 172.21.15.120:0/2608443788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 msgr2=0x7f933019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 0x7f933019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 -- 172.21.15.120:0/2608443788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933019f260 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f932e59c640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9320005940 tx=0x7f93200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932003f030 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933019f4f0 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9320003cb0 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320035070 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.839+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93301a5c10 con 0x7f9330103b70 2024-09-10T22:50:56.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.840+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330103f50 con 0x7f9330103b70 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.842+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f932003bbf0 con 0x7f9330103b70 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.842+0000 7f931f7fe640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 0x7f93180b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.843+0000 7f932dd9b640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 0x7f93180b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.843+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f9320106ab0 con 0x7f9330103b70 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.843+0000 7f932dd9b640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 0x7f93180b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9324005e10 tx=0x7f9324005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:56.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:56.845+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f93200cd8c0 con 0x7f9330103b70 2024-09-10T22:50:57.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.067+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f93301a5e10 con 0x7f9330103b70 2024-09-10T22:50:57.068 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:56 smithi120 ceph-mon[31247]: pgmap v296: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:57.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.071+0000 7f931f7fe640 1 -- 172.21.15.120:0/2608443788 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+294221 (secure 0 0 0) 0x7f93200cd1c0 con 0x7f9330103b70 2024-09-10T22:50:57.077 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:57.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 msgr2=0x7f93180b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:57.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 0x7f93180b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9324005e10 tx=0x7f9324005da0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 msgr2=0x7f93301991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:57.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9320005940 tx=0x7f93200042f0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 shutdown_connections 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93180b5370 0x7f93180b7830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f933010d120 0x7f933019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9330104570 0x7f93301996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 --2- 172.21.15.120:0/2608443788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9330103b70 0x7f93301991b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 >> 172.21.15.120:0/2608443788 conn(0x7f93300fd700 msgr2=0x7f93300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 shutdown_connections 2024-09-10T22:50:57.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:57.074+0000 7f93349e7640 1 -- 172.21.15.120:0/2608443788 wait complete. 2024-09-10T22:50:57.080 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:56 smithi176 ceph-mon[34982]: pgmap v296: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:57.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:56 smithi153 ceph-mon[34951]: pgmap v296: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.5 KiB/s wr, 3 op/s 2024-09-10T22:50:57.778 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":578504839},{"type":"v1","addr":"172.21.15.120:6801","nonce":578504839}]},"active_addr":"172.21.15.120:6801/578504839","active_change":"2024-09-10T22:43:14.065938+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24137,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.120:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":1997299028}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2795140245}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":1977369792}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":148579461}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2704787382}]}]} 2024-09-10T22:50:57.787 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T22:50:57.788 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T22:50:57.788 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:50:58.041 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:58.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2608443788' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:50:58.161 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2608443788' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:50:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2608443788' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- 172.21.15.120:0/2384460373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64001076c0 msgr2=0x7f6400107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 --2- 172.21.15.120:0/2384460373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64001076c0 0x7f6400107b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009900 tx=0x7f63e802a410 comp rx=0 tx=0).stop 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- 172.21.15.120:0/2384460373 shutdown_connections 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 --2- 172.21.15.120:0/2384460373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6400108040 0x7f640010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 --2- 172.21.15.120:0/2384460373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64001076c0 0x7f6400107b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 --2- 172.21.15.120:0/2384460373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400101cb0 0x7f6400102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- 172.21.15.120:0/2384460373 >> 172.21.15.120:0/2384460373 conn(0x7f64000fd740 msgr2=0x7f64000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- 172.21.15.120:0/2384460373 shutdown_connections 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- 172.21.15.120:0/2384460373 wait complete. 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 Processor -- start 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.531+0000 7f6404bac640 1 -- start start 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 0x7f6400198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400108040 0x7f640019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111d00 con 0x7f6400101cb0 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111b80 con 0x7f64001076c0 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111e80 con 0x7f6400108040 2024-09-10T22:50:58.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40028/0 (socket says 172.21.15.120:40028) 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 -- 172.21.15.120:0/193908255 learned_addr learned my addr 172.21.15.120:0/193908255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fdd9b640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 0x7f6400198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fed9d640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400108040 0x7f640019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 -- 172.21.15.120:0/193908255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400108040 msgr2=0x7f640019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400108040 0x7f640019da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 -- 172.21.15.120:0/193908255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 msgr2=0x7f6400198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 0x7f6400198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 -- 172.21.15.120:0/193908255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640019e200 con 0x7f6400101cb0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fdd9b640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 0x7f6400198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63fe59c640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d990 tx=0x7f63f000de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f001b070 con 0x7f6400101cb0 2024-09-10T22:50:58.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f000edf0 con 0x7f6400101cb0 2024-09-10T22:50:58.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640019e4f0 con 0x7f6400101cb0 2024-09-10T22:50:58.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0014800 con 0x7f6400101cb0 2024-09-10T22:50:58.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.532+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640006b550 con 0x7f6400101cb0 2024-09-10T22:50:58.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.533+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6400102090 con 0x7f6400101cb0 2024-09-10T22:50:58.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.535+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f63f000e040 con 0x7f6400101cb0 2024-09-10T22:50:58.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.536+0000 7f63f77fe640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 0x7f63ec0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:58.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.536+0000 7f63fdd9b640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 0x7f63ec0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:58.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.536+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f63f00e7650 con 0x7f6400101cb0 2024-09-10T22:50:58.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.536+0000 7f63fdd9b640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 0x7f63ec0b78b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63e8000c00 tx=0x7f63e8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:58.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.539+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f63f00ae560 con 0x7f6400101cb0 2024-09-10T22:50:58.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.715+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f640006c190 con 0x7f6400101cb0 2024-09-10T22:50:58.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.717+0000 7f63f77fe640 1 -- 172.21.15.120:0/193908255 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22360 (secure 0 0 0) 0x7f63f00ade60 con 0x7f6400101cb0 2024-09-10T22:50:58.718 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:50:58.718 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":75,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:50:27.621440+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:50:58.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 msgr2=0x7f63ec0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:58.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 0x7f63ec0b78b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63e8000c00 tx=0x7f63e8031400 comp rx=0 tx=0).stop 2024-09-10T22:50:58.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 msgr2=0x7f6400198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:58.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63f000d990 tx=0x7f63f000de50 comp rx=0 tx=0).stop 2024-09-10T22:50:58.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 shutdown_connections 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63ec0b53f0 0x7f63ec0b78b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6400108040 0x7f640019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64001076c0 0x7f6400198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 --2- 172.21.15.120:0/193908255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6400101cb0 0x7f6400198150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 >> 172.21.15.120:0/193908255 conn(0x7f64000fd740 msgr2=0x7f64000ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 shutdown_connections 2024-09-10T22:50:58.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:50:58.719+0000 7f6404bac640 1 -- 172.21.15.120:0/193908255 wait complete. 2024-09-10T22:50:59.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:58 smithi176 ceph-mon[34982]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-10T22:50:59.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:50:58 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/193908255' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:59.173 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:58 smithi120 ceph-mon[31247]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-10T22:50:59.173 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:50:58 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/193908255' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:59.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:58 smithi153 ceph-mon[34951]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-10T22:50:59.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:50:58 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/193908255' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:50:59.512 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T22:50:59.512 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:50:59.760 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:00.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 -- 172.21.15.120:0/2616948151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 msgr2=0x7f34a410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:00.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 --2- 172.21.15.120:0/2616948151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a410f990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f349c009cd0 tx=0x7f349c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 -- 172.21.15.120:0/2616948151 shutdown_connections 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 --2- 172.21.15.120:0/2616948151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a410f990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 --2- 172.21.15.120:0/2616948151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34a4108910 0x7f34a4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 --2- 172.21.15.120:0/2616948151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 0x7f34a4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 -- 172.21.15.120:0/2616948151 >> 172.21.15.120:0/2616948151 conn(0x7f34a4076f20 msgr2=0x7f34a4077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 -- 172.21.15.120:0/2616948151 shutdown_connections 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.258+0000 7f34a9cca640 1 -- 172.21.15.120:0/2616948151 wait complete. 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 Processor -- start 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 -- start start 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 0x7f34a4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34a4108910 0x7f34a41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:00.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4112dd0 con 0x7f34a4109290 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4112c50 con 0x7f34a4101810 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a9cca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4112f50 con 0x7f34a4108910 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40044/0 (socket says 172.21.15.120:40044) 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 -- 172.21.15.120:0/1630388681 learned_addr learned my addr 172.21.15.120:0/1630388681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a37fe640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 0x7f34a4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 -- 172.21.15.120:0/1630388681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34a4108910 msgr2=0x7f34a41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34a4108910 0x7f34a41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 -- 172.21.15.120:0/1630388681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 msgr2=0x7f34a4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 0x7f34a4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.259+0000 7f34a3fff640 1 -- 172.21.15.120:0/1630388681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a419f230 con 0x7f34a4109290 2024-09-10T22:51:00.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a3fff640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f349c005cc0 tx=0x7f349c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c004240 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c0305a0 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a419f4c0 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0466c0 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.260+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a5be0 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.261+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a4101bf0 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.262+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f349c030070 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.263+0000 7f34a0ff9640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 0x7f348c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.263+0000 7f34a37fe640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 0x7f348c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.263+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f349c02e030 con 0x7f34a4109290 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.263+0000 7f34a37fe640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 0x7f348c0b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f349400abe0 tx=0x7f349400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:00.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.266+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f349c04b060 con 0x7f34a4109290 2024-09-10T22:51:00.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.443+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34a41a5ee0 con 0x7f34a4109290 2024-09-10T22:51:00.444 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:00 smithi120 ceph-mon[31247]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T22:51:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.444+0000 7f34a0ff9640 1 -- 172.21.15.120:0/1630388681 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22360 (secure 0 0 0) 0x7f349c0cde40 con 0x7f34a4109290 2024-09-10T22:51:00.445 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:00.446 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":75,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:50:27.621440+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.446+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 msgr2=0x7f348c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.446+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 0x7f348c0b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f349400abe0 tx=0x7f349400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.446+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 msgr2=0x7f34a419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.446+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f349c005cc0 tx=0x7f349c003ba0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 shutdown_connections 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f348c0b5370 0x7f348c0b7830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34a4109290 0x7f34a419ea60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34a4108910 0x7f34a41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 --2- 172.21.15.120:0/1630388681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34a4101810 0x7f34a4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 >> 172.21.15.120:0/1630388681 conn(0x7f34a4076f20 msgr2=0x7f34a40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 shutdown_connections 2024-09-10T22:51:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:00.447+0000 7f34a9cca640 1 -- 172.21.15.120:0/1630388681 wait complete. 2024-09-10T22:51:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:00 smithi176 ceph-mon[34982]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T22:51:00.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:00 smithi153 ceph-mon[34951]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T22:51:01.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T22:51:01.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T22:51:01.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T22:51:01.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T22:51:01.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T22:51:01.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T22:51:01.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T22:51:01.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T22:51:01.117 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T22:51:01.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T22:51:01.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T22:51:01.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T22:51:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:01 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1630388681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:01 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1630388681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:01 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1630388681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:01.770 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:01.795 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- 172.21.15.120:0/2155067348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 msgr2=0x7f833810f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- 172.21.15.120:0/2155067348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833810f580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8330009ca0 tx=0x7f833002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- 172.21.15.120:0/2155067348 shutdown_connections 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- 172.21.15.120:0/2155067348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833810f580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- 172.21.15.120:0/2155067348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83381045e0 0x7f833810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- 172.21.15.120:0/2155067348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8338103cc0 0x7f83381040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- 172.21.15.120:0/2155067348 >> 172.21.15.120:0/2155067348 conn(0x7f83380fd700 msgr2=0x7f83380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- 172.21.15.120:0/2155067348 shutdown_connections 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- 172.21.15.120:0/2155067348 wait complete. 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 Processor -- start 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- start start 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8338103cc0 0x7f8338199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83381045e0 0x7f8338199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112df0 con 0x7f833810d120 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.367+0000 7f833f1eb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112c70 con 0x7f8338103cc0 2024-09-10T22:51:02.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833f1eb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112f70 con 0x7f83381045e0 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40062/0 (socket says 172.21.15.120:40062) 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 -- 172.21.15.120:0/2932593870 learned_addr learned my addr 172.21.15.120:0/2932593870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833cf87640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8338103cc0 0x7f8338199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f8337fff640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83381045e0 0x7f8338199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 -- 172.21.15.120:0/2932593870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83381045e0 msgr2=0x7f8338199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83381045e0 0x7f8338199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 -- 172.21.15.120:0/2932593870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8338103cc0 msgr2=0x7f8338199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8338103cc0 0x7f8338199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833819f3b0 con 0x7f833810d120 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833d788640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833819ebe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8330005cc0 tx=0x7f833002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330003cf0 con 0x7f833810d120 2024-09-10T22:51:02.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f833f1eb640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833819f640 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83300044f0 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.368+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833002fd10 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.369+0000 7f833f1eb640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a5de0 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.369+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8318001030 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.371+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8330004040 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8335ffb640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83200b5370 0x7f83200b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f833cf87640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83200b5370 0x7f83200b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f83301073b0 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8335ffb640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] conn(0x7f83200beab0 0x7f83200c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8330046e70 con 0x7f83200beab0 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8330107890 con 0x7f833810d120 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f833cf87640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83200b5370 0x7f83200b7830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83280058c0 tx=0x7f832800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8337fff640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] conn(0x7f83200beab0 0x7f83200c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:02.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.372+0000 7f8337fff640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] conn(0x7f83200beab0 0x7f83200c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:02.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.373+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== osd.3 v2:172.21.15.120:6810/1845328035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8330046e70 con 0x7f83200beab0 2024-09-10T22:51:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 --> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8318002dd0 con 0x7f83200beab0 2024-09-10T22:51:02.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f8335ffb640 1 -- 172.21.15.120:0/2932593870 <== osd.3 v2:172.21.15.120:6810/1845328035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8318002dd0 con 0x7f83200beab0 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] conn(0x7f83200beab0 msgr2=0x7f83200c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] conn(0x7f83200beab0 0x7f83200c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83200b5370 msgr2=0x7f83200b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83200b5370 0x7f83200b7830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f83280058c0 tx=0x7f832800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 msgr2=0x7f833819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 --2- 172.21.15.120:0/2932593870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f833810d120 0x7f833819ebe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8330005cc0 tx=0x7f833002f470 comp rx=0 tx=0).stop 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f833d788640 1 -- 172.21.15.120:0/2932593870 reap_dead start 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 shutdown_connections 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.387+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 >> 172.21.15.120:0/2932593870 conn(0x7f83380fd700 msgr2=0x7f833810f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.388+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 shutdown_connections 2024-09-10T22:51:02.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:02.388+0000 7f831f7fe640 1 -- 172.21.15.120:0/2932593870 wait complete. 2024-09-10T22:51:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:02 smithi176 ceph-mon[34982]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T22:51:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:02 smithi120 ceph-mon[31247]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T22:51:02.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:02 smithi153 ceph-mon[34951]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T22:51:02.867 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- 172.21.15.120:0/2390052339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0109150 msgr2=0x7f58b010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 --2- 172.21.15.120:0/2390052339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0109150 0x7f58b010f950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009ca0 tx=0x7f58a802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- 172.21.15.120:0/2390052339 shutdown_connections 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 --2- 172.21.15.120:0/2390052339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b0109150 0x7f58b010f950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 --2- 172.21.15.120:0/2390052339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b01087d0 0x7f58b0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 --2- 172.21.15.120:0/2390052339 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b0102dc0 0x7f58b01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- 172.21.15.120:0/2390052339 >> 172.21.15.120:0/2390052339 conn(0x7f58b00fe830 msgr2=0x7f58b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- 172.21.15.120:0/2390052339 shutdown_connections 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- 172.21.15.120:0/2390052339 wait complete. 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 Processor -- start 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.195+0000 7f58b70b8640 1 -- start start 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102dc0 0x7f58b00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 0x7f58b00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:03.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b0109150 0x7f58b0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0112e10 con 0x7f58b01087d0 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0112c90 con 0x7f58b0102dc0 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0112f90 con 0x7f58b0109150 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 0x7f58b00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 0x7f58b00759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38314/0 (socket says 172.21.15.120:38314) 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 -- 172.21.15.120:0/824312736 learned_addr learned my addr 172.21.15.120:0/824312736 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b4e54640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102dc0 0x7f58b00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b5655640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b0109150 0x7f58b0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 -- 172.21.15.120:0/824312736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b0109150 msgr2=0x7f58b0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b0109150 0x7f58b0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 -- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102dc0 msgr2=0x7f58b00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102dc0 0x7f58b00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b01a7620 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b4e54640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102dc0 0x7f58b00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58affff640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 0x7f58b00759e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c920 tx=0x7f589c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c007c70 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b01a7910 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589c007e10 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.196+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c018820 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.197+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b01a7ea0 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.197+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58b00638a0 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.199+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f589c00f450 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.199+0000 7f58adffb640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58a00b5370 0x7f58a00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.199+0000 7f58b4e54640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58a00b5370 0x7f58a00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.199+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f589c0e6190 con 0x7f58b01087d0 2024-09-10T22:51:03.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.200+0000 7f58adffb640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] conn(0x7f58a00beab0 0x7f58a00c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.200+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f589c01be50 con 0x7f58a00beab0 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.200+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f589c0e6670 con 0x7f58b01087d0 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.200+0000 7f58b4e54640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58a00b5370 0x7f58a00b7830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58a40058c0 tx=0x7f58a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.200+0000 7f58b5655640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] conn(0x7f58a00beab0 0x7f58a00c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.201+0000 7f58b5655640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] conn(0x7f58a00beab0 0x7f58a00c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:03.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.201+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== osd.7 v2:172.21.15.153:6816/1702194586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f589c01be50 con 0x7f58a00beab0 2024-09-10T22:51:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.215+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 --> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58b0063ca0 con 0x7f58a00beab0 2024-09-10T22:51:03.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.215+0000 7f58adffb640 1 -- 172.21.15.120:0/824312736 <== osd.7 v2:172.21.15.153:6816/1702194586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58b0063ca0 con 0x7f58a00beab0 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] conn(0x7f58a00beab0 msgr2=0x7f58a00c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] conn(0x7f58a00beab0 0x7f58a00c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58a00b5370 msgr2=0x7f58a00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58a00b5370 0x7f58a00b7830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58a40058c0 tx=0x7f58a4005850 comp rx=0 tx=0).stop 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 msgr2=0x7f58b00759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:03.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 --2- 172.21.15.120:0/824312736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58b01087d0 0x7f58b00759e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c920 tx=0x7f589c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:51:03.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b5655640 1 -- 172.21.15.120:0/824312736 reap_dead start 2024-09-10T22:51:03.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 shutdown_connections 2024-09-10T22:51:03.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 >> 172.21.15.120:0/824312736 conn(0x7f58b00fe830 msgr2=0x7f58b0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:03.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 shutdown_connections 2024-09-10T22:51:03.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:03.216+0000 7f58b70b8640 1 -- 172.21.15.120:0/824312736 wait complete. 2024-09-10T22:51:03.483 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.485 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.485 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.486 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.487 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.487 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:03.490 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:04.042 INFO:teuthology.orchestra.run.smithi120.stdout:197568495655 2024-09-10T22:51:04.042 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T22:51:04.433 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:04 smithi120 ceph-mon[31247]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:04.439 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:04.474 INFO:teuthology.orchestra.run.smithi120.stdout:103079215163 2024-09-10T22:51:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T22:51:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:04 smithi176 ceph-mon[34982]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:04 smithi153 ceph-mon[34951]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:05.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 -- 172.21.15.120:0/1619316536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e8109150 msgr2=0x7f55e810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 --2- 172.21.15.120:0/1619316536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e8109150 0x7f55e810f950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009ca0 tx=0x7f55e002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 -- 172.21.15.120:0/1619316536 shutdown_connections 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 --2- 172.21.15.120:0/1619316536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e8109150 0x7f55e810f950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 --2- 172.21.15.120:0/1619316536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e81087d0 0x7f55e8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 --2- 172.21.15.120:0/1619316536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e8102dc0 0x7f55e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 -- 172.21.15.120:0/1619316536 >> 172.21.15.120:0/1619316536 conn(0x7f55e80fe870 msgr2=0x7f55e8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 -- 172.21.15.120:0/1619316536 shutdown_connections 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.321+0000 7f55ecb81640 1 -- 172.21.15.120:0/1619316536 wait complete. 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 Processor -- start 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 -- start start 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e8102dc0 0x7f55e81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 0x7f55e8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:05.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e8109150 0x7f55e819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8112dd0 con 0x7f55e81087d0 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8112c50 con 0x7f55e8109150 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55ecb81640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8112f50 con 0x7f55e8102dc0 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 0x7f55e8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 0x7f55e8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38336/0 (socket says 172.21.15.120:38336) 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 -- 172.21.15.120:0/3550239641 learned_addr learned my addr 172.21.15.120:0/3550239641 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55e659c640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e8102dc0 0x7f55e81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55e6d9d640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e8109150 0x7f55e819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 -- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e8102dc0 msgr2=0x7f55e81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e8102dc0 0x7f55e81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 -- 172.21.15.120:0/3550239641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e8109150 msgr2=0x7f55e819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e8109150 0x7f55e819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55dffff640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e819f290 con 0x7f55e81087d0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.322+0000 7f55e6d9d640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55e8109150 0x7f55e819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55dffff640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 0x7f55e8199720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55d000d950 tx=0x7f55d000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0017070 con 0x7f55e81087d0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e819f580 con 0x7f55e81087d0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d000de80 con 0x7f55e81087d0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000f620 con 0x7f55e81087d0 2024-09-10T22:51:05.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.323+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e81a5c40 con 0x7f55e81087d0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.324+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55e80ffaf0 con 0x7f55e81087d0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.325+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f55d0010460 con 0x7f55e81087d0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55df7fe640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55d40b5370 0x7f55d40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55e659c640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55d40b5370 0x7f55d40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f55d00e5820 con 0x7f55e81087d0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55df7fe640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] conn(0x7f55d40beab0 0x7f55d40c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55d0004470 con 0x7f55d40beab0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f55d00ea2c0 con 0x7f55e81087d0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55e659c640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55d40b5370 0x7f55d40b7830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55d800abe0 tx=0x7f55d800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.326+0000 7f55e6d9d640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] conn(0x7f55d40beab0 0x7f55d40c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.327+0000 7f55e6d9d640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] conn(0x7f55d40beab0 0x7f55d40c0ed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:05.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.328+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== osd.2 v2:172.21.15.176:6800/1904007733 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55d0004470 con 0x7f55d40beab0 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.341+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 --> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55e8104070 con 0x7f55d40beab0 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55df7fe640 1 -- 172.21.15.120:0/3550239641 <== osd.2 v2:172.21.15.176:6800/1904007733 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55e8104070 con 0x7f55d40beab0 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] conn(0x7f55d40beab0 msgr2=0x7f55d40c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] conn(0x7f55d40beab0 0x7f55d40c0ed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55d40b5370 msgr2=0x7f55d40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55d40b5370 0x7f55d40b7830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55d800abe0 tx=0x7f55d800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 msgr2=0x7f55e8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 --2- 172.21.15.120:0/3550239641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55e81087d0 0x7f55e8199720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55d000d950 tx=0x7f55d000dd50 comp rx=0 tx=0).stop 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55e6d9d640 1 -- 172.21.15.120:0/3550239641 reap_dead start 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 shutdown_connections 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 >> 172.21.15.120:0/3550239641 conn(0x7f55e80fe870 msgr2=0x7f55e80ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 shutdown_connections 2024-09-10T22:51:05.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.342+0000 7f55ecb81640 1 -- 172.21.15.120:0/3550239641 wait complete. 2024-09-10T22:51:06.306 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:06 smithi120 ceph-mon[31247]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:06 smithi176 ceph-mon[34982]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:06.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:06 smithi153 ceph-mon[34951]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 -- 172.21.15.120:0/3836955829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c108020 msgr2=0x7f0b6c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 --2- 172.21.15.120:0/3836955829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c108020 0x7f0b6c10e820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ca0 tx=0x7f0b6402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 -- 172.21.15.120:0/3836955829 shutdown_connections 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 --2- 172.21.15.120:0/3836955829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c108020 0x7f0b6c10e820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 --2- 172.21.15.120:0/3836955829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b6c1076a0 0x7f0b6c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 --2- 172.21.15.120:0/3836955829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c101c90 0x7f0b6c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 -- 172.21.15.120:0/3836955829 >> 172.21.15.120:0/3836955829 conn(0x7f0b6c0fd700 msgr2=0x7f0b6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 -- 172.21.15.120:0/3836955829 shutdown_connections 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.666+0000 7f0b719d2640 1 -- 172.21.15.120:0/3836955829 wait complete. 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 Processor -- start 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 -- start start 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 0x7f0b6c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c1076a0 0x7f0b6c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b6c108020 0x7f0b6c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c112dc0 con 0x7f0b6c101c90 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c112c40 con 0x7f0b6c1076a0 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b719d2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c112f40 con 0x7f0b6c108020 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 0x7f0b6c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 0x7f0b6c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38378/0 (socket says 172.21.15.120:38378) 2024-09-10T22:51:06.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 -- 172.21.15.120:0/2475704739 learned_addr learned my addr 172.21.15.120:0/2475704739 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b63fff640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c1076a0 0x7f0b6c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 -- 172.21.15.120:0/2475704739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b6c108020 msgr2=0x7f0b6c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b6c108020 0x7f0b6c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 -- 172.21.15.120:0/2475704739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c1076a0 msgr2=0x7f0b6c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c1076a0 0x7f0b6c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c19f260 con 0x7f0b6c101c90 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b63fff640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b6c1076a0 0x7f0b6c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.667+0000 7f0b6affd640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 0x7f0b6c1991b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c920 tx=0x7f0b5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c007c70 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c007e10 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c19f550 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c018820 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c1a5c10 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.668+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b6c0fe920 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.670+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0b5c00f450 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b709d0640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b580b5370 0x7f0b580b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b63fff640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b580b5370 0x7f0b580b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f0b5c0e61a0 con 0x7f0b6c101c90 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b709d0640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] conn(0x7f0b580beab0 0x7f0b580c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b580c1580 con 0x7f0b580beab0 2024-09-10T22:51:06.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0e6680 con 0x7f0b6c101c90 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b6b7fe640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] conn(0x7f0b580beab0 0x7f0b580c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.671+0000 7f0b63fff640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b580b5370 0x7f0b580b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005e10 tx=0x7f0b54005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.672+0000 7f0b6b7fe640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] conn(0x7f0b580beab0 0x7f0b580c0ed0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.672+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== osd.0 v2:172.21.15.120:6802/2370065006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b580c1580 con 0x7f0b580beab0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.685+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 --> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b6c102f40 con 0x7f0b580beab0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b709d0640 1 -- 172.21.15.120:0/2475704739 <== osd.0 v2:172.21.15.120:6802/2370065006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b6c102f40 con 0x7f0b580beab0 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] conn(0x7f0b580beab0 msgr2=0x7f0b580c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] conn(0x7f0b580beab0 0x7f0b580c0ed0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b580b5370 msgr2=0x7f0b580b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b580b5370 0x7f0b580b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b54005e10 tx=0x7f0b54005da0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 msgr2=0x7f0b6c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 --2- 172.21.15.120:0/2475704739 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b6c101c90 0x7f0b6c1991b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c920 tx=0x7f0b5c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b6b7fe640 1 -- 172.21.15.120:0/2475704739 reap_dead start 2024-09-10T22:51:06.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 shutdown_connections 2024-09-10T22:51:06.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 >> 172.21.15.120:0/2475704739 conn(0x7f0b6c0fd700 msgr2=0x7f0b6c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 shutdown_connections 2024-09-10T22:51:06.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.686+0000 7f0b719d2640 1 -- 172.21.15.120:0/2475704739 wait complete. 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 -- 172.21.15.120:0/1604056642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 msgr2=0x7ff55810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/1604056642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55810e860 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff550009ca0 tx=0x7ff55002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 -- 172.21.15.120:0/1604056642 shutdown_connections 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/1604056642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55810e860 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/1604056642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5581076e0 0x7ff558107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/1604056642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff558101cd0 0x7ff5581020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 -- 172.21.15.120:0/1604056642 >> 172.21.15.120:0/1604056642 conn(0x7ff5580fd740 msgr2=0x7ff5580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 -- 172.21.15.120:0/1604056642 shutdown_connections 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.216+0000 7ff55cc3c640 1 -- 172.21.15.120:0/1604056642 wait complete. 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 Processor -- start 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 -- start start 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff558101cd0 0x7ff55819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5581076e0 0x7ff55819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff558111ce0 con 0x7ff558108060 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff558111b60 con 0x7ff5581076e0 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55cc3c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff558111e60 con 0x7ff558101cd0 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38406/0 (socket says 172.21.15.120:38406) 2024-09-10T22:51:07.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 -- 172.21.15.120:0/995172356 learned_addr learned my addr 172.21.15.120:0/995172356 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff54ffff640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5581076e0 0x7ff55819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55659c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff558101cd0 0x7ff55819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 -- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff558101cd0 msgr2=0x7ff55819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff558101cd0 0x7ff55819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 -- 172.21.15.120:0/995172356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5581076e0 msgr2=0x7ff55819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5581076e0 0x7ff55819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff556d9d640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5581a5470 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.217+0000 7ff55659c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff558101cd0 0x7ff55819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff556d9d640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55819ab30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff550005cc0 tx=0x7ff55002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550003bf0 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5581a5700 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff550003d90 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55002fbc0 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.218+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5581a5c40 con 0x7ff558108060 2024-09-10T22:51:07.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.219+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5580638e0 con 0x7ff558108060 2024-09-10T22:51:07.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.220+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff550004610 con 0x7ff558108060 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff54f7fe640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5440b5370 0x7ff5440b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff55659c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5440b5370 0x7ff5440b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff550107250 con 0x7ff558108060 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff54f7fe640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] conn(0x7ff5440beab0 0x7ff5440c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5440c1580 con 0x7ff5440beab0 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff550107730 con 0x7ff558108060 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.221+0000 7ff55659c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5440b5370 0x7ff5440b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff548009b40 tx=0x7ff548005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.222+0000 7ff54ffff640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] conn(0x7ff5440beab0 0x7ff5440c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.222+0000 7ff54ffff640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] conn(0x7ff5440beab0 0x7ff5440c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.223+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== osd.11 v2:172.21.15.176:6824/2240071367 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5440c1580 con 0x7ff5440beab0 2024-09-10T22:51:07.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.236+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 --> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff558063d60 con 0x7ff5440beab0 2024-09-10T22:51:07.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff54f7fe640 1 -- 172.21.15.120:0/995172356 <== osd.11 v2:172.21.15.176:6824/2240071367 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff558063d60 con 0x7ff5440beab0 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] conn(0x7ff5440beab0 msgr2=0x7ff5440c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] conn(0x7ff5440beab0 0x7ff5440c0ed0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5440b5370 msgr2=0x7ff5440b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5440b5370 0x7ff5440b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff548009b40 tx=0x7ff548005e50 comp rx=0 tx=0).stop 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 msgr2=0x7ff55819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 --2- 172.21.15.120:0/995172356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff558108060 0x7ff55819ab30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff550005cc0 tx=0x7ff55002f380 comp rx=0 tx=0).stop 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff556d9d640 1 -- 172.21.15.120:0/995172356 reap_dead start 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 shutdown_connections 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 >> 172.21.15.120:0/995172356 conn(0x7ff5580fd740 msgr2=0x7ff5580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 shutdown_connections 2024-09-10T22:51:07.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:07.237+0000 7ff55cc3c640 1 -- 172.21.15.120:0/995172356 wait complete. 2024-09-10T22:51:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- 172.21.15.120:0/2074388867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c109130 msgr2=0x7f040c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 --2- 172.21.15.120:0/2074388867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c109130 0x7f040c10f930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0404009ca0 tx=0x7f040402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- 172.21.15.120:0/2074388867 shutdown_connections 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 --2- 172.21.15.120:0/2074388867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c109130 0x7f040c10f930 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 --2- 172.21.15.120:0/2074388867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f040c1087b0 0x7f040c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 --2- 172.21.15.120:0/2074388867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f040c102da0 0x7f040c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- 172.21.15.120:0/2074388867 >> 172.21.15.120:0/2074388867 conn(0x7f040c0fe810 msgr2=0x7f040c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- 172.21.15.120:0/2074388867 shutdown_connections 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- 172.21.15.120:0/2074388867 wait complete. 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 Processor -- start 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.210+0000 7f0411e8e640 1 -- start start 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 0x7f040c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f040c1087b0 0x7f040c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f040c109130 0x7f040c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f040c112df0 con 0x7f040c102da0 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f040c112c70 con 0x7f040c109130 2024-09-10T22:51:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0411e8e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f040c112f70 con 0x7f040c1087b0 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 0x7f040c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 0x7f040c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38324/0 (socket says 172.21.15.120:38324) 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 -- 172.21.15.120:0/3546078991 learned_addr learned my addr 172.21.15.120:0/3546078991 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040bfff640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f040c109130 0x7f040c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f0402ffd640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f040c1087b0 0x7f040c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 -- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f040c1087b0 msgr2=0x7f040c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f040c1087b0 0x7f040c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 -- 172.21.15.120:0/3546078991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f040c109130 msgr2=0x7f040c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f040c109130 0x7f040c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c1a7640 con 0x7f040c102da0 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.211+0000 7f040b7fe640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 0x7f040c0754a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d770 tx=0x7f03fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc017070 con 0x7f040c102da0 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03fc0054d0 con 0x7f040c102da0 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c1a7930 con 0x7f040c102da0 2024-09-10T22:51:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004c30 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c1a7e70 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.212+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f040c0638e0 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.214+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f03fc004dd0 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f04097fa640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f03f80b53f0 0x7f03f80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f0402ffd640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f03f80b53f0 0x7f03f80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f03fc0e5a10 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f04097fa640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] conn(0x7f03f80beb30 0x7f03f80c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03fc021e50 con 0x7f03f80beb30 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f03fc0e5ef0 con 0x7f040c102da0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f0402ffd640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f03f80b53f0 0x7f03f80b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f040c076f90 tx=0x7f03f400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.215+0000 7f040bfff640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] conn(0x7f03f80beb30 0x7f03f80c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.216+0000 7f040bfff640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] conn(0x7f03f80beb30 0x7f03f80c0f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.217+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== osd.5 v2:172.21.15.176:6808/2804384590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03fc021e50 con 0x7f03f80beb30 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.230+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 --> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f040c063d60 con 0x7f03f80beb30 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f04097fa640 1 -- 172.21.15.120:0/3546078991 <== osd.5 v2:172.21.15.176:6808/2804384590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f040c063d60 con 0x7f03f80beb30 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] conn(0x7f03f80beb30 msgr2=0x7f03f80c0f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] conn(0x7f03f80beb30 0x7f03f80c0f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f03f80b53f0 msgr2=0x7f03f80b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f03f80b53f0 0x7f03f80b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f040c076f90 tx=0x7f03f400a4c0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 msgr2=0x7f040c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 --2- 172.21.15.120:0/3546078991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f040c102da0 0x7f040c0754a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d770 tx=0x7f03fc00dc30 comp rx=0 tx=0).stop 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f040bfff640 1 -- 172.21.15.120:0/3546078991 reap_dead start 2024-09-10T22:51:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 shutdown_connections 2024-09-10T22:51:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 >> 172.21.15.120:0/3546078991 conn(0x7f040c0fe810 msgr2=0x7f040c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 shutdown_connections 2024-09-10T22:51:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:05.231+0000 7f0411e8e640 1 -- 172.21.15.120:0/3546078991 wait complete. 2024-09-10T22:51:08.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/855869862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d58102dc0 msgr2=0x7f6d581031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/855869862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d58102dc0 0x7f6d581031a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009900 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/855869862 shutdown_connections 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/855869862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d58109150 0x7f6d5810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/855869862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d581087d0 0x7f6d58108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/855869862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d58102dc0 0x7f6d581031a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/855869862 >> 172.21.15.120:0/855869862 conn(0x7f6d580fe830 msgr2=0x7f6d58100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/855869862 shutdown_connections 2024-09-10T22:51:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.579+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/855869862 wait complete. 2024-09-10T22:51:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 Processor -- start 2024-09-10T22:51:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 -- start start 2024-09-10T22:51:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d58102dc0 0x7f6d580754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 0x7f6d580759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d58109150 0x7f6d58075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112e10 con 0x7f6d581087d0 2024-09-10T22:51:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112c90 con 0x7f6d58102dc0 2024-09-10T22:51:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5f4d6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112f90 con 0x7f6d58109150 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 0x7f6d580759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 0x7f6d580759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38362/0 (socket says 172.21.15.120:38362) 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 -- 172.21.15.120:0/2695208201 learned_addr learned my addr 172.21.15.120:0/2695208201 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5d272640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d58102dc0 0x7f6d580754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5da73640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d58109150 0x7f6d58075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 -- 172.21.15.120:0/2695208201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d58109150 msgr2=0x7f6d58075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d58109150 0x7f6d58075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 -- 172.21.15.120:0/2695208201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d58102dc0 msgr2=0x7f6d580754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d58102dc0 0x7f6d580754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.580+0000 7f6d5ca71640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581a7710 con 0x7f6d581087d0 2024-09-10T22:51:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d5ca71640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 0x7f6d580759e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800d950 tx=0x7f6d4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48017070 con 0x7f6d581087d0 2024-09-10T22:51:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48009c30 con 0x7f6d581087d0 2024-09-10T22:51:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800f620 con 0x7f6d581087d0 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581a7a00 con 0x7f6d581087d0 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.581+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581a7fc0 con 0x7f6d581087d0 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.582+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d580638a0 con 0x7f6d581087d0 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.583+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f6d48010460 con 0x7f6d581087d0 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.584+0000 7f6d4e7fc640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d440b5370 0x7f6d440b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.584+0000 7f6d5d272640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d440b5370 0x7f6d440b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.584+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6d480e5ca0 con 0x7f6d581087d0 2024-09-10T22:51:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.584+0000 7f6d4e7fc640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] conn(0x7f6d440beab0 0x7f6d440c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.585+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d48022e70 con 0x7f6d440beab0 2024-09-10T22:51:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.585+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6d480e6180 con 0x7f6d581087d0 2024-09-10T22:51:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.585+0000 7f6d5da73640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] conn(0x7f6d440beab0 0x7f6d440c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.585+0000 7f6d5d272640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d440b5370 0x7f6d440b7830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d54031440 tx=0x7f6d54005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.585+0000 7f6d5da73640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] conn(0x7f6d440beab0 0x7f6d440c0ed0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.586+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== osd.6 v2:172.21.15.120:6818/2500547812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d48022e70 con 0x7f6d440beab0 2024-09-10T22:51:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.600+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 --> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d58063d20 con 0x7f6d440beab0 2024-09-10T22:51:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.600+0000 7f6d4e7fc640 1 -- 172.21.15.120:0/2695208201 <== osd.6 v2:172.21.15.120:6818/2500547812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d58063d20 con 0x7f6d440beab0 2024-09-10T22:51:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.600+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] conn(0x7f6d440beab0 msgr2=0x7f6d440c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.600+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] conn(0x7f6d440beab0 0x7f6d440c0ed0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d440b5370 msgr2=0x7f6d440b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d440b5370 0x7f6d440b7830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d54031440 tx=0x7f6d54005b50 comp rx=0 tx=0).stop 2024-09-10T22:51:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 msgr2=0x7f6d580759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 --2- 172.21.15.120:0/2695208201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d581087d0 0x7f6d580759e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800d950 tx=0x7f6d4800de10 comp rx=0 tx=0).stop 2024-09-10T22:51:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5da73640 1 -- 172.21.15.120:0/2695208201 reap_dead start 2024-09-10T22:51:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 shutdown_connections 2024-09-10T22:51:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 >> 172.21.15.120:0/2695208201 conn(0x7f6d580fe830 msgr2=0x7f6d58100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 shutdown_connections 2024-09-10T22:51:08.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:06.601+0000 7f6d5f4d6640 1 -- 172.21.15.120:0/2695208201 wait complete. 2024-09-10T22:51:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:08 smithi176 ceph-mon[34982]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:08.689 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:08 smithi120 ceph-mon[31247]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:08.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:08 smithi153 ceph-mon[34951]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:51:08.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 -- 172.21.15.120:0/286629678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 msgr2=0x7f8c30102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 --2- 172.21.15.120:0/286629678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30102070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009900 tx=0x7f8c2c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 -- 172.21.15.120:0/286629678 shutdown_connections 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 --2- 172.21.15.120:0/286629678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c30108020 0x7f8c3010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 --2- 172.21.15.120:0/286629678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 0x7f8c30107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 --2- 172.21.15.120:0/286629678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30102070 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 -- 172.21.15.120:0/286629678 >> 172.21.15.120:0/286629678 conn(0x7f8c300fd700 msgr2=0x7f8c300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 -- 172.21.15.120:0/286629678 shutdown_connections 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.769+0000 7f8c37989640 1 -- 172.21.15.120:0/286629678 wait complete. 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 Processor -- start 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 -- start start 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 0x7f8c301986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c30108020 0x7f8c3019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30111ce0 con 0x7f8c30101c90 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30111b60 con 0x7f8c301076a0 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c37989640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30111e60 con 0x7f8c30108020 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38418/0 (socket says 172.21.15.120:38418) 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 -- 172.21.15.120:0/849116609 learned_addr learned my addr 172.21.15.120:0/849116609 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35f26640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c30108020 0x7f8c3019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c34f24640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 0x7f8c301986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 -- 172.21.15.120:0/849116609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c30108020 msgr2=0x7f8c3019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c30108020 0x7f8c3019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 -- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 msgr2=0x7f8c301986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 0x7f8c301986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3019e220 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c34f24640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c301076a0 0x7f8c301986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.770+0000 7f8c35725640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30198170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c03f030 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2c003800 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3019e4b0 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c035070 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301a4bd0 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.771+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c300638e0 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.773+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8c2c031b10 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.773+0000 7f8c267fc640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c1c0b53f0 0x7f8c1c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c34f24640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c1c0b53f0 0x7f8c1c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f8c2c106bd0 con 0x7f8c30101c90 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c267fc640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] conn(0x7f8c1c0beb30 0x7f8c1c0c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c1c0c1600 con 0x7f8c1c0beb30 2024-09-10T22:51:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c1070b0 con 0x7f8c30101c90 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c34f24640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c1c0b53f0 0x7f8c1c0b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c30101100 tx=0x7f8c20005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c35f26640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] conn(0x7f8c1c0beb30 0x7f8c1c0c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.774+0000 7f8c35f26640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] conn(0x7f8c1c0beb30 0x7f8c1c0c0f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.775+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== osd.4 v2:172.21.15.153:6808/2098592997 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c1c0c1600 con 0x7f8c1c0beb30 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.789+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 --> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c30063ce0 con 0x7f8c1c0beb30 2024-09-10T22:51:08.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.789+0000 7f8c267fc640 1 -- 172.21.15.120:0/849116609 <== osd.4 v2:172.21.15.153:6808/2098592997 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c30063ce0 con 0x7f8c1c0beb30 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] conn(0x7f8c1c0beb30 msgr2=0x7f8c1c0c0f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] conn(0x7f8c1c0beb30 0x7f8c1c0c0f50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c1c0b53f0 msgr2=0x7f8c1c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c1c0b53f0 0x7f8c1c0b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c30101100 tx=0x7f8c20005d90 comp rx=0 tx=0).stop 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 msgr2=0x7f8c30198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 --2- 172.21.15.120:0/849116609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c30101c90 0x7f8c30198170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c35f26640 1 -- 172.21.15.120:0/849116609 reap_dead start 2024-09-10T22:51:08.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 shutdown_connections 2024-09-10T22:51:08.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 >> 172.21.15.120:0/849116609 conn(0x7f8c300fd700 msgr2=0x7f8c300ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:08.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 shutdown_connections 2024-09-10T22:51:08.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:08.790+0000 7f8c37989640 1 -- 172.21.15.120:0/849116609 wait complete. 2024-09-10T22:51:09.578 INFO:teuthology.orchestra.run.smithi120.stdout:34359738445 2024-09-10T22:51:09.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T22:51:09.996 INFO:teuthology.orchestra.run.smithi120.stdout:150323855410 2024-09-10T22:51:09.997 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T22:51:10.490 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:10 smithi120 ceph-mon[31247]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T22:51:10.518 INFO:teuthology.orchestra.run.smithi120.stdout:73014444098 2024-09-10T22:51:10.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T22:51:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:10 smithi176 ceph-mon[34982]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T22:51:10.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:10 smithi153 ceph-mon[34951]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T22:51:10.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- 172.21.15.120:0/1660169822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d81076a0 msgr2=0x7f59d8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/1660169822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d81076a0 0x7f59d8107ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009900 tx=0x7f59d402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:10.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- 172.21.15.120:0/1660169822 shutdown_connections 2024-09-10T22:51:10.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/1660169822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d8108020 0x7f59d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/1660169822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d81076a0 0x7f59d8107ae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/1660169822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8101c90 0x7f59d8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- 172.21.15.120:0/1660169822 >> 172.21.15.120:0/1660169822 conn(0x7f59d80fd700 msgr2=0x7f59d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- 172.21.15.120:0/1660169822 shutdown_connections 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- 172.21.15.120:0/1660169822 wait complete. 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 Processor -- start 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.804+0000 7f59dfe4b640 1 -- start start 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 0x7f59d81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d81076a0 0x7f59d8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8108020 0x7f59d819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8111ce0 con 0x7f59d8101c90 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8111b60 con 0x7f59d81076a0 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dfe4b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8111e60 con 0x7f59d8108020 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 0x7f59d81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:10.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 0x7f59d81992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38428/0 (socket says 172.21.15.120:38428) 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 -- 172.21.15.120:0/2016750037 learned_addr learned my addr 172.21.15.120:0/2016750037 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dd3e6640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d81076a0 0x7f59d8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59de3e8640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8108020 0x7f59d819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 -- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8108020 msgr2=0x7f59d819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8108020 0x7f59d819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 -- 172.21.15.120:0/2016750037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d81076a0 msgr2=0x7f59d8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d81076a0 0x7f59d8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d819f380 con 0x7f59d8101c90 2024-09-10T22:51:10.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59de3e8640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59d8108020 0x7f59d819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59dd3e6640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59d81076a0 0x7f59d8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.805+0000 7f59ddbe7640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 0x7f59d81992d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f59c800b4a0 tx=0x7f59c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.806+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8014070 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.806+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d819f670 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.806+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d80fdb60 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.806+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c8010ac0 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.806+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8017c30 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.807+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59bc001030 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.808+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f59c8017360 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59ceffd640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59c40b5470 0x7f59c40b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59dd3e6640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59c40b5470 0x7f59c40b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f59c80e5850 con 0x7f59d8101c90 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59ceffd640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] conn(0x7f59c40beba0 0x7f59c40c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:10.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59c8020e70 con 0x7f59c40beba0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f59c80e5c40 con 0x7f59d8101c90 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59dd3e6640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59c40b5470 0x7f59c40b7930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d4031440 tx=0x7f59d40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.809+0000 7f59de3e8640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] conn(0x7f59c40beba0 0x7f59c40c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.810+0000 7f59de3e8640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] conn(0x7f59c40beba0 0x7f59c40c0fc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.811+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== osd.8 v2:172.21.15.176:6816/2769671146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59c8020e70 con 0x7f59c40beba0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.824+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 --> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59bc002dd0 con 0x7f59c40beba0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59ceffd640 1 -- 172.21.15.120:0/2016750037 <== osd.8 v2:172.21.15.176:6816/2769671146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59bc002dd0 con 0x7f59c40beba0 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] conn(0x7f59c40beba0 msgr2=0x7f59c40c0fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] conn(0x7f59c40beba0 0x7f59c40c0fc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59c40b5470 msgr2=0x7f59c40b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59c40b5470 0x7f59c40b7930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d4031440 tx=0x7f59d40095f0 comp rx=0 tx=0).stop 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 msgr2=0x7f59d81992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 --2- 172.21.15.120:0/2016750037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59d8101c90 0x7f59d81992d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f59c800b4a0 tx=0x7f59c800b960 comp rx=0 tx=0).stop 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59de3e8640 1 -- 172.21.15.120:0/2016750037 reap_dead start 2024-09-10T22:51:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 shutdown_connections 2024-09-10T22:51:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 >> 172.21.15.120:0/2016750037 conn(0x7f59d80fd700 msgr2=0x7f59d8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 shutdown_connections 2024-09-10T22:51:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:10.825+0000 7f59dfe4b640 1 -- 172.21.15.120:0/2016750037 wait complete. 2024-09-10T22:51:11.460 INFO:teuthology.orchestra.run.smithi120.stdout:176093659182 2024-09-10T22:51:11.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T22:51:11.919 INFO:teuthology.orchestra.run.smithi120.stdout:296352743444 2024-09-10T22:51:11.919 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 -- 172.21.15.120:0/3186339743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 msgr2=0x7fbb04103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 --2- 172.21.15.120:0/3186339743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb04103180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009900 tx=0x7fbaf402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 -- 172.21.15.120:0/3186339743 shutdown_connections 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 --2- 172.21.15.120:0/3186339743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb04109130 0x7fbb0410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 --2- 172.21.15.120:0/3186339743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb041087b0 0x7fbb04108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 --2- 172.21.15.120:0/3186339743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb04103180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 -- 172.21.15.120:0/3186339743 >> 172.21.15.120:0/3186339743 conn(0x7fbb040fe850 msgr2=0x7fbb04100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 -- 172.21.15.120:0/3186339743 shutdown_connections 2024-09-10T22:51:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 -- 172.21.15.120:0/3186339743 wait complete. 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.071+0000 7fbb09737640 1 Processor -- start 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 -- start start 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb0419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb041087b0 0x7fbb0419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb04109130 0x7fbb041a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112e30 con 0x7fbb04102da0 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112cb0 con 0x7fbb041087b0 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb09737640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112fb0 con 0x7fbb04109130 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb0419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb0419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38434/0 (socket says 172.21.15.120:38434) 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 -- 172.21.15.120:0/3385080942 learned_addr learned my addr 172.21.15.120:0/3385080942 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb027fc640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb041087b0 0x7fbb0419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb037fe640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb04109130 0x7fbb041a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 -- 172.21.15.120:0/3385080942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb04109130 msgr2=0x7fbb041a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb04109130 0x7fbb041a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 -- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb041087b0 msgr2=0x7fbb0419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb041087b0 0x7fbb0419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a1710 con 0x7fbb04102da0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb037fe640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb04109130 0x7fbb041a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.072+0000 7fbb02ffd640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb0419b660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006010 tx=0x7fbaf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.073+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf403f030 con 0x7fbb04102da0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.073+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf4003800 con 0x7fbb04102da0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.073+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a19a0 con 0x7fbb04102da0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.073+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4035070 con 0x7fbb04102da0 2024-09-10T22:51:12.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.073+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a80c0 con 0x7fbb04102da0 2024-09-10T22:51:12.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.074+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb040638a0 con 0x7fbb04102da0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.075+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fbaf404c060 con 0x7fbb04102da0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.076+0000 7fbaebfff640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbaec0b5370 0x7fbaec0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbb027fc640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbaec0b5370 0x7fbaec0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fbaf4106980 con 0x7fbb04102da0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbaebfff640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] conn(0x7fbaec0beab0 0x7fbaec0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbaf4044e50 con 0x7fbaec0beab0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbaf4106d30 con 0x7fbb04102da0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbb027fc640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbaec0b5370 0x7fbaec0b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb04108530 tx=0x7fbaf8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.077+0000 7fbb037fe640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] conn(0x7fbaec0beab0 0x7fbaec0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:12.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.078+0000 7fbb037fe640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] conn(0x7fbaec0beab0 0x7fbaec0c0ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:12.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.078+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== osd.1 v2:172.21.15.153:6800/2556493584 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf4044e50 con 0x7fbaec0beab0 2024-09-10T22:51:12.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.095+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 --> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb04063d20 con 0x7fbaec0beab0 2024-09-10T22:51:12.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbaebfff640 1 -- 172.21.15.120:0/3385080942 <== osd.1 v2:172.21.15.153:6800/2556493584 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb04063d20 con 0x7fbaec0beab0 2024-09-10T22:51:12.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] conn(0x7fbaec0beab0 msgr2=0x7fbaec0c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] conn(0x7fbaec0beab0 0x7fbaec0c0ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbaec0b5370 msgr2=0x7fbaec0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbaec0b5370 0x7fbaec0b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb04108530 tx=0x7fbaf8009110 comp rx=0 tx=0).stop 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 msgr2=0x7fbb0419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 --2- 172.21.15.120:0/3385080942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb04102da0 0x7fbb0419b660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006010 tx=0x7fbaf40042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb037fe640 1 -- 172.21.15.120:0/3385080942 reap_dead start 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 shutdown_connections 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 >> 172.21.15.120:0/3385080942 conn(0x7fbb040fe850 msgr2=0x7fbb04075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 shutdown_connections 2024-09-10T22:51:12.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:12.096+0000 7fbb09737640 1 -- 172.21.15.120:0/3385080942 wait complete. 2024-09-10T22:51:12.497 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:12 smithi120 ceph-mon[31247]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:51:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:12 smithi176 ceph-mon[34982]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:51:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:12 smithi153 ceph-mon[34951]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:51:12.904 INFO:teuthology.orchestra.run.smithi120.stdout:55834574920 2024-09-10T22:51:12.905 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T22:51:13.374 INFO:teuthology.orchestra.run.smithi120.stdout:227633266725 2024-09-10T22:51:13.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T22:51:13.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 -- 172.21.15.120:0/75840357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c0109150 msgr2=0x7f14c010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 --2- 172.21.15.120:0/75840357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c0109150 0x7f14c010f950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009ca0 tx=0x7f14b802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 -- 172.21.15.120:0/75840357 shutdown_connections 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 --2- 172.21.15.120:0/75840357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c0109150 0x7f14c010f950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 --2- 172.21.15.120:0/75840357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c01087d0 0x7f14c0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 --2- 172.21.15.120:0/75840357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0102dc0 0x7f14c01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 -- 172.21.15.120:0/75840357 >> 172.21.15.120:0/75840357 conn(0x7f14c00fe830 msgr2=0x7f14c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 -- 172.21.15.120:0/75840357 shutdown_connections 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.814+0000 7f14c5303640 1 -- 172.21.15.120:0/75840357 wait complete. 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 Processor -- start 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 -- start start 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c0102dc0 0x7f14c01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:13.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 0x7f14c0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0109150 0x7f14c019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0112e50 con 0x7f14c01087d0 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0112cd0 con 0x7f14c0102dc0 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14c5303640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0112fd0 con 0x7f14c0109150 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14be7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 0x7f14c0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14be7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 0x7f14c0199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49770/0 (socket says 172.21.15.120:49770) 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14be7fc640 1 -- 172.21.15.120:0/615553103 learned_addr learned my addr 172.21.15.120:0/615553103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14beffd640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c0102dc0 0x7f14c01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.815+0000 7f14bf7fe640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0109150 0x7f14c019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 -- 172.21.15.120:0/615553103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0109150 msgr2=0x7f14c019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0109150 0x7f14c019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 -- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c0102dc0 msgr2=0x7f14c01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c0102dc0 0x7f14c01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c019f490 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14bf7fe640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c0109150 0x7f14c019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14beffd640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14c0102dc0 0x7f14c01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14be7fc640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 0x7f14c0199920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b4a0 tx=0x7f14b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4014070 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c019f780 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4010ac0 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4017c30 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.816+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01a5e40 con 0x7f14c01087d0 2024-09-10T22:51:13.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.817+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14c00638e0 con 0x7f14c01087d0 2024-09-10T22:51:13.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.819+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f14b4010610 con 0x7f14c01087d0 2024-09-10T22:51:13.824 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14a7fff640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14a80b5370 0x7f14a80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f14b40e56d0 con 0x7f14c01087d0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14beffd640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14a80b5370 0x7f14a80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14a7fff640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] conn(0x7f14a80beab0 0x7f14a80c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14b400ee50 con 0x7f14a80beab0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.820+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f14b40e5a80 con 0x7f14c01087d0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.821+0000 7f14bf7fe640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] conn(0x7f14a80beab0 0x7f14a80c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.821+0000 7f14beffd640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14a80b5370 0x7f14a80b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009930 tx=0x7f14b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.821+0000 7f14bf7fe640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] conn(0x7f14a80beab0 0x7f14a80c0ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:13.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.822+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== osd.10 v2:172.21.15.153:6824/718699262 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b400ee50 con 0x7f14a80beab0 2024-09-10T22:51:13.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.836+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 --> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14c0063ce0 con 0x7f14a80beab0 2024-09-10T22:51:13.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.836+0000 7f14a7fff640 1 -- 172.21.15.120:0/615553103 <== osd.10 v2:172.21.15.153:6824/718699262 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f14c0063ce0 con 0x7f14a80beab0 2024-09-10T22:51:13.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] conn(0x7f14a80beab0 msgr2=0x7f14a80c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] conn(0x7f14a80beab0 0x7f14a80c0ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14a80b5370 msgr2=0x7f14a80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14a80b5370 0x7f14a80b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009930 tx=0x7f14b0008040 comp rx=0 tx=0).stop 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 msgr2=0x7f14c0199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 --2- 172.21.15.120:0/615553103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14c01087d0 0x7f14c0199920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b4a0 tx=0x7f14b400b960 comp rx=0 tx=0).stop 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14bf7fe640 1 -- 172.21.15.120:0/615553103 reap_dead start 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 shutdown_connections 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 >> 172.21.15.120:0/615553103 conn(0x7f14c00fe830 msgr2=0x7f14c0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 shutdown_connections 2024-09-10T22:51:13.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:13.837+0000 7f14c5303640 1 -- 172.21.15.120:0/615553103 wait complete. 2024-09-10T22:51:14.406 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-10T22:51:14.407 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:14 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T22:51:14.434 INFO:teuthology.orchestra.run.smithi120.stdout:128849018936 2024-09-10T22:51:14.435 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-10T22:51:14.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:14 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T22:51:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:14.507 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-10T22:51:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:14 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T22:51:14.953 INFO:teuthology.orchestra.run.smithi120.stdout:274877906970 2024-09-10T22:51:14.953 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T22:51:14.998 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:15.000 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:15.001 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:15.002 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:15.005 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:15.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:15 smithi120 ceph-mon[31247]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:51:15.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T22:51:15.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-10T22:51:15.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:51:15.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:51:15.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T22:51:15.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T22:51:15.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T22:51:15.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:51:15.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:15.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:15 smithi153 ceph-mon[34951]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:51:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T22:51:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-10T22:51:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:15.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:15 smithi176 ceph-mon[34982]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 -- 172.21.15.120:0/3977842111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528108010 msgr2=0x7ff52810e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 --2- 172.21.15.120:0/3977842111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528108010 0x7ff52810e810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff520009ca0 tx=0x7ff52002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 -- 172.21.15.120:0/3977842111 shutdown_connections 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 --2- 172.21.15.120:0/3977842111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528108010 0x7ff52810e810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 --2- 172.21.15.120:0/3977842111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 0x7ff528107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 --2- 172.21.15.120:0/3977842111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528101c80 0x7ff528102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 -- 172.21.15.120:0/3977842111 >> 172.21.15.120:0/3977842111 conn(0x7ff5280fd6d0 msgr2=0x7ff5280ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 -- 172.21.15.120:0/3977842111 shutdown_connections 2024-09-10T22:51:16.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 -- 172.21.15.120:0/3977842111 wait complete. 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.004+0000 7ff52db99640 1 Processor -- start 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 -- start start 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 0x7ff5280fe110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528108010 0x7ff5280fe650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:16.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff528111d10 con 0x7ff528101c80 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff528111b90 con 0x7ff528108010 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff52db99640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff528111e90 con 0x7ff528107690 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49784/0 (socket says 172.21.15.120:49784) 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 -- 172.21.15.120:0/4081732572 learned_addr learned my addr 172.21.15.120:0/4081732572 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff526ffd640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 0x7ff5280fe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff527fff640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528108010 0x7ff5280fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 -- 172.21.15.120:0/4081732572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 msgr2=0x7ff5280fe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 0x7ff5280fe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 -- 172.21.15.120:0/4081732572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528108010 msgr2=0x7ff5280fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528108010 0x7ff5280fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.005+0000 7ff5277fe640 1 -- 172.21.15.120:0/4081732572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281a77d0 con 0x7ff528101c80 2024-09-10T22:51:16.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff5277fe640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff51800b4a0 tx=0x7ff51800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518015020 con 0x7ff528101c80 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff518011ce0 con 0x7ff528101c80 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281a7ac0 con 0x7ff528101c80 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518010dc0 con 0x7ff528101c80 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.006+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281a8050 con 0x7ff528101c80 2024-09-10T22:51:16.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.007+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5281020e0 con 0x7ff528101c80 2024-09-10T22:51:16.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.009+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff518011830 con 0x7ff528101c80 2024-09-10T22:51:16.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.010+0000 7ff524ff9640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 0x7ff5100b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:16.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.010+0000 7ff526ffd640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 0x7ff5100b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:16.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.010+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff5180e5780 con 0x7ff528101c80 2024-09-10T22:51:16.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.010+0000 7ff526ffd640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 0x7ff5100b78b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ff6c0 tx=0x7ff51c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:16.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.016+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff51801f020 con 0x7ff528101c80 2024-09-10T22:51:16.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.204+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff5281a8430 con 0x7ff528101c80 2024-09-10T22:51:16.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.204+0000 7ff524ff9640 1 -- 172.21.15.120:0/4081732572 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5180ac510 con 0x7ff528101c80 2024-09-10T22:51:16.207 INFO:teuthology.orchestra.run.smithi120.stdout:34359738447 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 msgr2=0x7ff5100b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 0x7ff5100b78b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ff6c0 tx=0x7ff51c009110 comp rx=0 tx=0).stop 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 msgr2=0x7ff5280fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff51800b4a0 tx=0x7ff51800b8a0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 shutdown_connections 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff5100b53f0 0x7ff5100b78b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff528108010 0x7ff5280fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff528107690 0x7ff5280fe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 --2- 172.21.15.120:0/4081732572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff528101c80 0x7ff5280fdbd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 >> 172.21.15.120:0/4081732572 conn(0x7ff5280fd6d0 msgr2=0x7ff52806b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 shutdown_connections 2024-09-10T22:51:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:16.207+0000 7ff52db99640 1 -- 172.21.15.120:0/4081732572 wait complete. 2024-09-10T22:51:17.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-10T22:51:17.299 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:17.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:17 smithi120 ceph-mon[31247]: pgmap v308: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:17.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:17 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4081732572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:51:17.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:17 smithi153 ceph-mon[34951]: pgmap v308: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:17.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:17 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4081732572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:51:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:17 smithi176 ceph-mon[34982]: pgmap v308: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:17 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4081732572' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:51:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 -- 172.21.15.120:0/2137307789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 msgr2=0x7fe60c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 --2- 172.21.15.120:0/2137307789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c10f930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ca0 tx=0x7fe60402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 -- 172.21.15.120:0/2137307789 shutdown_connections 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 --2- 172.21.15.120:0/2137307789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c10f930 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 --2- 172.21.15.120:0/2137307789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 0x7fe60c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 --2- 172.21.15.120:0/2137307789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe60c101220 0x7fe60c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 -- 172.21.15.120:0/2137307789 >> 172.21.15.120:0/2137307789 conn(0x7fe60c078100 msgr2=0x7fe60c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 -- 172.21.15.120:0/2137307789 shutdown_connections 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.683+0000 7fe612972640 1 -- 172.21.15.120:0/2137307789 wait complete. 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 Processor -- start 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 -- start start 2024-09-10T22:51:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe60c101220 0x7fe60c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 0x7fe60c102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c112df0 con 0x7fe60c105ef0 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c112c70 con 0x7fe60c101220 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe612972640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c112f70 con 0x7fe60c105570 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe610f0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe610f0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49798/0 (socket says 172.21.15.120:49798) 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe610f0f640 1 -- 172.21.15.120:0/1688566731 learned_addr learned my addr 172.21.15.120:0/1688566731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.684+0000 7fe60bfff640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe60c101220 0x7fe60c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe60b7fe640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 0x7fe60c102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 -- 172.21.15.120:0/1688566731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 msgr2=0x7fe60c102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 0x7fe60c102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 -- 172.21.15.120:0/1688566731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe60c101220 msgr2=0x7fe60c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe60c101220 0x7fe60c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c1a7680 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe60b7fe640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe60c105570 0x7fe60c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe610f0f640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c102c70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe604005cc0 tx=0x7fe60402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604003cf0 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6040044f0 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c1a7910 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60402fc30 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.685+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1a7ed0 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.686+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe60c0638e0 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.687+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fe604004040 con 0x7fe60c105ef0 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe6097fa640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5f40b5370 0x7fe5f40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe60bfff640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5f40b5370 0x7fe5f40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fe604107490 con 0x7fe60c105ef0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe6097fa640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] conn(0x7fe5f40bed90 0x7fe5f40c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe60403fec0 con 0x7fe5f40bed90 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe6041078e0 con 0x7fe60c105ef0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe60b7fe640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] conn(0x7fe5f40bed90 0x7fe5f40c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.688+0000 7fe60bfff640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5f40b5370 0x7fe5f40b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00abe0 tx=0x7fe5fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.689+0000 7fe60b7fe640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] conn(0x7fe5f40bed90 0x7fe5f40c1230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.689+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== osd.9 v2:172.21.15.120:6826/283565809 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe60403fec0 con 0x7fe5f40bed90 2024-09-10T22:51:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 --> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe60c063d60 con 0x7fe5f40bed90 2024-09-10T22:51:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe6097fa640 1 -- 172.21.15.120:0/1688566731 <== osd.9 v2:172.21.15.120:6826/283565809 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe60c063d60 con 0x7fe5f40bed90 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] conn(0x7fe5f40bed90 msgr2=0x7fe5f40c1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] conn(0x7fe5f40bed90 0x7fe5f40c1230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5f40b5370 msgr2=0x7fe5f40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5f40b5370 0x7fe5f40b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00abe0 tx=0x7fe5fc00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 msgr2=0x7fe60c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe612972640 1 --2- 172.21.15.120:0/1688566731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe60c105ef0 0x7fe60c102c70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe604005cc0 tx=0x7fe60402f470 comp rx=0 tx=0).stop 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.703+0000 7fe610f0f640 1 -- 172.21.15.120:0/1688566731 reap_dead start 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.704+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 shutdown_connections 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.704+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 >> 172.21.15.120:0/1688566731 conn(0x7fe60c078100 msgr2=0x7fe60c108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.704+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 shutdown_connections 2024-09-10T22:51:17.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.704+0000 7fe612972640 1 -- 172.21.15.120:0/1688566731 wait complete. 2024-09-10T22:51:18.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 -- 172.21.15.120:0/1116900189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1340103b70 msgr2=0x7f1340103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 --2- 172.21.15.120:0/1116900189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1340103b70 0x7f1340103f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1330009900 tx=0x7f133002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 -- 172.21.15.120:0/1116900189 shutdown_connections 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 --2- 172.21.15.120:0/1116900189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f134010d120 0x7f134010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 --2- 172.21.15.120:0/1116900189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340104570 0x7f134010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 --2- 172.21.15.120:0/1116900189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1340103b70 0x7f1340103f50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 -- 172.21.15.120:0/1116900189 >> 172.21.15.120:0/1116900189 conn(0x7f13400fd700 msgr2=0x7f13400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 -- 172.21.15.120:0/1116900189 shutdown_connections 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.051+0000 7f134629a640 1 -- 172.21.15.120:0/1116900189 wait complete. 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 Processor -- start 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 -- start start 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340103b70 0x7f13401019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1340104570 0x7f1340101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112db0 con 0x7f134010d120 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112c30 con 0x7f1340103b70 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f134629a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112f30 con 0x7f1340104570 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f1344837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f1344837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49848/0 (socket says 172.21.15.120:49848) 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f1344837640 1 -- 172.21.15.120:0/2506437929 learned_addr learned my addr 172.21.15.120:0/2506437929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.052+0000 7f133ffff640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340103b70 0x7f13401019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1337fff640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1340104570 0x7f1340101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 -- 172.21.15.120:0/2506437929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1340104570 msgr2=0x7f1340101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1340104570 0x7f1340101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 -- 172.21.15.120:0/2506437929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340103b70 msgr2=0x7f13401019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340103b70 0x7f13401019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 -- 172.21.15.120:0/2506437929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13401a76a0 con 0x7f134010d120 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f1344837640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f133800cad0 tx=0x7f133800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133800ae20 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13401a7990 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1338005020 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133800f6b0 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.053+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401a7f20 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.054+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1340103f50 con 0x7f134010d120 2024-09-10T22:51:18.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.057+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f13380054f0 con 0x7f134010d120 2024-09-10T22:51:18.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.058+0000 7f133dffb640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 0x7f132c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.058+0000 7f133ffff640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 0x7f132c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.058+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f13380e6160 con 0x7f134010d120 2024-09-10T22:51:18.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.059+0000 7f133ffff640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 0x7f132c0b78b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1330005ab0 tx=0x7f1330005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.062+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1338014030 con 0x7f134010d120 2024-09-10T22:51:18.066 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:18.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.252+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f13401a8300 con 0x7f134010d120 2024-09-10T22:51:18.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.253+0000 7f133dffb640 1 -- 172.21.15.120:0/2506437929 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13380ace90 con 0x7f134010d120 2024-09-10T22:51:18.255 INFO:teuthology.orchestra.run.smithi120.stdout:73014444101 2024-09-10T22:51:18.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 msgr2=0x7f132c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 0x7f132c0b78b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1330005ab0 tx=0x7f1330005a20 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 msgr2=0x7f1340102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f133800cad0 tx=0x7f133800cf90 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 shutdown_connections 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f132c0b53f0 0x7f132c0b78b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f134010d120 0x7f1340102470 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1340104570 0x7f1340101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 --2- 172.21.15.120:0/2506437929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1340103b70 0x7f13401019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 >> 172.21.15.120:0/2506437929 conn(0x7f13400fd700 msgr2=0x7f134010f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 shutdown_connections 2024-09-10T22:51:18.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.255+0000 7f134629a640 1 -- 172.21.15.120:0/2506437929 wait complete. 2024-09-10T22:51:18.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 -- 172.21.15.120:0/1866513164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c102d90 msgr2=0x7ff34c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 --2- 172.21.15.120:0/1866513164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c102d90 0x7ff34c103170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009900 tx=0x7ff33c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:18.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 -- 172.21.15.120:0/1866513164 shutdown_connections 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 --2- 172.21.15.120:0/1866513164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 --2- 172.21.15.120:0/1866513164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c1087a0 0x7ff34c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 --2- 172.21.15.120:0/1866513164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c102d90 0x7ff34c103170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 -- 172.21.15.120:0/1866513164 >> 172.21.15.120:0/1866513164 conn(0x7ff34c0fe820 msgr2=0x7ff34c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 -- 172.21.15.120:0/1866513164 shutdown_connections 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.296+0000 7ff352057640 1 -- 172.21.15.120:0/1866513164 wait complete. 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 Processor -- start 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 -- start start 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 0x7ff34c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112da0 con 0x7ff34c1087a0 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112c20 con 0x7ff34c102d90 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff352057640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112f20 con 0x7ff34c109120 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff34affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff34affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49868/0 (socket says 172.21.15.120:49868) 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.297+0000 7ff34affd640 1 -- 172.21.15.120:0/2334349917 learned_addr learned my addr 172.21.15.120:0/2334349917 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34b7fe640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 0x7ff34c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34bfff640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 -- 172.21.15.120:0/2334349917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 msgr2=0x7ff34c19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 -- 172.21.15.120:0/2334349917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 msgr2=0x7ff34c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 0x7ff34c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 -- 172.21.15.120:0/2334349917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c19f2a0 con 0x7ff34c1087a0 2024-09-10T22:51:18.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34b7fe640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 0x7ff34c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34bfff640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff34affd640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d950 tx=0x7ff34000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340017030 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff340004020 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c19f590 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400047a0 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.298+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a5c50 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.299+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c103170 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.300+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff340009a30 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.301+0000 7ff348ff9640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 0x7ff3340b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.301+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff3400e5ef0 con 0x7ff34c1087a0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.301+0000 7ff34b7fe640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 0x7ff3340b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.302+0000 7ff34b7fe640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 0x7ff3340b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff33c031440 tx=0x7ff33c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.304+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff3400acc80 con 0x7ff34c1087a0 2024-09-10T22:51:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:18 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1104052119' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:51:18.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:18 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1104052119' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:51:18.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.488+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff34c19a320 con 0x7ff34c1087a0 2024-09-10T22:51:18.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.488+0000 7ff348ff9640 1 -- 172.21.15.120:0/2334349917 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff34c19a320 con 0x7ff34c1087a0 2024-09-10T22:51:18.491 INFO:teuthology.orchestra.run.smithi120.stdout:197568495658 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 msgr2=0x7ff3340b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 0x7ff3340b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff33c031440 tx=0x7ff33c005a20 comp rx=0 tx=0).stop 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 msgr2=0x7ff34c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d950 tx=0x7ff34000de10 comp rx=0 tx=0).stop 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 shutdown_connections 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff3340b5370 0x7ff3340b7830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff34c109120 0x7ff34c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff34c1087a0 0x7ff34c199730 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 --2- 172.21.15.120:0/2334349917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff34c102d90 0x7ff34c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 >> 172.21.15.120:0/2334349917 conn(0x7ff34c0fe820 msgr2=0x7ff34c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 shutdown_connections 2024-09-10T22:51:18.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.491+0000 7ff352057640 1 -- 172.21.15.120:0/2334349917 wait complete. 2024-09-10T22:51:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:18 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1104052119' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:51:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 -- 172.21.15.120:0/912749790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38108020 msgr2=0x7f0c3810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/912749790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38108020 0x7f0c3810e820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009ca0 tx=0x7f0c3002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 -- 172.21.15.120:0/912749790 shutdown_connections 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/912749790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38108020 0x7f0c3810e820 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/912749790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c381076a0 0x7f0c38107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/912749790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c38101c90 0x7f0c38102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 -- 172.21.15.120:0/912749790 >> 172.21.15.120:0/912749790 conn(0x7f0c380fd700 msgr2=0x7f0c380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 -- 172.21.15.120:0/912749790 shutdown_connections 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.550+0000 7f0c3e98e640 1 -- 172.21.15.120:0/912749790 wait complete. 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 Processor -- start 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 -- start start 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c381076a0 0x7f0c3819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c38108020 0x7f0c381a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111ce0 con 0x7f0c38101c90 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111b60 con 0x7f0c38108020 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c3e98e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111e60 con 0x7f0c381076a0 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.551+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49882/0 (socket says 172.21.15.120:49882) 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 -- 172.21.15.120:0/1337381920 learned_addr learned my addr 172.21.15.120:0/1337381920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c3cf2b640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c38108020 0x7f0c381a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c377fe640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c381076a0 0x7f0c3819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 -- 172.21.15.120:0/1337381920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c381076a0 msgr2=0x7f0c3819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c381076a0 0x7f0c3819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 -- 172.21.15.120:0/1337381920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c38108020 msgr2=0x7f0c381a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c38108020 0x7f0c381a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 -- 172.21.15.120:0/1337381920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c381a1490 con 0x7f0c38101c90 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c37fff640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800d770 tx=0x7f0c2800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28017070 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c28009820 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c381a1780 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28004bc0 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.552+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a7e40 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.553+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38102070 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.555+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0c28005220 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.556+0000 7f0c357fa640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 0x7f0c200b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.556+0000 7f0c377fe640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 0x7f0c200b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.556+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f0c280e5b50 con 0x7f0c38101c90 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.556+0000 7f0c377fe640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 0x7f0c200b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c38107050 tx=0x7f0c2c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.559+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c280ac8e0 con 0x7f0c38101c90 2024-09-10T22:51:18.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.742+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0c381a8270 con 0x7f0c38101c90 2024-09-10T22:51:18.744 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.742+0000 7f0c357fa640 1 -- 172.21.15.120:0/1337381920 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c280ac1e0 con 0x7f0c38101c90 2024-09-10T22:51:18.745 INFO:teuthology.orchestra.run.smithi120.stdout:150323855412 2024-09-10T22:51:18.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 msgr2=0x7f0c200b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 0x7f0c200b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c38107050 tx=0x7f0c2c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 msgr2=0x7f0c3819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800d770 tx=0x7f0c2800db70 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 shutdown_connections 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0c200b5370 0x7f0c200b7830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c38108020 0x7f0c381a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c381076a0 0x7f0c3819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 --2- 172.21.15.120:0/1337381920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c38101c90 0x7f0c3819b3e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 >> 172.21.15.120:0/1337381920 conn(0x7f0c380fd700 msgr2=0x7f0c38100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 shutdown_connections 2024-09-10T22:51:18.747 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.744+0000 7f0c3e98e640 1 -- 172.21.15.120:0/1337381920 wait complete. 2024-09-10T22:51:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:19 smithi120 ceph-mon[31247]: pgmap v309: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:19 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2506437929' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:51:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:19 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2334349917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:51:19.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:19 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1337381920' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:51:19.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:19 smithi153 ceph-mon[34951]: pgmap v309: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:19.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:19 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2506437929' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:51:19.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:19 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2334349917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:51:19.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:19 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1337381920' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:51:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:19 smithi176 ceph-mon[34982]: pgmap v309: 97 pgs: 1 activating, 1 peering, 95 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:19 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2506437929' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:51:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:19 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2334349917' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:51:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:19 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1337381920' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:51:20.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495655 got 197568495658 for osd.7 2024-09-10T22:51:20.072 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:20.654 INFO:teuthology.orchestra.run.smithi120.stdout:253403070498 2024-09-10T22:51:20.654 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T22:51:20.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.899+0000 7fac2f646640 1 -- 172.21.15.120:0/2597379027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac28104600 msgr2=0x7fac2810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.899+0000 7fac2f646640 1 --2- 172.21.15.120:0/2597379027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac28104600 0x7fac2810caf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac18009900 tx=0x7fac1802a410 comp rx=0 tx=0).stop 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- 172.21.15.120:0/2597379027 shutdown_connections 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- 172.21.15.120:0/2597379027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac2810d140 0x7fac2810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- 172.21.15.120:0/2597379027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac28104600 0x7fac2810caf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- 172.21.15.120:0/2597379027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac281040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- 172.21.15.120:0/2597379027 >> 172.21.15.120:0/2597379027 conn(0x7fac280fd700 msgr2=0x7fac280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- 172.21.15.120:0/2597379027 shutdown_connections 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- 172.21.15.120:0/2597379027 wait complete. 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 Processor -- start 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- start start 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac28199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac28104600 0x7fac28199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac28112df0 con 0x7fac2810d140 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac28112c70 con 0x7fac28103ce0 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2f646640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fac28112f70 con 0x7fac28104600 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.900+0000 7fac2dbe3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49824/0 (socket says 172.21.15.120:49824) 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 -- 172.21.15.120:0/1104052119 learned_addr learned my addr 172.21.15.120:0/1104052119 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2cbe1640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac28104600 0x7fac28199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2d3e2640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac28199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 -- 172.21.15.120:0/1104052119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac28104600 msgr2=0x7fac28199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac28104600 0x7fac28199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 -- 172.21.15.120:0/1104052119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 msgr2=0x7fac28199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac28199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 -- 172.21.15.120:0/1104052119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2819f3d0 con 0x7fac2810d140 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stdout:296352743446 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2d3e2640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac28199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2dbe3640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac2000b970 tx=0x7fac2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20019070 con 0x7fac2810d140 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac20014440 con 0x7fac2810d140 2024-09-10T22:51:20.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2819f6c0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200075d0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.901+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac281a5f50 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.902+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac281040c0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.903+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fac2000cb40 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.904+0000 7fac1e7fc640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 0x7fac140b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.905+0000 7fac2d3e2640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 0x7fac140b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.905+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fac200e59a0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.905+0000 7fac2d3e2640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 0x7fac140b78b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac24009b40 tx=0x7fac24005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:17.908+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac200ac6d0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.102+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fac281a6230 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.102+0000 7fac1e7fc640 1 -- 172.21.15.120:0/1104052119 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fac200abfd0 con 0x7fac2810d140 2024-09-10T22:51:20.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 msgr2=0x7fac140b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 0x7fac140b78b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac24009b40 tx=0x7fac24005e50 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 msgr2=0x7fac2819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac2000b970 tx=0x7fac2000be30 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 shutdown_connections 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac140b53f0 0x7fac140b78b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac2810d140 0x7fac2819ec00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac28104600 0x7fac28199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 --2- 172.21.15.120:0/1104052119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac28103ce0 0x7fac28199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 >> 172.21.15.120:0/1104052119 conn(0x7fac280fd700 msgr2=0x7fac280fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.104+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 shutdown_connections 2024-09-10T22:51:20.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:18.105+0000 7fac2f646640 1 -- 172.21.15.120:0/1104052119 wait complete. 2024-09-10T22:51:21.377 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:21 smithi120 ceph-mon[31247]: pgmap v310: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:21.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743444 got 296352743446 for osd.11 2024-09-10T22:51:21.403 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:21.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:21 smithi153 ceph-mon[34951]: pgmap v310: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:21 smithi176 ceph-mon[34982]: pgmap v310: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:21.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 -- 172.21.15.120:0/1173172081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 msgr2=0x7fdb94108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/1173172081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94108c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009900 tx=0x7fdb7c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 -- 172.21.15.120:0/1173172081 shutdown_connections 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/1173172081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/1173172081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94108c10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/1173172081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 0x7fdb941031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 -- 172.21.15.120:0/1173172081 >> 172.21.15.120:0/1173172081 conn(0x7fdb940fe830 msgr2=0x7fdb94100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 -- 172.21.15.120:0/1173172081 shutdown_connections 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.866+0000 7fdb98c0c640 1 -- 172.21.15.120:0/1173172081 wait complete. 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 Processor -- start 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 -- start start 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 0x7fdb941992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:21.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94112e10 con 0x7fdb941087d0 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94112c90 con 0x7fdb94109150 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb98c0c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94112f90 con 0x7fdb94102dc0 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb89d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49890/0 (socket says 172.21.15.120:49890) 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb89d9b640 1 -- 172.21.15.120:0/4219773947 learned_addr learned my addr 172.21.15.120:0/4219773947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.867+0000 7fdb92d9d640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 -- 172.21.15.120:0/4219773947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 msgr2=0x7fdb941992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb9259c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 0x7fdb941992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 0x7fdb941992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 -- 172.21.15.120:0/4219773947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 msgr2=0x7fdb9419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:21.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 -- 172.21.15.120:0/4219773947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9419f3a0 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb92d9d640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb89d9b640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006010 tx=0x7fdb7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c03f030 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c003800 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9419f630 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c035070 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.868+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941a5d50 con 0x7fdb941087d0 2024-09-10T22:51:21.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.869+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb941031a0 con 0x7fdb941087d0 2024-09-10T22:51:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.870+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fdb7c031b10 con 0x7fdb941087d0 2024-09-10T22:51:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.871+0000 7fdb8bfff640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 0x7fdb800b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.872+0000 7fdb9259c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 0x7fdb800b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.872+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fdb7c106a10 con 0x7fdb941087d0 2024-09-10T22:51:21.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.872+0000 7fdb9259c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 0x7fdb800b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400a810 tx=0x7fdb84008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:21.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:21.875+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb7c0cd740 con 0x7fdb941087d0 2024-09-10T22:51:21.915 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:22.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.065+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdb9419a420 con 0x7fdb941087d0 2024-09-10T22:51:22.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.065+0000 7fdb8bfff640 1 -- 172.21.15.120:0/4219773947 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb9419a420 con 0x7fdb941087d0 2024-09-10T22:51:22.068 INFO:teuthology.orchestra.run.smithi120.stdout:176093659184 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.067+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 msgr2=0x7fdb800b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.067+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 0x7fdb800b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400a810 tx=0x7fdb84008040 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.067+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 msgr2=0x7fdb94199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.067+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006010 tx=0x7fdb7c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 shutdown_connections 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb800b5370 0x7fdb800b7830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb94109150 0x7fdb9419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb941087d0 0x7fdb94199830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 --2- 172.21.15.120:0/4219773947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb94102dc0 0x7fdb941992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 >> 172.21.15.120:0/4219773947 conn(0x7fdb940fe830 msgr2=0x7fdb94100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 shutdown_connections 2024-09-10T22:51:22.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.068+0000 7fdb98c0c640 1 -- 172.21.15.120:0/4219773947 wait complete. 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 -- 172.21.15.120:0/1431197544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e74108160 msgr2=0x7f8e7410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 --2- 172.21.15.120:0/1431197544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e74108160 0x7f8e7410e890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 -- 172.21.15.120:0/1431197544 shutdown_connections 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 --2- 172.21.15.120:0/1431197544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e74108160 0x7f8e7410e890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 --2- 172.21.15.120:0/1431197544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741077c0 0x7f8e74107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 --2- 172.21.15.120:0/1431197544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e741029d0 0x7f8e74102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 -- 172.21.15.120:0/1431197544 >> 172.21.15.120:0/1431197544 conn(0x7f8e74076e50 msgr2=0x7f8e74077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 -- 172.21.15.120:0/1431197544 shutdown_connections 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.183+0000 7f8e79918640 1 -- 172.21.15.120:0/1431197544 wait complete. 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 Processor -- start 2024-09-10T22:51:22.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 -- start start 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 0x7f8e74197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 0x7f8e74198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111cc0 con 0x7f8e741077c0 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111b40 con 0x7f8e741029d0 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e79918640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111e40 con 0x7f8e74108160 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e727fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e727fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49908/0 (socket says 172.21.15.120:49908) 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.184+0000 7f8e727fc640 1 -- 172.21.15.120:0/1820313664 learned_addr learned my addr 172.21.15.120:0/1820313664 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e737fe640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 0x7f8e74198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e72ffd640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 0x7f8e74197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 -- 172.21.15.120:0/1820313664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 msgr2=0x7f8e74198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 0x7f8e74198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 -- 172.21.15.120:0/1820313664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 msgr2=0x7f8e74197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 0x7f8e74197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 -- 172.21.15.120:0/1820313664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e741a7690 con 0x7f8e741077c0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e737fe640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 0x7f8e74198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e72ffd640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 0x7f8e74197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e727fc640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800db80 tx=0x7f8e6800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68017070 con 0x7f8e741077c0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e68003b70 con 0x7f8e741077c0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680104c0 con 0x7f8e741077c0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e741a7980 con 0x7f8e741077c0 2024-09-10T22:51:22.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.185+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a7e70 con 0x7f8e741077c0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.186+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e74102db0 con 0x7f8e741077c0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.187+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8e68023060 con 0x7f8e741077c0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.188+0000 7f8e78916640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 0x7f8e5c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.188+0000 7f8e72ffd640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 0x7f8e5c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.188+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f8e680e5a40 con 0x7f8e741077c0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.189+0000 7f8e72ffd640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 0x7f8e5c0b78b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009b40 tx=0x7f8e64005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.192+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8e680ac7d0 con 0x7f8e741077c0 2024-09-10T22:51:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.371+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8e7402e620 con 0x7f8e741077c0 2024-09-10T22:51:22.374 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4219773947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:51:22.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.372+0000 7f8e78916640 1 -- 172.21.15.120:0/1820313664 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e680ac0d0 con 0x7f8e741077c0 2024-09-10T22:51:22.375 INFO:teuthology.orchestra.run.smithi120.stdout:103079215167 2024-09-10T22:51:22.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 msgr2=0x7f8e5c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 0x7f8e5c0b78b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009b40 tx=0x7f8e64005e50 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 msgr2=0x7f8e741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800db80 tx=0x7f8e6800b630 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 shutdown_connections 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8e5c0b53f0 0x7f8e5c0b78b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e74108160 0x7f8e74198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e741077c0 0x7f8e741983c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.374+0000 7f8e79918640 1 --2- 172.21.15.120:0/1820313664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e741029d0 0x7f8e74197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.375+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 >> 172.21.15.120:0/1820313664 conn(0x7f8e74076e50 msgr2=0x7f8e74100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.375+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 shutdown_connections 2024-09-10T22:51:22.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:22.375+0000 7f8e79918640 1 -- 172.21.15.120:0/1820313664 wait complete. 2024-09-10T22:51:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4219773947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:51:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4219773947' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:51:23.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215163 got 103079215167 for osd.3 2024-09-10T22:51:23.064 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:23 smithi120 ceph-mon[31247]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:23.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:23 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1820313664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:51:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:23 smithi153 ceph-mon[34951]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:23.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:23 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1820313664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:51:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:23.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855410 got 150323855412 for osd.5 2024-09-10T22:51:23.509 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:23.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:23 smithi176 ceph-mon[34982]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:23.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:23 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1820313664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:51:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/2948084175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8108040 msgr2=0x7fe5f810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/2948084175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8108040 0x7fe5f810e840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009ca0 tx=0x7fe5f002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/2948084175 shutdown_connections 2024-09-10T22:51:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/2948084175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8108040 0x7fe5f810e840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/2948084175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/2948084175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/2948084175 >> 172.21.15.120:0/2948084175 conn(0x7fe5f80fd700 msgr2=0x7fe5f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.897+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/2948084175 shutdown_connections 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/2948084175 wait complete. 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 Processor -- start 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 -- start start 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8108040 0x7fe5f819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f8111d00 con 0x7fe5f8101cb0 2024-09-10T22:51:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f8111b80 con 0x7fe5f81076c0 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5fe6e4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f8111e80 con 0x7fe5f8108040 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35126/0 (socket says 172.21.15.120:35126) 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.898+0000 7fe5f7fff640 1 -- 172.21.15.120:0/3954998848 learned_addr learned my addr 172.21.15.120:0/3954998848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5fcc81640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8108040 0x7fe5f819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5effff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 -- 172.21.15.120:0/3954998848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8108040 msgr2=0x7fe5f819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8108040 0x7fe5f819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 -- 172.21.15.120:0/3954998848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 msgr2=0x7fe5f819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 -- 172.21.15.120:0/3954998848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f81a53f0 con 0x7fe5f8101cb0 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5effff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f7fff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c920 tx=0x7fe5e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8007c70 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f81a56e0 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e8007e10 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8018820 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.899+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f81a5cf0 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.900+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f8102090 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.901+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fe5e800f450 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.902+0000 7fe5f5ffb640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 0x7fe5e40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.902+0000 7fe5effff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 0x7fe5e40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.902+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fe5e80e61b0 con 0x7fe5f8101cb0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.902+0000 7fe5effff640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 0x7fe5e40b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0006fd0 tx=0x7fe5e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:23.905+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe5e8014030 con 0x7fe5f8101cb0 2024-09-10T22:51:24.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.086+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe5f806b8c0 con 0x7fe5f8101cb0 2024-09-10T22:51:24.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.086+0000 7fe5f5ffb640 1 -- 172.21.15.120:0/3954998848 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5e80acfd0 con 0x7fe5f8101cb0 2024-09-10T22:51:24.089 INFO:teuthology.orchestra.run.smithi120.stdout:55834574922 2024-09-10T22:51:24.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 msgr2=0x7fe5e40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 0x7fe5e40b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0006fd0 tx=0x7fe5e0008040 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 msgr2=0x7fe5f8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c920 tx=0x7fe5e800cde0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 shutdown_connections 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5e40b5370 0x7fe5e40b7830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5f8108040 0x7fe5f819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5f81076c0 0x7fe5f819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 --2- 172.21.15.120:0/3954998848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5f8101cb0 0x7fe5f8199ff0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.088+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 >> 172.21.15.120:0/3954998848 conn(0x7fe5f80fd700 msgr2=0x7fe5f80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.089+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 shutdown_connections 2024-09-10T22:51:24.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.089+0000 7fe5fe6e4640 1 -- 172.21.15.120:0/3954998848 wait complete. 2024-09-10T22:51:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:24 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3954998848' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:51:24.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:24 smithi120 ceph-mon[31247]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:24 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3954998848' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:51:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:24 smithi153 ceph-mon[34951]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:24 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3954998848' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:51:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:24 smithi176 ceph-mon[34982]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:24.610 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444098 got 73014444101 for osd.2 2024-09-10T22:51:24.610 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 -- 172.21.15.120:0/3079913395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 msgr2=0x7f95d8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 --2- 172.21.15.120:0/3079913395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8108c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009900 tx=0x7f95c002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 -- 172.21.15.120:0/3079913395 shutdown_connections 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 --2- 172.21.15.120:0/3079913395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 --2- 172.21.15.120:0/3079913395 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8108c30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 --2- 172.21.15.120:0/3079913395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 -- 172.21.15.120:0/3079913395 >> 172.21.15.120:0/3079913395 conn(0x7f95d80fe830 msgr2=0x7f95d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 -- 172.21.15.120:0/3079913395 shutdown_connections 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.785+0000 7f95dd43b640 1 -- 172.21.15.120:0/3079913395 wait complete. 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 Processor -- start 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 -- start start 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8112e30 con 0x7f95d81087f0 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8112cb0 con 0x7f95d8102de0 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95dd43b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8112fb0 con 0x7f95d8109170 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35152/0 (socket says 172.21.15.120:35152) 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.786+0000 7f95d67fc640 1 -- 172.21.15.120:0/1252625340 learned_addr learned my addr 172.21.15.120:0/1252625340 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d6ffd640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 -- 172.21.15.120:0/1252625340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 msgr2=0x7f95d819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d77fe640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 -- 172.21.15.120:0/1252625340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 msgr2=0x7f95d8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 -- 172.21.15.120:0/1252625340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d819f3f0 con 0x7f95d81087f0 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d6ffd640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d77fe640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95d67fc640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006010 tx=0x7f95c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c003f030 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d819f680 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c0003800 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0035070 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.787+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a5da0 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.788+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d81031c0 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.789+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f95c0031b10 con 0x7f95d81087f0 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.790+0000 7f95cffff640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 0x7f95c40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.791+0000 7f95d6ffd640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 0x7f95c40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.791+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f95c01069b0 con 0x7f95d81087f0 2024-09-10T22:51:24.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.791+0000 7f95d6ffd640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 0x7f95c40b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f95c8008e00 tx=0x7f95c8008350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.794+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f95c00cd740 con 0x7f95d81087f0 2024-09-10T22:51:24.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.975+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f95d810af50 con 0x7f95d81087f0 2024-09-10T22:51:24.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.975+0000 7f95cffff640 1 -- 172.21.15.120:0/1252625340 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f95c00cd040 con 0x7f95d81087f0 2024-09-10T22:51:24.978 INFO:teuthology.orchestra.run.smithi120.stdout:128849018939 2024-09-10T22:51:24.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 msgr2=0x7f95c40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 0x7f95c40b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f95c8008e00 tx=0x7f95c8008350 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 msgr2=0x7f95d8199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006010 tx=0x7f95c00042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 shutdown_connections 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95c40b5370 0x7f95c40b7830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95d8109170 0x7f95d819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d81087f0 0x7f95d8199880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 --2- 172.21.15.120:0/1252625340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95d8102de0 0x7f95d8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.977+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 >> 172.21.15.120:0/1252625340 conn(0x7f95d80fe830 msgr2=0x7f95d8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.978+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 shutdown_connections 2024-09-10T22:51:24.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:24.978+0000 7f95dd43b640 1 -- 172.21.15.120:0/1252625340 wait complete. 2024-09-10T22:51:25.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1252625340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:51:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1252625340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:51:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1252625340' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:51:25.882 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659182 got 176093659184 for osd.6 2024-09-10T22:51:25.882 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:26.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018936 got 128849018939 for osd.4 2024-09-10T22:51:26.341 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:26 smithi120 ceph-mon[31247]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 743 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:26.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:26 smithi153 ceph-mon[34951]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 743 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:26.474 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:26 smithi176 ceph-mon[34982]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 743 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:51:26.710 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:26.711 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:26.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2024-09-10T22:51:26.748 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3661713319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 msgr2=0x7ff0f41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3661713319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f41020b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009900 tx=0x7ff0e402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3661713319 shutdown_connections 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3661713319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3661713319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 0x7ff0f4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3661713319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f41020b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3661713319 >> 172.21.15.120:0/3661713319 conn(0x7ff0f40fd700 msgr2=0x7ff0f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3661713319 shutdown_connections 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.235+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3661713319 wait complete. 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 Processor -- start 2024-09-10T22:51:27.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 -- start start 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 0x7ff0f419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f41a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111d20 con 0x7ff0f4101cd0 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111ba0 con 0x7ff0f4108060 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f9a04640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111ea0 con 0x7ff0f41076e0 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35176/0 (socket says 172.21.15.120:35176) 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 -- 172.21.15.120:0/3983066026 learned_addr learned my addr 172.21.15.120:0/3983066026 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0ea7fc640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 0x7ff0f419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f37fe640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f41a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 -- 172.21.15.120:0/3983066026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 msgr2=0x7ff0f419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 0x7ff0f419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 -- 172.21.15.120:0/3983066026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 msgr2=0x7ff0f41a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f41a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.236+0000 7ff0f2ffd640 1 -- 172.21.15.120:0/3983066026 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41a5650 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f37fe640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f41a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f2ffd640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005940 tx=0x7ff0e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e403f030 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a5820 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e4003cb0 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4035070 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.237+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a5c70 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.238+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f41020b0 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.239+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff0e403bbf0 con 0x7ff0f4101cd0 2024-09-10T22:51:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.240+0000 7ff0f8a02640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.240+0000 7ff0ea7fc640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.240+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff0e4106ae0 con 0x7ff0f4101cd0 2024-09-10T22:51:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.240+0000 7ff0ea7fc640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 0x7ff0e00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0f406c0f0 tx=0x7ff0dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.243+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff0e40cd870 con 0x7ff0f4101cd0 2024-09-10T22:51:27.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.423+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff0f402e5c0 con 0x7ff0f4101cd0 2024-09-10T22:51:27.426 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:27 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:27.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.423+0000 7ff0f8a02640 1 -- 172.21.15.120:0/3983066026 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff0e40cd170 con 0x7ff0f4101cd0 2024-09-10T22:51:27.426 INFO:teuthology.orchestra.run.smithi120.stdout:227633266728 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 msgr2=0x7ff0e00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 0x7ff0e00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0f406c0f0 tx=0x7ff0dc00b390 comp rx=0 tx=0).stop 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 msgr2=0x7ff0f419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005940 tx=0x7ff0e40042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 shutdown_connections 2024-09-10T22:51:27.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0f4108060 0x7ff0f41a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0f41076e0 0x7ff0f419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 --2- 172.21.15.120:0/3983066026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0f4101cd0 0x7ff0f419a090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 >> 172.21.15.120:0/3983066026 conn(0x7ff0f40fd700 msgr2=0x7ff0f40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 shutdown_connections 2024-09-10T22:51:27.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.426+0000 7ff0f9a04640 1 -- 172.21.15.120:0/3983066026 wait complete. 2024-09-10T22:51:27.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:27 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:27.509 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:27 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:27.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 -- 172.21.15.120:0/491187561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c104570 msgr2=0x7f8c1c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 --2- 172.21.15.120:0/491187561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c104570 0x7f8c1c10cb50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009900 tx=0x7f8c0c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 -- 172.21.15.120:0/491187561 shutdown_connections 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 --2- 172.21.15.120:0/491187561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c10d120 0x7f8c1c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 --2- 172.21.15.120:0/491187561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c104570 0x7f8c1c10cb50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 --2- 172.21.15.120:0/491187561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c103b70 0x7f8c1c103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 -- 172.21.15.120:0/491187561 >> 172.21.15.120:0/491187561 conn(0x7f8c1c0fd700 msgr2=0x7f8c1c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 -- 172.21.15.120:0/491187561 shutdown_connections 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.892+0000 7f8c233c1640 1 -- 172.21.15.120:0/491187561 wait complete. 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 Processor -- start 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 -- start start 2024-09-10T22:51:27.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c103b70 0x7f8c1c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c104570 0x7f8c1c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c112d60 con 0x7f8c1c10d120 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c112be0 con 0x7f8c1c103b70 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c233c1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c112ee0 con 0x7f8c1c104570 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35206/0 (socket says 172.21.15.120:35206) 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 -- 172.21.15.120:0/2047663687 learned_addr learned my addr 172.21.15.120:0/2047663687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2115d640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c103b70 0x7f8c1c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2095c640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c104570 0x7f8c1c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 -- 172.21.15.120:0/2047663687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c104570 msgr2=0x7f8c1c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c104570 0x7f8c1c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 -- 172.21.15.120:0/2047663687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c103b70 msgr2=0x7f8c1c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c103b70 0x7f8c1c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.893+0000 7f8c2195e640 1 -- 172.21.15.120:0/2047663687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c19f2e0 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c2195e640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400eaa0 tx=0x7f8c1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400cdb0 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c14005020 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c19f5d0 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140116d0 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.894+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1a5c90 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.895+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c103f50 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.896+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8c1400f040 con 0x7f8c1c10d120 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.897+0000 7f8c127fc640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 0x7f8c080b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.897+0000 7f8c2115d640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 0x7f8c080b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:27.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.897+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f8c140e5fb0 con 0x7f8c1c10d120 2024-09-10T22:51:27.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.897+0000 7f8c2115d640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 0x7f8c080b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800bbe0 tx=0x7f8c1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:27.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:27.900+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8c14016030 con 0x7f8c1c10d120 2024-09-10T22:51:28.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.077+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8c1c02e5c0 con 0x7f8c1c10d120 2024-09-10T22:51:28.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.077+0000 7f8c127fc640 1 -- 172.21.15.120:0/2047663687 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8c140acd40 con 0x7f8c1c10d120 2024-09-10T22:51:28.080 INFO:teuthology.orchestra.run.smithi120.stdout:274877906972 2024-09-10T22:51:28.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 msgr2=0x7f8c080b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 0x7f8c080b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800bbe0 tx=0x7f8c1800b390 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 msgr2=0x7f8c1c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400eaa0 tx=0x7f8c1400ef60 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 shutdown_connections 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c080b5370 0x7f8c080b7830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c1c10d120 0x7f8c1c19eb10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c104570 0x7f8c1c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 --2- 172.21.15.120:0/2047663687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c1c103b70 0x7f8c1c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 >> 172.21.15.120:0/2047663687 conn(0x7f8c1c0fd700 msgr2=0x7f8c1c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 shutdown_connections 2024-09-10T22:51:28.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.079+0000 7f8c233c1640 1 -- 172.21.15.120:0/2047663687 wait complete. 2024-09-10T22:51:28.378 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3983066026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:51:28.379 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2047663687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:51:28.379 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:28 smithi120 ceph-mon[31247]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T22:51:28.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266725 got 227633266728 for osd.8 2024-09-10T22:51:28.405 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3983066026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:51:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2047663687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:51:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:28 smithi176 ceph-mon[34982]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T22:51:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3983066026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:51:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2047663687' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:51:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:28 smithi153 ceph-mon[34951]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T22:51:29.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 -- 172.21.15.120:0/3692870195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8050108040 msgr2=0x7f805010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:29.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 --2- 172.21.15.120:0/3692870195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8050108040 0x7f805010e840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8048009ca0 tx=0x7f804802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 -- 172.21.15.120:0/3692870195 shutdown_connections 2024-09-10T22:51:29.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 --2- 172.21.15.120:0/3692870195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8050108040 0x7f805010e840 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 --2- 172.21.15.120:0/3692870195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80501076c0 0x7f8050107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 --2- 172.21.15.120:0/3692870195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 -- 172.21.15.120:0/3692870195 >> 172.21.15.120:0/3692870195 conn(0x7f80500fd700 msgr2=0x7f80500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 -- 172.21.15.120:0/3692870195 shutdown_connections 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.795+0000 7f80558b9640 1 -- 172.21.15.120:0/3692870195 wait complete. 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 Processor -- start 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 -- start start 2024-09-10T22:51:29.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 0x7f805019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8050111d50 con 0x7f80501076c0 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8050111bd0 con 0x7f8050101cb0 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f80558b9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8050111ed0 con 0x7f8050108040 2024-09-10T22:51:29.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35214/0 (socket says 172.21.15.120:35214) 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 -- 172.21.15.120:0/646340366 learned_addr learned my addr 172.21.15.120:0/646340366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f804effd640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f804f7fe640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 0x7f805019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 -- 172.21.15.120:0/646340366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 msgr2=0x7f805019da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 0x7f805019da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 -- 172.21.15.120:0/646340366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 msgr2=0x7f8050198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f8047fff640 1 -- 172.21.15.120:0/646340366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805019e1d0 con 0x7f80501076c0 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f804f7fe640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 0x7f805019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.796+0000 7f804effd640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f8047fff640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f803800b4a0 tx=0x7f803800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038014070 con 0x7f80501076c0 2024-09-10T22:51:29.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805019e4c0 con 0x7f80501076c0 2024-09-10T22:51:29.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8038010ac0 con 0x7f80501076c0 2024-09-10T22:51:29.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038017c30 con 0x7f80501076c0 2024-09-10T22:51:29.091 INFO:teuthology.orchestra.run.smithi120.stdout:253403070500 2024-09-10T22:51:29.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.797+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501a4d50 con 0x7f80501076c0 2024-09-10T22:51:29.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.798+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8050102090 con 0x7f80501076c0 2024-09-10T22:51:29.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.800+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8038010610 con 0x7f80501076c0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.800+0000 7f80548b7640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 0x7f803c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.800+0000 7f804effd640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 0x7f803c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.801+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f80380e56b0 con 0x7f80501076c0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.801+0000 7f804effd640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 0x7f803c0b7830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f804000abe0 tx=0x7f804000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.803+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80380ac440 con 0x7f80501076c0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.984+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8050199250 con 0x7f80501076c0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.984+0000 7f80548b7640 1 -- 172.21.15.120:0/646340366 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f80380abd40 con 0x7f80501076c0 2024-09-10T22:51:29.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 msgr2=0x7f803c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 0x7f803c0b7830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f804000abe0 tx=0x7f804000a5c0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 msgr2=0x7f8050198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f803800b4a0 tx=0x7f803800b960 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 shutdown_connections 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f803c0b5370 0x7f803c0b7830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8050108040 0x7f805019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80501076c0 0x7f8050198660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 --2- 172.21.15.120:0/646340366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8050101cb0 0x7f8050198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 >> 172.21.15.120:0/646340366 conn(0x7f80500fd700 msgr2=0x7f80500ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 shutdown_connections 2024-09-10T22:51:29.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:28.986+0000 7f80558b9640 1 -- 172.21.15.120:0/646340366 wait complete. 2024-09-10T22:51:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:29 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:29 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/646340366' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:51:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:29 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:29 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/646340366' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:51:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:29 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:29 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/646340366' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:51:29.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906972 for osd.10 2024-09-10T22:51:29.724 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:30.171 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070498 got 253403070500 for osd.9 2024-09-10T22:51:30.171 DEBUG:teuthology.parallel:result is None 2024-09-10T22:51:30.171 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:51:30.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:51:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:30 smithi120 ceph-mon[31247]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:30 smithi176 ceph-mon[34982]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:30.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:30 smithi153 ceph-mon[34951]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:30.937 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:31.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 -- 172.21.15.120:0/3253429869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 msgr2=0x7f845810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 --2- 172.21.15.120:0/3253429869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845810f590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ca0 tx=0x7f845002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 -- 172.21.15.120:0/3253429869 shutdown_connections 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 --2- 172.21.15.120:0/3253429869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845810f590 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 --2- 172.21.15.120:0/3253429869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84581045d0 0x7f845810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 --2- 172.21.15.120:0/3253429869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8458103bd0 0x7f8458103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 -- 172.21.15.120:0/3253429869 >> 172.21.15.120:0/3253429869 conn(0x7f84580fd720 msgr2=0x7f84580ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 -- 172.21.15.120:0/3253429869 shutdown_connections 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.428+0000 7f845ca88640 1 -- 172.21.15.120:0/3253429869 wait complete. 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 Processor -- start 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 -- start start 2024-09-10T22:51:31.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 0x7f84581993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84581045d0 0x7f8458199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112e60 con 0x7f845810d180 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112ce0 con 0x7f8458103bd0 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f845ca88640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112fe0 con 0x7f84581045d0 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f8456d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f8456d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35240/0 (socket says 172.21.15.120:35240) 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.429+0000 7f8456d9d640 1 -- 172.21.15.120:0/296081693 learned_addr learned my addr 172.21.15.120:0/296081693 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8455d9b640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84581045d0 0x7f8458199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f845659c640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 0x7f84581993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 -- 172.21.15.120:0/296081693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84581045d0 msgr2=0x7f8458199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:31.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84581045d0 0x7f8458199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 -- 172.21.15.120:0/296081693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 msgr2=0x7f84581993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 0x7f84581993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 -- 172.21.15.120:0/296081693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845819f480 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f845659c640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 0x7f84581993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f8456d9d640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8450031a20 tx=0x7f8450031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450003cf0 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84500044f0 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845819f710 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845002fce0 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.430+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84581a5eb0 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.431+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8458104030 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.432+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8450004040 con 0x7f845810d180 2024-09-10T22:51:31.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.434+0000 7f844f7fe640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 0x7f84440b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:31.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.434+0000 7f845659c640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 0x7f84440b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:31.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.434+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f84501073a0 con 0x7f845810d180 2024-09-10T22:51:31.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.434+0000 7f845659c640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 0x7f84440b7830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8448006fd0 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:31.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.437+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f845003b410 con 0x7f845810d180 2024-09-10T22:51:31.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.629+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8458063590 con 0x7f84440b5370 2024-09-10T22:51:31.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.633+0000 7f844f7fe640 1 -- 172.21.15.120:0/296081693 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258629 (secure 0 0 0) 0x7f8458063590 con 0x7f84440b5370 2024-09-10T22:51:31.637 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 msgr2=0x7f84440b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 0x7f84440b7830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8448006fd0 tx=0x7f8448008040 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 msgr2=0x7f845819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8450031a20 tx=0x7f8450031a50 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 shutdown_connections 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f84440b5370 0x7f84440b7830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f845810d180 0x7f845819ecb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84581045d0 0x7f8458199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 --2- 172.21.15.120:0/296081693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8458103bd0 0x7f84581993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:31.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 >> 172.21.15.120:0/296081693 conn(0x7f84580fd720 msgr2=0x7f84580fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:31.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 shutdown_connections 2024-09-10T22:51:31.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:31.637+0000 7f845ca88640 1 -- 172.21.15.120:0/296081693 wait complete. 2024-09-10T22:51:32.291 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-10T22:51:30.162268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653150,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":195,"num_read_kb":186,"num_write":207,"num_write_kb":2238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377696,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564128,"statfs":{"total":1151940427776,"available":1151553667072,"internally_reserved":0,"allocated":12005376,"data_stored":6815514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156012+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:51:15.156012+0000","last_peered":"2024-09-10T22:51:15.156012+0000","last_clean":"2024-09-10T22:51:15.156012+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:51:15.156012+0000","last_undegraded":"2024-09-10T22:51:15.156012+0000","last_fullsized":"2024-09-10T22:51:15.156012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154505+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:51:15.154505+0000","last_peered":"2024-09-10T22:51:15.154505+0000","last_clean":"2024-09-10T22:51:15.154505+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:51:15.154505+0000","last_undegraded":"2024-09-10T22:51:15.154505+0000","last_fullsized":"2024-09-10T22:51:15.154505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153680+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:51:15.153680+0000","last_peered":"2024-09-10T22:51:15.153680+0000","last_clean":"2024-09-10T22:51:15.153680+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:51:15.153680+0000","last_undegraded":"2024-09-10T22:51:15.153680+0000","last_fullsized":"2024-09-10T22:51:15.153680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153157+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:51:15.153157+0000","last_peered":"2024-09-10T22:51:15.153157+0000","last_clean":"2024-09-10T22:51:15.153157+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:51:15.153157+0000","last_undegraded":"2024-09-10T22:51:15.153157+0000","last_fullsized":"2024-09-10T22:51:15.153157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161803+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:51:15.161803+0000","last_peered":"2024-09-10T22:51:15.161803+0000","last_clean":"2024-09-10T22:51:15.161803+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:51:15.161803+0000","last_undegraded":"2024-09-10T22:51:15.161803+0000","last_fullsized":"2024-09-10T22:51:15.161803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150981+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.150981+0000","last_peered":"2024-09-10T22:51:15.150981+0000","last_clean":"2024-09-10T22:51:15.150981+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:51:15.150981+0000","last_undegraded":"2024-09-10T22:51:15.150981+0000","last_fullsized":"2024-09-10T22:51:15.150981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154759+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:51:15.154759+0000","last_peered":"2024-09-10T22:51:15.154759+0000","last_clean":"2024-09-10T22:51:15.154759+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:51:15.154759+0000","last_undegraded":"2024-09-10T22:51:15.154759+0000","last_fullsized":"2024-09-10T22:51:15.154759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294963+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:51:15.294963+0000","last_peered":"2024-09-10T22:51:15.294963+0000","last_clean":"2024-09-10T22:51:15.294963+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:51:15.294963+0000","last_undegraded":"2024-09-10T22:51:15.294963+0000","last_fullsized":"2024-09-10T22:51:15.294963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296097+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:51:15.296097+0000","last_peered":"2024-09-10T22:51:15.296097+0000","last_clean":"2024-09-10T22:51:15.296097+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:51:15.296097+0000","last_undegraded":"2024-09-10T22:51:15.296097+0000","last_fullsized":"2024-09-10T22:51:15.296097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151443+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:51:15.151443+0000","last_peered":"2024-09-10T22:51:15.151443+0000","last_clean":"2024-09-10T22:51:15.151443+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:51:15.151443+0000","last_undegraded":"2024-09-10T22:51:15.151443+0000","last_fullsized":"2024-09-10T22:51:15.151443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155194+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:51:15.155194+0000","last_peered":"2024-09-10T22:51:15.155194+0000","last_clean":"2024-09-10T22:51:15.155194+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:51:15.155194+0000","last_undegraded":"2024-09-10T22:51:15.155194+0000","last_fullsized":"2024-09-10T22:51:15.155194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153263+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:51:15.153263+0000","last_peered":"2024-09-10T22:51:15.153263+0000","last_clean":"2024-09-10T22:51:15.153263+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:51:15.153263+0000","last_undegraded":"2024-09-10T22:51:15.153263+0000","last_fullsized":"2024-09-10T22:51:15.153263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154928+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:51:15.154928+0000","last_peered":"2024-09-10T22:51:15.154928+0000","last_clean":"2024-09-10T22:51:15.154928+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:51:15.154928+0000","last_undegraded":"2024-09-10T22:51:15.154928+0000","last_fullsized":"2024-09-10T22:51:15.154928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153640+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:51:15.153640+0000","last_peered":"2024-09-10T22:51:15.153640+0000","last_clean":"2024-09-10T22:51:15.153640+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:51:15.153640+0000","last_undegraded":"2024-09-10T22:51:15.153640+0000","last_fullsized":"2024-09-10T22:51:15.153640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153570+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:51:15.153570+0000","last_peered":"2024-09-10T22:51:15.153570+0000","last_clean":"2024-09-10T22:51:15.153570+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:51:15.153570+0000","last_undegraded":"2024-09-10T22:51:15.153570+0000","last_fullsized":"2024-09-10T22:51:15.153570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295653+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:51:15.295653+0000","last_peered":"2024-09-10T22:51:15.295653+0000","last_clean":"2024-09-10T22:51:15.295653+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:51:15.295653+0000","last_undegraded":"2024-09-10T22:51:15.295653+0000","last_fullsized":"2024-09-10T22:51:15.295653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156076+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:51:15.156076+0000","last_peered":"2024-09-10T22:51:15.156076+0000","last_clean":"2024-09-10T22:51:15.156076+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:51:15.156076+0000","last_undegraded":"2024-09-10T22:51:15.156076+0000","last_fullsized":"2024-09-10T22:51:15.156076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150803+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:51:15.150803+0000","last_peered":"2024-09-10T22:51:15.150803+0000","last_clean":"2024-09-10T22:51:15.150803+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:51:15.150803+0000","last_undegraded":"2024-09-10T22:51:15.150803+0000","last_fullsized":"2024-09-10T22:51:15.150803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162560+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:51:15.162560+0000","last_peered":"2024-09-10T22:51:15.162560+0000","last_clean":"2024-09-10T22:51:15.162560+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:51:15.162560+0000","last_undegraded":"2024-09-10T22:51:15.162560+0000","last_fullsized":"2024-09-10T22:51:15.162560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155723+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:51:15.155723+0000","last_peered":"2024-09-10T22:51:15.155723+0000","last_clean":"2024-09-10T22:51:15.155723+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:51:15.155723+0000","last_undegraded":"2024-09-10T22:51:15.155723+0000","last_fullsized":"2024-09-10T22:51:15.155723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155039+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:51:15.155039+0000","last_peered":"2024-09-10T22:51:15.155039+0000","last_clean":"2024-09-10T22:51:15.155039+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:51:15.155039+0000","last_undegraded":"2024-09-10T22:51:15.155039+0000","last_fullsized":"2024-09-10T22:51:15.155039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295743+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:51:15.295743+0000","last_peered":"2024-09-10T22:51:15.295743+0000","last_clean":"2024-09-10T22:51:15.295743+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:51:15.295743+0000","last_undegraded":"2024-09-10T22:51:15.295743+0000","last_fullsized":"2024-09-10T22:51:15.295743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294977+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:51:15.294977+0000","last_peered":"2024-09-10T22:51:15.294977+0000","last_clean":"2024-09-10T22:51:15.294977+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:51:15.294977+0000","last_undegraded":"2024-09-10T22:51:15.294977+0000","last_fullsized":"2024-09-10T22:51:15.294977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295461+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:51:15.295461+0000","last_peered":"2024-09-10T22:51:15.295461+0000","last_clean":"2024-09-10T22:51:15.295461+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:51:15.295461+0000","last_undegraded":"2024-09-10T22:51:15.295461+0000","last_fullsized":"2024-09-10T22:51:15.295461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295171+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:51:15.295171+0000","last_peered":"2024-09-10T22:51:15.295171+0000","last_clean":"2024-09-10T22:51:15.295171+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:51:15.295171+0000","last_undegraded":"2024-09-10T22:51:15.295171+0000","last_fullsized":"2024-09-10T22:51:15.295171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162515+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:51:15.162515+0000","last_peered":"2024-09-10T22:51:15.162515+0000","last_clean":"2024-09-10T22:51:15.162515+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:51:15.162515+0000","last_undegraded":"2024-09-10T22:51:15.162515+0000","last_fullsized":"2024-09-10T22:51:15.162515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156495+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:51:15.156495+0000","last_peered":"2024-09-10T22:51:15.156495+0000","last_clean":"2024-09-10T22:51:15.156495+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:51:15.156495+0000","last_undegraded":"2024-09-10T22:51:15.156495+0000","last_fullsized":"2024-09-10T22:51:15.156495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154828+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:51:15.154828+0000","last_peered":"2024-09-10T22:51:15.154828+0000","last_clean":"2024-09-10T22:51:15.154828+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:51:15.154828+0000","last_undegraded":"2024-09-10T22:51:15.154828+0000","last_fullsized":"2024-09-10T22:51:15.154828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154404+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:51:15.154404+0000","last_peered":"2024-09-10T22:51:15.154404+0000","last_clean":"2024-09-10T22:51:15.154404+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:51:15.154404+0000","last_undegraded":"2024-09-10T22:51:15.154404+0000","last_fullsized":"2024-09-10T22:51:15.154404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161965+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:51:15.161965+0000","last_peered":"2024-09-10T22:51:15.161965+0000","last_clean":"2024-09-10T22:51:15.161965+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:51:15.161965+0000","last_undegraded":"2024-09-10T22:51:15.161965+0000","last_fullsized":"2024-09-10T22:51:15.161965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162068+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:51:15.162068+0000","last_peered":"2024-09-10T22:51:15.162068+0000","last_clean":"2024-09-10T22:51:15.162068+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:51:15.162068+0000","last_undegraded":"2024-09-10T22:51:15.162068+0000","last_fullsized":"2024-09-10T22:51:15.162068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155610+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:51:15.155610+0000","last_peered":"2024-09-10T22:51:15.155610+0000","last_clean":"2024-09-10T22:51:15.155610+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:51:15.155610+0000","last_undegraded":"2024-09-10T22:51:15.155610+0000","last_fullsized":"2024-09-10T22:51:15.155610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161948+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:51:15.161948+0000","last_peered":"2024-09-10T22:51:15.161948+0000","last_clean":"2024-09-10T22:51:15.161948+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:51:15.161948+0000","last_undegraded":"2024-09-10T22:51:15.161948+0000","last_fullsized":"2024-09-10T22:51:15.161948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295741+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:51:15.295741+0000","last_peered":"2024-09-10T22:51:15.295741+0000","last_clean":"2024-09-10T22:51:15.295741+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:51:15.295741+0000","last_undegraded":"2024-09-10T22:51:15.295741+0000","last_fullsized":"2024-09-10T22:51:15.295741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296700+0000","last_change":"2024-09-10T22:51:15.296217+0000","last_active":"2024-09-10T22:51:15.296700+0000","last_peered":"2024-09-10T22:51:15.296700+0000","last_clean":"2024-09-10T22:51:15.296700+0000","last_became_active":"2024-09-10T22:51:15.295842+0000","last_became_peered":"2024-09-10T22:51:15.295842+0000","last_unstale":"2024-09-10T22:51:15.296700+0000","last_undegraded":"2024-09-10T22:51:15.296700+0000","last_fullsized":"2024-09-10T22:51:15.296700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:32.464281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151051+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:51:15.151051+0000","last_peered":"2024-09-10T22:51:15.151051+0000","last_clean":"2024-09-10T22:51:15.151051+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:51:15.151051+0000","last_undegraded":"2024-09-10T22:51:15.151051+0000","last_fullsized":"2024-09-10T22:51:15.151051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153986+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:51:15.153986+0000","last_peered":"2024-09-10T22:51:15.153986+0000","last_clean":"2024-09-10T22:51:15.153986+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:51:15.153986+0000","last_undegraded":"2024-09-10T22:51:15.153986+0000","last_fullsized":"2024-09-10T22:51:15.153986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295017+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:51:15.295017+0000","last_peered":"2024-09-10T22:51:15.295017+0000","last_clean":"2024-09-10T22:51:15.295017+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:51:15.295017+0000","last_undegraded":"2024-09-10T22:51:15.295017+0000","last_fullsized":"2024-09-10T22:51:15.295017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295730+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:51:15.295730+0000","last_peered":"2024-09-10T22:51:15.295730+0000","last_clean":"2024-09-10T22:51:15.295730+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:51:15.295730+0000","last_undegraded":"2024-09-10T22:51:15.295730+0000","last_fullsized":"2024-09-10T22:51:15.295730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150604+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:51:15.150604+0000","last_peered":"2024-09-10T22:51:15.150604+0000","last_clean":"2024-09-10T22:51:15.150604+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:51:15.150604+0000","last_undegraded":"2024-09-10T22:51:15.150604+0000","last_fullsized":"2024-09-10T22:51:15.150604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151268+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:51:15.151268+0000","last_peered":"2024-09-10T22:51:15.151268+0000","last_clean":"2024-09-10T22:51:15.151268+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:51:15.151268+0000","last_undegraded":"2024-09-10T22:51:15.151268+0000","last_fullsized":"2024-09-10T22:51:15.151268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295974+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:51:15.295974+0000","last_peered":"2024-09-10T22:51:15.295974+0000","last_clean":"2024-09-10T22:51:15.295974+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:51:15.295974+0000","last_undegraded":"2024-09-10T22:51:15.295974+0000","last_fullsized":"2024-09-10T22:51:15.295974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155301+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:51:15.155301+0000","last_peered":"2024-09-10T22:51:15.155301+0000","last_clean":"2024-09-10T22:51:15.155301+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:51:15.155301+0000","last_undegraded":"2024-09-10T22:51:15.155301+0000","last_fullsized":"2024-09-10T22:51:15.155301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296977+0000","last_change":"2024-09-10T22:51:15.296922+0000","last_active":"2024-09-10T22:51:15.296977+0000","last_peered":"2024-09-10T22:51:15.296977+0000","last_clean":"2024-09-10T22:51:15.296977+0000","last_became_active":"2024-09-10T22:51:15.296658+0000","last_became_peered":"2024-09-10T22:51:15.296658+0000","last_unstale":"2024-09-10T22:51:15.296977+0000","last_undegraded":"2024-09-10T22:51:15.296977+0000","last_fullsized":"2024-09-10T22:51:15.296977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:18.932940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295896+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:51:15.295896+0000","last_peered":"2024-09-10T22:51:15.295896+0000","last_clean":"2024-09-10T22:51:15.295896+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:51:15.295896+0000","last_undegraded":"2024-09-10T22:51:15.295896+0000","last_fullsized":"2024-09-10T22:51:15.295896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295846+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:51:15.295846+0000","last_peered":"2024-09-10T22:51:15.295846+0000","last_clean":"2024-09-10T22:51:15.295846+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:51:15.295846+0000","last_undegraded":"2024-09-10T22:51:15.295846+0000","last_fullsized":"2024-09-10T22:51:15.295846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":163,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:26.132188+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:51:26.132188+0000","last_peered":"2024-09-10T22:51:26.132188+0000","last_clean":"2024-09-10T22:51:26.132188+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:51:26.132188+0000","last_undegraded":"2024-09-10T22:51:26.132188+0000","last_fullsized":"2024-09-10T22:51:26.132188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":116,"num_write":27,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162574+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:51:15.162574+0000","last_peered":"2024-09-10T22:51:15.162574+0000","last_clean":"2024-09-10T22:51:15.162574+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:51:15.162574+0000","last_undegraded":"2024-09-10T22:51:15.162574+0000","last_fullsized":"2024-09-10T22:51:15.162574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162287+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:51:15.162287+0000","last_peered":"2024-09-10T22:51:15.162287+0000","last_clean":"2024-09-10T22:51:15.162287+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:51:15.162287+0000","last_undegraded":"2024-09-10T22:51:15.162287+0000","last_fullsized":"2024-09-10T22:51:15.162287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154237+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:51:15.154237+0000","last_peered":"2024-09-10T22:51:15.154237+0000","last_clean":"2024-09-10T22:51:15.154237+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:51:15.154237+0000","last_undegraded":"2024-09-10T22:51:15.154237+0000","last_fullsized":"2024-09-10T22:51:15.154237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295672+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:51:15.295672+0000","last_peered":"2024-09-10T22:51:15.295672+0000","last_clean":"2024-09-10T22:51:15.295672+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:51:15.295672+0000","last_undegraded":"2024-09-10T22:51:15.295672+0000","last_fullsized":"2024-09-10T22:51:15.295672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156806+0000","last_change":"2024-09-10T22:51:15.156697+0000","last_active":"2024-09-10T22:51:15.156806+0000","last_peered":"2024-09-10T22:51:15.156806+0000","last_clean":"2024-09-10T22:51:15.156806+0000","last_became_active":"2024-09-10T22:51:15.156396+0000","last_became_peered":"2024-09-10T22:51:15.156396+0000","last_unstale":"2024-09-10T22:51:15.156806+0000","last_undegraded":"2024-09-10T22:51:15.156806+0000","last_fullsized":"2024-09-10T22:51:15.156806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:26.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.152948+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:51:15.152948+0000","last_peered":"2024-09-10T22:51:15.152948+0000","last_clean":"2024-09-10T22:51:15.152948+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:51:15.152948+0000","last_undegraded":"2024-09-10T22:51:15.152948+0000","last_fullsized":"2024-09-10T22:51:15.152948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154683+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:51:15.154683+0000","last_peered":"2024-09-10T22:51:15.154683+0000","last_clean":"2024-09-10T22:51:15.154683+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:51:15.154683+0000","last_undegraded":"2024-09-10T22:51:15.154683+0000","last_fullsized":"2024-09-10T22:51:15.154683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162012+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:51:15.162012+0000","last_peered":"2024-09-10T22:51:15.162012+0000","last_clean":"2024-09-10T22:51:15.162012+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:51:15.162012+0000","last_undegraded":"2024-09-10T22:51:15.162012+0000","last_fullsized":"2024-09-10T22:51:15.162012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295528+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:51:15.295528+0000","last_peered":"2024-09-10T22:51:15.295528+0000","last_clean":"2024-09-10T22:51:15.295528+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:51:15.295528+0000","last_undegraded":"2024-09-10T22:51:15.295528+0000","last_fullsized":"2024-09-10T22:51:15.295528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162997+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:51:15.162997+0000","last_peered":"2024-09-10T22:51:15.162997+0000","last_clean":"2024-09-10T22:51:15.162997+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:51:15.162997+0000","last_undegraded":"2024-09-10T22:51:15.162997+0000","last_fullsized":"2024-09-10T22:51:15.162997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155841+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:51:15.155841+0000","last_peered":"2024-09-10T22:51:15.155841+0000","last_clean":"2024-09-10T22:51:15.155841+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:51:15.155841+0000","last_undegraded":"2024-09-10T22:51:15.155841+0000","last_fullsized":"2024-09-10T22:51:15.155841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154030+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:51:15.154030+0000","last_peered":"2024-09-10T22:51:15.154030+0000","last_clean":"2024-09-10T22:51:15.154030+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:51:15.154030+0000","last_undegraded":"2024-09-10T22:51:15.154030+0000","last_fullsized":"2024-09-10T22:51:15.154030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155400+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:51:15.155400+0000","last_peered":"2024-09-10T22:51:15.155400+0000","last_clean":"2024-09-10T22:51:15.155400+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:51:15.155400+0000","last_undegraded":"2024-09-10T22:51:15.155400+0000","last_fullsized":"2024-09-10T22:51:15.155400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296015+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:51:15.296015+0000","last_peered":"2024-09-10T22:51:15.296015+0000","last_clean":"2024-09-10T22:51:15.296015+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:51:15.296015+0000","last_undegraded":"2024-09-10T22:51:15.296015+0000","last_fullsized":"2024-09-10T22:51:15.296015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155019+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:51:15.155019+0000","last_peered":"2024-09-10T22:51:15.155019+0000","last_clean":"2024-09-10T22:51:15.155019+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:51:15.155019+0000","last_undegraded":"2024-09-10T22:51:15.155019+0000","last_fullsized":"2024-09-10T22:51:15.155019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.158692+0000","last_change":"2024-09-10T22:51:15.158070+0000","last_active":"2024-09-10T22:51:15.158692+0000","last_peered":"2024-09-10T22:51:15.158692+0000","last_clean":"2024-09-10T22:51:15.158692+0000","last_became_active":"2024-09-10T22:51:15.157711+0000","last_became_peered":"2024-09-10T22:51:15.157711+0000","last_unstale":"2024-09-10T22:51:15.158692+0000","last_undegraded":"2024-09-10T22:51:15.158692+0000","last_fullsized":"2024-09-10T22:51:15.158692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:11.308860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155239+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.155239+0000","last_peered":"2024-09-10T22:51:15.155239+0000","last_clean":"2024-09-10T22:51:15.155239+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:51:15.155239+0000","last_undegraded":"2024-09-10T22:51:15.155239+0000","last_fullsized":"2024-09-10T22:51:15.155239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154834+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:51:15.154834+0000","last_peered":"2024-09-10T22:51:15.154834+0000","last_clean":"2024-09-10T22:51:15.154834+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:51:15.154834+0000","last_undegraded":"2024-09-10T22:51:15.154834+0000","last_fullsized":"2024-09-10T22:51:15.154834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153104+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:51:15.153104+0000","last_peered":"2024-09-10T22:51:15.153104+0000","last_clean":"2024-09-10T22:51:15.153104+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:51:15.153104+0000","last_undegraded":"2024-09-10T22:51:15.153104+0000","last_fullsized":"2024-09-10T22:51:15.153104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295351+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:51:15.295351+0000","last_peered":"2024-09-10T22:51:15.295351+0000","last_clean":"2024-09-10T22:51:15.295351+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:51:15.295351+0000","last_undegraded":"2024-09-10T22:51:15.295351+0000","last_fullsized":"2024-09-10T22:51:15.295351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154832+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:51:15.154832+0000","last_peered":"2024-09-10T22:51:15.154832+0000","last_clean":"2024-09-10T22:51:15.154832+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:51:15.154832+0000","last_undegraded":"2024-09-10T22:51:15.154832+0000","last_fullsized":"2024-09-10T22:51:15.154832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.165878+0000","last_change":"2024-09-10T22:51:15.165814+0000","last_active":"2024-09-10T22:51:15.165878+0000","last_peered":"2024-09-10T22:51:15.165878+0000","last_clean":"2024-09-10T22:51:15.165878+0000","last_became_active":"2024-09-10T22:51:15.165305+0000","last_became_peered":"2024-09-10T22:51:15.165305+0000","last_unstale":"2024-09-10T22:51:15.165878+0000","last_undegraded":"2024-09-10T22:51:15.165878+0000","last_fullsized":"2024-09-10T22:51:15.165878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:14.680823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154077+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:51:15.154077+0000","last_peered":"2024-09-10T22:51:15.154077+0000","last_clean":"2024-09-10T22:51:15.154077+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:51:15.154077+0000","last_undegraded":"2024-09-10T22:51:15.154077+0000","last_fullsized":"2024-09-10T22:51:15.154077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295816+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:51:15.295816+0000","last_peered":"2024-09-10T22:51:15.295816+0000","last_clean":"2024-09-10T22:51:15.295816+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:51:15.295816+0000","last_undegraded":"2024-09-10T22:51:15.295816+0000","last_fullsized":"2024-09-10T22:51:15.295816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155014+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:51:15.155014+0000","last_peered":"2024-09-10T22:51:15.155014+0000","last_clean":"2024-09-10T22:51:15.155014+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:51:15.155014+0000","last_undegraded":"2024-09-10T22:51:15.155014+0000","last_fullsized":"2024-09-10T22:51:15.155014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151167+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:51:15.151167+0000","last_peered":"2024-09-10T22:51:15.151167+0000","last_clean":"2024-09-10T22:51:15.151167+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:51:15.151167+0000","last_undegraded":"2024-09-10T22:51:15.151167+0000","last_fullsized":"2024-09-10T22:51:15.151167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150766+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:51:15.150766+0000","last_peered":"2024-09-10T22:51:15.150766+0000","last_clean":"2024-09-10T22:51:15.150766+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:51:15.150766+0000","last_undegraded":"2024-09-10T22:51:15.150766+0000","last_fullsized":"2024-09-10T22:51:15.150766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151181+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:51:15.151181+0000","last_peered":"2024-09-10T22:51:15.151181+0000","last_clean":"2024-09-10T22:51:15.151181+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:51:15.151181+0000","last_undegraded":"2024-09-10T22:51:15.151181+0000","last_fullsized":"2024-09-10T22:51:15.151181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162786+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:51:15.162786+0000","last_peered":"2024-09-10T22:51:15.162786+0000","last_clean":"2024-09-10T22:51:15.162786+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:51:15.162786+0000","last_undegraded":"2024-09-10T22:51:15.162786+0000","last_fullsized":"2024-09-10T22:51:15.162786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154876+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:51:15.154876+0000","last_peered":"2024-09-10T22:51:15.154876+0000","last_clean":"2024-09-10T22:51:15.154876+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:51:15.154876+0000","last_undegraded":"2024-09-10T22:51:15.154876+0000","last_fullsized":"2024-09-10T22:51:15.154876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150866+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:51:15.150866+0000","last_peered":"2024-09-10T22:51:15.150866+0000","last_clean":"2024-09-10T22:51:15.150866+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:51:15.150866+0000","last_undegraded":"2024-09-10T22:51:15.150866+0000","last_fullsized":"2024-09-10T22:51:15.150866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295230+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:51:15.295230+0000","last_peered":"2024-09-10T22:51:15.295230+0000","last_clean":"2024-09-10T22:51:15.295230+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:51:15.295230+0000","last_undegraded":"2024-09-10T22:51:15.295230+0000","last_fullsized":"2024-09-10T22:51:15.295230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153127+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:51:15.153127+0000","last_peered":"2024-09-10T22:51:15.153127+0000","last_clean":"2024-09-10T22:51:15.153127+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:51:15.153127+0000","last_undegraded":"2024-09-10T22:51:15.153127+0000","last_fullsized":"2024-09-10T22:51:15.153127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154511+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:51:15.154511+0000","last_peered":"2024-09-10T22:51:15.154511+0000","last_clean":"2024-09-10T22:51:15.154511+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:51:15.154511+0000","last_undegraded":"2024-09-10T22:51:15.154511+0000","last_fullsized":"2024-09-10T22:51:15.154511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295736+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:51:15.295736+0000","last_peered":"2024-09-10T22:51:15.295736+0000","last_clean":"2024-09-10T22:51:15.295736+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:51:15.295736+0000","last_undegraded":"2024-09-10T22:51:15.295736+0000","last_fullsized":"2024-09-10T22:51:15.295736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161616+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:51:15.161616+0000","last_peered":"2024-09-10T22:51:15.161616+0000","last_clean":"2024-09-10T22:51:15.161616+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:51:15.161616+0000","last_undegraded":"2024-09-10T22:51:15.161616+0000","last_fullsized":"2024-09-10T22:51:15.161616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295097+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:51:15.295097+0000","last_peered":"2024-09-10T22:51:15.295097+0000","last_clean":"2024-09-10T22:51:15.295097+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:51:15.295097+0000","last_undegraded":"2024-09-10T22:51:15.295097+0000","last_fullsized":"2024-09-10T22:51:15.295097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155731+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:51:15.155731+0000","last_peered":"2024-09-10T22:51:15.155731+0000","last_clean":"2024-09-10T22:51:15.155731+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:51:15.155731+0000","last_undegraded":"2024-09-10T22:51:15.155731+0000","last_fullsized":"2024-09-10T22:51:15.155731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.297065+0000","last_change":"2024-09-10T22:51:15.296938+0000","last_active":"2024-09-10T22:51:15.297065+0000","last_peered":"2024-09-10T22:51:15.297065+0000","last_clean":"2024-09-10T22:51:15.297065+0000","last_became_active":"2024-09-10T22:51:15.296291+0000","last_became_peered":"2024-09-10T22:51:15.296291+0000","last_unstale":"2024-09-10T22:51:15.297065+0000","last_undegraded":"2024-09-10T22:51:15.297065+0000","last_fullsized":"2024-09-10T22:51:15.297065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:26.752740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295243+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:51:15.295243+0000","last_peered":"2024-09-10T22:51:15.295243+0000","last_clean":"2024-09-10T22:51:15.295243+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:51:15.295243+0000","last_undegraded":"2024-09-10T22:51:15.295243+0000","last_fullsized":"2024-09-10T22:51:15.295243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161953+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:51:15.161953+0000","last_peered":"2024-09-10T22:51:15.161953+0000","last_clean":"2024-09-10T22:51:15.161953+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:51:15.161953+0000","last_undegraded":"2024-09-10T22:51:15.161953+0000","last_fullsized":"2024-09-10T22:51:15.161953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154833+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:51:15.154833+0000","last_peered":"2024-09-10T22:51:15.154833+0000","last_clean":"2024-09-10T22:51:15.154833+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:51:15.154833+0000","last_undegraded":"2024-09-10T22:51:15.154833+0000","last_fullsized":"2024-09-10T22:51:15.154833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295992+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:51:15.295992+0000","last_peered":"2024-09-10T22:51:15.295992+0000","last_clean":"2024-09-10T22:51:15.295992+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:51:15.295992+0000","last_undegraded":"2024-09-10T22:51:15.295992+0000","last_fullsized":"2024-09-10T22:51:15.295992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162544+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:51:15.162544+0000","last_peered":"2024-09-10T22:51:15.162544+0000","last_clean":"2024-09-10T22:51:15.162544+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:51:15.162544+0000","last_undegraded":"2024-09-10T22:51:15.162544+0000","last_fullsized":"2024-09-10T22:51:15.162544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295742+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:51:15.295742+0000","last_peered":"2024-09-10T22:51:15.295742+0000","last_clean":"2024-09-10T22:51:15.295742+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:51:15.295742+0000","last_undegraded":"2024-09-10T22:51:15.295742+0000","last_fullsized":"2024-09-10T22:51:15.295742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.173507+0000","last_change":"2024-09-10T22:51:15.173004+0000","last_active":"2024-09-10T22:51:15.173507+0000","last_peered":"2024-09-10T22:51:15.173507+0000","last_clean":"2024-09-10T22:51:15.173507+0000","last_became_active":"2024-09-10T22:51:15.165748+0000","last_became_peered":"2024-09-10T22:51:15.165748+0000","last_unstale":"2024-09-10T22:51:15.173507+0000","last_undegraded":"2024-09-10T22:51:15.173507+0000","last_fullsized":"2024-09-10T22:51:15.173507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:38.001944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153491+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:51:15.153491+0000","last_peered":"2024-09-10T22:51:15.153491+0000","last_clean":"2024-09-10T22:51:15.153491+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:51:15.153491+0000","last_undegraded":"2024-09-10T22:51:15.153491+0000","last_fullsized":"2024-09-10T22:51:15.153491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295425+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:51:15.295425+0000","last_peered":"2024-09-10T22:51:15.295425+0000","last_clean":"2024-09-10T22:51:15.295425+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:51:15.295425+0000","last_undegraded":"2024-09-10T22:51:15.295425+0000","last_fullsized":"2024-09-10T22:51:15.295425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295318+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:51:15.295318+0000","last_peered":"2024-09-10T22:51:15.295318+0000","last_clean":"2024-09-10T22:51:15.295318+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:51:15.295318+0000","last_undegraded":"2024-09-10T22:51:15.295318+0000","last_fullsized":"2024-09-10T22:51:15.295318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150890+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:51:15.150890+0000","last_peered":"2024-09-10T22:51:15.150890+0000","last_clean":"2024-09-10T22:51:15.150890+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:51:15.150890+0000","last_undegraded":"2024-09-10T22:51:15.150890+0000","last_fullsized":"2024-09-10T22:51:15.150890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62548,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":119,"num_read_kb":122,"num_write":82,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":187644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1507328,"data_stored":1057101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":831488,"data_stored":405254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":405276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495661,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1429504,"data_stored":995739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855415,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1437696,"data_stored":995339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":404694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444103,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":466456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":897024,"data_stored":466422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:51:32.297 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:51:32.568 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:32 smithi176 ceph-mon[34982]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:32 smithi176 ceph-mon[34982]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:32.689 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:32 smithi120 ceph-mon[31247]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:32.689 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:32 smithi120 ceph-mon[31247]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:32.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:32 smithi153 ceph-mon[34951]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:32.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:32 smithi153 ceph-mon[34951]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:33.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 -- 172.21.15.120:0/76320528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b24103b50 msgr2=0x7f9b24103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 --2- 172.21.15.120:0/76320528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b24103b50 0x7f9b24103f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009900 tx=0x7f9b1402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:33.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 -- 172.21.15.120:0/76320528 shutdown_connections 2024-09-10T22:51:33.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 --2- 172.21.15.120:0/76320528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b2410d100 0x7f9b2410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 --2- 172.21.15.120:0/76320528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24104550 0x7f9b2410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 --2- 172.21.15.120:0/76320528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b24103b50 0x7f9b24103f30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 -- 172.21.15.120:0/76320528 >> 172.21.15.120:0/76320528 conn(0x7f9b240fd700 msgr2=0x7f9b240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 -- 172.21.15.120:0/76320528 shutdown_connections 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.056+0000 7f9b28faa640 1 -- 172.21.15.120:0/76320528 wait complete. 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 Processor -- start 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 -- start start 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24103b50 0x7f9b24199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b24104550 0x7f9b24199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24112d60 con 0x7f9b2410d100 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24112be0 con 0x7f9b24103b50 2024-09-10T22:51:33.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b28faa640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24112ee0 con 0x7f9b24104550 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b22d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b22d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35246/0 (socket says 172.21.15.120:35246) 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b22d9d640 1 -- 172.21.15.120:0/1274492537 learned_addr learned my addr 172.21.15.120:0/1274492537 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b21d9b640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b24104550 0x7f9b24199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.057+0000 7f9b2259c640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24103b50 0x7f9b24199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 -- 172.21.15.120:0/1274492537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b24104550 msgr2=0x7f9b24199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b24104550 0x7f9b24199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 -- 172.21.15.120:0/1274492537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24103b50 msgr2=0x7f9b24199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24103b50 0x7f9b24199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 -- 172.21.15.120:0/1274492537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2419f2c0 con 0x7f9b2410d100 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b22d9d640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00cce0 tx=0x7f9b1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00f450 con 0x7f9b2410d100 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1c00faa0 con 0x7f9b2410d100 2024-09-10T22:51:33.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2419f5b0 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c018b70 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.058+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a5c70 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.059+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b24103f30 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.061+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f9b1c00f5f0 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.061+0000 7f9b137fe640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 0x7f9b0c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.062+0000 7f9b2259c640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 0x7f9b0c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.062+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f9b1c0e6410 con 0x7f9b2410d100 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.062+0000 7f9b2259c640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 0x7f9b0c0b78b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9b14031440 tx=0x7f9b14005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.065+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9b1c014030 con 0x7f9b2410d100 2024-09-10T22:51:33.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.236+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b240ffa20 con 0x7f9b0c0b53f0 2024-09-10T22:51:33.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.240+0000 7f9b137fe640 1 -- 172.21.15.120:0/1274492537 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258629 (secure 0 0 0) 0x7f9b240ffa20 con 0x7f9b0c0b53f0 2024-09-10T22:51:33.244 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:33.245 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:51:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 msgr2=0x7f9b0c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 0x7f9b0c0b78b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9b14031440 tx=0x7f9b14005e80 comp rx=0 tx=0).stop 2024-09-10T22:51:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 msgr2=0x7f9b2419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00cce0 tx=0x7f9b1c00a620 comp rx=0 tx=0).stop 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 shutdown_connections 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9b0c0b53f0 0x7f9b0c0b78b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b2410d100 0x7f9b2419eaf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b24104550 0x7f9b24199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 --2- 172.21.15.120:0/1274492537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b24103b50 0x7f9b24199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.243+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 >> 172.21.15.120:0/1274492537 conn(0x7f9b240fd700 msgr2=0x7f9b240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.244+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 shutdown_connections 2024-09-10T22:51:33.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:33.244+0000 7f9b28faa640 1 -- 172.21.15.120:0/1274492537 wait complete. 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:33.548 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:33.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:33.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.869 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.869 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:33.869 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:33.869 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:51:33.884 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T22:51:32.162812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653150,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":205,"num_read_kb":191,"num_write":207,"num_write_kb":2238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377696,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564128,"statfs":{"total":1151940427776,"available":1151553667072,"internally_reserved":0,"allocated":12005376,"data_stored":6815514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003219"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156012+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:51:15.156012+0000","last_peered":"2024-09-10T22:51:15.156012+0000","last_clean":"2024-09-10T22:51:15.156012+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:51:15.156012+0000","last_undegraded":"2024-09-10T22:51:15.156012+0000","last_fullsized":"2024-09-10T22:51:15.156012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154505+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:51:15.154505+0000","last_peered":"2024-09-10T22:51:15.154505+0000","last_clean":"2024-09-10T22:51:15.154505+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:51:15.154505+0000","last_undegraded":"2024-09-10T22:51:15.154505+0000","last_fullsized":"2024-09-10T22:51:15.154505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153680+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:51:15.153680+0000","last_peered":"2024-09-10T22:51:15.153680+0000","last_clean":"2024-09-10T22:51:15.153680+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:51:15.153680+0000","last_undegraded":"2024-09-10T22:51:15.153680+0000","last_fullsized":"2024-09-10T22:51:15.153680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153157+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:51:15.153157+0000","last_peered":"2024-09-10T22:51:15.153157+0000","last_clean":"2024-09-10T22:51:15.153157+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:51:15.153157+0000","last_undegraded":"2024-09-10T22:51:15.153157+0000","last_fullsized":"2024-09-10T22:51:15.153157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161803+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:51:15.161803+0000","last_peered":"2024-09-10T22:51:15.161803+0000","last_clean":"2024-09-10T22:51:15.161803+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:51:15.161803+0000","last_undegraded":"2024-09-10T22:51:15.161803+0000","last_fullsized":"2024-09-10T22:51:15.161803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150981+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.150981+0000","last_peered":"2024-09-10T22:51:15.150981+0000","last_clean":"2024-09-10T22:51:15.150981+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:51:15.150981+0000","last_undegraded":"2024-09-10T22:51:15.150981+0000","last_fullsized":"2024-09-10T22:51:15.150981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154759+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:51:15.154759+0000","last_peered":"2024-09-10T22:51:15.154759+0000","last_clean":"2024-09-10T22:51:15.154759+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:51:15.154759+0000","last_undegraded":"2024-09-10T22:51:15.154759+0000","last_fullsized":"2024-09-10T22:51:15.154759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294963+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:51:15.294963+0000","last_peered":"2024-09-10T22:51:15.294963+0000","last_clean":"2024-09-10T22:51:15.294963+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:51:15.294963+0000","last_undegraded":"2024-09-10T22:51:15.294963+0000","last_fullsized":"2024-09-10T22:51:15.294963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296097+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:51:15.296097+0000","last_peered":"2024-09-10T22:51:15.296097+0000","last_clean":"2024-09-10T22:51:15.296097+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:51:15.296097+0000","last_undegraded":"2024-09-10T22:51:15.296097+0000","last_fullsized":"2024-09-10T22:51:15.296097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151443+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:51:15.151443+0000","last_peered":"2024-09-10T22:51:15.151443+0000","last_clean":"2024-09-10T22:51:15.151443+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:51:15.151443+0000","last_undegraded":"2024-09-10T22:51:15.151443+0000","last_fullsized":"2024-09-10T22:51:15.151443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155194+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:51:15.155194+0000","last_peered":"2024-09-10T22:51:15.155194+0000","last_clean":"2024-09-10T22:51:15.155194+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:51:15.155194+0000","last_undegraded":"2024-09-10T22:51:15.155194+0000","last_fullsized":"2024-09-10T22:51:15.155194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153263+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:51:15.153263+0000","last_peered":"2024-09-10T22:51:15.153263+0000","last_clean":"2024-09-10T22:51:15.153263+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:51:15.153263+0000","last_undegraded":"2024-09-10T22:51:15.153263+0000","last_fullsized":"2024-09-10T22:51:15.153263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154928+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:51:15.154928+0000","last_peered":"2024-09-10T22:51:15.154928+0000","last_clean":"2024-09-10T22:51:15.154928+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:51:15.154928+0000","last_undegraded":"2024-09-10T22:51:15.154928+0000","last_fullsized":"2024-09-10T22:51:15.154928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153640+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:51:15.153640+0000","last_peered":"2024-09-10T22:51:15.153640+0000","last_clean":"2024-09-10T22:51:15.153640+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:51:15.153640+0000","last_undegraded":"2024-09-10T22:51:15.153640+0000","last_fullsized":"2024-09-10T22:51:15.153640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153570+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:51:15.153570+0000","last_peered":"2024-09-10T22:51:15.153570+0000","last_clean":"2024-09-10T22:51:15.153570+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:51:15.153570+0000","last_undegraded":"2024-09-10T22:51:15.153570+0000","last_fullsized":"2024-09-10T22:51:15.153570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295653+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:51:15.295653+0000","last_peered":"2024-09-10T22:51:15.295653+0000","last_clean":"2024-09-10T22:51:15.295653+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:51:15.295653+0000","last_undegraded":"2024-09-10T22:51:15.295653+0000","last_fullsized":"2024-09-10T22:51:15.295653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156076+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:51:15.156076+0000","last_peered":"2024-09-10T22:51:15.156076+0000","last_clean":"2024-09-10T22:51:15.156076+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:51:15.156076+0000","last_undegraded":"2024-09-10T22:51:15.156076+0000","last_fullsized":"2024-09-10T22:51:15.156076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150803+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:51:15.150803+0000","last_peered":"2024-09-10T22:51:15.150803+0000","last_clean":"2024-09-10T22:51:15.150803+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:51:15.150803+0000","last_undegraded":"2024-09-10T22:51:15.150803+0000","last_fullsized":"2024-09-10T22:51:15.150803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162560+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:51:15.162560+0000","last_peered":"2024-09-10T22:51:15.162560+0000","last_clean":"2024-09-10T22:51:15.162560+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:51:15.162560+0000","last_undegraded":"2024-09-10T22:51:15.162560+0000","last_fullsized":"2024-09-10T22:51:15.162560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155723+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:51:15.155723+0000","last_peered":"2024-09-10T22:51:15.155723+0000","last_clean":"2024-09-10T22:51:15.155723+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:51:15.155723+0000","last_undegraded":"2024-09-10T22:51:15.155723+0000","last_fullsized":"2024-09-10T22:51:15.155723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155039+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:51:15.155039+0000","last_peered":"2024-09-10T22:51:15.155039+0000","last_clean":"2024-09-10T22:51:15.155039+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:51:15.155039+0000","last_undegraded":"2024-09-10T22:51:15.155039+0000","last_fullsized":"2024-09-10T22:51:15.155039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295743+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:51:15.295743+0000","last_peered":"2024-09-10T22:51:15.295743+0000","last_clean":"2024-09-10T22:51:15.295743+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:51:15.295743+0000","last_undegraded":"2024-09-10T22:51:15.295743+0000","last_fullsized":"2024-09-10T22:51:15.295743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294977+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:51:15.294977+0000","last_peered":"2024-09-10T22:51:15.294977+0000","last_clean":"2024-09-10T22:51:15.294977+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:51:15.294977+0000","last_undegraded":"2024-09-10T22:51:15.294977+0000","last_fullsized":"2024-09-10T22:51:15.294977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295461+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:51:15.295461+0000","last_peered":"2024-09-10T22:51:15.295461+0000","last_clean":"2024-09-10T22:51:15.295461+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:51:15.295461+0000","last_undegraded":"2024-09-10T22:51:15.295461+0000","last_fullsized":"2024-09-10T22:51:15.295461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295171+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:51:15.295171+0000","last_peered":"2024-09-10T22:51:15.295171+0000","last_clean":"2024-09-10T22:51:15.295171+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:51:15.295171+0000","last_undegraded":"2024-09-10T22:51:15.295171+0000","last_fullsized":"2024-09-10T22:51:15.295171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162515+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:51:15.162515+0000","last_peered":"2024-09-10T22:51:15.162515+0000","last_clean":"2024-09-10T22:51:15.162515+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:51:15.162515+0000","last_undegraded":"2024-09-10T22:51:15.162515+0000","last_fullsized":"2024-09-10T22:51:15.162515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156495+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:51:15.156495+0000","last_peered":"2024-09-10T22:51:15.156495+0000","last_clean":"2024-09-10T22:51:15.156495+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:51:15.156495+0000","last_undegraded":"2024-09-10T22:51:15.156495+0000","last_fullsized":"2024-09-10T22:51:15.156495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154828+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:51:15.154828+0000","last_peered":"2024-09-10T22:51:15.154828+0000","last_clean":"2024-09-10T22:51:15.154828+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:51:15.154828+0000","last_undegraded":"2024-09-10T22:51:15.154828+0000","last_fullsized":"2024-09-10T22:51:15.154828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154404+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:51:15.154404+0000","last_peered":"2024-09-10T22:51:15.154404+0000","last_clean":"2024-09-10T22:51:15.154404+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:51:15.154404+0000","last_undegraded":"2024-09-10T22:51:15.154404+0000","last_fullsized":"2024-09-10T22:51:15.154404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161965+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:51:15.161965+0000","last_peered":"2024-09-10T22:51:15.161965+0000","last_clean":"2024-09-10T22:51:15.161965+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:51:15.161965+0000","last_undegraded":"2024-09-10T22:51:15.161965+0000","last_fullsized":"2024-09-10T22:51:15.161965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162068+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:51:15.162068+0000","last_peered":"2024-09-10T22:51:15.162068+0000","last_clean":"2024-09-10T22:51:15.162068+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:51:15.162068+0000","last_undegraded":"2024-09-10T22:51:15.162068+0000","last_fullsized":"2024-09-10T22:51:15.162068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155610+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:51:15.155610+0000","last_peered":"2024-09-10T22:51:15.155610+0000","last_clean":"2024-09-10T22:51:15.155610+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:51:15.155610+0000","last_undegraded":"2024-09-10T22:51:15.155610+0000","last_fullsized":"2024-09-10T22:51:15.155610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161948+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:51:15.161948+0000","last_peered":"2024-09-10T22:51:15.161948+0000","last_clean":"2024-09-10T22:51:15.161948+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:51:15.161948+0000","last_undegraded":"2024-09-10T22:51:15.161948+0000","last_fullsized":"2024-09-10T22:51:15.161948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295741+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:51:15.295741+0000","last_peered":"2024-09-10T22:51:15.295741+0000","last_clean":"2024-09-10T22:51:15.295741+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:51:15.295741+0000","last_undegraded":"2024-09-10T22:51:15.295741+0000","last_fullsized":"2024-09-10T22:51:15.295741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296700+0000","last_change":"2024-09-10T22:51:15.296217+0000","last_active":"2024-09-10T22:51:15.296700+0000","last_peered":"2024-09-10T22:51:15.296700+0000","last_clean":"2024-09-10T22:51:15.296700+0000","last_became_active":"2024-09-10T22:51:15.295842+0000","last_became_peered":"2024-09-10T22:51:15.295842+0000","last_unstale":"2024-09-10T22:51:15.296700+0000","last_undegraded":"2024-09-10T22:51:15.296700+0000","last_fullsized":"2024-09-10T22:51:15.296700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:32.464281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151051+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:51:15.151051+0000","last_peered":"2024-09-10T22:51:15.151051+0000","last_clean":"2024-09-10T22:51:15.151051+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:51:15.151051+0000","last_undegraded":"2024-09-10T22:51:15.151051+0000","last_fullsized":"2024-09-10T22:51:15.151051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153986+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:51:15.153986+0000","last_peered":"2024-09-10T22:51:15.153986+0000","last_clean":"2024-09-10T22:51:15.153986+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:51:15.153986+0000","last_undegraded":"2024-09-10T22:51:15.153986+0000","last_fullsized":"2024-09-10T22:51:15.153986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295017+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:51:15.295017+0000","last_peered":"2024-09-10T22:51:15.295017+0000","last_clean":"2024-09-10T22:51:15.295017+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:51:15.295017+0000","last_undegraded":"2024-09-10T22:51:15.295017+0000","last_fullsized":"2024-09-10T22:51:15.295017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295730+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:51:15.295730+0000","last_peered":"2024-09-10T22:51:15.295730+0000","last_clean":"2024-09-10T22:51:15.295730+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:51:15.295730+0000","last_undegraded":"2024-09-10T22:51:15.295730+0000","last_fullsized":"2024-09-10T22:51:15.295730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150604+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:51:15.150604+0000","last_peered":"2024-09-10T22:51:15.150604+0000","last_clean":"2024-09-10T22:51:15.150604+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:51:15.150604+0000","last_undegraded":"2024-09-10T22:51:15.150604+0000","last_fullsized":"2024-09-10T22:51:15.150604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151268+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:51:15.151268+0000","last_peered":"2024-09-10T22:51:15.151268+0000","last_clean":"2024-09-10T22:51:15.151268+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:51:15.151268+0000","last_undegraded":"2024-09-10T22:51:15.151268+0000","last_fullsized":"2024-09-10T22:51:15.151268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295974+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:51:15.295974+0000","last_peered":"2024-09-10T22:51:15.295974+0000","last_clean":"2024-09-10T22:51:15.295974+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:51:15.295974+0000","last_undegraded":"2024-09-10T22:51:15.295974+0000","last_fullsized":"2024-09-10T22:51:15.295974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155301+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:51:15.155301+0000","last_peered":"2024-09-10T22:51:15.155301+0000","last_clean":"2024-09-10T22:51:15.155301+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:51:15.155301+0000","last_undegraded":"2024-09-10T22:51:15.155301+0000","last_fullsized":"2024-09-10T22:51:15.155301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296977+0000","last_change":"2024-09-10T22:51:15.296922+0000","last_active":"2024-09-10T22:51:15.296977+0000","last_peered":"2024-09-10T22:51:15.296977+0000","last_clean":"2024-09-10T22:51:15.296977+0000","last_became_active":"2024-09-10T22:51:15.296658+0000","last_became_peered":"2024-09-10T22:51:15.296658+0000","last_unstale":"2024-09-10T22:51:15.296977+0000","last_undegraded":"2024-09-10T22:51:15.296977+0000","last_fullsized":"2024-09-10T22:51:15.296977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:18.932940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295896+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:51:15.295896+0000","last_peered":"2024-09-10T22:51:15.295896+0000","last_clean":"2024-09-10T22:51:15.295896+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:51:15.295896+0000","last_undegraded":"2024-09-10T22:51:15.295896+0000","last_fullsized":"2024-09-10T22:51:15.295896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295846+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:51:15.295846+0000","last_peered":"2024-09-10T22:51:15.295846+0000","last_clean":"2024-09-10T22:51:15.295846+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:51:15.295846+0000","last_undegraded":"2024-09-10T22:51:15.295846+0000","last_fullsized":"2024-09-10T22:51:15.295846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":173,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:31.139047+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:51:31.139047+0000","last_peered":"2024-09-10T22:51:31.139047+0000","last_clean":"2024-09-10T22:51:31.139047+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:51:31.139047+0000","last_undegraded":"2024-09-10T22:51:31.139047+0000","last_fullsized":"2024-09-10T22:51:31.139047+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":123,"num_read_kb":121,"num_write":27,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162574+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:51:15.162574+0000","last_peered":"2024-09-10T22:51:15.162574+0000","last_clean":"2024-09-10T22:51:15.162574+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:51:15.162574+0000","last_undegraded":"2024-09-10T22:51:15.162574+0000","last_fullsized":"2024-09-10T22:51:15.162574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162287+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:51:15.162287+0000","last_peered":"2024-09-10T22:51:15.162287+0000","last_clean":"2024-09-10T22:51:15.162287+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:51:15.162287+0000","last_undegraded":"2024-09-10T22:51:15.162287+0000","last_fullsized":"2024-09-10T22:51:15.162287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154237+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:51:15.154237+0000","last_peered":"2024-09-10T22:51:15.154237+0000","last_clean":"2024-09-10T22:51:15.154237+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:51:15.154237+0000","last_undegraded":"2024-09-10T22:51:15.154237+0000","last_fullsized":"2024-09-10T22:51:15.154237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295672+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:51:15.295672+0000","last_peered":"2024-09-10T22:51:15.295672+0000","last_clean":"2024-09-10T22:51:15.295672+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:51:15.295672+0000","last_undegraded":"2024-09-10T22:51:15.295672+0000","last_fullsized":"2024-09-10T22:51:15.295672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156806+0000","last_change":"2024-09-10T22:51:15.156697+0000","last_active":"2024-09-10T22:51:15.156806+0000","last_peered":"2024-09-10T22:51:15.156806+0000","last_clean":"2024-09-10T22:51:15.156806+0000","last_became_active":"2024-09-10T22:51:15.156396+0000","last_became_peered":"2024-09-10T22:51:15.156396+0000","last_unstale":"2024-09-10T22:51:15.156806+0000","last_undegraded":"2024-09-10T22:51:15.156806+0000","last_fullsized":"2024-09-10T22:51:15.156806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:26.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.152948+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:51:15.152948+0000","last_peered":"2024-09-10T22:51:15.152948+0000","last_clean":"2024-09-10T22:51:15.152948+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:51:15.152948+0000","last_undegraded":"2024-09-10T22:51:15.152948+0000","last_fullsized":"2024-09-10T22:51:15.152948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154683+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:51:15.154683+0000","last_peered":"2024-09-10T22:51:15.154683+0000","last_clean":"2024-09-10T22:51:15.154683+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:51:15.154683+0000","last_undegraded":"2024-09-10T22:51:15.154683+0000","last_fullsized":"2024-09-10T22:51:15.154683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162012+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:51:15.162012+0000","last_peered":"2024-09-10T22:51:15.162012+0000","last_clean":"2024-09-10T22:51:15.162012+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:51:15.162012+0000","last_undegraded":"2024-09-10T22:51:15.162012+0000","last_fullsized":"2024-09-10T22:51:15.162012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295528+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:51:15.295528+0000","last_peered":"2024-09-10T22:51:15.295528+0000","last_clean":"2024-09-10T22:51:15.295528+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:51:15.295528+0000","last_undegraded":"2024-09-10T22:51:15.295528+0000","last_fullsized":"2024-09-10T22:51:15.295528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162997+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:51:15.162997+0000","last_peered":"2024-09-10T22:51:15.162997+0000","last_clean":"2024-09-10T22:51:15.162997+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:51:15.162997+0000","last_undegraded":"2024-09-10T22:51:15.162997+0000","last_fullsized":"2024-09-10T22:51:15.162997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155841+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:51:15.155841+0000","last_peered":"2024-09-10T22:51:15.155841+0000","last_clean":"2024-09-10T22:51:15.155841+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:51:15.155841+0000","last_undegraded":"2024-09-10T22:51:15.155841+0000","last_fullsized":"2024-09-10T22:51:15.155841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154030+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:51:15.154030+0000","last_peered":"2024-09-10T22:51:15.154030+0000","last_clean":"2024-09-10T22:51:15.154030+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:51:15.154030+0000","last_undegraded":"2024-09-10T22:51:15.154030+0000","last_fullsized":"2024-09-10T22:51:15.154030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155400+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:51:15.155400+0000","last_peered":"2024-09-10T22:51:15.155400+0000","last_clean":"2024-09-10T22:51:15.155400+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:51:15.155400+0000","last_undegraded":"2024-09-10T22:51:15.155400+0000","last_fullsized":"2024-09-10T22:51:15.155400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296015+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:51:15.296015+0000","last_peered":"2024-09-10T22:51:15.296015+0000","last_clean":"2024-09-10T22:51:15.296015+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:51:15.296015+0000","last_undegraded":"2024-09-10T22:51:15.296015+0000","last_fullsized":"2024-09-10T22:51:15.296015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155019+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:51:15.155019+0000","last_peered":"2024-09-10T22:51:15.155019+0000","last_clean":"2024-09-10T22:51:15.155019+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:51:15.155019+0000","last_undegraded":"2024-09-10T22:51:15.155019+0000","last_fullsized":"2024-09-10T22:51:15.155019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.158692+0000","last_change":"2024-09-10T22:51:15.158070+0000","last_active":"2024-09-10T22:51:15.158692+0000","last_peered":"2024-09-10T22:51:15.158692+0000","last_clean":"2024-09-10T22:51:15.158692+0000","last_became_active":"2024-09-10T22:51:15.157711+0000","last_became_peered":"2024-09-10T22:51:15.157711+0000","last_unstale":"2024-09-10T22:51:15.158692+0000","last_undegraded":"2024-09-10T22:51:15.158692+0000","last_fullsized":"2024-09-10T22:51:15.158692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:11.308860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155239+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.155239+0000","last_peered":"2024-09-10T22:51:15.155239+0000","last_clean":"2024-09-10T22:51:15.155239+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:51:15.155239+0000","last_undegraded":"2024-09-10T22:51:15.155239+0000","last_fullsized":"2024-09-10T22:51:15.155239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154834+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:51:15.154834+0000","last_peered":"2024-09-10T22:51:15.154834+0000","last_clean":"2024-09-10T22:51:15.154834+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:51:15.154834+0000","last_undegraded":"2024-09-10T22:51:15.154834+0000","last_fullsized":"2024-09-10T22:51:15.154834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153104+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:51:15.153104+0000","last_peered":"2024-09-10T22:51:15.153104+0000","last_clean":"2024-09-10T22:51:15.153104+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:51:15.153104+0000","last_undegraded":"2024-09-10T22:51:15.153104+0000","last_fullsized":"2024-09-10T22:51:15.153104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295351+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:51:15.295351+0000","last_peered":"2024-09-10T22:51:15.295351+0000","last_clean":"2024-09-10T22:51:15.295351+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:51:15.295351+0000","last_undegraded":"2024-09-10T22:51:15.295351+0000","last_fullsized":"2024-09-10T22:51:15.295351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154832+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:51:15.154832+0000","last_peered":"2024-09-10T22:51:15.154832+0000","last_clean":"2024-09-10T22:51:15.154832+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:51:15.154832+0000","last_undegraded":"2024-09-10T22:51:15.154832+0000","last_fullsized":"2024-09-10T22:51:15.154832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.165878+0000","last_change":"2024-09-10T22:51:15.165814+0000","last_active":"2024-09-10T22:51:15.165878+0000","last_peered":"2024-09-10T22:51:15.165878+0000","last_clean":"2024-09-10T22:51:15.165878+0000","last_became_active":"2024-09-10T22:51:15.165305+0000","last_became_peered":"2024-09-10T22:51:15.165305+0000","last_unstale":"2024-09-10T22:51:15.165878+0000","last_undegraded":"2024-09-10T22:51:15.165878+0000","last_fullsized":"2024-09-10T22:51:15.165878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:14.680823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154077+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:51:15.154077+0000","last_peered":"2024-09-10T22:51:15.154077+0000","last_clean":"2024-09-10T22:51:15.154077+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:51:15.154077+0000","last_undegraded":"2024-09-10T22:51:15.154077+0000","last_fullsized":"2024-09-10T22:51:15.154077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295816+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:51:15.295816+0000","last_peered":"2024-09-10T22:51:15.295816+0000","last_clean":"2024-09-10T22:51:15.295816+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:51:15.295816+0000","last_undegraded":"2024-09-10T22:51:15.295816+0000","last_fullsized":"2024-09-10T22:51:15.295816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155014+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:51:15.155014+0000","last_peered":"2024-09-10T22:51:15.155014+0000","last_clean":"2024-09-10T22:51:15.155014+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:51:15.155014+0000","last_undegraded":"2024-09-10T22:51:15.155014+0000","last_fullsized":"2024-09-10T22:51:15.155014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151167+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:51:15.151167+0000","last_peered":"2024-09-10T22:51:15.151167+0000","last_clean":"2024-09-10T22:51:15.151167+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:51:15.151167+0000","last_undegraded":"2024-09-10T22:51:15.151167+0000","last_fullsized":"2024-09-10T22:51:15.151167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150766+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:51:15.150766+0000","last_peered":"2024-09-10T22:51:15.150766+0000","last_clean":"2024-09-10T22:51:15.150766+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:51:15.150766+0000","last_undegraded":"2024-09-10T22:51:15.150766+0000","last_fullsized":"2024-09-10T22:51:15.150766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151181+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:51:15.151181+0000","last_peered":"2024-09-10T22:51:15.151181+0000","last_clean":"2024-09-10T22:51:15.151181+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:51:15.151181+0000","last_undegraded":"2024-09-10T22:51:15.151181+0000","last_fullsized":"2024-09-10T22:51:15.151181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162786+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:51:15.162786+0000","last_peered":"2024-09-10T22:51:15.162786+0000","last_clean":"2024-09-10T22:51:15.162786+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:51:15.162786+0000","last_undegraded":"2024-09-10T22:51:15.162786+0000","last_fullsized":"2024-09-10T22:51:15.162786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154876+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:51:15.154876+0000","last_peered":"2024-09-10T22:51:15.154876+0000","last_clean":"2024-09-10T22:51:15.154876+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:51:15.154876+0000","last_undegraded":"2024-09-10T22:51:15.154876+0000","last_fullsized":"2024-09-10T22:51:15.154876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150866+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:51:15.150866+0000","last_peered":"2024-09-10T22:51:15.150866+0000","last_clean":"2024-09-10T22:51:15.150866+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:51:15.150866+0000","last_undegraded":"2024-09-10T22:51:15.150866+0000","last_fullsized":"2024-09-10T22:51:15.150866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295230+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:51:15.295230+0000","last_peered":"2024-09-10T22:51:15.295230+0000","last_clean":"2024-09-10T22:51:15.295230+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:51:15.295230+0000","last_undegraded":"2024-09-10T22:51:15.295230+0000","last_fullsized":"2024-09-10T22:51:15.295230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153127+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:51:15.153127+0000","last_peered":"2024-09-10T22:51:15.153127+0000","last_clean":"2024-09-10T22:51:15.153127+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:51:15.153127+0000","last_undegraded":"2024-09-10T22:51:15.153127+0000","last_fullsized":"2024-09-10T22:51:15.153127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154511+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:51:15.154511+0000","last_peered":"2024-09-10T22:51:15.154511+0000","last_clean":"2024-09-10T22:51:15.154511+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:51:15.154511+0000","last_undegraded":"2024-09-10T22:51:15.154511+0000","last_fullsized":"2024-09-10T22:51:15.154511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295736+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:51:15.295736+0000","last_peered":"2024-09-10T22:51:15.295736+0000","last_clean":"2024-09-10T22:51:15.295736+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:51:15.295736+0000","last_undegraded":"2024-09-10T22:51:15.295736+0000","last_fullsized":"2024-09-10T22:51:15.295736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161616+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:51:15.161616+0000","last_peered":"2024-09-10T22:51:15.161616+0000","last_clean":"2024-09-10T22:51:15.161616+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:51:15.161616+0000","last_undegraded":"2024-09-10T22:51:15.161616+0000","last_fullsized":"2024-09-10T22:51:15.161616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295097+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:51:15.295097+0000","last_peered":"2024-09-10T22:51:15.295097+0000","last_clean":"2024-09-10T22:51:15.295097+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:51:15.295097+0000","last_undegraded":"2024-09-10T22:51:15.295097+0000","last_fullsized":"2024-09-10T22:51:15.295097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155731+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:51:15.155731+0000","last_peered":"2024-09-10T22:51:15.155731+0000","last_clean":"2024-09-10T22:51:15.155731+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:51:15.155731+0000","last_undegraded":"2024-09-10T22:51:15.155731+0000","last_fullsized":"2024-09-10T22:51:15.155731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.297065+0000","last_change":"2024-09-10T22:51:15.296938+0000","last_active":"2024-09-10T22:51:15.297065+0000","last_peered":"2024-09-10T22:51:15.297065+0000","last_clean":"2024-09-10T22:51:15.297065+0000","last_became_active":"2024-09-10T22:51:15.296291+0000","last_became_peered":"2024-09-10T22:51:15.296291+0000","last_unstale":"2024-09-10T22:51:15.297065+0000","last_undegraded":"2024-09-10T22:51:15.297065+0000","last_fullsized":"2024-09-10T22:51:15.297065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:26.752740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295243+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:51:15.295243+0000","last_peered":"2024-09-10T22:51:15.295243+0000","last_clean":"2024-09-10T22:51:15.295243+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:51:15.295243+0000","last_undegraded":"2024-09-10T22:51:15.295243+0000","last_fullsized":"2024-09-10T22:51:15.295243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161953+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:51:15.161953+0000","last_peered":"2024-09-10T22:51:15.161953+0000","last_clean":"2024-09-10T22:51:15.161953+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:51:15.161953+0000","last_undegraded":"2024-09-10T22:51:15.161953+0000","last_fullsized":"2024-09-10T22:51:15.161953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154833+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:51:15.154833+0000","last_peered":"2024-09-10T22:51:15.154833+0000","last_clean":"2024-09-10T22:51:15.154833+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:51:15.154833+0000","last_undegraded":"2024-09-10T22:51:15.154833+0000","last_fullsized":"2024-09-10T22:51:15.154833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295992+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:51:15.295992+0000","last_peered":"2024-09-10T22:51:15.295992+0000","last_clean":"2024-09-10T22:51:15.295992+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:51:15.295992+0000","last_undegraded":"2024-09-10T22:51:15.295992+0000","last_fullsized":"2024-09-10T22:51:15.295992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162544+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:51:15.162544+0000","last_peered":"2024-09-10T22:51:15.162544+0000","last_clean":"2024-09-10T22:51:15.162544+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:51:15.162544+0000","last_undegraded":"2024-09-10T22:51:15.162544+0000","last_fullsized":"2024-09-10T22:51:15.162544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295742+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:51:15.295742+0000","last_peered":"2024-09-10T22:51:15.295742+0000","last_clean":"2024-09-10T22:51:15.295742+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:51:15.295742+0000","last_undegraded":"2024-09-10T22:51:15.295742+0000","last_fullsized":"2024-09-10T22:51:15.295742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.173507+0000","last_change":"2024-09-10T22:51:15.173004+0000","last_active":"2024-09-10T22:51:15.173507+0000","last_peered":"2024-09-10T22:51:15.173507+0000","last_clean":"2024-09-10T22:51:15.173507+0000","last_became_active":"2024-09-10T22:51:15.165748+0000","last_became_peered":"2024-09-10T22:51:15.165748+0000","last_unstale":"2024-09-10T22:51:15.173507+0000","last_undegraded":"2024-09-10T22:51:15.173507+0000","last_fullsized":"2024-09-10T22:51:15.173507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:38.001944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153491+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:51:15.153491+0000","last_peered":"2024-09-10T22:51:15.153491+0000","last_clean":"2024-09-10T22:51:15.153491+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:51:15.153491+0000","last_undegraded":"2024-09-10T22:51:15.153491+0000","last_fullsized":"2024-09-10T22:51:15.153491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295425+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:51:15.295425+0000","last_peered":"2024-09-10T22:51:15.295425+0000","last_clean":"2024-09-10T22:51:15.295425+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:51:15.295425+0000","last_undegraded":"2024-09-10T22:51:15.295425+0000","last_fullsized":"2024-09-10T22:51:15.295425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295318+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:51:15.295318+0000","last_peered":"2024-09-10T22:51:15.295318+0000","last_clean":"2024-09-10T22:51:15.295318+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:51:15.295318+0000","last_undegraded":"2024-09-10T22:51:15.295318+0000","last_fullsized":"2024-09-10T22:51:15.295318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150890+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:51:15.150890+0000","last_peered":"2024-09-10T22:51:15.150890+0000","last_clean":"2024-09-10T22:51:15.150890+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:51:15.150890+0000","last_undegraded":"2024-09-10T22:51:15.150890+0000","last_fullsized":"2024-09-10T22:51:15.150890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62548,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":129,"num_read_kb":127,"num_write":82,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":187644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1507328,"data_stored":1057101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":831488,"data_stored":405254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":405276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495661,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1429504,"data_stored":995739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855415,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1437696,"data_stored":995339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":404694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":466456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":897024,"data_stored":466422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:51:33.949 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:51:33.949 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:51:33.949 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:51:33.949 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T22:51:34.196 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:34.611 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:34 smithi120 ceph-mon[31247]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:34.612 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:34 smithi120 ceph-mon[31247]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:34.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:34 smithi153 ceph-mon[34951]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:34.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:34 smithi153 ceph-mon[34951]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:34.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 -- 172.21.15.120:0/2487493303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d81076a0 msgr2=0x7f85d8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 --2- 172.21.15.120:0/2487493303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d81076a0 0x7f85d8107ae0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009900 tx=0x7f85c002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 -- 172.21.15.120:0/2487493303 shutdown_connections 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 --2- 172.21.15.120:0/2487493303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d8108020 0x7f85d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 --2- 172.21.15.120:0/2487493303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d81076a0 0x7f85d8107ae0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 --2- 172.21.15.120:0/2487493303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85d8101c90 0x7f85d8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 -- 172.21.15.120:0/2487493303 >> 172.21.15.120:0/2487493303 conn(0x7f85d80fd700 msgr2=0x7f85d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 -- 172.21.15.120:0/2487493303 shutdown_connections 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.690+0000 7f85dca9f640 1 -- 172.21.15.120:0/2487493303 wait complete. 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 Processor -- start 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 -- start start 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d81076a0 0x7f85d819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:34.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85d8108020 0x7f85d819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111c70 con 0x7f85d8101c90 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111af0 con 0x7f85d81076a0 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85dca9f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111df0 con 0x7f85d8108020 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34150/0 (socket says 172.21.15.120:34150) 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 -- 172.21.15.120:0/724179989 learned_addr learned my addr 172.21.15.120:0/724179989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85cffff640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d81076a0 0x7f85d819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 -- 172.21.15.120:0/724179989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85d8108020 msgr2=0x7f85d819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85d8108020 0x7f85d819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 -- 172.21.15.120:0/724179989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d81076a0 msgr2=0x7f85d819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d81076a0 0x7f85d819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 -- 172.21.15.120:0/724179989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a5440 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85d659c640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c920 tx=0x7f85c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.691+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8007c70 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.692+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a5730 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.692+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c8007e10 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.692+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8018820 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.692+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a5c40 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.692+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8102070 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.694+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f85c800f450 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.694+0000 7f85cf7fe640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 0x7f85c40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.695+0000 7f85cffff640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 0x7f85c40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.695+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f85c80e61b0 con 0x7f85d8101c90 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.695+0000 7f85cffff640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 0x7f85c40b7830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85c0031440 tx=0x7f85c00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:34.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.697+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f85c8014030 con 0x7f85d8101c90 2024-09-10T22:51:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:34 smithi176 ceph-mon[34982]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:51:34.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:34 smithi176 ceph-mon[34982]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:34.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.907+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f85d806aec0 con 0x7f85d8101c90 2024-09-10T22:51:34.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.908+0000 7f85cf7fe640 1 -- 172.21.15.120:0/724179989 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f85d806aec0 con 0x7f85d8101c90 2024-09-10T22:51:34.909 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:34.909 INFO:teuthology.orchestra.run.smithi120.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:51:34.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 msgr2=0x7f85c40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 0x7f85c40b7830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85c0031440 tx=0x7f85c00095f0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 msgr2=0x7f85d819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c920 tx=0x7f85c800cde0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 shutdown_connections 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85c40b5370 0x7f85c40b7830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85d8108020 0x7f85d819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d81076a0 0x7f85d819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 --2- 172.21.15.120:0/724179989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85d8101c90 0x7f85d819a060 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 >> 172.21.15.120:0/724179989 conn(0x7f85d80fd700 msgr2=0x7f85d80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 shutdown_connections 2024-09-10T22:51:34.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:34.910+0000 7f85dca9f640 1 -- 172.21.15.120:0/724179989 wait complete. 2024-09-10T22:51:35.578 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:35 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/724179989' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:51:35.607 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:51:35.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:35.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:35 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/724179989' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:51:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:35 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/724179989' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:51:35.854 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 -- 172.21.15.120:0/2350744719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 msgr2=0x7f33c810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 --2- 172.21.15.120:0/2350744719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c810cac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009900 tx=0x7f33bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 -- 172.21.15.120:0/2350744719 shutdown_connections 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 --2- 172.21.15.120:0/2350744719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c810d110 0x7f33c810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 --2- 172.21.15.120:0/2350744719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c810cac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 --2- 172.21.15.120:0/2350744719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c8103cb0 0x7f33c8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 -- 172.21.15.120:0/2350744719 >> 172.21.15.120:0/2350744719 conn(0x7f33c80fd6d0 msgr2=0x7f33c80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 -- 172.21.15.120:0/2350744719 shutdown_connections 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.342+0000 7f33ce513640 1 -- 172.21.15.120:0/2350744719 wait complete. 2024-09-10T22:51:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 Processor -- start 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 -- start start 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c8103cb0 0x7f33c8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 0x7f33c819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8112dc0 con 0x7f33c81045d0 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8112c40 con 0x7f33c810d110 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33ce513640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8112f40 con 0x7f33c8103cb0 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34178/0 (socket says 172.21.15.120:34178) 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.343+0000 7f33c77fe640 1 -- 172.21.15.120:0/2667635604 learned_addr learned my addr 172.21.15.120:0/2667635604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33ccab0640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 0x7f33c819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c7fff640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c8103cb0 0x7f33c8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 -- 172.21.15.120:0/2667635604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c8103cb0 msgr2=0x7f33c8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c8103cb0 0x7f33c8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 -- 172.21.15.120:0/2667635604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 msgr2=0x7f33c819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 0x7f33c819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 -- 172.21.15.120:0/2667635604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c819f3c0 con 0x7f33c81045d0 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33ccab0640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 0x7f33c819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c77fe640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004270 tx=0x7f33bc0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc03f030 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc0044c0 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c819f650 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc035070 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.344+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81a5d70 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.345+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8104090 con 0x7f33c81045d0 2024-09-10T22:51:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.346+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f33bc03bbe0 con 0x7f33c81045d0 2024-09-10T22:51:36.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.347+0000 7f33c57fa640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 0x7f33b00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:36.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.347+0000 7f33c7fff640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 0x7f33b00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:36.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.347+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f33bc106a10 con 0x7f33c81045d0 2024-09-10T22:51:36.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.347+0000 7f33c7fff640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 0x7f33b00b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f33b80058c0 tx=0x7f33b8005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:36.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.350+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f33bc0cd7a0 con 0x7f33c81045d0 2024-09-10T22:51:36.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.562+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c81a6130 con 0x7f33c81045d0 2024-09-10T22:51:36.564 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:36 smithi120 ceph-mon[31247]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:36.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.563+0000 7f33c57fa640 1 -- 172.21.15.120:0/2667635604 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f33bc0cd0a0 con 0x7f33c81045d0 2024-09-10T22:51:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:36.565 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:51:36.566 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:36.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.565+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 msgr2=0x7f33b00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:36.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.565+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 0x7f33b00b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f33b80058c0 tx=0x7f33b8005c60 comp rx=0 tx=0).stop 2024-09-10T22:51:36.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.565+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 msgr2=0x7f33c8199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:36.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.565+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33bc004270 tx=0x7f33bc0042a0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 shutdown_connections 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33b00b5370 0x7f33b00b7830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33c810d110 0x7f33c819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33c81045d0 0x7f33c8199850 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 --2- 172.21.15.120:0/2667635604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33c8103cb0 0x7f33c8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 >> 172.21.15.120:0/2667635604 conn(0x7f33c80fd6d0 msgr2=0x7f33c80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 shutdown_connections 2024-09-10T22:51:36.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:36.566+0000 7f33ce513640 1 -- 172.21.15.120:0/2667635604 wait complete. 2024-09-10T22:51:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:36 smithi153 ceph-mon[34951]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:36.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:36 smithi176 ceph-mon[34982]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:37.233 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:37.481 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:37.601 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:37 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2667635604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:37 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2667635604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:37 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2667635604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.942+0000 7fdc0e57b640 1 -- 172.21.15.120:0/1063668180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 msgr2=0x7fdc081040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.942+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/1063668180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc081040c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009900 tx=0x7fdbf802a410 comp rx=0 tx=0).stop 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 -- 172.21.15.120:0/1063668180 shutdown_connections 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/1063668180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc0810d140 0x7fdc0810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/1063668180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc08104600 0x7fdc0810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/1063668180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc081040c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 -- 172.21.15.120:0/1063668180 >> 172.21.15.120:0/1063668180 conn(0x7fdc080fd700 msgr2=0x7fdc080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 -- 172.21.15.120:0/1063668180 shutdown_connections 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 -- 172.21.15.120:0/1063668180 wait complete. 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 Processor -- start 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 -- start start 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.943+0000 7fdc0e57b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:37.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 0x7fdc081998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 0x7fdc0819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112e10 con 0x7fdc08103ce0 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112c90 con 0x7fdc08104600 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08112f90 con 0x7fdc0810d140 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34198/0 (socket says 172.21.15.120:34198) 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 -- 172.21.15.120:0/3415977834 learned_addr learned my addr 172.21.15.120:0/3415977834 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0cb18640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 0x7fdc0819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc077fe640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 0x7fdc081998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 -- 172.21.15.120:0/3415977834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 msgr2=0x7fdc0819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 0x7fdc0819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:37.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 -- 172.21.15.120:0/3415977834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 msgr2=0x7fdc081998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 0x7fdc081998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 -- 172.21.15.120:0/3415977834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0819f410 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0cb18640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 0x7fdc0819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc077fe640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 0x7fdc081998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc07fff640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006010 tx=0x7fdbf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf803f030 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0819f6a0 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf8003800 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8035070 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.944+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc081a5dc0 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.945+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc081040c0 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.947+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fdbf8031b10 con 0x7fdc08103ce0 2024-09-10T22:51:37.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.947+0000 7fdc057fa640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 0x7fdbf40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.947+0000 7fdc077fe640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 0x7fdbf40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.947+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fdbf81069b0 con 0x7fdc08103ce0 2024-09-10T22:51:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.948+0000 7fdc077fe640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 0x7fdbf40b7830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006fd0 tx=0x7fdbf0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:37.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:37.950+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdbf80cd740 con 0x7fdc08103ce0 2024-09-10T22:51:38.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.164+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc0819a490 con 0x7fdc08103ce0 2024-09-10T22:51:38.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.165+0000 7fdc057fa640 1 -- 172.21.15.120:0/3415977834 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7fdbf80cd040 con 0x7fdc08103ce0 2024-09-10T22:51:38.167 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:38.167 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:51:38.167 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 msgr2=0x7fdbf40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 0x7fdbf40b7830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0006fd0 tx=0x7fdbf0008040 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 msgr2=0x7fdc08199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006010 tx=0x7fdbf80042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 shutdown_connections 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf40b5370 0x7fdbf40b7830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc0810d140 0x7fdc0819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc08104600 0x7fdc081998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 --2- 172.21.15.120:0/3415977834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc08103ce0 0x7fdc08199360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:38.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 >> 172.21.15.120:0/3415977834 conn(0x7fdc080fd700 msgr2=0x7fdc080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:38.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.167+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 shutdown_connections 2024-09-10T22:51:38.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:38.168+0000 7fdc0e57b640 1 -- 172.21.15.120:0/3415977834 wait complete. 2024-09-10T22:51:38.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:38 smithi120 ceph-mon[31247]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:38.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:38 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3415977834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:38 smithi153 ceph-mon[34951]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:38 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3415977834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:38.789 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:51:38.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:38 smithi176 ceph-mon[34982]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:38.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:38 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3415977834' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:39.035 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:39.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 -- 172.21.15.120:0/4013647999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0108040 msgr2=0x7fdcc010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:39.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 --2- 172.21.15.120:0/4013647999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0108040 0x7fdcc010e840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009900 tx=0x7fdcb002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:39.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 -- 172.21.15.120:0/4013647999 shutdown_connections 2024-09-10T22:51:39.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 --2- 172.21.15.120:0/4013647999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc0108040 0x7fdcc010e840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 --2- 172.21.15.120:0/4013647999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc01076c0 0x7fdcc0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 --2- 172.21.15.120:0/4013647999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 -- 172.21.15.120:0/4013647999 >> 172.21.15.120:0/4013647999 conn(0x7fdcc00fd700 msgr2=0x7fdcc00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 -- 172.21.15.120:0/4013647999 shutdown_connections 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.503+0000 7fdcc4927640 1 -- 172.21.15.120:0/4013647999 wait complete. 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 Processor -- start 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 -- start start 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc00fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 0x7fdcc00fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111d00 con 0x7fdcc01076c0 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111b80 con 0x7fdcc0101cb0 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcc4927640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111e80 con 0x7fdcc0108040 2024-09-10T22:51:39.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34220/0 (socket says 172.21.15.120:34220) 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 -- 172.21.15.120:0/847357401 learned_addr learned my addr 172.21.15.120:0/847357401 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbe59c640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc00fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbed9d640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 0x7fdcc00fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 -- 172.21.15.120:0/847357401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 msgr2=0x7fdcc00fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 0x7fdcc00fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 -- 172.21.15.120:0/847357401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 msgr2=0x7fdcc00fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc00fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbdd9b640 1 -- 172.21.15.120:0/847357401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a76f0 con 0x7fdcc01076c0 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbed9d640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 0x7fdcc00fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.504+0000 7fdcbe59c640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcbdd9b640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c890 tx=0x7fdca800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:39.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8004220 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca80043c0 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a79e0 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8004a00 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.505+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a7fa0 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.506+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0102090 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.507+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fdca80079b0 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.508+0000 7fdcb77fe640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 0x7fdcac0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.508+0000 7fdcbe59c640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 0x7fdcac0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.508+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fdca80e5f10 con 0x7fdcc01076c0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.508+0000 7fdcbe59c640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 0x7fdcac0b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009fa0 tx=0x7fdcb8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:39.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.511+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdca8014030 con 0x7fdcc01076c0 2024-09-10T22:51:39.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.687+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc01a83d0 con 0x7fdcc01076c0 2024-09-10T22:51:39.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.689+0000 7fdcb77fe640 1 -- 172.21.15.120:0/847357401 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22746 (secure 0 0 0) 0x7fdca80acca0 con 0x7fdcc01076c0 2024-09-10T22:51:39.690 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:39.690 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":77,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:51:15.147106+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:51:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 msgr2=0x7fdcac0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 0x7fdcac0b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009fa0 tx=0x7fdcb8009570 comp rx=0 tx=0).stop 2024-09-10T22:51:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 msgr2=0x7fdcc00fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:39.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c890 tx=0x7fdca800cd50 comp rx=0 tx=0).stop 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 shutdown_connections 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdcac0b5370 0x7fdcac0b7830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcc0108040 0x7fdcc00fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcc01076c0 0x7fdcc00fe140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 --2- 172.21.15.120:0/847357401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc0101cb0 0x7fdcc00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 >> 172.21.15.120:0/847357401 conn(0x7fdcc00fd700 msgr2=0x7fdcc006b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 shutdown_connections 2024-09-10T22:51:39.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:39.691+0000 7fdcc4927640 1 -- 172.21.15.120:0/847357401 wait complete. 2024-09-10T22:51:39.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:39 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/847357401' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:39 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/847357401' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:40.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:39 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/847357401' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:51:40.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:40.637 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:40.921 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:40 smithi120 ceph-mon[31247]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:41.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:40 smithi176 ceph-mon[34982]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 -- 172.21.15.120:0/467060865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 msgr2=0x7f16c4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 --2- 172.21.15.120:0/467060865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4107ae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009900 tx=0x7f16b402a410 comp rx=0 tx=0).stop 2024-09-10T22:51:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 -- 172.21.15.120:0/467060865 shutdown_connections 2024-09-10T22:51:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 --2- 172.21.15.120:0/467060865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4108020 0x7f16c410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 --2- 172.21.15.120:0/467060865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4107ae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 --2- 172.21.15.120:0/467060865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4101c90 0x7f16c4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 -- 172.21.15.120:0/467060865 >> 172.21.15.120:0/467060865 conn(0x7f16c40fd700 msgr2=0x7f16c40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 -- 172.21.15.120:0/467060865 shutdown_connections 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.115+0000 7f16cbb77640 1 -- 172.21.15.120:0/467060865 wait complete. 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 Processor -- start 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 -- start start 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 0x7f16c41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4108020 0x7f16c419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111ce0 con 0x7f16c41076a0 2024-09-10T22:51:41.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111b60 con 0x7f16c4101c90 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16cbb77640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111e60 con 0x7f16c4108020 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9112640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9112640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34248/0 (socket says 172.21.15.120:34248) 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9112640 1 -- 172.21.15.120:0/3875223173 learned_addr learned my addr 172.21.15.120:0/3875223173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9913640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 0x7f16c41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16ca114640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4108020 0x7f16c419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9112640 1 -- 172.21.15.120:0/3875223173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4108020 msgr2=0x7f16c419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.116+0000 7f16c9112640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4108020 0x7f16c419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16c9112640 1 -- 172.21.15.120:0/3875223173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 msgr2=0x7f16c41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16c9112640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 0x7f16c41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16c9112640 1 -- 172.21.15.120:0/3875223173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419e2a0 con 0x7f16c41076a0 2024-09-10T22:51:41.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16c9913640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 0x7f16c41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16c9112640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b403f030 con 0x7f16c41076a0 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b4003800 con 0x7f16c41076a0 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c419e530 con 0x7f16c41076a0 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4035070 con 0x7f16c41076a0 2024-09-10T22:51:41.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.117+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a4c50 con 0x7f16c41076a0 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.118+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c4102070 con 0x7f16c41076a0 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.119+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f16b4031b10 con 0x7f16c41076a0 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.120+0000 7f16baffd640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 0x7f16b00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.120+0000 7f16c9913640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 0x7f16b00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.120+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f16b41066a0 con 0x7f16c41076a0 2024-09-10T22:51:41.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.121+0000 7f16c9913640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 0x7f16b00b78b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009930 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:41.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.124+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f16b40cd430 con 0x7f16c41076a0 2024-09-10T22:51:41.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:40 smithi153 ceph-mon[34951]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:41.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.442+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16c4109870 con 0x7f16c41076a0 2024-09-10T22:51:41.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.443+0000 7f16baffd640 1 -- 172.21.15.120:0/3875223173 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f16b40ccd30 con 0x7f16c41076a0 2024-09-10T22:51:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:41.445 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:51:41.445 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:41.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 msgr2=0x7f16b00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:41.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 0x7f16b00b78b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009930 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 msgr2=0x7f16c4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b40042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 shutdown_connections 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16b00b53f0 0x7f16b00b78b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16c4108020 0x7f16c419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16c41076a0 0x7f16c4198730 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 --2- 172.21.15.120:0/3875223173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16c4101c90 0x7f16c41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 >> 172.21.15.120:0/3875223173 conn(0x7f16c40fd700 msgr2=0x7f16c40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.445+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 shutdown_connections 2024-09-10T22:51:41.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:41.446+0000 7f16cbb77640 1 -- 172.21.15.120:0/3875223173 wait complete. 2024-09-10T22:51:42.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:41 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3875223173' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:42.097 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:41 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3875223173' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:42.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:50:27.621402+0000', 'modified': '2024-09-10T22:50:36.741193+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6837/1046536521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 1046536521}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 1046536521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/3141658633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 3141658633}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 3141658633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24550, 'qdb_cluster': [24550]} 2024-09-10T22:51:42.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:51:42.128 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T22:51:42.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:41 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3875223173' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:42.379 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:42.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.898+0000 7f2204d4f640 1 -- 172.21.15.120:0/2653952240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200101830 msgr2=0x7f2200101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.898+0000 7f2204d4f640 1 --2- 172.21.15.120:0/2653952240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200101830 0x7f2200101c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009ca0 tx=0x7f21f002a950 comp rx=0 tx=0).stop 2024-09-10T22:51:42.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 -- 172.21.15.120:0/2653952240 shutdown_connections 2024-09-10T22:51:42.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 --2- 172.21.15.120:0/2653952240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f220010f3e0 0x7f22001117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 --2- 172.21.15.120:0/2653952240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2200102150 0x7f220010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 --2- 172.21.15.120:0/2653952240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200101830 0x7f2200101c10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 -- 172.21.15.120:0/2653952240 >> 172.21.15.120:0/2653952240 conn(0x7f22000fd700 msgr2=0x7f22000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 -- 172.21.15.120:0/2653952240 shutdown_connections 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 -- 172.21.15.120:0/2653952240 wait complete. 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 Processor -- start 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.899+0000 7f2204d4f640 1 -- start start 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2200101830 0x7f220019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220010f3e0 0x7f22001a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2200115020 con 0x7f2200102150 2024-09-10T22:51:42.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2200114ea0 con 0x7f220010f3e0 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22001151a0 con 0x7f2200101830 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34276/0 (socket says 172.21.15.120:34276) 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 -- 172.21.15.120:0/801030431 learned_addr learned my addr 172.21.15.120:0/801030431 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fed9d640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220010f3e0 0x7f22001a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fe59c640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2200101830 0x7f220019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 -- 172.21.15.120:0/801030431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2200101830 msgr2=0x7f220019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2200101830 0x7f220019b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 -- 172.21.15.120:0/801030431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220010f3e0 msgr2=0x7f22001a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220010f3e0 0x7f22001a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 -- 172.21.15.120:0/801030431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22001a16a0 con 0x7f2200102150 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b4a0 tx=0x7f21e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21f77fe640 1 -- 172.21.15.120:0/801030431 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8014070 con 0x7f2200102150 2024-09-10T22:51:42.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21f77fe640 1 -- 172.21.15.120:0/801030431 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e8010ac0 con 0x7f2200102150 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001a1990 con 0x7f2200102150 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f21f77fe640 1 -- 172.21.15.120:0/801030431 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017c30 con 0x7f2200102150 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.900+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a80d0 con 0x7f2200102150 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f21f77fe640 1 -- 172.21.15.120:0/801030431 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f21e8010610 con 0x7f2200102150 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 Processor -- start 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 -- start start 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x557266220110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x55726622c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557266197c10 con 0x55726622c5e0 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557266187040 con 0x55726621fd30 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.903+0000 7f2207124740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55726618ae50 con 0x557266220650 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34286/0 (socket says 172.21.15.120:34286) 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 -- 172.21.15.120:0/3627835444 learned_addr learned my addr 172.21.15.120:0/3627835444 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fed9d640 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x557266220110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fe59c640 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x55726622c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 -- 172.21.15.120:0/3627835444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 msgr2=0x55726622c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x55726622c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 -- 172.21.15.120:0/3627835444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 msgr2=0x557266220110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x557266220110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 -- 172.21.15.120:0/3627835444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557265ee5540 con 0x55726622c5e0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21f77fe640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 0x7f21ec0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fed9d640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 0x7f21ec0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21e80ebc20 tx=0x7f21e80ebd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd00ece09bb1733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21f77fe640 1 -- 172.21.15.120:0/801030431 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f21e80e5730 con 0x7f2200102150 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.904+0000 7f21f4ff9640 1 -- 172.21.15.120:0/3627835444 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80a83b0 con 0x55726622c5e0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f21f4ff9640 1 -- 172.21.15.120:0/3627835444 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e8017360 con 0x55726622c5e0 2024-09-10T22:51:42.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f21f4ff9640 1 -- 172.21.15.120:0/3627835444 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80092b0 con 0x55726622c5e0 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 -- 172.21.15.120:0/3627835444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 msgr2=0x55726622e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f21fed9d640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 0x7f21ec0b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b9d0 tx=0x7f21f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21e80ebc20 tx=0x7f21e80ebd10 comp rx=0 tx=0).stop 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 -- 172.21.15.120:0/3627835444 shutdown_connections 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622e9d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x55726622c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 --2- 172.21.15.120:0/3627835444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x557266220110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 -- 172.21.15.120:0/3627835444 >> 172.21.15.120:0/3627835444 conn(0x55726621db60 msgr2=0x55726621ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 -- 172.21.15.120:0/3627835444 shutdown_connections 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.905+0000 7f2207124740 1 -- 172.21.15.120:0/3627835444 wait complete. 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 -- start start 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 Processor -- start 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x5572662295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x557266229ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557265fe85f0 con 0x55726622c5e0 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557266122010 con 0x55726621fd30 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55726622f290 con 0x557266220650 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f2207124740 10 client.0 _ll_register_callbacks cb 0x557266181340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34294/0 (socket says 172.21.15.120:34294) 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 -- 172.21.15.120:0/954548468 learned_addr learned my addr 172.21.15.120:0/954548468 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fe59c640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x5572662295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 -- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 msgr2=0x557266229ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x557266229ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 -- 172.21.15.120:0/954548468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 msgr2=0x5572662295a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x5572662295a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.906+0000 7f21fed9d640 1 -- 172.21.15.120:0/954548468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726614fa00 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21fed9d640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ff90 tx=0x7f21f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742eb578fc5f1f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f801b050 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f21f8015820 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557266199b60 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f21f8015a20 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 10 client.15099 fetch_fsmap learned FSMap version 19 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55726623ae80 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557266199b60 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f21f8020c50 con 0x55726622c5e0 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 10 client.15099 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:51:42.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 20 client.15099 populate_metadata read hostname 'smithi120' 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55726623d5f0 0x55726623ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 4 client.15099 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.907+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557266195610 con 0x55726623d5f0 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.908+0000 7f21fe59c640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55726623d5f0 0x55726623ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.908+0000 7f21e5ffb640 10 client.15099 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.908+0000 7f21fe59c640 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55726623d5f0 0x55726623ea60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539b6802312672ef server_cookie=28e6275d98ace6c7 in_seq=0 out_seq=0 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.909+0000 7f21e5ffb640 1 -- 172.21.15.120:0/954548468 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557266195610 con 0x55726623d5f0 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.909+0000 7f21e5ffb640 10 client.15099 handle_command_reply: tid=0 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.909+0000 7f2207124740 1 client.15099 shutdown 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 msgr2=0x55726622a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ff90 tx=0x7f21f80046c0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 shutdown_connections 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55726623d5f0 0x55726623ea60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622a020 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557266220650 0x557266229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 --2- 172.21.15.120:0/954548468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55726621fd30 0x5572662295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 >> 172.21.15.120:0/954548468 conn(0x55726621db60 msgr2=0x557266223700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f21e5ffb640 0 client.15099 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 shutdown_connections 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 1 -- 172.21.15.120:0/954548468 wait complete. 2024-09-10T22:51:42.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.910+0000 7f2207124740 20 client.15099 trim_cache size 0 max 16384 2024-09-10T22:51:42.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 Processor -- start 2024-09-10T22:51:42.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 -- start start 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726621fd30 0x55726621ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266223c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55726623cda0 con 0x55726622c5e0 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55726623d470 con 0x557266220650 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f2207124740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55726623db40 con 0x55726621fd30 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34296/0 (socket says 172.21.15.120:34296) 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 -- 172.21.15.120:0/3107622303 learned_addr learned my addr 172.21.15.120:0/3107622303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 -- 172.21.15.120:0/3107622303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726621fd30 msgr2=0x55726621ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726621fd30 0x55726621ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 -- 172.21.15.120:0/3107622303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 msgr2=0x557266223c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266223c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 -- 172.21.15.120:0/3107622303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726621e5e0 con 0x55726622c5e0 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.928+0000 7f21fed9d640 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21f800bfd0 tx=0x7f21f800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75e63edae6f2a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f21e7fff640 1 -- 172.21.15.120:0/3107622303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f803a070 con 0x55726622c5e0 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f21e7fff640 1 -- 172.21.15.120:0/3107622303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f8025070 con 0x55726622c5e0 2024-09-10T22:51:42.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f21e7fff640 1 -- 172.21.15.120:0/3107622303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f801b040 con 0x55726622c5e0 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 -- 172.21.15.120:0/3107622303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 msgr2=0x55726622b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21f800bfd0 tx=0x7f21f800b7e0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 -- 172.21.15.120:0/3107622303 shutdown_connections 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726622c5e0 0x55726622b2e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266223c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 --2- 172.21.15.120:0/3107622303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726621fd30 0x55726621ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 -- 172.21.15.120:0/3107622303 >> 172.21.15.120:0/3107622303 conn(0x55726621db60 msgr2=0x55726622abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 -- 172.21.15.120:0/3107622303 shutdown_connections 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.929+0000 7f2207124740 1 -- 172.21.15.120:0/3107622303 wait complete. 2024-09-10T22:51:42.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 -- start start 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 Processor -- start 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266241500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726622c5e0 0x557266246750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55726623e8e0 con 0x55726621fd30 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55726623e210 con 0x557266220650 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55726623efb0 con 0x55726622c5e0 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f2207124740 10 client.0 _ll_register_callbacks cb 0x7f21f800fc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34300/0 (socket says 172.21.15.120:34300) 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 -- 172.21.15.120:0/2592286846 learned_addr learned my addr 172.21.15.120:0/2592286846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fed9d640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726622c5e0 0x557266246750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266241500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 -- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726622c5e0 msgr2=0x557266246750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726622c5e0 0x557266246750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 -- 172.21.15.120:0/2592286846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 msgr2=0x557266241500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266241500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fe59c640 1 -- 172.21.15.120:0/2592286846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557266246c90 con 0x55726621fd30 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.930+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266241500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21fe59c640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f21f0030040 tx=0x7f21f002f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da63240a5c1d3b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0034040 con 0x55726621fd30 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f21f002fa70 con 0x55726621fd30 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f8025070 con 0x55726621fd30 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f21f0054070 con 0x55726621fd30 2024-09-10T22:51:42.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 10 client.15105 fetch_fsmap learned FSMap version 19 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5572662aa650 con 0x55726621fd30 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f8025070 con 0x55726621fd30 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f21f00046c0 con 0x55726621fd30 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 10 client.15105 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 20 client.15105 populate_metadata read hostname 'smithi120' 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5572662af1e0 0x5572662adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 4 client.15105 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.931+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "status"}) -- 0x557266192a40 con 0x5572662af1e0 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5572662af1e0 0x5572662adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21fdd9b640 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5572662af1e0 0x5572662adef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135a9b9a75d15faa server_cookie=43e53322e96cbbb6 in_seq=0 out_seq=0 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21e5ffb640 10 client.15105 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:51:42.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21e5ffb640 1 -- 172.21.15.120:0/2592286846 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557266192a40 con 0x5572662af1e0 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21e5ffb640 10 client.15105 handle_command_reply: tid=0 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 client.15105 shutdown 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 msgr2=0x557266240fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f21f0030040 tx=0x7f21f002f500 comp rx=0 tx=0).stop 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 shutdown_connections 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5572662af1e0 0x5572662adef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55726622c5e0 0x557266246750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557266220650 0x557266241500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f21e5ffb640 0 client.15105 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 --2- 172.21.15.120:0/2592286846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55726621fd30 0x557266240fc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.932+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 >> 172.21.15.120:0/2592286846 conn(0x55726621db60 msgr2=0x557266229d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 shutdown_connections 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2207124740 1 -- 172.21.15.120:0/2592286846 wait complete. 2024-09-10T22:51:42.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2207124740 20 client.15105 trim_cache size 0 max 16384 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "d2838936-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24550, 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 74.916245889999999, 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 19, 2024-09-10T22:51:42.940 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 77, 2024-09-10T22:51:42.941 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:51:42.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 92.976249538999994 2024-09-10T22:51:42.941 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:51:42.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 msgr2=0x7f21ec0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 0x7f21ec0b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b9d0 tx=0x7f21f8008040 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 msgr2=0x7f220019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b4a0 tx=0x7f21e800b960 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 shutdown_connections 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21ec0b5370 0x7f21ec0b7830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220010f3e0 0x7f22001a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2200102150 0x7f220019bb30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 --2- 172.21.15.120:0/801030431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2200101830 0x7f220019b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 >> 172.21.15.120:0/801030431 conn(0x7f22000fd700 msgr2=0x7f22000ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 shutdown_connections 2024-09-10T22:51:42.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:42.933+0000 7f2204d4f640 1 -- 172.21.15.120:0/801030431 wait complete. 2024-09-10T22:51:43.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:42 smithi176 ceph-mon[34982]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:43.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:42 smithi120 ceph-mon[31247]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:43.200 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:42 smithi153 ceph-mon[34951]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:43.702 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T22:51:43.702 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:51:43.711 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T22:51:43.711 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T22:51:43.956 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:44.370 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:44 smithi120 ceph-mon[31247]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 -- 172.21.15.120:0/2863440141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 msgr2=0x7fa11010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2863440141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11010aed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa100009cd0 tx=0x7fa10002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 -- 172.21.15.120:0/2863440141 shutdown_connections 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2863440141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11010aed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2863440141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2863440141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 -- 172.21.15.120:0/2863440141 >> 172.21.15.120:0/2863440141 conn(0x7fa1100fe320 msgr2=0x7fa110100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:44.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 -- 172.21.15.120:0/2863440141 shutdown_connections 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.448+0000 7fa114d1a640 1 -- 172.21.15.120:0/2863440141 wait complete. 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 Processor -- start 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 -- start start 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010e710 con 0x7fa110104550 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010e590 con 0x7fa110103b90 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa114d1a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010e890 con 0x7fa110102a30 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44512/0 (socket says 172.21.15.120:44512) 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 -- 172.21.15.120:0/2336483547 learned_addr learned my addr 172.21.15.120:0/2336483547 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:44.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10dd9b640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10e59c640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 -- 172.21.15.120:0/2336483547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 msgr2=0x7fa110198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 -- 172.21.15.120:0/2336483547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 msgr2=0x7fa110199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.449+0000 7fa10ed9d640 1 -- 172.21.15.120:0/2336483547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11019a950 con 0x7fa110104550 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa10dd9b640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa10e59c640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa10ed9d640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1000317d0 tx=0x7fa100003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100004510 con 0x7fa110104550 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11019abe0 con 0x7fa110104550 2024-09-10T22:51:44.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1000046b0 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10003b770 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.450+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a1320 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.451+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa110109ce0 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.453+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa10004c020 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.454+0000 7fa1077fe640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 0x7fa0fc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.454+0000 7fa10e59c640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 0x7fa0fc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.454+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fa10002e030 con 0x7fa110104550 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.454+0000 7fa10e59c640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 0x7fa0fc0b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa108006fd0 tx=0x7fa108008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:44.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.457+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa1000cdee0 con 0x7fa110104550 2024-09-10T22:51:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:44 smithi176 ceph-mon[34982]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:44.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.647+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa110063590 con 0x7fa0fc0b5370 2024-09-10T22:51:44.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.649+0000 7fa1077fe640 1 -- 172.21.15.120:0/2336483547 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa110063590 con 0x7fa0fc0b5370 2024-09-10T22:51:44.650 INFO:teuthology.orchestra.run.smithi120.stdout:Backend: cephadm 2024-09-10T22:51:44.650 INFO:teuthology.orchestra.run.smithi120.stdout:Available: Yes 2024-09-10T22:51:44.650 INFO:teuthology.orchestra.run.smithi120.stdout:Paused: No 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.650+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 msgr2=0x7fa0fc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.650+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 0x7fa0fc0b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa108006fd0 tx=0x7fa108008040 comp rx=0 tx=0).stop 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.650+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 msgr2=0x7fa11019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.650+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1000317d0 tx=0x7fa100003f10 comp rx=0 tx=0).stop 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 shutdown_connections 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0fc0b5370 0x7fa0fc0b7830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa110104550 0x7fa11019a0b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa110103b90 0x7fa110199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 --2- 172.21.15.120:0/2336483547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa110102a30 0x7fa110198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:44.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 >> 172.21.15.120:0/2336483547 conn(0x7fa1100fe320 msgr2=0x7fa110106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:44.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 shutdown_connections 2024-09-10T22:51:44.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:44.651+0000 7fa114d1a640 1 -- 172.21.15.120:0/2336483547 wait complete. 2024-09-10T22:51:44.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:44 smithi153 ceph-mon[34951]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:45.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T22:51:45.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:45 smithi120 ceph-mon[31247]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.530 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:45 smithi176 ceph-mon[34982]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:45 smithi153 ceph-mon[34951]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- 172.21.15.120:0/3447875613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb078105ba0 msgr2=0x7fb078108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/3447875613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb078105ba0 0x7fb078108010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009ca0 tx=0x7fb06c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- 172.21.15.120:0/3447875613 shutdown_connections 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/3447875613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078108750 0x7fb07810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/3447875613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb078105ba0 0x7fb078108010 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/3447875613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb07806c100 0x7fb078105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- 172.21.15.120:0/3447875613 >> 172.21.15.120:0/3447875613 conn(0x7fb0780fd200 msgr2=0x7fb0780ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- 172.21.15.120:0/3447875613 shutdown_connections 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- 172.21.15.120:0/3447875613 wait complete. 2024-09-10T22:51:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 Processor -- start 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.967+0000 7fb07fb5c640 1 -- start start 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 0x7fb078199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 0x7fb07819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810e700 con 0x7fb07806c100 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810e580 con 0x7fb078108750 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810e880 con 0x7fb078105ba0 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44524/0 (socket says 172.21.15.120:44524) 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 -- 172.21.15.120:0/1777552207 learned_addr learned my addr 172.21.15.120:0/1777552207 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d0f7640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 0x7fb078199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07e0f9640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 0x7fb07819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:45.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 -- 172.21.15.120:0/1777552207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 msgr2=0x7fb078199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 0x7fb078199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 -- 172.21.15.120:0/1777552207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 msgr2=0x7fb07819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 0x7fb07819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 -- 172.21.15.120:0/1777552207 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07819aa00 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d0f7640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 0x7fb078199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07e0f9640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 0x7fb07819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07d8f8640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c890 tx=0x7fb07000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070004220 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0700043c0 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07819acf0 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.968+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070004a00 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.969+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0781a13d0 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.969+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0780fecf0 con 0x7fb07806c100 2024-09-10T22:51:45.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.971+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fb0700079b0 con 0x7fb07806c100 2024-09-10T22:51:45.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.971+0000 7fb076ffd640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 0x7fb0600b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:45.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.971+0000 7fb07d0f7640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 0x7fb0600b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:45.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.971+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fb0700e5f00 con 0x7fb07806c100 2024-09-10T22:51:45.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.972+0000 7fb07d0f7640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 0x7fb0600b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb06c031b50 tx=0x7fb06c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:45.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:45.974+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb070014030 con 0x7fb07806c100 2024-09-10T22:51:46.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.158+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb078063590 con 0x7fb0600b53f0 2024-09-10T22:51:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.170+0000 7fb076ffd640 1 -- 172.21.15.120:0/1777552207 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb078063590 con 0x7fb0600b53f0 2024-09-10T22:51:46.171 INFO:teuthology.orchestra.run.smithi120.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T22:51:46.171 INFO:teuthology.orchestra.run.smithi120.stdout:mds.a smithi120 running (90s) 13s ago 89s 16.8M - 19.3.0-4854-g8f219c47 589dd18248e4 a339a6377d0d 2024-09-10T22:51:46.171 INFO:teuthology.orchestra.run.smithi120.stdout:mds.b smithi153 running (87s) 17s ago 86s 13.6M - 19.3.0-4854-g8f219c47 589dd18248e4 fc0eed164bbf 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.c smithi176 running (101s) 17s ago 101s 18.7M - 19.3.0-4854-g8f219c47 589dd18248e4 ec03888c56e8 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.d smithi120 running (109s) 13s ago 108s 13.8M - 19.3.0-4854-g8f219c47 589dd18248e4 928e6f513689 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.e smithi153 running (106s) 17s ago 105s 14.6M - 19.3.0-4854-g8f219c47 589dd18248e4 3573f74b2e93 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.f smithi176 running (103s) 17s ago 103s 14.6M - 19.3.0-4854-g8f219c47 589dd18248e4 a0730dd0bf86 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.g smithi120 running (93s) 13s ago 92s 16.1M - 19.3.0-4854-g8f219c47 589dd18248e4 0c133ff96da9 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.h smithi153 running (98s) 17s ago 98s 13.8M - 19.3.0-4854-g8f219c47 589dd18248e4 862be588a002 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.i smithi176 running (96s) 17s ago 96s 28.6M - 19.3.0-4854-g8f219c47 589dd18248e4 b5fbd3613a2f 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.j smithi120 running (112s) 13s ago 111s 16.8M - 19.3.0-4854-g8f219c47 589dd18248e4 121daa7bfd4d 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.k smithi153 running (115s) 17s ago 114s 14.6M - 19.3.0-4854-g8f219c47 589dd18248e4 11c10d3cdf2c 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mds.l smithi176 running (117s) 17s ago 117s 14.7M - 19.3.0-4854-g8f219c47 589dd18248e4 c6f065d6ac54 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.x smithi120 *:9283,8765 running (9m) 13s ago 9m 525M - 19.3.0-4854-g8f219c47 589dd18248e4 cbfc6d50a4ef 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.y smithi153 *:8443,8765 running (7m) 17s ago 7m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 ad94726c6825 2024-09-10T22:51:46.172 INFO:teuthology.orchestra.run.smithi120.stdout:mgr.z smithi176 *:8443,8765 running (7m) 17s ago 7m 457M - 19.3.0-4854-g8f219c47 589dd18248e4 7e60e763bb8d 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:mon.a smithi120 running (9m) 13s ago 9m 51.3M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 065319f19c34 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:mon.b smithi153 running (7m) 17s ago 7m 41.4M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 380d42ee9192 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:mon.c smithi176 running (7m) 17s ago 7m 41.8M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 bdd517694a84 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 smithi120 running (6m) 13s ago 6m 55.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 6fa927c931cb 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 smithi153 running (6m) 17s ago 6m 55.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 bba203d55b4f 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 smithi176 running (6m) 17s ago 6m 59.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 fb9c48a8a596 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 smithi120 running (5m) 13s ago 5m 50.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 cef1dcd187f3 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 smithi153 running (5m) 17s ago 5m 45.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 bacdf0805b3a 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 smithi176 running (4m) 17s ago 4m 47.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 78728c6dded3 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 smithi120 running (4m) 13s ago 4m 52.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e07b7d07d5ef 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 smithi153 running (3m) 17s ago 3m 50.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 368742f6335d 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.8 smithi176 running (3m) 17s ago 3m 55.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 788a69201c07 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.9 smithi120 running (3m) 13s ago 3m 50.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 3b24c2a068a3 2024-09-10T22:51:46.173 INFO:teuthology.orchestra.run.smithi120.stdout:osd.10 smithi153 running (2m) 17s ago 2m 46.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 ba4f35ac7d96 2024-09-10T22:51:46.174 INFO:teuthology.orchestra.run.smithi120.stdout:osd.11 smithi176 running (2m) 17s ago 2m 44.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f2696778bdd3 2024-09-10T22:51:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 msgr2=0x7fb0600b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 0x7fb0600b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb06c031b50 tx=0x7fb06c0093f0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 msgr2=0x7fb078198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c890 tx=0x7fb07000cd50 comp rx=0 tx=0).stop 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 shutdown_connections 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0600b53f0 0x7fb0600b78b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb078108750 0x7fb07819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb078105ba0 0x7fb078199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 --2- 172.21.15.120:0/1777552207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb07806c100 0x7fb078198ed0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 >> 172.21.15.120:0/1777552207 conn(0x7fb0780fd200 msgr2=0x7fb078103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 shutdown_connections 2024-09-10T22:51:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:46.172+0000 7fb07fb5c640 1 -- 172.21.15.120:0/1777552207 wait complete. 2024-09-10T22:51:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:46 smithi176 ceph-mon[34982]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:46 smithi176 ceph-mon[34982]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:46.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:46 smithi120 ceph-mon[31247]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:46.575 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:46 smithi120 ceph-mon[31247]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:46 smithi153 ceph-mon[34951]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:46 smithi153 ceph-mon[34951]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:46.897 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T22:51:47.146 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 -- 172.21.15.120:0/4207982694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 msgr2=0x7f7590109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 --2- 172.21.15.120:0/4207982694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f7590109de0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7588009cd0 tx=0x7f758802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 -- 172.21.15.120:0/4207982694 shutdown_connections 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 --2- 172.21.15.120:0/4207982694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f7590109de0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 --2- 172.21.15.120:0/4207982694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590102aa0 0x7f7590102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 --2- 172.21.15.120:0/4207982694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590101940 0x7f7590101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 -- 172.21.15.120:0/4207982694 >> 172.21.15.120:0/4207982694 conn(0x7f75900fd210 msgr2=0x7f75900ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 -- 172.21.15.120:0/4207982694 shutdown_connections 2024-09-10T22:51:47.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.676+0000 7f7597da6640 1 -- 172.21.15.120:0/4207982694 wait complete. 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 Processor -- start 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 -- start start 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 0x7f7590198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590102aa0 0x7f7590199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f759010d620 con 0x7f7590103460 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f759010d4a0 con 0x7f7590101940 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7597da6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f759010d7a0 con 0x7f7590102aa0 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44538/0 (socket says 172.21.15.120:44538) 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 -- 172.21.15.120:0/1781398842 learned_addr learned my addr 172.21.15.120:0/1781398842 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7595b42640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 0x7f7590198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7595341640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590102aa0 0x7f7590199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:47.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 -- 172.21.15.120:0/1781398842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590102aa0 msgr2=0x7f7590199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590102aa0 0x7f7590199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 -- 172.21.15.120:0/1781398842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 msgr2=0x7f7590198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 0x7f7590198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 -- 172.21.15.120:0/1781398842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759019aa50 con 0x7f7590103460 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7595b42640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 0x7f7590198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.677+0000 7f7596343640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f75880097b0 tx=0x7f7588003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.678+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588004450 con 0x7f7590103460 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.678+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7588003e30 con 0x7f7590103460 2024-09-10T22:51:47.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.678+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759019ace0 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.678+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588045670 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.678+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a1420 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.679+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75901a15f0 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.680+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f7588030070 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.681+0000 7f7586ffd640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 0x7f757c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.681+0000 7f7595b42640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 0x7f757c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.681+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f758802e030 con 0x7f7590103460 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.681+0000 7f7595b42640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 0x7f757c0b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:47.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.685+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f758804a060 con 0x7f7590103460 2024-09-10T22:51:47.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.868+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f75900ff380 con 0x7f757c0b53f0 2024-09-10T22:51:47.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.874+0000 7f7586ffd640 1 -- 172.21.15.120:0/1781398842 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f75900ff380 con 0x7f757c0b53f0 2024-09-10T22:51:47.875 INFO:teuthology.orchestra.run.smithi120.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T22:51:47.875 INFO:teuthology.orchestra.run.smithi120.stdout:mds.all 12/12 19s ago 2m smithi120=a;smithi120=d;smithi120=g;smithi120=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi176=c;smithi176=f;smithi176=i;smithi176=l;count:12 2024-09-10T22:51:47.875 INFO:teuthology.orchestra.run.smithi120.stdout:mgr 3/3 19s ago 7m smithi120=x;smithi153=y;smithi176=z;count:3 2024-09-10T22:51:47.875 INFO:teuthology.orchestra.run.smithi120.stdout:mon 3/3 19s ago 8m smithi120:172.21.15.120=a;smithi153:172.21.15.153=b;smithi176:172.21.15.176=c;count:3 2024-09-10T22:51:47.876 INFO:teuthology.orchestra.run.smithi120.stdout:osd 12 19s ago - 2024-09-10T22:51:47.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 msgr2=0x7f757c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:47.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 0x7f757c0b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).stop 2024-09-10T22:51:47.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 msgr2=0x7f759019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f75880097b0 tx=0x7f7588003c10 comp rx=0 tx=0).stop 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 shutdown_connections 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f757c0b53f0 0x7f757c0b78b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7590103460 0x7f759019a1b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7590102aa0 0x7f7590199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 --2- 172.21.15.120:0/1781398842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7590101940 0x7f7590198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 >> 172.21.15.120:0/1781398842 conn(0x7f75900fd210 msgr2=0x7f75900fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 shutdown_connections 2024-09-10T22:51:47.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:47.876+0000 7f7597da6640 1 -- 172.21.15.120:0/1781398842 wait complete. 2024-09-10T22:51:48.570 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:48 smithi120 ceph-mon[31247]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:48.570 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:48 smithi120 ceph-mon[31247]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:48 smithi176 ceph-mon[34982]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:48 smithi176 ceph-mon[34982]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:48.596 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T22:51:48.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:48 smithi153 ceph-mon[34951]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:48.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:48 smithi153 ceph-mon[34951]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:48.841 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:49.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 -- 172.21.15.120:0/1955475328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098101910 msgr2=0x7f7098101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:49.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 --2- 172.21.15.120:0/1955475328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098101910 0x7f7098101d10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7088009930 tx=0x7f708802a420 comp rx=0 tx=0).stop 2024-09-10T22:51:49.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 -- 172.21.15.120:0/1955475328 shutdown_connections 2024-09-10T22:51:49.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 --2- 172.21.15.120:0/1955475328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098103430 0x7f7098109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 --2- 172.21.15.120:0/1955475328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098102a70 0x7f7098102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 --2- 172.21.15.120:0/1955475328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098101910 0x7f7098101d10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 -- 172.21.15.120:0/1955475328 >> 172.21.15.120:0/1955475328 conn(0x7f70980fd1e0 msgr2=0x7f70980ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 -- 172.21.15.120:0/1955475328 shutdown_connections 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 -- 172.21.15.120:0/1955475328 wait complete. 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.311+0000 7f709d971640 1 Processor -- start 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 -- start start 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098101910 0x7f7098197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098102a70 0x7f7098198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f709810d590 con 0x7f7098103430 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f709810d410 con 0x7f7098102a70 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f709d971640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f709810d710 con 0x7f7098101910 2024-09-10T22:51:49.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44554/0 (socket says 172.21.15.120:44554) 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 -- 172.21.15.120:0/4275773773 learned_addr learned my addr 172.21.15.120:0/4275773773 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f7096ffd640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098101910 0x7f7098197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70967fc640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098102a70 0x7f7098198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 -- 172.21.15.120:0/4275773773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098101910 msgr2=0x7f7098197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098101910 0x7f7098197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 -- 172.21.15.120:0/4275773773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098102a70 msgr2=0x7f7098198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098102a70 0x7f7098198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 -- 172.21.15.120:0/4275773773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7098199850 con 0x7f7098103430 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.312+0000 7f70977fe640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7090007cb0 tx=0x7f709000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.313+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709000eb30 con 0x7f7098103430 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.313+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709000ecd0 con 0x7f7098103430 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.313+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7098199b40 con 0x7f7098103430 2024-09-10T22:51:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.313+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900153f0 con 0x7f7098103430 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.313+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709806ac80 con 0x7f7098103430 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.314+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7098199da0 con 0x7f7098103430 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.316+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f7090014070 con 0x7f7098103430 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.316+0000 7f709c96f640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 0x7f70800b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.317+0000 7f7096ffd640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 0x7f70800b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.317+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f70900e7920 con 0x7f7098103430 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.317+0000 7f7096ffd640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 0x7f70800b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f7088031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.320+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f70900ae6b0 con 0x7f7098103430 2024-09-10T22:51:49.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.499+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f70981087b0 con 0x7f70800b5370 2024-09-10T22:51:49.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.500+0000 7f709c96f640 1 -- 172.21.15.120:0/4275773773 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f70981087b0 con 0x7f70800b5370 2024-09-10T22:51:49.501 INFO:teuthology.orchestra.run.smithi120.stdout:HOST ADDR LABELS STATUS 2024-09-10T22:51:49.502 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120 172.21.15.120 2024-09-10T22:51:49.502 INFO:teuthology.orchestra.run.smithi120.stdout:smithi153 172.21.15.153 2024-09-10T22:51:49.502 INFO:teuthology.orchestra.run.smithi120.stdout:smithi176 172.21.15.176 2024-09-10T22:51:49.502 INFO:teuthology.orchestra.run.smithi120.stdout:3 hosts in cluster 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 msgr2=0x7f70800b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 0x7f70800b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f7088031430 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 msgr2=0x7f7098198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7090007cb0 tx=0x7f709000e420 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 shutdown_connections 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70800b5370 0x7f70800b7830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7098103430 0x7f7098198fb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7098102a70 0x7f7098198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 --2- 172.21.15.120:0/4275773773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7098101910 0x7f7098197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.502+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 >> 172.21.15.120:0/4275773773 conn(0x7f70980fd1e0 msgr2=0x7f7098105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.503+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 shutdown_connections 2024-09-10T22:51:49.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:49.503+0000 7f709d971640 1 -- 172.21.15.120:0/4275773773 wait complete. 2024-09-10T22:51:50.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T22:51:50.434 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:50.555 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:50 smithi120 ceph-mon[31247]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:50.555 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:50 smithi120 ceph-mon[31247]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:50 smithi176 ceph-mon[34982]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:50 smithi176 ceph-mon[34982]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:50 smithi153 ceph-mon[34951]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:50 smithi153 ceph-mon[34951]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 -- 172.21.15.120:0/3120716583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44102aa0 msgr2=0x7fde44102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 --2- 172.21.15.120:0/3120716583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44102aa0 0x7fde44102f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fde34009930 tx=0x7fde3402a420 comp rx=0 tx=0).stop 2024-09-10T22:51:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 -- 172.21.15.120:0/3120716583 shutdown_connections 2024-09-10T22:51:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 --2- 172.21.15.120:0/3120716583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 0x7fde44109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:50.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 --2- 172.21.15.120:0/3120716583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44102aa0 0x7fde44102f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 --2- 172.21.15.120:0/3120716583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44101940 0x7fde44101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 -- 172.21.15.120:0/3120716583 >> 172.21.15.120:0/3120716583 conn(0x7fde440fd210 msgr2=0x7fde440ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 -- 172.21.15.120:0/3120716583 shutdown_connections 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 -- 172.21.15.120:0/3120716583 wait complete. 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.929+0000 7fde49067640 1 Processor -- start 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 -- start start 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44102aa0 0x7fde4419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 0x7fde441a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d5c0 con 0x7fde44101940 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d440 con 0x7fde44103460 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d740 con 0x7fde44102aa0 2024-09-10T22:51:50.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44572/0 (socket says 172.21.15.120:44572) 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 -- 172.21.15.120:0/3529316840 learned_addr learned my addr 172.21.15.120:0/3529316840 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde4259c640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44102aa0 0x7fde4419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 -- 172.21.15.120:0/3529316840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44102aa0 msgr2=0x7fde4419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44102aa0 0x7fde4419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde4359e640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 0x7fde441a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 -- 172.21.15.120:0/3529316840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 msgr2=0x7fde441a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 0x7fde441a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 -- 172.21.15.120:0/3529316840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde441a0aa0 con 0x7fde44101940 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde42d9d640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00d770 tx=0x7fde2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde44101940 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde441a0d90 con 0x7fde44101940 2024-09-10T22:51:50.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.930+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2c0054d0 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.931+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c004c30 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.931+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a1210 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.931+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde44101d40 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.934+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fde2c004dd0 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.934+0000 7fde3bfff640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 0x7fde300b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.934+0000 7fde4259c640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 0x7fde300b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.934+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fde2c0e5bc0 con 0x7fde44101940 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.935+0000 7fde4259c640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 0x7fde300b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde34009900 tx=0x7fde34031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:50.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:50.937+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fde2c0ac950 con 0x7fde44101940 2024-09-10T22:51:51.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.120+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fde44063650 con 0x7fde300b5370 2024-09-10T22:51:51.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.122+0000 7fde3bfff640 1 -- 172.21.15.120:0/3529316840 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fde44063650 con 0x7fde300b5370 2024-09-10T22:51:51.123 INFO:teuthology.orchestra.run.smithi120.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T22:51:51.123 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T22:51:51.123 INFO:teuthology.orchestra.run.smithi120.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T22:51:51.123 INFO:teuthology.orchestra.run.smithi120.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 msgr2=0x7fde300b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 0x7fde300b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde34009900 tx=0x7fde34031430 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 msgr2=0x7fde4406bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00d770 tx=0x7fde2c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 shutdown_connections 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde300b5370 0x7fde300b7830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde44103460 0x7fde441a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde44102aa0 0x7fde4419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 --2- 172.21.15.120:0/3529316840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde44101940 0x7fde4406bd80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 >> 172.21.15.120:0/3529316840 conn(0x7fde440fd210 msgr2=0x7fde44105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 shutdown_connections 2024-09-10T22:51:51.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:51.124+0000 7fde49067640 1 -- 172.21.15.120:0/3529316840 wait complete. 2024-09-10T22:51:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:51 smithi120 ceph-mon[31247]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:51 smithi176 ceph-mon[34982]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:51.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:51 smithi153 ceph-mon[34951]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:51:51.847 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:51:51.857 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T22:51:51.857 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T22:51:52.106 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:52.557 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:52 smithi120 ceph-mon[31247]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:52 smithi176 ceph-mon[34982]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:52.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 -- 172.21.15.120:0/1321660765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034101910 msgr2=0x7fb034101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 --2- 172.21.15.120:0/1321660765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034101910 0x7fb034101d10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb028009930 tx=0x7fb02802a420 comp rx=0 tx=0).stop 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 -- 172.21.15.120:0/1321660765 shutdown_connections 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 --2- 172.21.15.120:0/1321660765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034103430 0x7fb034109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 --2- 172.21.15.120:0/1321660765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034102a70 0x7fb034102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 --2- 172.21.15.120:0/1321660765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034101910 0x7fb034101d10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.635+0000 7fb03b07e640 1 -- 172.21.15.120:0/1321660765 >> 172.21.15.120:0/1321660765 conn(0x7fb0340fd1e0 msgr2=0x7fb0340ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- 172.21.15.120:0/1321660765 shutdown_connections 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- 172.21.15.120:0/1321660765 wait complete. 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 Processor -- start 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- start start 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034101910 0x7fb03406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:52.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 0x7fb0341a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d570 con 0x7fb034102a70 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d3f0 con 0x7fb034101910 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb03b07e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d6f0 con 0x7fb034103430 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb033fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb033fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44582/0 (socket says 172.21.15.120:44582) 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.636+0000 7fb033fff640 1 -- 172.21.15.120:0/2657364610 learned_addr learned my addr 172.21.15.120:0/2657364610 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb03961b640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 0x7fb0341a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb038e1a640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034101910 0x7fb03406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 -- 172.21.15.120:0/2657364610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 msgr2=0x7fb0341a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 0x7fb0341a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 -- 172.21.15.120:0/2657364610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034101910 msgr2=0x7fb03406b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:52.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034101910 0x7fb03406b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 -- 172.21.15.120:0/2657364610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0341a09f0 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb03961b640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 0x7fb0341a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb033fff640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c890 tx=0x7fb02000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020004220 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0341a0ce0 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0200043c0 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020004a00 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.637+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0341a1210 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.638+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb034108d60 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.640+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fb0200079b0 con 0x7fb034102a70 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.640+0000 7fb031ffb640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 0x7fb0240b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.640+0000 7fb038e1a640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 0x7fb0240b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:52.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.640+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fb0200e5bc0 con 0x7fb034102a70 2024-09-10T22:51:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.641+0000 7fb038e1a640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 0x7fb0240b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb028009230 tx=0x7fb028000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:52.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.644+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb020014030 con 0x7fb034102a70 2024-09-10T22:51:52.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:52 smithi153 ceph-mon[34951]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:52.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.857+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb03402e5c0 con 0x7fb034102a70 2024-09-10T22:51:52.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.857+0000 7fb031ffb640 1 -- 172.21.15.120:0/2657364610 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3058 (secure 0 0 0) 0x7fb0200ac8f0 con 0x7fb034102a70 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:e19 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:btime 2024-09-10T22:50:37:038717+0000 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:legacy client fscid: 1 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:Filesystem 'cephfs' (1) 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:fs_name cephfs 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 19 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-10T22:50:27.621402+0000 2024-09-10T22:51:52.859 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-10T22:50:36.741193+0000 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:tableserver 0 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:root 0 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:session_timeout 300 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:session_autoclose 300 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:max_file_size 1099511627776 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:max_xattr_size 65536 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:required_client_features {} 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:last_failure 0 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:last_failure_osd_epoch 0 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:max_mds 1 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:in 0 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:up {0=24550} 2024-09-10T22:51:52.860 INFO:teuthology.orchestra.run.smithi120.stdout:failed 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:damaged 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:stopped 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:data_pools [3] 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:metadata_pool 2 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:inline_data disabled 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:balancer 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:bal_rank_mask -1 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:standby_count_wanted 0 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:qdb_cluster leader: 24550 members: 24550 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.i{0:24550} state up:active seq 6 addr [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c{0:24535} state up:standby-replay seq 1 addr [v2:172.21.15.176:6836/1046536521,v1:172.21.15.176:6837/1046536521] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:52.861 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:Standby daemons: 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.j{-1:14673} state up:standby seq 1 addr [v2:172.21.15.120:6834/1392508310,v1:172.21.15.120:6835/1392508310] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d{-1:14685} state up:standby seq 1 addr [v2:172.21.15.120:6836/3319509867,v1:172.21.15.120:6837/3319509867] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.g{-1:14736} state up:standby seq 1 addr [v2:172.21.15.120:6838/987205566,v1:172.21.15.120:6839/987205566] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a{-1:14748} state up:standby seq 1 addr [v2:172.21.15.120:6840/420929141,v1:172.21.15.120:6841/420929141] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.k{-1:24467} state up:standby seq 1 addr [v2:172.21.15.153:6832/811375010,v1:172.21.15.153:6833/811375010] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.e{-1:24485} state up:standby seq 1 addr [v2:172.21.15.153:6834/1790379992,v1:172.21.15.153:6835/1790379992] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.l{-1:24502} state up:standby seq 1 addr [v2:172.21.15.176:6832/1099927074,v1:172.21.15.176:6833/1099927074] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.h{-1:24503} state up:standby seq 1 addr [v2:172.21.15.153:6836/1833265650,v1:172.21.15.153:6837/1833265650] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b{-1:24524} state up:standby seq 1 addr [v2:172.21.15.153:6838/2945283653,v1:172.21.15.153:6839/2945283653] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.862 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.f{-1:24526} state up:standby seq 1 addr [v2:172.21.15.176:6834/4233369048,v1:172.21.15.176:6835/4233369048] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 msgr2=0x7fb0240b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 0x7fb0240b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb028009230 tx=0x7fb028000bc0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 msgr2=0x7fb03419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c890 tx=0x7fb02000cd50 comp rx=0 tx=0).stop 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 shutdown_connections 2024-09-10T22:51:52.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0240b5370 0x7fb0240b7830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb034103430 0x7fb0341a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb034102a70 0x7fb03419db40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 --2- 172.21.15.120:0/2657364610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb034101910 0x7fb03406b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.860+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 >> 172.21.15.120:0/2657364610 conn(0x7fb0340fd1e0 msgr2=0x7fb034105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.861+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 shutdown_connections 2024-09-10T22:51:52.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:52.861+0000 7fb03b07e640 1 -- 172.21.15.120:0/2657364610 wait complete. 2024-09-10T22:51:53.537 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:53 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2657364610' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:51:53.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T22:51:53.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:53 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2657364610' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:51:53.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:53 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2657364610' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:51:53.811 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:54.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.261+0000 7f4a83eed640 1 -- 172.21.15.120:0/844643023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c102a30 msgr2=0x7f4a7c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:54.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- 172.21.15.120:0/844643023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c102a30 0x7f4a7c102e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009930 tx=0x7f4a7802a420 comp rx=0 tx=0).stop 2024-09-10T22:51:54.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- 172.21.15.120:0/844643023 shutdown_connections 2024-09-10T22:51:54.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- 172.21.15.120:0/844643023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c104550 0x7f4a7c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- 172.21.15.120:0/844643023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c103b90 0x7f4a7c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- 172.21.15.120:0/844643023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c102a30 0x7f4a7c102e30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- 172.21.15.120:0/844643023 >> 172.21.15.120:0/844643023 conn(0x7f4a7c0fe320 msgr2=0x7f4a7c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- 172.21.15.120:0/844643023 shutdown_connections 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- 172.21.15.120:0/844643023 wait complete. 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 Processor -- start 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- start start 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 0x7f4a7c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c103b90 0x7f4a7c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10e710 con 0x7f4a7c104550 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10e590 con 0x7f4a7c103b90 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.262+0000 7f4a83eed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10e890 con 0x7f4a7c102a30 2024-09-10T22:51:54.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56040/0 (socket says 172.21.15.120:56040) 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 -- 172.21.15.120:0/3818433243 learned_addr learned my addr 172.21.15.120:0/3818433243 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a81488640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c103b90 0x7f4a7c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a81c89640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 0x7f4a7c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 -- 172.21.15.120:0/3818433243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 msgr2=0x7f4a7c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 0x7f4a7c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 -- 172.21.15.120:0/3818433243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c103b90 msgr2=0x7f4a7c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c103b90 0x7f4a7c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 -- 172.21.15.120:0/3818433243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c19a940 con 0x7f4a7c104550 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a81c89640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 0x7f4a7c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a8248a640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400cce0 tx=0x7f4a7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:54.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400f450 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7400faa0 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74018b70 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c19ac30 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.263+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c1a1310 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.264+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c109db0 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.265+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f4a7400f5f0 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.266+0000 7f4a6effd640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 0x7f4a680b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.266+0000 7f4a81c89640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 0x7f4a680b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.266+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f4a740e6610 con 0x7f4a7c104550 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.267+0000 7f4a81c89640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 0x7f4a680b7830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a780096c0 tx=0x7f4a78000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.269+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a74014030 con 0x7f4a7c104550 2024-09-10T22:51:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:54 smithi120 ceph-mon[31247]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.447+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4a7c02e810 con 0x7f4a7c104550 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.448+0000 7f4a6effd640 1 -- 172.21.15.120:0/3818433243 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5286 (secure 0 0 0) 0x7f4a740ad430 con 0x7f4a7c104550 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:epoch 77 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:fsid d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:created 2024-09-10T22:42:26.976696+0000 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:modified 2024-09-10T22:51:15.147106+0000 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:crush_version 26 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:full_ratio 0.95 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:backfillfull_ratio 0.9 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:nearfull_ratio 0.85 2024-09-10T22:51:54.450 INFO:teuthology.orchestra.run.smithi120.stdout:require_min_compat_client luminous 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:min_compat_client luminous 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:require_osd_release squid 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:stretch_mode_enabled false 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:max_osd 12 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6802/2370065006,v1:172.21.15.120:6803/2370065006] [v2:172.21.15.120:6804/2370065006,v1:172.21.15.120:6805/2370065006] exists,up f1918891-2bc5-4422-b3ff-eaf029ff3161 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.1 up in weight 1 up_from 13 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/2556493584,v1:172.21.15.153:6801/2556493584] [v2:172.21.15.153:6802/2556493584,v1:172.21.15.153:6803/2556493584] exists,up 8cb0107b-f7e1-48bd-8916-bd98f0075444 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.2 up in weight 1 up_from 17 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/1904007733,v1:172.21.15.176:6801/1904007733] [v2:172.21.15.176:6802/1904007733,v1:172.21.15.176:6803/1904007733] exists,up fcf71dbb-0b3e-4778-9735-44deb5142b73 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.3 up in weight 1 up_from 24 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6810/1845328035,v1:172.21.15.120:6811/1845328035] [v2:172.21.15.120:6812/1845328035,v1:172.21.15.120:6813/1845328035] exists,up 422694c8-3c8d-4cae-987f-829b757f91eb 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.4 up in weight 1 up_from 30 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/2098592997,v1:172.21.15.153:6809/2098592997] [v2:172.21.15.153:6810/2098592997,v1:172.21.15.153:6811/2098592997] exists,up b5b35613-0bf7-4f22-8640-e73ac586d2a8 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.5 up in weight 1 up_from 35 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/2804384590,v1:172.21.15.176:6809/2804384590] [v2:172.21.15.176:6810/2804384590,v1:172.21.15.176:6811/2804384590] exists,up c3110d3e-1def-43cf-afae-30ca5fb29647 2024-09-10T22:51:54.451 INFO:teuthology.orchestra.run.smithi120.stdout:osd.6 up in weight 1 up_from 41 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6818/2500547812,v1:172.21.15.120:6819/2500547812] [v2:172.21.15.120:6820/2500547812,v1:172.21.15.120:6821/2500547812] exists,up 8062f8c3-d12f-4df5-8ef5-4286a273c161 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.7 up in weight 1 up_from 46 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/1702194586,v1:172.21.15.153:6817/1702194586] [v2:172.21.15.153:6818/1702194586,v1:172.21.15.153:6819/1702194586] exists,up 44778377-474e-4621-bd97-1cb74a97869f 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.8 up in weight 1 up_from 53 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/2769671146,v1:172.21.15.176:6817/2769671146] [v2:172.21.15.176:6818/2769671146,v1:172.21.15.176:6819/2769671146] exists,up 9699f84b-810d-46f9-b062-ea48812dafea 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.9 up in weight 1 up_from 59 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6826/283565809,v1:172.21.15.120:6827/283565809] [v2:172.21.15.120:6828/283565809,v1:172.21.15.120:6829/283565809] exists,up 6f143c9a-4955-4d2d-9670-42a73378a865 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.10 up in weight 1 up_from 64 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/718699262,v1:172.21.15.153:6825/718699262] [v2:172.21.15.153:6826/718699262,v1:172.21.15.153:6827/718699262] exists,up 7801ec3e-95f4-4dcb-9134-5d9dbf72145f 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/2240071367,v1:172.21.15.176:6825/2240071367] [v2:172.21.15.176:6826/2240071367,v1:172.21.15.176:6827/2240071367] exists,up bbced0d0-788b-48aa-9271-9cc721683b01 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.a [2,11] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.c [2,4] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 2.1d [8,11] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.1 [4,9] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.13 [4,9] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.17 [4,8] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:pg_upmap_items 3.1b [5,9] 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6800/3131521405 expires 2024-09-11T22:43:14.065833+0000 2024-09-10T22:51:54.452 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/1124015461 expires 2024-09-11T22:43:14.065833+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/959921448 expires 2024-09-11T22:43:14.065833+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/1170023971 expires 2024-09-11T22:43:14.065833+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6801/872559236 expires 2024-09-11T22:42:50.195302+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6800/872559236 expires 2024-09-11T22:42:50.195302+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:6801/3131521405 expires 2024-09-11T22:43:14.065833+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/2219003248 expires 2024-09-11T22:42:50.195302+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/2465372008 expires 2024-09-11T22:42:50.195302+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:blocklist 172.21.15.120:0/1252718387 expires 2024-09-11T22:42:50.195302+0000 2024-09-10T22:51:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 msgr2=0x7f4a680b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 0x7f4a680b7830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a780096c0 tx=0x7f4a78000bc0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 msgr2=0x7f4a7c19a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400cce0 tx=0x7f4a7400a620 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 shutdown_connections 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4a680b5370 0x7f4a680b7830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a7c104550 0x7f4a7c19a0a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a7c103b90 0x7f4a7c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 --2- 172.21.15.120:0/3818433243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a7c102a30 0x7f4a7c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 >> 172.21.15.120:0/3818433243 conn(0x7f4a7c0fe320 msgr2=0x7f4a7c106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 shutdown_connections 2024-09-10T22:51:54.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:54.451+0000 7f4a83eed640 1 -- 172.21.15.120:0/3818433243 wait complete. 2024-09-10T22:51:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:54 smithi176 ceph-mon[34982]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:54.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:54 smithi153 ceph-mon[34951]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:55.082 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T22:51:55.095 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:55.343 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:55.496 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:55 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3818433243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:51:55.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:55 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3818433243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:51:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:55 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3818433243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 -- 172.21.15.120:0/387332935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460101cd0 msgr2=0x7f74601020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 --2- 172.21.15.120:0/387332935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460101cd0 0x7f74601020b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f745c009900 tx=0x7f745c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 -- 172.21.15.120:0/387332935 shutdown_connections 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 --2- 172.21.15.120:0/387332935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460108060 0x7f746010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 --2- 172.21.15.120:0/387332935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 0x7f7460107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 --2- 172.21.15.120:0/387332935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460101cd0 0x7f74601020b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 -- 172.21.15.120:0/387332935 >> 172.21.15.120:0/387332935 conn(0x7f74600fd700 msgr2=0x7f74600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 -- 172.21.15.120:0/387332935 shutdown_connections 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.818+0000 7f7468076640 1 -- 172.21.15.120:0/387332935 wait complete. 2024-09-10T22:51:55.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 Processor -- start 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 -- start start 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460101cd0 0x7f74601981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 0x7f74601986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7460111d20 con 0x7f7460108060 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7460111ba0 con 0x7f74601076e0 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.819+0000 7f7468076640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7460111ea0 con 0x7f7460101cd0 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56058/0 (socket says 172.21.15.120:56058) 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 -- 172.21.15.120:0/2308246698 learned_addr learned my addr 172.21.15.120:0/2308246698 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7465e12640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460101cd0 0x7f74601981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7465611640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 0x7f74601986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:55.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 -- 172.21.15.120:0/2308246698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460101cd0 msgr2=0x7f74601981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460101cd0 0x7f74601981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 -- 172.21.15.120:0/2308246698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 msgr2=0x7f74601986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 0x7f74601986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 -- 172.21.15.120:0/2308246698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746019e260 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7466613640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f745800b970 tx=0x7f745800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458019070 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7458014440 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746019e550 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580075d0 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.820+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601a4c90 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.821+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74601020b0 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.822+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f74580145f0 con 0x7f7460108060 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.823+0000 7f7456ffd640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 0x7f744c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:55.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.823+0000 7f7465e12640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 0x7f744c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:55.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.823+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f74580e5410 con 0x7f7460108060 2024-09-10T22:51:55.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.824+0000 7f7465e12640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 0x7f744c0b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f745c031440 tx=0x7f745c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:55.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:55.827+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74580abb50 con 0x7f7460108060 2024-09-10T22:51:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.037+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f746002e5c0 con 0x7f7460108060 2024-09-10T22:51:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.037+0000 7f7456ffd640 1 -- 172.21.15.120:0/2308246698 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f74580b0580 con 0x7f7460108060 2024-09-10T22:51:56.040 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:56.040 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:51:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 msgr2=0x7f744c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 0x7f744c0b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f745c031440 tx=0x7f745c005e80 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 msgr2=0x7f746019da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f745800b970 tx=0x7f745800bd70 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 shutdown_connections 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f744c0b5370 0x7f744c0b7830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7460108060 0x7f746019da90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74601076e0 0x7f74601986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 --2- 172.21.15.120:0/2308246698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7460101cd0 0x7f74601981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:56.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 >> 172.21.15.120:0/2308246698 conn(0x7f74600fd700 msgr2=0x7f74600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:56.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 shutdown_connections 2024-09-10T22:51:56.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:56.040+0000 7f7468076640 1 -- 172.21.15.120:0/2308246698 wait complete. 2024-09-10T22:51:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:56 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2308246698' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:56 smithi176 ceph-mon[34982]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:56.600 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:56 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2308246698' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:56.600 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:56 smithi153 ceph-mon[34951]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:56.612 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:56 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2308246698' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:56.612 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:56 smithi120 ceph-mon[31247]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:51:56.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:57.219 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/3381243847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d881045c0 msgr2=0x7f0d8810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/3381243847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d881045c0 0x7f0d8810cab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009ca0 tx=0x7f0d7802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/3381243847 shutdown_connections 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/3381243847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d8810d100 0x7f0d8810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/3381243847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d881045c0 0x7f0d8810cab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/3381243847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d88104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/3381243847 >> 172.21.15.120:0/3381243847 conn(0x7f0d880fd700 msgr2=0x7f0d880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/3381243847 shutdown_connections 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/3381243847 wait complete. 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 Processor -- start 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 -- start start 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d8819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:57.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881045c0 0x7f0d8819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.702+0000 7f0d8eb6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8eb6a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88112dd0 con 0x7f0d8810d100 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8eb6a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88112c50 con 0x7f0d88103ca0 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8eb6a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88112f50 con 0x7f0d881045c0 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56072/0 (socket says 172.21.15.120:56072) 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 -- 172.21.15.120:0/307458206 learned_addr learned my addr 172.21.15.120:0/307458206 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d87fff640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881045c0 0x7f0d8819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:57.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8c906640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d8819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 -- 172.21.15.120:0/307458206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881045c0 msgr2=0x7f0d8819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881045c0 0x7f0d8819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 -- 172.21.15.120:0/307458206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 msgr2=0x7f0d8819b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d8819b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 -- 172.21.15.120:0/307458206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881a1730 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8c906640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d8819b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8d107640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d750 tx=0x7f0d7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70017070 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d700054d0 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881a1a20 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004c60 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.703+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a82b0 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.704+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8806b1b0 con 0x7f0d8810d100 2024-09-10T22:51:57.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.706+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0d70004e00 con 0x7f0d8810d100 2024-09-10T22:51:57.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.707+0000 7f0d85ffb640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 0x7f0d740b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:57.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.707+0000 7f0d8c906640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 0x7f0d740b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:57.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.707+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f0d700e5b20 con 0x7f0d8810d100 2024-09-10T22:51:57.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.707+0000 7f0d8c906640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 0x7f0d740b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006fd0 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:57.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.710+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0d700ac930 con 0x7f0d8810d100 2024-09-10T22:51:57.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.922+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d88104080 con 0x7f0d8810d100 2024-09-10T22:51:57.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.923+0000 7f0d85ffb640 1 -- 172.21.15.120:0/307458206 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f0d700ac230 con 0x7f0d8810d100 2024-09-10T22:51:57.925 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:57.925 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:51:57.926 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 msgr2=0x7f0d740b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 0x7f0d740b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006fd0 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 msgr2=0x7f0d881a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d750 tx=0x7f0d7000dc10 comp rx=0 tx=0).stop 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 shutdown_connections 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0d740b53f0 0x7f0d740b78b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d8810d100 0x7f0d881a0f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881045c0 0x7f0d8819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 --2- 172.21.15.120:0/307458206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d88103ca0 0x7f0d8819b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:57.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 >> 172.21.15.120:0/307458206 conn(0x7f0d880fd700 msgr2=0x7f0d880ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:57.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.925+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 shutdown_connections 2024-09-10T22:51:57.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:57.926+0000 7f0d8eb6a640 1 -- 172.21.15.120:0/307458206 wait complete. 2024-09-10T22:51:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/307458206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/307458206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:58.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/307458206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:58.650 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:51:58.896 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:59.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:58 smithi176 ceph-mon[34982]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:59.324 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:58 smithi120 ceph-mon[31247]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:59.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4254207266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 msgr2=0x7f9c5810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4254207266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c5810cb10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c02a410 comp rx=0 tx=0).stop 2024-09-10T22:51:59.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4254207266 shutdown_connections 2024-09-10T22:51:59.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4254207266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c5810d0e0 0x7f9c5810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4254207266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c5810cb10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4254207266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c58103b30 0x7f9c58103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.401+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4254207266 >> 172.21.15.120:0/4254207266 conn(0x7f9c580fd700 msgr2=0x7f9c580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4254207266 shutdown_connections 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4254207266 wait complete. 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 Processor -- start 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 -- start start 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 0x7f9c58199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 0x7f9c5819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.402+0000 7f9c5da0f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112da0 con 0x7f9c58104530 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5da0f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112c20 con 0x7f9c5810d0e0 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5da0f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112f20 con 0x7f9c58103b30 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56100/0 (socket says 172.21.15.120:56100) 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 -- 172.21.15.120:0/4211905934 learned_addr learned my addr 172.21.15.120:0/4211905934 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c56ffd640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 0x7f9c58199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c577fe640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 0x7f9c5819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 -- 172.21.15.120:0/4211905934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 msgr2=0x7f9c58199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 0x7f9c58199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 -- 172.21.15.120:0/4211905934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 msgr2=0x7f9c5819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 0x7f9c5819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 -- 172.21.15.120:0/4211905934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5819f420 con 0x7f9c58104530 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c56ffd640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 0x7f9c58199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c577fe640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 0x7f9c5819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c567fc640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c03f030 con 0x7f9c58104530 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c4c003800 con 0x7f9c58104530 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c035070 con 0x7f9c58104530 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5819f6b0 con 0x7f9c58104530 2024-09-10T22:51:59.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.403+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581a5dd0 con 0x7f9c58104530 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.404+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58103f10 con 0x7f9c58104530 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.406+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f9c4c031b10 con 0x7f9c58104530 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.407+0000 7f9c5ca0d640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 0x7f9c400b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.407+0000 7f9c56ffd640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 0x7f9c400b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.407+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f9c4c1069b0 con 0x7f9c58104530 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.407+0000 7f9c56ffd640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 0x7f9c400b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c48007eb0 tx=0x7f9c480073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.410+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9c4c0cd740 con 0x7f9c58104530 2024-09-10T22:51:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:58 smithi153 ceph-mon[34951]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:51:59.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.587+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c5819a4a0 con 0x7f9c58104530 2024-09-10T22:51:59.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.588+0000 7f9c5ca0d640 1 -- 172.21.15.120:0/4211905934 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22746 (secure 0 0 0) 0x7f9c5819a4a0 con 0x7f9c58104530 2024-09-10T22:51:59.591 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:51:59.591 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":77,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:51:15.147106+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 msgr2=0x7f9c400b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 0x7f9c400b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c48007eb0 tx=0x7f9c480073c0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 msgr2=0x7f9c581998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 shutdown_connections 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9c400b5370 0x7f9c400b7830 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c5810d0e0 0x7f9c5819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c58104530 0x7f9c581998b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 --2- 172.21.15.120:0/4211905934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c58103b30 0x7f9c58199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 >> 172.21.15.120:0/4211905934 conn(0x7f9c580fd700 msgr2=0x7f9c580fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 shutdown_connections 2024-09-10T22:51:59.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:51:59.591+0000 7f9c5da0f640 1 -- 172.21.15.120:0/4211905934 wait complete. 2024-09-10T22:52:00.280 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:51:59 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4211905934' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:00.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:50:27.621402+0000', 'modified': '2024-09-10T22:50:36.741193+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6837/1046536521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 1046536521}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 1046536521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/3141658633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 3141658633}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 3141658633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24550, 'qdb_cluster': [24550]} 2024-09-10T22:52:00.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:52:00.308 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T22:52:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:51:59 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4211905934' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:00.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:51:59 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4211905934' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:00.554 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:01.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 -- 172.21.15.120:0/2845347803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d0101cd0 msgr2=0x7f73d01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/2845347803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d0101cd0 0x7f73d01020b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009900 tx=0x7f73c002a410 comp rx=0 tx=0).stop 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 -- 172.21.15.120:0/2845347803 shutdown_connections 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/2845347803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 0x7f73d010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/2845347803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01076e0 0x7f73d0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/2845347803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d0101cd0 0x7f73d01020b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 -- 172.21.15.120:0/2845347803 >> 172.21.15.120:0/2845347803 conn(0x7f73d00fd700 msgr2=0x7f73d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 -- 172.21.15.120:0/2845347803 shutdown_connections 2024-09-10T22:52:01.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.094+0000 7f73d5bf0640 1 -- 172.21.15.120:0/2845347803 wait complete. 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 Processor -- start 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 -- start start 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d0101cd0 0x7f73d01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 0x7f73d019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111d20 con 0x7f73d01076e0 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111ba0 con 0x7f73d0108060 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73d5bf0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0111ea0 con 0x7f73d0101cd0 2024-09-10T22:52:01.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56116/0 (socket says 172.21.15.120:56116) 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 -- 172.21.15.120:0/1614315938 learned_addr learned my addr 172.21.15.120:0/1614315938 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73cffff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 0x7f73d019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 -- 172.21.15.120:0/1614315938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d0101cd0 msgr2=0x7f73d01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d0101cd0 0x7f73d01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 -- 172.21.15.120:0/1614315938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 msgr2=0x7f73d019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 0x7f73d019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 -- 172.21.15.120:0/1614315938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d019f390 con 0x7f73d01076e0 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.095+0000 7f73c7fff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b4a0 tx=0x7f73b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.096+0000 7f73cd7fa640 1 -- 172.21.15.120:0/1614315938 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8014070 con 0x7f73d01076e0 2024-09-10T22:52:01.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.096+0000 7f73cd7fa640 1 -- 172.21.15.120:0/1614315938 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b8010ac0 con 0x7f73d01076e0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.096+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d019f680 con 0x7f73d01076e0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.096+0000 7f73cd7fa640 1 -- 172.21.15.120:0/1614315938 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8017c30 con 0x7f73d01076e0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.096+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a5dc0 con 0x7f73d01076e0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.098+0000 7f73cd7fa640 1 -- 172.21.15.120:0/1614315938 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f73b8010610 con 0x7f73d01076e0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.098+0000 7f73d7fc5740 1 Processor -- start 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.098+0000 7f73d7fc5740 1 -- start start 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b94d21bc10 con 0x55b94d2b49f0 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b94d20b040 con 0x55b94d2a4650 2024-09-10T22:52:01.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b94d20ee50 con 0x55b94d2a3d30 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73c7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:56418/0 (socket says 172.21.15.120:56418) 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 -- 172.21.15.120:0/320763201 learned_addr learned my addr 172.21.15.120:0/320763201 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cffff640 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 -- 172.21.15.120:0/320763201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 msgr2=0x55b94d2a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 -- 172.21.15.120:0/320763201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 msgr2=0x55b94d2b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 -- 172.21.15.120:0/320763201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94cf69540 con 0x55b94d2a4650 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73c7fff640 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cd7fa640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 0x7f73bc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cffff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 0x7f73bc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73c0032bd0 tx=0x7f73c0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610eee6fb255a688 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.099+0000 7f73cd7fa640 1 -- 172.21.15.120:0/1614315938 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f73b80e5320 con 0x7f73d01076e0 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73c5ffb640 1 -- 172.21.15.120:0/320763201 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0040070 con 0x55b94d2a4650 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73c5ffb640 1 -- 172.21.15.120:0/320763201 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00316a0 con 0x55b94d2a4650 2024-09-10T22:52:01.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73c5ffb640 1 -- 172.21.15.120:0/320763201 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0035040 con 0x55b94d2a4650 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 -- 172.21.15.120:0/320763201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 msgr2=0x55b94d2b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73cffff640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 0x7f73bc0b78b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73c8008e40 tx=0x7f73c80085f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73c0032bd0 tx=0x7f73c0000bc0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 -- 172.21.15.120:0/320763201 shutdown_connections 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2b44b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/320763201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.100+0000 7f73d7fc5740 1 -- 172.21.15.120:0/320763201 >> 172.21.15.120:0/320763201 conn(0x55b94d2a1b60 msgr2=0x55b94d2a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 -- 172.21.15.120:0/320763201 shutdown_connections 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 -- 172.21.15.120:0/320763201 wait complete. 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 -- start start 2024-09-10T22:52:01.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 Processor -- start 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 0x55b94d2b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b94d06c5f0 con 0x55b94d2a3d30 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.101+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b94d1a6010 con 0x55b94d2b49f0 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2b7640 con 0x55b94d2a4650 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73d7fc5740 10 client.0 _ll_register_callbacks cb 0x55b94d205340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56126/0 (socket says 172.21.15.120:56126) 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 -- 172.21.15.120:0/2995138139 learned_addr learned my addr 172.21.15.120:0/2995138139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cffff640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 0x55b94d2b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73c7fff640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 -- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 msgr2=0x55b94d2b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 -- 172.21.15.120:0/2995138139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 msgr2=0x55b94d2b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 0x55b94d2b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 -- 172.21.15.120:0/2995138139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94d1d3a00 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cffff640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 0x55b94d2b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73c00038a0 tx=0x7f73c002aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea3d3cdd0bf5d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0035040 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f73c0045660 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c00316a0 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f73c0040040 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73d7fc5740 10 client.15177 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.102+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b94d2c3160 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00316a0 con 0x55b94d2a3d30 2024-09-10T22:52:01.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f73c004f970 con 0x55b94d2a3d30 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 10 client.15177 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 10 client.15177 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 20 client.15177 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d2c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 4 client.15177 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b94d21db60 con 0x55b94d2c44b0 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.103+0000 7f73c7fff640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d2c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.104+0000 7f73c7fff640 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d2c6690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36597d4200f7b620 server_cookie=fe09b3a2e5be2cd5 in_seq=0 out_seq=0 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.104+0000 7f73aeffd640 10 client.15177 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.104+0000 7f73aeffd640 1 -- 172.21.15.120:0/2995138139 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b94d21db60 con 0x55b94d2c44b0 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.104+0000 7f73aeffd640 10 client.15177 handle_command_reply: tid=0 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.104+0000 7f73d7fc5740 1 client.15177 shutdown 2024-09-10T22:52:01.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 msgr2=0x55b94d2b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73c00038a0 tx=0x7f73c002aba0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 shutdown_connections 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d2c6690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73aeffd640 0 client.15177 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2b49f0 0x55b94d2b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2995138139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2a3d30 0x55b94d2b1920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 >> 172.21.15.120:0/2995138139 conn(0x55b94d2a1b60 msgr2=0x55b94d2a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 shutdown_connections 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2995138139 wait complete. 2024-09-10T22:52:01.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.105+0000 7f73d7fc5740 20 client.15177 trim_cache size 0 max 16384 2024-09-10T22:52:01.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.122+0000 7f73d7fc5740 1 Processor -- start 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.122+0000 7f73d7fc5740 1 -- start start 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a3d30 0x55b94d2af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c49d0 con 0x55b94d2b49f0 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c50a0 con 0x55b94d2a3d30 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c5770 con 0x55b94d2a4650 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56128/0 (socket says 172.21.15.120:56128) 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 -- 172.21.15.120:0/252310770 learned_addr learned my addr 172.21.15.120:0/252310770 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73c7fff640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a3d30 0x55b94d2af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 -- 172.21.15.120:0/252310770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 msgr2=0x55b94d2b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 -- 172.21.15.120:0/252310770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a3d30 msgr2=0x55b94d2af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a3d30 0x55b94d2af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 -- 172.21.15.120:0/252310770 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94d2b4100 con 0x55b94d2b49f0 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73c7fff640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73cffff640 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73c801e4a0 tx=0x7f73c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13888a148420bd1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73c57fa640 1 -- 172.21.15.120:0/252310770 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c802b070 con 0x55b94d2b49f0 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73c57fa640 1 -- 172.21.15.120:0/252310770 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c801ccf0 con 0x55b94d2b49f0 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.123+0000 7f73c57fa640 1 -- 172.21.15.120:0/252310770 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8026d80 con 0x55b94d2b49f0 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- 172.21.15.120:0/252310770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 msgr2=0x55b94d2a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73c801e4a0 tx=0x7f73c8014000 comp rx=0 tx=0).stop 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- 172.21.15.120:0/252310770 shutdown_connections 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a14f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a4650 0x55b94d2b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/252310770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a3d30 0x55b94d2af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- 172.21.15.120:0/252310770 >> 172.21.15.120:0/252310770 conn(0x55b94d2a1b60 msgr2=0x55b94d2a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- 172.21.15.120:0/252310770 shutdown_connections 2024-09-10T22:52:01.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- 172.21.15.120:0/252310770 wait complete. 2024-09-10T22:52:01.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.124+0000 7f73d7fc5740 1 -- start start 2024-09-10T22:52:01.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 Processor -- start 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c6510 con 0x55b94d2b49f0 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c5e40 con 0x55b94d2a4650 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b94d2c6be0 con 0x55b94d2a3d30 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73d7fc5740 10 client.0 _ll_register_callbacks cb 0x7f73b80e8d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56144/0 (socket says 172.21.15.120:56144) 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 -- 172.21.15.120:0/2320683396 learned_addr learned my addr 172.21.15.120:0/2320683396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:01.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73c7fff640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 -- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 msgr2=0x55b94d2aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 -- 172.21.15.120:0/2320683396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 msgr2=0x55b94d2af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73cffff640 1 -- 172.21.15.120:0/2320683396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94d2af910 con 0x55b94d2b49f0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.125+0000 7f73c7fff640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73cffff640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f73c8026900 tx=0x7f73c8026d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5506ac637e261d13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8035030 con 0x55b94d2b49f0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f73c8004a40 con 0x55b94d2b49f0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c801ccf0 con 0x55b94d2b49f0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f73c802b040 con 0x55b94d2b49f0 2024-09-10T22:52:01.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 10 client.15183 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b94d2cde50 con 0x55b94d2b49f0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c801ccf0 con 0x55b94d2b49f0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f73c802e030 con 0x55b94d2b49f0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 10 client.15183 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 10 client.15183 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 20 client.15183 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d335690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 4 client.15183 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.126+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "status"}) -- 0x55b94d219610 con 0x55b94d2c44b0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d335690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73cf7fe640 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d335690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d3c4336aa06fe0 server_cookie=f855cf7731fbefe9 in_seq=0 out_seq=0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73aeffd640 10 client.15183 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73aeffd640 1 -- 172.21.15.120:0/2320683396 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b94d219610 con 0x55b94d2c44b0 2024-09-10T22:52:01.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73aeffd640 10 client.15183 handle_command_reply: tid=0 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 client.15183 shutdown 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 msgr2=0x55b94d2a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f73c8026900 tx=0x7f73c8026d30 comp rx=0 tx=0).stop 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 shutdown_connections 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b94d2c44b0 0x55b94d335690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b94d2b49f0 0x55b94d2a4d80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b94d2a4650 0x55b94d2af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 --2- 172.21.15.120:0/2320683396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b94d2a3d30 0x55b94d2aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73aeffd640 0 client.15183 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 >> 172.21.15.120:0/2320683396 conn(0x55b94d2a1b60 msgr2=0x55b94d2b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.127+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 shutdown_connections 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d7fc5740 1 -- 172.21.15.120:0/2320683396 wait complete. 2024-09-10T22:52:01.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d7fc5740 20 client.15183 trim_cache size 0 max 16384 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "cluster_fsid": "d2838936-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "whoami": 0, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "id": 24550, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "want_state": "up:active", 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "state": "up:active", 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "fs_name": "cephfs", 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "rank_uptime": 93.112126148000002, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "mdsmap_epoch": 19, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch": 77, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:52:01.136 INFO:teuthology.orchestra.run.smithi120.stdout: "uptime": 111.172129434 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 msgr2=0x7f73bc0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 0x7f73bc0b78b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73c8008e40 tx=0x7f73c80085f0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 msgr2=0x7f73d0199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b4a0 tx=0x7f73b800b960 comp rx=0 tx=0).stop 2024-09-10T22:52:01.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 shutdown_connections 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73bc0b53f0 0x7f73bc0b78b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d0108060 0x7f73d019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d01076e0 0x7f73d0199820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 --2- 172.21.15.120:0/1614315938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d0101cd0 0x7f73d01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 >> 172.21.15.120:0/1614315938 conn(0x7f73d00fd700 msgr2=0x7f73d006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 shutdown_connections 2024-09-10T22:52:01.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:01.128+0000 7f73d5bf0640 1 -- 172.21.15.120:0/1614315938 wait complete. 2024-09-10T22:52:01.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:00 smithi176 ceph-mon[34982]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:01.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:00 smithi120 ceph-mon[31247]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:01.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:00 smithi153 ceph-mon[34951]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:01.884 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T22:52:01.909 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T22:52:01.909 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T22:52:01.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:01.909 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T22:52:01.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:01.926 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-10T22:52:01.993 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-09-10T22:52:01.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:52:01.995 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-10T22:52:01.995 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:52:01.995 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:01.995 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:52:02.283 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:02.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:02 smithi120 ceph-mon[31247]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:52:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:02 smithi176 ceph-mon[34982]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:52:02.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:02 smithi153 ceph-mon[34951]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 -- 172.21.15.120:0/1729184193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 msgr2=0x7f48a41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 --2- 172.21.15.120:0/1729184193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a41031a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009900 tx=0x7f48a002a410 comp rx=0 tx=0).stop 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 -- 172.21.15.120:0/1729184193 shutdown_connections 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 --2- 172.21.15.120:0/1729184193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 0x7f48a410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 --2- 172.21.15.120:0/1729184193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 0x7f48a4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 --2- 172.21.15.120:0/1729184193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a41031a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 -- 172.21.15.120:0/1729184193 >> 172.21.15.120:0/1729184193 conn(0x7f48a40fe830 msgr2=0x7f48a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 -- 172.21.15.120:0/1729184193 shutdown_connections 2024-09-10T22:52:02.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.754+0000 7f48ab85a640 1 -- 172.21.15.120:0/1729184193 wait complete. 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 Processor -- start 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 -- start start 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 0x7f48a4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 0x7f48a419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112e50 con 0x7f48a4102dc0 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112cd0 con 0x7f48a41087d0 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48ab85a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112fd0 con 0x7f48a4109150 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56160/0 (socket says 172.21.15.120:56160) 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 -- 172.21.15.120:0/2141226457 learned_addr learned my addr 172.21.15.120:0/2141226457 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a8df5640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 0x7f48a4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a9df7640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 0x7f48a419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:02.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 -- 172.21.15.120:0/2141226457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 msgr2=0x7f48a419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 0x7f48a419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 -- 172.21.15.120:0/2141226457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 msgr2=0x7f48a4199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 0x7f48a4199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 -- 172.21.15.120:0/2141226457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a419f4f0 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.755+0000 7f48a95f6640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006010 tx=0x7f48a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.756+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a003f030 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.756+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48a0003800 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.756+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a419f780 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.756+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0035070 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.756+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41a5ea0 con 0x7f48a4102dc0 2024-09-10T22:52:02.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.757+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a4103220 con 0x7f48a4102dc0 2024-09-10T22:52:02.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.758+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f48a0031b10 con 0x7f48a4102dc0 2024-09-10T22:52:02.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.759+0000 7f48967fc640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 0x7f48900b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:02.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.759+0000 7f48a8df5640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 0x7f48900b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:02.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.759+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f48a0106950 con 0x7f48a4102dc0 2024-09-10T22:52:02.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.760+0000 7f48a8df5640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 0x7f48900b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48a41082f0 tx=0x7f4898008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:02.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.762+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f48a00cd6e0 con 0x7f48a4102dc0 2024-09-10T22:52:02.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.977+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48a402e5c0 con 0x7f48a4102dc0 2024-09-10T22:52:02.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.978+0000 7f48967fc640 1 -- 172.21.15.120:0/2141226457 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f48a00ccfe0 con 0x7f48a4102dc0 2024-09-10T22:52:02.980 INFO:teuthology.orchestra.run.smithi120.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 msgr2=0x7f48900b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 0x7f48900b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48a41082f0 tx=0x7f4898008040 comp rx=0 tx=0).stop 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 msgr2=0x7f48a4199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006010 tx=0x7f48a00042f0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 shutdown_connections 2024-09-10T22:52:02.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48900b53f0 0x7f48900b78b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48a4109150 0x7f48a419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a41087d0 0x7f48a4199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 --2- 172.21.15.120:0/2141226457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48a4102dc0 0x7f48a4199440 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 >> 172.21.15.120:0/2141226457 conn(0x7f48a40fe830 msgr2=0x7f48a4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 shutdown_connections 2024-09-10T22:52:02.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:02.980+0000 7f48ab85a640 1 -- 172.21.15.120:0/2141226457 wait complete. 2024-09-10T22:52:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:03 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2141226457' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:52:03.623 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:03 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2141226457' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:52:03.652 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:52:03.652 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:52:03.652 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-09-10T22:52:03.652 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:52:03.653 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:03.653 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:52:03.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:52:03.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:52:03.653 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:52:03.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:03.653 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-09-10T22:52:03.669 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:52:03.669 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:52:03.669 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:52:03.669 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-09-10T22:52:03.670 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link noprefixroute 2024-09-10T22:52:03.671 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:52:03.671 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:52:03.671 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:52:03.671 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T22:52:03.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:52:03.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-09-10T22:52:03.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-09-10T22:52:03.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:52:03.672 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T22:52:03.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:03 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2141226457' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:52:03.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:03.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:03.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:03.883 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:52:03.908 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-09-10T22:52:03.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:03.910 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-09-10T22:52:03.983 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-09-10T22:52:03.983 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:52:03.984 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:52:03.984 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:52:03.984 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:52:03.984 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:52:03.984 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T22:52:03.984 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:52:03.984 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:52:03.985 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:52:03.985 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T22:52:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:04.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:04.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:04.285 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T22:52:04.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:04.302 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-09-10T22:52:04.356 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:52:04.357 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T22:52:04.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:52:04.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T22:52:04.357 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T22:52:04.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:04 smithi120 ceph-mon[31247]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:04.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:04.474 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T22:52:04.475 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T22:52:04.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:04 smithi176 ceph-mon[34982]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:04.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:04.600 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:52:04.601 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-09-10T22:52:04.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-09-10T22:52:04.601 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:52:04.601 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-09-10T22:52:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:04.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:04.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:52:04.677 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:04.693 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:52:04.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:52:04.693 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:04.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:04 smithi153 ceph-mon[34951]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:04.748 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T22:52:04.748 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T22:52:04.748 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/64d96756-0a18-479f-8469-4787856ad6c3 2024-09-10T22:52:04.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:04.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/64d96756-0a18-479f-8469-4787856ad6c3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-10T22:52:04.851 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-10T22:52:04.851 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-10T22:52:04.851 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-09-10T22:52:04.976 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-10T22:52:04.976 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-10T22:52:04.976 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-09-10T22:52:04.976 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.120:3300,172.21.15.153:3300,172.21.15.176:3300" 2024-09-10T22:52:04.976 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.120:3300,172.21.15.153:3300,172.21.15.176:3300:/volumes/qa/sv_1/64d96756-0a18-479f-8469-4787856ad6c3 2024-09-10T22:52:04.977 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=d2838936-6fc5-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T22:52:04.978 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T22:52:04.978 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:52:04.978 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:52:04.978 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:52:04.978 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:52:04.979 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:52:04.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:52:04.979 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:05.008 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:52:05.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:52:05.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:05.112 INFO:teuthology.orchestra.run.smithi120.stdout:client15192 2024-09-10T22:52:05.112 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:52:05.112 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:52:05.130 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:52:05.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:52:05.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:05.233 INFO:teuthology.orchestra.run.smithi120.stdout:client15192 2024-09-10T22:52:05.233 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:52:05.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:52:05.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:05.297 INFO:teuthology.orchestra.run.smithi120.stdout:client15192 2024-09-10T22:52:05.297 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d2838936-6fc5-11ef-bcea-c7b262605968.client15192/status') 2024-09-10T22:52:05.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:52:05.362 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.15192 (3)192.168.0.1:0/3490718255 2024-09-10T22:52:05.362 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-09-10T22:52:05.362 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-09-10T22:52:05.362 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-09-10T22:52:05.362 INFO:teuthology.orchestra.run.smithi120.stderr:70 bytes copied, 2.1819e-05 s, 3.2 MB/s 2024-09-10T22:52:05.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:52:05.370 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T22:52:05.380 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T22:52:05.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 -- 172.21.15.120:0/3348885910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba94105af0 msgr2=0x7fba94107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 --2- 172.21.15.120:0/3348885910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba94105af0 0x7fba94107f60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba84009900 tx=0x7fba8402a410 comp rx=0 tx=0).stop 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 -- 172.21.15.120:0/3348885910 shutdown_connections 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 --2- 172.21.15.120:0/3348885910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba941086a0 0x7fba9410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 --2- 172.21.15.120:0/3348885910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba94105af0 0x7fba94107f60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 --2- 172.21.15.120:0/3348885910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 0x7fba94105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 -- 172.21.15.120:0/3348885910 >> 172.21.15.120:0/3348885910 conn(0x7fba940fd150 msgr2=0x7fba940ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 -- 172.21.15.120:0/3348885910 shutdown_connections 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 -- 172.21.15.120:0/3348885910 wait complete. 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.505+0000 7fba9c216640 1 Processor -- start 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- start start 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 0x7fba9419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba94105af0 0x7fba9419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410e5f0 con 0x7fba941086a0 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410e470 con 0x7fba94105af0 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410e770 con 0x7fba9406bec0 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56992/0 (socket says 172.21.15.120:56992) 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 -- 172.21.15.120:0/1957654609 learned_addr learned my addr 172.21.15.120:0/1957654609 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:05.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba99fb2640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 0x7fba9419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba997b1640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba94105af0 0x7fba9419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 -- 172.21.15.120:0/1957654609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 msgr2=0x7fba9419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 0x7fba9419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 -- 172.21.15.120:0/1957654609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba94105af0 msgr2=0x7fba9419b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba94105af0 0x7fba9419b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 -- 172.21.15.120:0/1957654609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9419cad0 con 0x7fba941086a0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9a7b3640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00eaa0 tx=0x7fba8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00cdb0 con 0x7fba941086a0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba8c005020 con 0x7fba941086a0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9419cdc0 con 0x7fba941086a0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0116d0 con 0x7fba941086a0 2024-09-10T22:52:05.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.506+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941a3490 con 0x7fba941086a0 2024-09-10T22:52:05.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.507+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94105f50 con 0x7fba941086a0 2024-09-10T22:52:05.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.509+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fba8c00f040 con 0x7fba941086a0 2024-09-10T22:52:05.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.510+0000 7fba8affd640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 0x7fba800b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.510+0000 7fba99fb2640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 0x7fba800b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.510+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fba8c0e5f60 con 0x7fba941086a0 2024-09-10T22:52:05.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.510+0000 7fba99fb2640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 0x7fba800b78b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba90009bd0 tx=0x7fba90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:05.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.512+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fba8c016030 con 0x7fba941086a0 2024-09-10T22:52:05.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.690+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba941a3820 con 0x7fba941086a0 2024-09-10T22:52:05.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.691+0000 7fba8affd640 1 -- 172.21.15.120:0/1957654609 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22746 (secure 0 0 0) 0x7fba8c0ac640 con 0x7fba941086a0 2024-09-10T22:52:05.693 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:05.693 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":77,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:51:15.147106+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 msgr2=0x7fba800b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 0x7fba800b78b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba90009bd0 tx=0x7fba90005f90 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 msgr2=0x7fba9419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00eaa0 tx=0x7fba8c00ef60 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 shutdown_connections 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fba800b53f0 0x7fba800b78b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba941086a0 0x7fba9419c210 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba94105af0 0x7fba9419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 --2- 172.21.15.120:0/1957654609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba9406bec0 0x7fba9419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 >> 172.21.15.120:0/1957654609 conn(0x7fba940fd150 msgr2=0x7fba940ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 shutdown_connections 2024-09-10T22:52:05.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.693+0000 7fba9c216640 1 -- 172.21.15.120:0/1957654609 wait complete. 2024-09-10T22:52:05.707 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:45:56.375659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:50:23.901262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:50:25.937324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:52:05.707 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:52:05.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 -- 172.21.15.120:0/3178641123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108680 msgr2=0x7fd36810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 --2- 172.21.15.120:0/3178641123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108680 0x7fd36810ab40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd360009900 tx=0x7fd36002a410 comp rx=0 tx=0).stop 2024-09-10T22:52:05.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 -- 172.21.15.120:0/3178641123 shutdown_connections 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 --2- 172.21.15.120:0/3178641123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108680 0x7fd36810ab40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 --2- 172.21.15.120:0/3178641123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368105af0 0x7fd368107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 --2- 172.21.15.120:0/3178641123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd3681055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 -- 172.21.15.120:0/3178641123 >> 172.21.15.120:0/3178641123 conn(0x7fd3680fd140 msgr2=0x7fd3680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 -- 172.21.15.120:0/3178641123 shutdown_connections 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.809+0000 7fd36d317640 1 -- 172.21.15.120:0/3178641123 wait complete. 2024-09-10T22:52:05.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 Processor -- start 2024-09-10T22:52:05.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 -- start start 2024-09-10T22:52:05.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd368102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368108680 0x7fd368101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd36810e680 con 0x7fd368105af0 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd36810e500 con 0x7fd36806af40 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd36d317640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd36810e800 con 0x7fd368108680 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd3667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57000/0 (socket says 172.21.15.120:57000) 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd3667fc640 1 -- 172.21.15.120:0/806566084 learned_addr learned my addr 172.21.15.120:0/806566084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.810+0000 7fd3677fe640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368108680 0x7fd368101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd366ffd640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd368102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 -- 172.21.15.120:0/806566084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368108680 msgr2=0x7fd368101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368108680 0x7fd368101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 -- 172.21.15.120:0/806566084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 msgr2=0x7fd368102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd368102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 -- 172.21.15.120:0/806566084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd368102300 con 0x7fd368105af0 2024-09-10T22:52:05.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd366ffd640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd368102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd3667fc640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d740 tx=0x7fd35c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c017070 con 0x7fd368105af0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c00de50 con 0x7fd368105af0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3681a2eb0 con 0x7fd368105af0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004ce0 con 0x7fd368105af0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.811+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3681a33f0 con 0x7fd368105af0 2024-09-10T22:52:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.812+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3680fec30 con 0x7fd368105af0 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.814+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fd35c004e80 con 0x7fd368105af0 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.815+0000 7fd34ffff640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 0x7fd3500b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.815+0000 7fd366ffd640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 0x7fd3500b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.815+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fd35c0e5b10 con 0x7fd368105af0 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.815+0000 7fd366ffd640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 0x7fd3500b78b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd358007eb0 tx=0x7fd3580073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:05.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.818+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd35c0ac920 con 0x7fd368105af0 2024-09-10T22:52:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.993+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd36802e7c0 con 0x7fd368105af0 2024-09-10T22:52:05.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.994+0000 7fd34ffff640 1 -- 172.21.15.120:0/806566084 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fd35c0ac220 con 0x7fd368105af0 2024-09-10T22:52:05.996 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-10T22:52:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 msgr2=0x7fd3500b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 0x7fd3500b78b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd358007eb0 tx=0x7fd3580073c0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 msgr2=0x7fd368101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d740 tx=0x7fd35c00dc00 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 shutdown_connections 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3500b53f0 0x7fd3500b78b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368108680 0x7fd368101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368105af0 0x7fd368101550 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 --2- 172.21.15.120:0/806566084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36806af40 0x7fd368102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 >> 172.21.15.120:0/806566084 conn(0x7fd3680fd140 msgr2=0x7fd368103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:05.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 shutdown_connections 2024-09-10T22:52:06.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:05.996+0000 7fd36d317640 1 -- 172.21.15.120:0/806566084 wait complete. 2024-09-10T22:52:06.008 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:05 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1957654609' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:06.009 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T22:52:06.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:05 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1957654609' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- 172.21.15.120:0/2642754873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 msgr2=0x7f64ac10af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- 172.21.15.120:0/2642754873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac10af80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009cd0 tx=0x7f64a402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- 172.21.15.120:0/2642754873 shutdown_connections 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- 172.21.15.120:0/2642754873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac10af80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- 172.21.15.120:0/2642754873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- 172.21.15.120:0/2642754873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- 172.21.15.120:0/2642754873 >> 172.21.15.120:0/2642754873 conn(0x7f64ac0fc8a0 msgr2=0x7f64ac0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- 172.21.15.120:0/2642754873 shutdown_connections 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- 172.21.15.120:0/2642754873 wait complete. 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 Processor -- start 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- start start 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e5e0 con 0x7f64ac101130 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e460 con 0x7f64ac1048e0 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.109+0000 7f64b1590640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e760 con 0x7f64ac103fa0 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57016/0 (socket says 172.21.15.120:57016) 2024-09-10T22:52:06.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 -- 172.21.15.120:0/61940208 learned_addr learned my addr 172.21.15.120:0/61940208 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64a3fff640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64ab7fe640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 -- 172.21.15.120:0/61940208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 msgr2=0x7f64ac1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 -- 172.21.15.120:0/61940208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 msgr2=0x7f64ac199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 -- 172.21.15.120:0/61940208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac19a8a0 con 0x7f64ac101130 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64a3fff640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64ab7fe640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64aaffd640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d770 tx=0x7f649c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c017070 con 0x7f64ac101130 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f649c0054d0 con 0x7f64ac101130 2024-09-10T22:52:06.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac19ab90 con 0x7f64ac101130 2024-09-10T22:52:06.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c004c30 con 0x7f64ac101130 2024-09-10T22:52:06.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.110+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1a1260 con 0x7f64ac101130 2024-09-10T22:52:06.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.111+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac109e00 con 0x7f64ac101130 2024-09-10T22:52:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.112+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f649c004dd0 con 0x7f64ac101130 2024-09-10T22:52:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.113+0000 7f64a8ff9640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 0x7f64980b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.114+0000 7f64a3fff640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 0x7f64980b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.114+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f649c0e5cc0 con 0x7f64ac101130 2024-09-10T22:52:06.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.114+0000 7f64a3fff640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 0x7f64980b7830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6494005d00 tx=0x7f6494005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:06.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.116+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f649c0aca50 con 0x7f64ac101130 2024-09-10T22:52:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:05 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1957654609' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:06.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.291+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64ac063d50 con 0x7f64ac101130 2024-09-10T22:52:06.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.291+0000 7f64a8ff9640 1 -- 172.21.15.120:0/61940208 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f649c0b5070 con 0x7f64ac101130 2024-09-10T22:52:06.294 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 32 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 msgr2=0x7f64980b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 0x7f64980b7830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6494005d00 tx=0x7f6494005c70 comp rx=0 tx=0).stop 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 msgr2=0x7f64ac198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d770 tx=0x7f649c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 shutdown_connections 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64980b5370 0x7f64980b7830 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ac1048e0 0x7f64ac199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ac103fa0 0x7f64ac1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 --2- 172.21.15.120:0/61940208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ac101130 0x7f64ac198d60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 >> 172.21.15.120:0/61940208 conn(0x7f64ac0fc8a0 msgr2=0x7f64ac106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 shutdown_connections 2024-09-10T22:52:06.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.294+0000 7f64b1590640 1 -- 172.21.15.120:0/61940208 wait complete. 2024-09-10T22:52:06.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/278467750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 msgr2=0x7fdbb8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/278467750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb8102e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009930 tx=0x7fdbac02a420 comp rx=0 tx=0).stop 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/278467750 shutdown_connections 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/278467750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb8103370 0x7fdbb8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/278467750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb8102e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/278467750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8101860 0x7fdbb8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/278467750 >> 172.21.15.120:0/278467750 conn(0x7fdbb80fd140 msgr2=0x7fdbb80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/278467750 shutdown_connections 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.407+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/278467750 wait complete. 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 Processor -- start 2024-09-10T22:52:06.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 -- start start 2024-09-10T22:52:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb8101860 0x7fdbb8197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 0x7fdbb8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d4d0 con 0x7fdbb81029b0 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d350 con 0x7fdbb8103370 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbe7d3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d650 con 0x7fdbb8101860 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57036/0 (socket says 172.21.15.120:57036) 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbb77fe640 1 -- 172.21.15.120:0/818623896 learned_addr learned my addr 172.21.15.120:0/818623896 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.408+0000 7fdbbcd70640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 0x7fdbb8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 -- 172.21.15.120:0/818623896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb8101860 msgr2=0x7fdbb8197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb8101860 0x7fdbb8197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 -- 172.21.15.120:0/818623896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 msgr2=0x7fdbb8198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 0x7fdbb8198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 -- 172.21.15.120:0/818623896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb81997c0 con 0x7fdbb81029b0 2024-09-10T22:52:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbbcd70640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 0x7fdbb8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb77fe640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009a50 tx=0x7fdbac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac03f030 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac0044b0 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb8199a50 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac035070 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.409+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb8199d60 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.410+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8108bd0 con 0x7fdbb81029b0 2024-09-10T22:52:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.412+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fdbac03bb90 con 0x7fdbb81029b0 2024-09-10T22:52:06.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.413+0000 7fdbb57fa640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 0x7fdba00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:06.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.413+0000 7fdbb7fff640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 0x7fdba00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:06.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.413+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fdbac106c60 con 0x7fdbb81029b0 2024-09-10T22:52:06.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.413+0000 7fdbb7fff640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 0x7fdba00b7830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ba80 tx=0x7fdba8005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:06.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.416+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdbac0cd990 con 0x7fdbb81029b0 2024-09-10T22:52:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.592+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbb802e5c0 con 0x7fdbb81029b0 2024-09-10T22:52:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.592+0000 7fdbb57fa640 1 -- 172.21.15.120:0/818623896 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7fdbac0cd290 con 0x7fdbb81029b0 2024-09-10T22:52:06.595 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 64 2024-09-10T22:52:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 msgr2=0x7fdba00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 0x7fdba00b7830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ba80 tx=0x7fdba8005ed0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 msgr2=0x7fdbb81982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009a50 tx=0x7fdbac031a50 comp rx=0 tx=0).stop 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 shutdown_connections 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdba00b5370 0x7fdba00b7830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbb8103370 0x7fdbb8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdbb81029b0 0x7fdbb81982d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 --2- 172.21.15.120:0/818623896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb8101860 0x7fdbb8197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 >> 172.21.15.120:0/818623896 conn(0x7fdbb80fd140 msgr2=0x7fdbb8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 shutdown_connections 2024-09-10T22:52:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:06.595+0000 7fdbbe7d3640 1 -- 172.21.15.120:0/818623896 wait complete. 2024-09-10T22:52:06.608 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T22:52:06.608 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:06.859 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:06.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/806566084' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:52:06.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:06 smithi120 ceph-mon[31247]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:06.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/61940208' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:52:06.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/818623896' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/806566084' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:06 smithi176 ceph-mon[34982]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:07.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/61940208' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/818623896' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/806566084' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:06 smithi153 ceph-mon[34951]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T22:52:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/61940208' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/818623896' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:52:07.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 -- 172.21.15.120:0/2157948642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be8108040 msgr2=0x7f4be810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2157948642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be8108040 0x7f4be810e840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009ca0 tx=0x7f4bd802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 -- 172.21.15.120:0/2157948642 shutdown_connections 2024-09-10T22:52:07.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2157948642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be8108040 0x7f4be810e840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2157948642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be81076c0 0x7f4be8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2157948642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8101cb0 0x7f4be8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 -- 172.21.15.120:0/2157948642 >> 172.21.15.120:0/2157948642 conn(0x7f4be80fd700 msgr2=0x7f4be80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 -- 172.21.15.120:0/2157948642 shutdown_connections 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.325+0000 7f4bece8b640 1 -- 172.21.15.120:0/2157948642 wait complete. 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 Processor -- start 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 -- start start 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 0x7f4be819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 0x7f4be819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8111d50 con 0x7f4be81076c0 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8111bd0 con 0x7f4be8101cb0 2024-09-10T22:52:07.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.326+0000 7f4bece8b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8111ed0 con 0x7f4be8108040 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57054/0 (socket says 172.21.15.120:57054) 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 -- 172.21.15.120:0/2974506405 learned_addr learned my addr 172.21.15.120:0/2974506405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be659c640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 0x7f4be819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be6d9d640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 0x7f4be819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 -- 172.21.15.120:0/2974506405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 msgr2=0x7f4be819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 0x7f4be819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 -- 172.21.15.120:0/2974506405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 msgr2=0x7f4be819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 0x7f4be819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 -- 172.21.15.120:0/2974506405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be81a5440 con 0x7f4be81076c0 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be659c640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 0x7f4be819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:07.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be6d9d640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 0x7f4be819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4be5d9b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e9f0 tx=0x7f4bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000cd80 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be81a5730 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd00108e0 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0018980 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.327+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be81a5cc0 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.328+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be8102090 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.330+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f4bd0010430 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.331+0000 7f4bdf7fe640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 0x7f4bd40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.331+0000 7f4be659c640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 0x7f4bd40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.331+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f4bd00e6660 con 0x7f4be81076c0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.332+0000 7f4be659c640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 0x7f4bd40b7830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4be0005fd0 tx=0x7f4be0005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.334+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4bd00ad3f0 con 0x7f4be81076c0 2024-09-10T22:52:07.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.545+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4be806b8a0 con 0x7f4be81076c0 2024-09-10T22:52:07.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.546+0000 7f4bdf7fe640 1 -- 172.21.15.120:0/2974506405 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f4be806b8a0 con 0x7f4be81076c0 2024-09-10T22:52:07.549 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:07.549 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:52:07.549 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:52:07.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 msgr2=0x7f4bd40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 0x7f4bd40b7830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4be0005fd0 tx=0x7f4be0005ea0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 msgr2=0x7f4be819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e9f0 tx=0x7f4bd000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 shutdown_connections 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4bd40b5370 0x7f4bd40b7830 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4be8108040 0x7f4be819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4be81076c0 0x7f4be819a5a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 --2- 172.21.15.120:0/2974506405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4be8101cb0 0x7f4be819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 >> 172.21.15.120:0/2974506405 conn(0x7f4be80fd700 msgr2=0x7f4be80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 shutdown_connections 2024-09-10T22:52:07.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:07.548+0000 7f4bece8b640 1 -- 172.21.15.120:0/2974506405 wait complete. 2024-09-10T22:52:07.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:07 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2974506405' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:08.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:07 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2974506405' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:08.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:07 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2974506405' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:08.255 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T22:52:08.255 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T22:52:08.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:52:08.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 -- 172.21.15.120:0/860625329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941033a0 msgr2=0x7fa494109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 --2- 172.21.15.120:0/860625329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941033a0 0x7fa494109d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009930 tx=0x7fa48c02a420 comp rx=0 tx=0).stop 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 -- 172.21.15.120:0/860625329 shutdown_connections 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 --2- 172.21.15.120:0/860625329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941033a0 0x7fa494109d20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 --2- 172.21.15.120:0/860625329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941029e0 0x7fa494102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 --2- 172.21.15.120:0/860625329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa494101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 -- 172.21.15.120:0/860625329 >> 172.21.15.120:0/860625329 conn(0x7fa4940fd150 msgr2=0x7fa4940ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 -- 172.21.15.120:0/860625329 shutdown_connections 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.355+0000 7fa49b952640 1 -- 172.21.15.120:0/860625329 wait complete. 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 Processor -- start 2024-09-10T22:52:08.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 -- start start 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa4941056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941033a0 0x7fa494104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa49410d500 con 0x7fa4941029e0 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa49410d380 con 0x7fa494101890 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa49b952640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa49410d680 con 0x7fa4941033a0 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa498eed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa498eed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57070/0 (socket says 172.21.15.120:57070) 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.356+0000 7fa498eed640 1 -- 172.21.15.120:0/2641887780 learned_addr learned my addr 172.21.15.120:0/2641887780 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa4996ee640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa4941056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 -- 172.21.15.120:0/2641887780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941033a0 msgr2=0x7fa494104460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa499eef640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941033a0 0x7fa494104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941033a0 0x7fa494104460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 -- 172.21.15.120:0/2641887780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 msgr2=0x7fa4941056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa4941056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 -- 172.21.15.120:0/2641887780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494104cd0 con 0x7fa4941029e0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa4996ee640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa4941056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa498eed640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c970 tx=0x7fa48400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484007bb0 con 0x7fa4941029e0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4941a2e00 con 0x7fa4941029e0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa484007d50 con 0x7fa4941029e0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400f450 con 0x7fa4941029e0 2024-09-10T22:52:08.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.357+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941a3270 con 0x7fa4941029e0 2024-09-10T22:52:08.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.358+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4941a3500 con 0x7fa4941029e0 2024-09-10T22:52:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.359+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa484004f00 con 0x7fa4941029e0 2024-09-10T22:52:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.360+0000 7fa48a7fc640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 0x7fa4800b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.360+0000 7fa4996ee640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 0x7fa4800b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.360+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fa4840e5e40 con 0x7fa4941029e0 2024-09-10T22:52:08.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.360+0000 7fa4996ee640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 0x7fa4800b7830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa490009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:08.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.363+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa484014030 con 0x7fa4941029e0 2024-09-10T22:52:08.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.538+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4941a36d0 con 0x7fa4800b5370 2024-09-10T22:52:08.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.542+0000 7fa48a7fc640 1 -- 172.21.15.120:0/2641887780 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258631 (secure 0 0 0) 0x7fa4941a36d0 con 0x7fa4800b5370 2024-09-10T22:52:08.547 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:08.548 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 msgr2=0x7fa4800b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 0x7fa4800b7830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa490009510 comp rx=0 tx=0).stop 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 msgr2=0x7fa494103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c970 tx=0x7fa48400ce30 comp rx=0 tx=0).stop 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 shutdown_connections 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4800b5370 0x7fa4800b7830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4941033a0 0x7fa494104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4941029e0 0x7fa494103f20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 --2- 172.21.15.120:0/2641887780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa494101890 0x7fa4941056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 >> 172.21.15.120:0/2641887780 conn(0x7fa4940fd150 msgr2=0x7fa49406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:08.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 shutdown_connections 2024-09-10T22:52:08.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.545+0000 7fa49b952640 1 -- 172.21.15.120:0/2641887780 wait complete. 2024-09-10T22:52:08.558 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-10T22:52:08.171758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653458,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":227,"num_write":209,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377684,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564140,"statfs":{"total":1151940427776,"available":1151553679360,"internally_reserved":0,"allocated":12005376,"data_stored":6816130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003084"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156012+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:51:15.156012+0000","last_peered":"2024-09-10T22:51:15.156012+0000","last_clean":"2024-09-10T22:51:15.156012+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:51:15.156012+0000","last_undegraded":"2024-09-10T22:51:15.156012+0000","last_fullsized":"2024-09-10T22:51:15.156012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154505+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:51:15.154505+0000","last_peered":"2024-09-10T22:51:15.154505+0000","last_clean":"2024-09-10T22:51:15.154505+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:51:15.154505+0000","last_undegraded":"2024-09-10T22:51:15.154505+0000","last_fullsized":"2024-09-10T22:51:15.154505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153680+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:51:15.153680+0000","last_peered":"2024-09-10T22:51:15.153680+0000","last_clean":"2024-09-10T22:51:15.153680+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:51:15.153680+0000","last_undegraded":"2024-09-10T22:51:15.153680+0000","last_fullsized":"2024-09-10T22:51:15.153680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153157+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:51:15.153157+0000","last_peered":"2024-09-10T22:51:15.153157+0000","last_clean":"2024-09-10T22:51:15.153157+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:51:15.153157+0000","last_undegraded":"2024-09-10T22:51:15.153157+0000","last_fullsized":"2024-09-10T22:51:15.153157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161803+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:51:15.161803+0000","last_peered":"2024-09-10T22:51:15.161803+0000","last_clean":"2024-09-10T22:51:15.161803+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:51:15.161803+0000","last_undegraded":"2024-09-10T22:51:15.161803+0000","last_fullsized":"2024-09-10T22:51:15.161803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150981+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.150981+0000","last_peered":"2024-09-10T22:51:15.150981+0000","last_clean":"2024-09-10T22:51:15.150981+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:51:15.150981+0000","last_undegraded":"2024-09-10T22:51:15.150981+0000","last_fullsized":"2024-09-10T22:51:15.150981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154759+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:51:15.154759+0000","last_peered":"2024-09-10T22:51:15.154759+0000","last_clean":"2024-09-10T22:51:15.154759+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:51:15.154759+0000","last_undegraded":"2024-09-10T22:51:15.154759+0000","last_fullsized":"2024-09-10T22:51:15.154759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294963+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:51:15.294963+0000","last_peered":"2024-09-10T22:51:15.294963+0000","last_clean":"2024-09-10T22:51:15.294963+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:51:15.294963+0000","last_undegraded":"2024-09-10T22:51:15.294963+0000","last_fullsized":"2024-09-10T22:51:15.294963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296097+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:51:15.296097+0000","last_peered":"2024-09-10T22:51:15.296097+0000","last_clean":"2024-09-10T22:51:15.296097+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:51:15.296097+0000","last_undegraded":"2024-09-10T22:51:15.296097+0000","last_fullsized":"2024-09-10T22:51:15.296097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151443+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:51:15.151443+0000","last_peered":"2024-09-10T22:51:15.151443+0000","last_clean":"2024-09-10T22:51:15.151443+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:51:15.151443+0000","last_undegraded":"2024-09-10T22:51:15.151443+0000","last_fullsized":"2024-09-10T22:51:15.151443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155194+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:51:15.155194+0000","last_peered":"2024-09-10T22:51:15.155194+0000","last_clean":"2024-09-10T22:51:15.155194+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:51:15.155194+0000","last_undegraded":"2024-09-10T22:51:15.155194+0000","last_fullsized":"2024-09-10T22:51:15.155194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153263+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:51:15.153263+0000","last_peered":"2024-09-10T22:51:15.153263+0000","last_clean":"2024-09-10T22:51:15.153263+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:51:15.153263+0000","last_undegraded":"2024-09-10T22:51:15.153263+0000","last_fullsized":"2024-09-10T22:51:15.153263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154928+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:51:15.154928+0000","last_peered":"2024-09-10T22:51:15.154928+0000","last_clean":"2024-09-10T22:51:15.154928+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:51:15.154928+0000","last_undegraded":"2024-09-10T22:51:15.154928+0000","last_fullsized":"2024-09-10T22:51:15.154928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153640+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:51:15.153640+0000","last_peered":"2024-09-10T22:51:15.153640+0000","last_clean":"2024-09-10T22:51:15.153640+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:51:15.153640+0000","last_undegraded":"2024-09-10T22:51:15.153640+0000","last_fullsized":"2024-09-10T22:51:15.153640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153570+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:51:15.153570+0000","last_peered":"2024-09-10T22:51:15.153570+0000","last_clean":"2024-09-10T22:51:15.153570+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:51:15.153570+0000","last_undegraded":"2024-09-10T22:51:15.153570+0000","last_fullsized":"2024-09-10T22:51:15.153570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295653+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:51:15.295653+0000","last_peered":"2024-09-10T22:51:15.295653+0000","last_clean":"2024-09-10T22:51:15.295653+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:51:15.295653+0000","last_undegraded":"2024-09-10T22:51:15.295653+0000","last_fullsized":"2024-09-10T22:51:15.295653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156076+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:51:15.156076+0000","last_peered":"2024-09-10T22:51:15.156076+0000","last_clean":"2024-09-10T22:51:15.156076+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:51:15.156076+0000","last_undegraded":"2024-09-10T22:51:15.156076+0000","last_fullsized":"2024-09-10T22:51:15.156076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150803+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:51:15.150803+0000","last_peered":"2024-09-10T22:51:15.150803+0000","last_clean":"2024-09-10T22:51:15.150803+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:51:15.150803+0000","last_undegraded":"2024-09-10T22:51:15.150803+0000","last_fullsized":"2024-09-10T22:51:15.150803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162560+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:51:15.162560+0000","last_peered":"2024-09-10T22:51:15.162560+0000","last_clean":"2024-09-10T22:51:15.162560+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:51:15.162560+0000","last_undegraded":"2024-09-10T22:51:15.162560+0000","last_fullsized":"2024-09-10T22:51:15.162560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155723+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:51:15.155723+0000","last_peered":"2024-09-10T22:51:15.155723+0000","last_clean":"2024-09-10T22:51:15.155723+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:51:15.155723+0000","last_undegraded":"2024-09-10T22:51:15.155723+0000","last_fullsized":"2024-09-10T22:51:15.155723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155039+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:51:15.155039+0000","last_peered":"2024-09-10T22:51:15.155039+0000","last_clean":"2024-09-10T22:51:15.155039+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:51:15.155039+0000","last_undegraded":"2024-09-10T22:51:15.155039+0000","last_fullsized":"2024-09-10T22:51:15.155039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295743+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:51:15.295743+0000","last_peered":"2024-09-10T22:51:15.295743+0000","last_clean":"2024-09-10T22:51:15.295743+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:51:15.295743+0000","last_undegraded":"2024-09-10T22:51:15.295743+0000","last_fullsized":"2024-09-10T22:51:15.295743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294977+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:51:15.294977+0000","last_peered":"2024-09-10T22:51:15.294977+0000","last_clean":"2024-09-10T22:51:15.294977+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:51:15.294977+0000","last_undegraded":"2024-09-10T22:51:15.294977+0000","last_fullsized":"2024-09-10T22:51:15.294977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295461+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:51:15.295461+0000","last_peered":"2024-09-10T22:51:15.295461+0000","last_clean":"2024-09-10T22:51:15.295461+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:51:15.295461+0000","last_undegraded":"2024-09-10T22:51:15.295461+0000","last_fullsized":"2024-09-10T22:51:15.295461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295171+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:51:15.295171+0000","last_peered":"2024-09-10T22:51:15.295171+0000","last_clean":"2024-09-10T22:51:15.295171+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:51:15.295171+0000","last_undegraded":"2024-09-10T22:51:15.295171+0000","last_fullsized":"2024-09-10T22:51:15.295171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162515+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:51:15.162515+0000","last_peered":"2024-09-10T22:51:15.162515+0000","last_clean":"2024-09-10T22:51:15.162515+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:51:15.162515+0000","last_undegraded":"2024-09-10T22:51:15.162515+0000","last_fullsized":"2024-09-10T22:51:15.162515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156495+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:51:15.156495+0000","last_peered":"2024-09-10T22:51:15.156495+0000","last_clean":"2024-09-10T22:51:15.156495+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:51:15.156495+0000","last_undegraded":"2024-09-10T22:51:15.156495+0000","last_fullsized":"2024-09-10T22:51:15.156495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154828+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:51:15.154828+0000","last_peered":"2024-09-10T22:51:15.154828+0000","last_clean":"2024-09-10T22:51:15.154828+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:51:15.154828+0000","last_undegraded":"2024-09-10T22:51:15.154828+0000","last_fullsized":"2024-09-10T22:51:15.154828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154404+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:51:15.154404+0000","last_peered":"2024-09-10T22:51:15.154404+0000","last_clean":"2024-09-10T22:51:15.154404+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:51:15.154404+0000","last_undegraded":"2024-09-10T22:51:15.154404+0000","last_fullsized":"2024-09-10T22:51:15.154404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161965+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:51:15.161965+0000","last_peered":"2024-09-10T22:51:15.161965+0000","last_clean":"2024-09-10T22:51:15.161965+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:51:15.161965+0000","last_undegraded":"2024-09-10T22:51:15.161965+0000","last_fullsized":"2024-09-10T22:51:15.161965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162068+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:51:15.162068+0000","last_peered":"2024-09-10T22:51:15.162068+0000","last_clean":"2024-09-10T22:51:15.162068+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:51:15.162068+0000","last_undegraded":"2024-09-10T22:51:15.162068+0000","last_fullsized":"2024-09-10T22:51:15.162068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155610+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:51:15.155610+0000","last_peered":"2024-09-10T22:51:15.155610+0000","last_clean":"2024-09-10T22:51:15.155610+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:51:15.155610+0000","last_undegraded":"2024-09-10T22:51:15.155610+0000","last_fullsized":"2024-09-10T22:51:15.155610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161948+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:51:15.161948+0000","last_peered":"2024-09-10T22:51:15.161948+0000","last_clean":"2024-09-10T22:51:15.161948+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:51:15.161948+0000","last_undegraded":"2024-09-10T22:51:15.161948+0000","last_fullsized":"2024-09-10T22:51:15.161948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295741+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:51:15.295741+0000","last_peered":"2024-09-10T22:51:15.295741+0000","last_clean":"2024-09-10T22:51:15.295741+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:51:15.295741+0000","last_undegraded":"2024-09-10T22:51:15.295741+0000","last_fullsized":"2024-09-10T22:51:15.295741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296700+0000","last_change":"2024-09-10T22:51:15.296217+0000","last_active":"2024-09-10T22:51:15.296700+0000","last_peered":"2024-09-10T22:51:15.296700+0000","last_clean":"2024-09-10T22:51:15.296700+0000","last_became_active":"2024-09-10T22:51:15.295842+0000","last_became_peered":"2024-09-10T22:51:15.295842+0000","last_unstale":"2024-09-10T22:51:15.296700+0000","last_undegraded":"2024-09-10T22:51:15.296700+0000","last_fullsized":"2024-09-10T22:51:15.296700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:32.464281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151051+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:51:15.151051+0000","last_peered":"2024-09-10T22:51:15.151051+0000","last_clean":"2024-09-10T22:51:15.151051+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:51:15.151051+0000","last_undegraded":"2024-09-10T22:51:15.151051+0000","last_fullsized":"2024-09-10T22:51:15.151051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153986+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:51:15.153986+0000","last_peered":"2024-09-10T22:51:15.153986+0000","last_clean":"2024-09-10T22:51:15.153986+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:51:15.153986+0000","last_undegraded":"2024-09-10T22:51:15.153986+0000","last_fullsized":"2024-09-10T22:51:15.153986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295017+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:51:15.295017+0000","last_peered":"2024-09-10T22:51:15.295017+0000","last_clean":"2024-09-10T22:51:15.295017+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:51:15.295017+0000","last_undegraded":"2024-09-10T22:51:15.295017+0000","last_fullsized":"2024-09-10T22:51:15.295017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295730+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:51:15.295730+0000","last_peered":"2024-09-10T22:51:15.295730+0000","last_clean":"2024-09-10T22:51:15.295730+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:51:15.295730+0000","last_undegraded":"2024-09-10T22:51:15.295730+0000","last_fullsized":"2024-09-10T22:51:15.295730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150604+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:51:15.150604+0000","last_peered":"2024-09-10T22:51:15.150604+0000","last_clean":"2024-09-10T22:51:15.150604+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:51:15.150604+0000","last_undegraded":"2024-09-10T22:51:15.150604+0000","last_fullsized":"2024-09-10T22:51:15.150604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151268+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:51:15.151268+0000","last_peered":"2024-09-10T22:51:15.151268+0000","last_clean":"2024-09-10T22:51:15.151268+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:51:15.151268+0000","last_undegraded":"2024-09-10T22:51:15.151268+0000","last_fullsized":"2024-09-10T22:51:15.151268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295974+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:51:15.295974+0000","last_peered":"2024-09-10T22:51:15.295974+0000","last_clean":"2024-09-10T22:51:15.295974+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:51:15.295974+0000","last_undegraded":"2024-09-10T22:51:15.295974+0000","last_fullsized":"2024-09-10T22:51:15.295974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155301+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:51:15.155301+0000","last_peered":"2024-09-10T22:51:15.155301+0000","last_clean":"2024-09-10T22:51:15.155301+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:51:15.155301+0000","last_undegraded":"2024-09-10T22:51:15.155301+0000","last_fullsized":"2024-09-10T22:51:15.155301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296977+0000","last_change":"2024-09-10T22:51:15.296922+0000","last_active":"2024-09-10T22:51:15.296977+0000","last_peered":"2024-09-10T22:51:15.296977+0000","last_clean":"2024-09-10T22:51:15.296977+0000","last_became_active":"2024-09-10T22:51:15.296658+0000","last_became_peered":"2024-09-10T22:51:15.296658+0000","last_unstale":"2024-09-10T22:51:15.296977+0000","last_undegraded":"2024-09-10T22:51:15.296977+0000","last_fullsized":"2024-09-10T22:51:15.296977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:18.932940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295896+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:51:15.295896+0000","last_peered":"2024-09-10T22:51:15.295896+0000","last_clean":"2024-09-10T22:51:15.295896+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:51:15.295896+0000","last_undegraded":"2024-09-10T22:51:15.295896+0000","last_fullsized":"2024-09-10T22:51:15.295896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295846+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:51:15.295846+0000","last_peered":"2024-09-10T22:51:15.295846+0000","last_clean":"2024-09-10T22:51:15.295846+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:51:15.295846+0000","last_undegraded":"2024-09-10T22:51:15.295846+0000","last_fullsized":"2024-09-10T22:51:15.295846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'29","reported_seq":246,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:52:06.196104+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:52:06.196104+0000","last_peered":"2024-09-10T22:52:06.196104+0000","last_clean":"2024-09-10T22:52:06.196104+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:52:06.196104+0000","last_undegraded":"2024-09-10T22:52:06.196104+0000","last_fullsized":"2024-09-10T22:52:06.196104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62058,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":194,"num_read_kb":157,"num_write":29,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162574+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:51:15.162574+0000","last_peered":"2024-09-10T22:51:15.162574+0000","last_clean":"2024-09-10T22:51:15.162574+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:51:15.162574+0000","last_undegraded":"2024-09-10T22:51:15.162574+0000","last_fullsized":"2024-09-10T22:51:15.162574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162287+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:51:15.162287+0000","last_peered":"2024-09-10T22:51:15.162287+0000","last_clean":"2024-09-10T22:51:15.162287+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:51:15.162287+0000","last_undegraded":"2024-09-10T22:51:15.162287+0000","last_fullsized":"2024-09-10T22:51:15.162287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154237+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:51:15.154237+0000","last_peered":"2024-09-10T22:51:15.154237+0000","last_clean":"2024-09-10T22:51:15.154237+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:51:15.154237+0000","last_undegraded":"2024-09-10T22:51:15.154237+0000","last_fullsized":"2024-09-10T22:51:15.154237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295672+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:51:15.295672+0000","last_peered":"2024-09-10T22:51:15.295672+0000","last_clean":"2024-09-10T22:51:15.295672+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:51:15.295672+0000","last_undegraded":"2024-09-10T22:51:15.295672+0000","last_fullsized":"2024-09-10T22:51:15.295672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156806+0000","last_change":"2024-09-10T22:51:15.156697+0000","last_active":"2024-09-10T22:51:15.156806+0000","last_peered":"2024-09-10T22:51:15.156806+0000","last_clean":"2024-09-10T22:51:15.156806+0000","last_became_active":"2024-09-10T22:51:15.156396+0000","last_became_peered":"2024-09-10T22:51:15.156396+0000","last_unstale":"2024-09-10T22:51:15.156806+0000","last_undegraded":"2024-09-10T22:51:15.156806+0000","last_fullsized":"2024-09-10T22:51:15.156806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:26.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.152948+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:51:15.152948+0000","last_peered":"2024-09-10T22:51:15.152948+0000","last_clean":"2024-09-10T22:51:15.152948+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:51:15.152948+0000","last_undegraded":"2024-09-10T22:51:15.152948+0000","last_fullsized":"2024-09-10T22:51:15.152948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154683+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:51:15.154683+0000","last_peered":"2024-09-10T22:51:15.154683+0000","last_clean":"2024-09-10T22:51:15.154683+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:51:15.154683+0000","last_undegraded":"2024-09-10T22:51:15.154683+0000","last_fullsized":"2024-09-10T22:51:15.154683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162012+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:51:15.162012+0000","last_peered":"2024-09-10T22:51:15.162012+0000","last_clean":"2024-09-10T22:51:15.162012+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:51:15.162012+0000","last_undegraded":"2024-09-10T22:51:15.162012+0000","last_fullsized":"2024-09-10T22:51:15.162012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295528+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:51:15.295528+0000","last_peered":"2024-09-10T22:51:15.295528+0000","last_clean":"2024-09-10T22:51:15.295528+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:51:15.295528+0000","last_undegraded":"2024-09-10T22:51:15.295528+0000","last_fullsized":"2024-09-10T22:51:15.295528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162997+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:51:15.162997+0000","last_peered":"2024-09-10T22:51:15.162997+0000","last_clean":"2024-09-10T22:51:15.162997+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:51:15.162997+0000","last_undegraded":"2024-09-10T22:51:15.162997+0000","last_fullsized":"2024-09-10T22:51:15.162997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155841+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:51:15.155841+0000","last_peered":"2024-09-10T22:51:15.155841+0000","last_clean":"2024-09-10T22:51:15.155841+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:51:15.155841+0000","last_undegraded":"2024-09-10T22:51:15.155841+0000","last_fullsized":"2024-09-10T22:51:15.155841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154030+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:51:15.154030+0000","last_peered":"2024-09-10T22:51:15.154030+0000","last_clean":"2024-09-10T22:51:15.154030+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:51:15.154030+0000","last_undegraded":"2024-09-10T22:51:15.154030+0000","last_fullsized":"2024-09-10T22:51:15.154030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155400+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:51:15.155400+0000","last_peered":"2024-09-10T22:51:15.155400+0000","last_clean":"2024-09-10T22:51:15.155400+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:51:15.155400+0000","last_undegraded":"2024-09-10T22:51:15.155400+0000","last_fullsized":"2024-09-10T22:51:15.155400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296015+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:51:15.296015+0000","last_peered":"2024-09-10T22:51:15.296015+0000","last_clean":"2024-09-10T22:51:15.296015+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:51:15.296015+0000","last_undegraded":"2024-09-10T22:51:15.296015+0000","last_fullsized":"2024-09-10T22:51:15.296015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155019+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:51:15.155019+0000","last_peered":"2024-09-10T22:51:15.155019+0000","last_clean":"2024-09-10T22:51:15.155019+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:51:15.155019+0000","last_undegraded":"2024-09-10T22:51:15.155019+0000","last_fullsized":"2024-09-10T22:51:15.155019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.158692+0000","last_change":"2024-09-10T22:51:15.158070+0000","last_active":"2024-09-10T22:51:15.158692+0000","last_peered":"2024-09-10T22:51:15.158692+0000","last_clean":"2024-09-10T22:51:15.158692+0000","last_became_active":"2024-09-10T22:51:15.157711+0000","last_became_peered":"2024-09-10T22:51:15.157711+0000","last_unstale":"2024-09-10T22:51:15.158692+0000","last_undegraded":"2024-09-10T22:51:15.158692+0000","last_fullsized":"2024-09-10T22:51:15.158692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:11.308860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155239+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.155239+0000","last_peered":"2024-09-10T22:51:15.155239+0000","last_clean":"2024-09-10T22:51:15.155239+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:51:15.155239+0000","last_undegraded":"2024-09-10T22:51:15.155239+0000","last_fullsized":"2024-09-10T22:51:15.155239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154891+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:51:15.154891+0000","last_peered":"2024-09-10T22:51:15.154891+0000","last_clean":"2024-09-10T22:51:15.154891+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:51:15.154891+0000","last_undegraded":"2024-09-10T22:51:15.154891+0000","last_fullsized":"2024-09-10T22:51:15.154891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153104+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:51:15.153104+0000","last_peered":"2024-09-10T22:51:15.153104+0000","last_clean":"2024-09-10T22:51:15.153104+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:51:15.153104+0000","last_undegraded":"2024-09-10T22:51:15.153104+0000","last_fullsized":"2024-09-10T22:51:15.153104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295351+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:51:15.295351+0000","last_peered":"2024-09-10T22:51:15.295351+0000","last_clean":"2024-09-10T22:51:15.295351+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:51:15.295351+0000","last_undegraded":"2024-09-10T22:51:15.295351+0000","last_fullsized":"2024-09-10T22:51:15.295351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154832+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:51:15.154832+0000","last_peered":"2024-09-10T22:51:15.154832+0000","last_clean":"2024-09-10T22:51:15.154832+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:51:15.154832+0000","last_undegraded":"2024-09-10T22:51:15.154832+0000","last_fullsized":"2024-09-10T22:51:15.154832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.165878+0000","last_change":"2024-09-10T22:51:15.165814+0000","last_active":"2024-09-10T22:51:15.165878+0000","last_peered":"2024-09-10T22:51:15.165878+0000","last_clean":"2024-09-10T22:51:15.165878+0000","last_became_active":"2024-09-10T22:51:15.165305+0000","last_became_peered":"2024-09-10T22:51:15.165305+0000","last_unstale":"2024-09-10T22:51:15.165878+0000","last_undegraded":"2024-09-10T22:51:15.165878+0000","last_fullsized":"2024-09-10T22:51:15.165878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:14.680823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154077+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:51:15.154077+0000","last_peered":"2024-09-10T22:51:15.154077+0000","last_clean":"2024-09-10T22:51:15.154077+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:51:15.154077+0000","last_undegraded":"2024-09-10T22:51:15.154077+0000","last_fullsized":"2024-09-10T22:51:15.154077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295816+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:51:15.295816+0000","last_peered":"2024-09-10T22:51:15.295816+0000","last_clean":"2024-09-10T22:51:15.295816+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:51:15.295816+0000","last_undegraded":"2024-09-10T22:51:15.295816+0000","last_fullsized":"2024-09-10T22:51:15.295816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155014+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:51:15.155014+0000","last_peered":"2024-09-10T22:51:15.155014+0000","last_clean":"2024-09-10T22:51:15.155014+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:51:15.155014+0000","last_undegraded":"2024-09-10T22:51:15.155014+0000","last_fullsized":"2024-09-10T22:51:15.155014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151167+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:51:15.151167+0000","last_peered":"2024-09-10T22:51:15.151167+0000","last_clean":"2024-09-10T22:51:15.151167+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:51:15.151167+0000","last_undegraded":"2024-09-10T22:51:15.151167+0000","last_fullsized":"2024-09-10T22:51:15.151167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150766+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:51:15.150766+0000","last_peered":"2024-09-10T22:51:15.150766+0000","last_clean":"2024-09-10T22:51:15.150766+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:51:15.150766+0000","last_undegraded":"2024-09-10T22:51:15.150766+0000","last_fullsized":"2024-09-10T22:51:15.150766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151181+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:51:15.151181+0000","last_peered":"2024-09-10T22:51:15.151181+0000","last_clean":"2024-09-10T22:51:15.151181+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:51:15.151181+0000","last_undegraded":"2024-09-10T22:51:15.151181+0000","last_fullsized":"2024-09-10T22:51:15.151181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162786+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:51:15.162786+0000","last_peered":"2024-09-10T22:51:15.162786+0000","last_clean":"2024-09-10T22:51:15.162786+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:51:15.162786+0000","last_undegraded":"2024-09-10T22:51:15.162786+0000","last_fullsized":"2024-09-10T22:51:15.162786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154876+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:51:15.154876+0000","last_peered":"2024-09-10T22:51:15.154876+0000","last_clean":"2024-09-10T22:51:15.154876+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:51:15.154876+0000","last_undegraded":"2024-09-10T22:51:15.154876+0000","last_fullsized":"2024-09-10T22:51:15.154876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150866+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:51:15.150866+0000","last_peered":"2024-09-10T22:51:15.150866+0000","last_clean":"2024-09-10T22:51:15.150866+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:51:15.150866+0000","last_undegraded":"2024-09-10T22:51:15.150866+0000","last_fullsized":"2024-09-10T22:51:15.150866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295230+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:51:15.295230+0000","last_peered":"2024-09-10T22:51:15.295230+0000","last_clean":"2024-09-10T22:51:15.295230+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:51:15.295230+0000","last_undegraded":"2024-09-10T22:51:15.295230+0000","last_fullsized":"2024-09-10T22:51:15.295230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153127+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:51:15.153127+0000","last_peered":"2024-09-10T22:51:15.153127+0000","last_clean":"2024-09-10T22:51:15.153127+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:51:15.153127+0000","last_undegraded":"2024-09-10T22:51:15.153127+0000","last_fullsized":"2024-09-10T22:51:15.153127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154511+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:51:15.154511+0000","last_peered":"2024-09-10T22:51:15.154511+0000","last_clean":"2024-09-10T22:51:15.154511+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:51:15.154511+0000","last_undegraded":"2024-09-10T22:51:15.154511+0000","last_fullsized":"2024-09-10T22:51:15.154511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295736+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:51:15.295736+0000","last_peered":"2024-09-10T22:51:15.295736+0000","last_clean":"2024-09-10T22:51:15.295736+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:51:15.295736+0000","last_undegraded":"2024-09-10T22:51:15.295736+0000","last_fullsized":"2024-09-10T22:51:15.295736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161616+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:51:15.161616+0000","last_peered":"2024-09-10T22:51:15.161616+0000","last_clean":"2024-09-10T22:51:15.161616+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:51:15.161616+0000","last_undegraded":"2024-09-10T22:51:15.161616+0000","last_fullsized":"2024-09-10T22:51:15.161616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295097+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:51:15.295097+0000","last_peered":"2024-09-10T22:51:15.295097+0000","last_clean":"2024-09-10T22:51:15.295097+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:51:15.295097+0000","last_undegraded":"2024-09-10T22:51:15.295097+0000","last_fullsized":"2024-09-10T22:51:15.295097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155731+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:51:15.155731+0000","last_peered":"2024-09-10T22:51:15.155731+0000","last_clean":"2024-09-10T22:51:15.155731+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:51:15.155731+0000","last_undegraded":"2024-09-10T22:51:15.155731+0000","last_fullsized":"2024-09-10T22:51:15.155731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.297065+0000","last_change":"2024-09-10T22:51:15.296938+0000","last_active":"2024-09-10T22:51:15.297065+0000","last_peered":"2024-09-10T22:51:15.297065+0000","last_clean":"2024-09-10T22:51:15.297065+0000","last_became_active":"2024-09-10T22:51:15.296291+0000","last_became_peered":"2024-09-10T22:51:15.296291+0000","last_unstale":"2024-09-10T22:51:15.297065+0000","last_undegraded":"2024-09-10T22:51:15.297065+0000","last_fullsized":"2024-09-10T22:51:15.297065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:26.752740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295243+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:51:15.295243+0000","last_peered":"2024-09-10T22:51:15.295243+0000","last_clean":"2024-09-10T22:51:15.295243+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:51:15.295243+0000","last_undegraded":"2024-09-10T22:51:15.295243+0000","last_fullsized":"2024-09-10T22:51:15.295243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161953+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:51:15.161953+0000","last_peered":"2024-09-10T22:51:15.161953+0000","last_clean":"2024-09-10T22:51:15.161953+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:51:15.161953+0000","last_undegraded":"2024-09-10T22:51:15.161953+0000","last_fullsized":"2024-09-10T22:51:15.161953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154833+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:51:15.154833+0000","last_peered":"2024-09-10T22:51:15.154833+0000","last_clean":"2024-09-10T22:51:15.154833+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:51:15.154833+0000","last_undegraded":"2024-09-10T22:51:15.154833+0000","last_fullsized":"2024-09-10T22:51:15.154833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295992+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:51:15.295992+0000","last_peered":"2024-09-10T22:51:15.295992+0000","last_clean":"2024-09-10T22:51:15.295992+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:51:15.295992+0000","last_undegraded":"2024-09-10T22:51:15.295992+0000","last_fullsized":"2024-09-10T22:51:15.295992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162544+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:51:15.162544+0000","last_peered":"2024-09-10T22:51:15.162544+0000","last_clean":"2024-09-10T22:51:15.162544+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:51:15.162544+0000","last_undegraded":"2024-09-10T22:51:15.162544+0000","last_fullsized":"2024-09-10T22:51:15.162544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295742+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:51:15.295742+0000","last_peered":"2024-09-10T22:51:15.295742+0000","last_clean":"2024-09-10T22:51:15.295742+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:51:15.295742+0000","last_undegraded":"2024-09-10T22:51:15.295742+0000","last_fullsized":"2024-09-10T22:51:15.295742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.173507+0000","last_change":"2024-09-10T22:51:15.173004+0000","last_active":"2024-09-10T22:51:15.173507+0000","last_peered":"2024-09-10T22:51:15.173507+0000","last_clean":"2024-09-10T22:51:15.173507+0000","last_became_active":"2024-09-10T22:51:15.165748+0000","last_became_peered":"2024-09-10T22:51:15.165748+0000","last_unstale":"2024-09-10T22:51:15.173507+0000","last_undegraded":"2024-09-10T22:51:15.173507+0000","last_fullsized":"2024-09-10T22:51:15.173507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:38.001944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153491+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:51:15.153491+0000","last_peered":"2024-09-10T22:51:15.153491+0000","last_clean":"2024-09-10T22:51:15.153491+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:51:15.153491+0000","last_undegraded":"2024-09-10T22:51:15.153491+0000","last_fullsized":"2024-09-10T22:51:15.153491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295425+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:51:15.295425+0000","last_peered":"2024-09-10T22:51:15.295425+0000","last_clean":"2024-09-10T22:51:15.295425+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:51:15.295425+0000","last_undegraded":"2024-09-10T22:51:15.295425+0000","last_fullsized":"2024-09-10T22:51:15.295425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295318+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:51:15.295318+0000","last_peered":"2024-09-10T22:51:15.295318+0000","last_clean":"2024-09-10T22:51:15.295318+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:51:15.295318+0000","last_undegraded":"2024-09-10T22:51:15.295318+0000","last_fullsized":"2024-09-10T22:51:15.295318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150890+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:51:15.150890+0000","last_peered":"2024-09-10T22:51:15.150890+0000","last_clean":"2024-09-10T22:51:15.150890+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:51:15.150890+0000","last_undegraded":"2024-09-10T22:51:15.150890+0000","last_fullsized":"2024-09-10T22:51:15.150890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62856,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":200,"num_read_kb":163,"num_write":84,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":188568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1507328,"data_stored":1057409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":831488,"data_stored":405254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":405276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1429504,"data_stored":995739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1437696,"data_stored":995339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":404694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":466764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":897024,"data_stored":466422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:52:08.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.695+0000 7ff51240c640 1 -- 172.21.15.120:0/4111876952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1029f0 msgr2=0x7ff50c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.695+0000 7ff51240c640 1 --2- 172.21.15.120:0/4111876952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1029f0 0x7ff50c102e70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff500009930 tx=0x7ff50002a420 comp rx=0 tx=0).stop 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- 172.21.15.120:0/4111876952 shutdown_connections 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- 172.21.15.120:0/4111876952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1033b0 0x7ff50c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- 172.21.15.120:0/4111876952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1029f0 0x7ff50c102e70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- 172.21.15.120:0/4111876952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1018a0 0x7ff50c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- 172.21.15.120:0/4111876952 >> 172.21.15.120:0/4111876952 conn(0x7ff50c0fd180 msgr2=0x7ff50c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- 172.21.15.120:0/4111876952 shutdown_connections 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- 172.21.15.120:0/4111876952 wait complete. 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 Processor -- start 2024-09-10T22:52:08.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- start start 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 0x7ff50c06ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 0x7ff50c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10d510 con 0x7ff50c1033b0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10d390 con 0x7ff50c1018a0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.696+0000 7ff51240c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c10d690 con 0x7ff50c1029f0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57086/0 (socket says 172.21.15.120:57086) 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 -- 172.21.15.120:0/2260454226 learned_addr learned my addr 172.21.15.120:0/2260454226 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff50bfff640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 0x7ff50c06ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff50b7fe640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 0x7ff50c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 -- 172.21.15.120:0/2260454226 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 msgr2=0x7ff50c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 0x7ff50c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 -- 172.21.15.120:0/2260454226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 msgr2=0x7ff50c06ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 0x7ff50c06ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 -- 172.21.15.120:0/2260454226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1a08f0 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff50bfff640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 0x7ff50c06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff50b7fe640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 0x7ff50c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5109a9640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff50400cce0 tx=0x7ff50400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400f450 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50400faa0 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1a0be0 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504018b70 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.697+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1a1130 con 0x7ff50c1033b0 2024-09-10T22:52:08.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.698+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c108b90 con 0x7ff50c1033b0 2024-09-10T22:52:08.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.699+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff50400f5f0 con 0x7ff50c1033b0 2024-09-10T22:52:08.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.700+0000 7ff5097fa640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 0x7ff4f40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:08.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.700+0000 7ff50bfff640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 0x7ff4f40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:08.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.700+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff5040e6500 con 0x7ff50c1033b0 2024-09-10T22:52:08.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.700+0000 7ff50bfff640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 0x7ff4f40b78b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0058c0 tx=0x7ff4fc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:08.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.704+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff504014030 con 0x7ff50c1033b0 2024-09-10T22:52:08.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.876+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff50c063590 con 0x7ff4f40b53f0 2024-09-10T22:52:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.880+0000 7ff5097fa640 1 -- 172.21.15.120:0/2260454226 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258631 (secure 0 0 0) 0x7ff50c063590 con 0x7ff4f40b53f0 2024-09-10T22:52:08.884 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:52:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 msgr2=0x7ff4f40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 0x7ff4f40b78b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0058c0 tx=0x7ff4fc005d30 comp rx=0 tx=0).stop 2024-09-10T22:52:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 msgr2=0x7ff50c1a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff50400cce0 tx=0x7ff50400a620 comp rx=0 tx=0).stop 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 shutdown_connections 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4f40b53f0 0x7ff4f40b78b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff50c1033b0 0x7ff50c1a0030 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff50c1029f0 0x7ff50c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 --2- 172.21.15.120:0/2260454226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c1018a0 0x7ff50c06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 >> 172.21.15.120:0/2260454226 conn(0x7ff50c0fd180 msgr2=0x7ff50c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 shutdown_connections 2024-09-10T22:52:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:08.883+0000 7ff51240c640 1 -- 172.21.15.120:0/2260454226 wait complete. 2024-09-10T22:52:08.895 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:08 smithi120 ceph-mon[31247]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T22:52:08.896 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-10T22:52:08.171758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653458,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":227,"num_write":209,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377684,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564140,"statfs":{"total":1151940427776,"available":1151553679360,"internally_reserved":0,"allocated":12005376,"data_stored":6816130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003084"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156012+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:51:15.156012+0000","last_peered":"2024-09-10T22:51:15.156012+0000","last_clean":"2024-09-10T22:51:15.156012+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:51:15.156012+0000","last_undegraded":"2024-09-10T22:51:15.156012+0000","last_fullsized":"2024-09-10T22:51:15.156012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154505+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:51:15.154505+0000","last_peered":"2024-09-10T22:51:15.154505+0000","last_clean":"2024-09-10T22:51:15.154505+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:51:15.154505+0000","last_undegraded":"2024-09-10T22:51:15.154505+0000","last_fullsized":"2024-09-10T22:51:15.154505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153680+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:51:15.153680+0000","last_peered":"2024-09-10T22:51:15.153680+0000","last_clean":"2024-09-10T22:51:15.153680+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:51:15.153680+0000","last_undegraded":"2024-09-10T22:51:15.153680+0000","last_fullsized":"2024-09-10T22:51:15.153680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153157+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:51:15.153157+0000","last_peered":"2024-09-10T22:51:15.153157+0000","last_clean":"2024-09-10T22:51:15.153157+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:51:15.153157+0000","last_undegraded":"2024-09-10T22:51:15.153157+0000","last_fullsized":"2024-09-10T22:51:15.153157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161803+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:51:15.161803+0000","last_peered":"2024-09-10T22:51:15.161803+0000","last_clean":"2024-09-10T22:51:15.161803+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:51:15.161803+0000","last_undegraded":"2024-09-10T22:51:15.161803+0000","last_fullsized":"2024-09-10T22:51:15.161803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150981+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.150981+0000","last_peered":"2024-09-10T22:51:15.150981+0000","last_clean":"2024-09-10T22:51:15.150981+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:51:15.150981+0000","last_undegraded":"2024-09-10T22:51:15.150981+0000","last_fullsized":"2024-09-10T22:51:15.150981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154759+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:51:15.154759+0000","last_peered":"2024-09-10T22:51:15.154759+0000","last_clean":"2024-09-10T22:51:15.154759+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:51:15.154759+0000","last_undegraded":"2024-09-10T22:51:15.154759+0000","last_fullsized":"2024-09-10T22:51:15.154759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294963+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:51:15.294963+0000","last_peered":"2024-09-10T22:51:15.294963+0000","last_clean":"2024-09-10T22:51:15.294963+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:51:15.294963+0000","last_undegraded":"2024-09-10T22:51:15.294963+0000","last_fullsized":"2024-09-10T22:51:15.294963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296097+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:51:15.296097+0000","last_peered":"2024-09-10T22:51:15.296097+0000","last_clean":"2024-09-10T22:51:15.296097+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:51:15.296097+0000","last_undegraded":"2024-09-10T22:51:15.296097+0000","last_fullsized":"2024-09-10T22:51:15.296097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151443+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:51:15.151443+0000","last_peered":"2024-09-10T22:51:15.151443+0000","last_clean":"2024-09-10T22:51:15.151443+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:51:15.151443+0000","last_undegraded":"2024-09-10T22:51:15.151443+0000","last_fullsized":"2024-09-10T22:51:15.151443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155194+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:51:15.155194+0000","last_peered":"2024-09-10T22:51:15.155194+0000","last_clean":"2024-09-10T22:51:15.155194+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:51:15.155194+0000","last_undegraded":"2024-09-10T22:51:15.155194+0000","last_fullsized":"2024-09-10T22:51:15.155194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153263+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:51:15.153263+0000","last_peered":"2024-09-10T22:51:15.153263+0000","last_clean":"2024-09-10T22:51:15.153263+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:51:15.153263+0000","last_undegraded":"2024-09-10T22:51:15.153263+0000","last_fullsized":"2024-09-10T22:51:15.153263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154928+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:51:15.154928+0000","last_peered":"2024-09-10T22:51:15.154928+0000","last_clean":"2024-09-10T22:51:15.154928+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:51:15.154928+0000","last_undegraded":"2024-09-10T22:51:15.154928+0000","last_fullsized":"2024-09-10T22:51:15.154928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153640+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:51:15.153640+0000","last_peered":"2024-09-10T22:51:15.153640+0000","last_clean":"2024-09-10T22:51:15.153640+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:51:15.153640+0000","last_undegraded":"2024-09-10T22:51:15.153640+0000","last_fullsized":"2024-09-10T22:51:15.153640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153570+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:51:15.153570+0000","last_peered":"2024-09-10T22:51:15.153570+0000","last_clean":"2024-09-10T22:51:15.153570+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:51:15.153570+0000","last_undegraded":"2024-09-10T22:51:15.153570+0000","last_fullsized":"2024-09-10T22:51:15.153570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295653+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:51:15.295653+0000","last_peered":"2024-09-10T22:51:15.295653+0000","last_clean":"2024-09-10T22:51:15.295653+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:51:15.295653+0000","last_undegraded":"2024-09-10T22:51:15.295653+0000","last_fullsized":"2024-09-10T22:51:15.295653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156076+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:51:15.156076+0000","last_peered":"2024-09-10T22:51:15.156076+0000","last_clean":"2024-09-10T22:51:15.156076+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:51:15.156076+0000","last_undegraded":"2024-09-10T22:51:15.156076+0000","last_fullsized":"2024-09-10T22:51:15.156076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150803+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:51:15.150803+0000","last_peered":"2024-09-10T22:51:15.150803+0000","last_clean":"2024-09-10T22:51:15.150803+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:51:15.150803+0000","last_undegraded":"2024-09-10T22:51:15.150803+0000","last_fullsized":"2024-09-10T22:51:15.150803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162560+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:51:15.162560+0000","last_peered":"2024-09-10T22:51:15.162560+0000","last_clean":"2024-09-10T22:51:15.162560+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:51:15.162560+0000","last_undegraded":"2024-09-10T22:51:15.162560+0000","last_fullsized":"2024-09-10T22:51:15.162560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155723+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:51:15.155723+0000","last_peered":"2024-09-10T22:51:15.155723+0000","last_clean":"2024-09-10T22:51:15.155723+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:51:15.155723+0000","last_undegraded":"2024-09-10T22:51:15.155723+0000","last_fullsized":"2024-09-10T22:51:15.155723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155039+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:51:15.155039+0000","last_peered":"2024-09-10T22:51:15.155039+0000","last_clean":"2024-09-10T22:51:15.155039+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:51:15.155039+0000","last_undegraded":"2024-09-10T22:51:15.155039+0000","last_fullsized":"2024-09-10T22:51:15.155039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295743+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:51:15.295743+0000","last_peered":"2024-09-10T22:51:15.295743+0000","last_clean":"2024-09-10T22:51:15.295743+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:51:15.295743+0000","last_undegraded":"2024-09-10T22:51:15.295743+0000","last_fullsized":"2024-09-10T22:51:15.295743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294977+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:51:15.294977+0000","last_peered":"2024-09-10T22:51:15.294977+0000","last_clean":"2024-09-10T22:51:15.294977+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:51:15.294977+0000","last_undegraded":"2024-09-10T22:51:15.294977+0000","last_fullsized":"2024-09-10T22:51:15.294977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295461+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:51:15.295461+0000","last_peered":"2024-09-10T22:51:15.295461+0000","last_clean":"2024-09-10T22:51:15.295461+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:51:15.295461+0000","last_undegraded":"2024-09-10T22:51:15.295461+0000","last_fullsized":"2024-09-10T22:51:15.295461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295171+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:51:15.295171+0000","last_peered":"2024-09-10T22:51:15.295171+0000","last_clean":"2024-09-10T22:51:15.295171+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:51:15.295171+0000","last_undegraded":"2024-09-10T22:51:15.295171+0000","last_fullsized":"2024-09-10T22:51:15.295171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162515+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:51:15.162515+0000","last_peered":"2024-09-10T22:51:15.162515+0000","last_clean":"2024-09-10T22:51:15.162515+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:51:15.162515+0000","last_undegraded":"2024-09-10T22:51:15.162515+0000","last_fullsized":"2024-09-10T22:51:15.162515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156495+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:51:15.156495+0000","last_peered":"2024-09-10T22:51:15.156495+0000","last_clean":"2024-09-10T22:51:15.156495+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:51:15.156495+0000","last_undegraded":"2024-09-10T22:51:15.156495+0000","last_fullsized":"2024-09-10T22:51:15.156495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154828+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:51:15.154828+0000","last_peered":"2024-09-10T22:51:15.154828+0000","last_clean":"2024-09-10T22:51:15.154828+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:51:15.154828+0000","last_undegraded":"2024-09-10T22:51:15.154828+0000","last_fullsized":"2024-09-10T22:51:15.154828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154404+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:51:15.154404+0000","last_peered":"2024-09-10T22:51:15.154404+0000","last_clean":"2024-09-10T22:51:15.154404+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:51:15.154404+0000","last_undegraded":"2024-09-10T22:51:15.154404+0000","last_fullsized":"2024-09-10T22:51:15.154404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161965+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:51:15.161965+0000","last_peered":"2024-09-10T22:51:15.161965+0000","last_clean":"2024-09-10T22:51:15.161965+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:51:15.161965+0000","last_undegraded":"2024-09-10T22:51:15.161965+0000","last_fullsized":"2024-09-10T22:51:15.161965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162068+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:51:15.162068+0000","last_peered":"2024-09-10T22:51:15.162068+0000","last_clean":"2024-09-10T22:51:15.162068+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:51:15.162068+0000","last_undegraded":"2024-09-10T22:51:15.162068+0000","last_fullsized":"2024-09-10T22:51:15.162068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155610+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:51:15.155610+0000","last_peered":"2024-09-10T22:51:15.155610+0000","last_clean":"2024-09-10T22:51:15.155610+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:51:15.155610+0000","last_undegraded":"2024-09-10T22:51:15.155610+0000","last_fullsized":"2024-09-10T22:51:15.155610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161948+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:51:15.161948+0000","last_peered":"2024-09-10T22:51:15.161948+0000","last_clean":"2024-09-10T22:51:15.161948+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:51:15.161948+0000","last_undegraded":"2024-09-10T22:51:15.161948+0000","last_fullsized":"2024-09-10T22:51:15.161948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295741+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:51:15.295741+0000","last_peered":"2024-09-10T22:51:15.295741+0000","last_clean":"2024-09-10T22:51:15.295741+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:51:15.295741+0000","last_undegraded":"2024-09-10T22:51:15.295741+0000","last_fullsized":"2024-09-10T22:51:15.295741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296700+0000","last_change":"2024-09-10T22:51:15.296217+0000","last_active":"2024-09-10T22:51:15.296700+0000","last_peered":"2024-09-10T22:51:15.296700+0000","last_clean":"2024-09-10T22:51:15.296700+0000","last_became_active":"2024-09-10T22:51:15.295842+0000","last_became_peered":"2024-09-10T22:51:15.295842+0000","last_unstale":"2024-09-10T22:51:15.296700+0000","last_undegraded":"2024-09-10T22:51:15.296700+0000","last_fullsized":"2024-09-10T22:51:15.296700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:32.464281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151051+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:51:15.151051+0000","last_peered":"2024-09-10T22:51:15.151051+0000","last_clean":"2024-09-10T22:51:15.151051+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:51:15.151051+0000","last_undegraded":"2024-09-10T22:51:15.151051+0000","last_fullsized":"2024-09-10T22:51:15.151051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153986+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:51:15.153986+0000","last_peered":"2024-09-10T22:51:15.153986+0000","last_clean":"2024-09-10T22:51:15.153986+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:51:15.153986+0000","last_undegraded":"2024-09-10T22:51:15.153986+0000","last_fullsized":"2024-09-10T22:51:15.153986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295017+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:51:15.295017+0000","last_peered":"2024-09-10T22:51:15.295017+0000","last_clean":"2024-09-10T22:51:15.295017+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:51:15.295017+0000","last_undegraded":"2024-09-10T22:51:15.295017+0000","last_fullsized":"2024-09-10T22:51:15.295017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295730+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:51:15.295730+0000","last_peered":"2024-09-10T22:51:15.295730+0000","last_clean":"2024-09-10T22:51:15.295730+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:51:15.295730+0000","last_undegraded":"2024-09-10T22:51:15.295730+0000","last_fullsized":"2024-09-10T22:51:15.295730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150604+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:51:15.150604+0000","last_peered":"2024-09-10T22:51:15.150604+0000","last_clean":"2024-09-10T22:51:15.150604+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:51:15.150604+0000","last_undegraded":"2024-09-10T22:51:15.150604+0000","last_fullsized":"2024-09-10T22:51:15.150604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151268+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:51:15.151268+0000","last_peered":"2024-09-10T22:51:15.151268+0000","last_clean":"2024-09-10T22:51:15.151268+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:51:15.151268+0000","last_undegraded":"2024-09-10T22:51:15.151268+0000","last_fullsized":"2024-09-10T22:51:15.151268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295974+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:51:15.295974+0000","last_peered":"2024-09-10T22:51:15.295974+0000","last_clean":"2024-09-10T22:51:15.295974+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:51:15.295974+0000","last_undegraded":"2024-09-10T22:51:15.295974+0000","last_fullsized":"2024-09-10T22:51:15.295974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155301+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:51:15.155301+0000","last_peered":"2024-09-10T22:51:15.155301+0000","last_clean":"2024-09-10T22:51:15.155301+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:51:15.155301+0000","last_undegraded":"2024-09-10T22:51:15.155301+0000","last_fullsized":"2024-09-10T22:51:15.155301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296977+0000","last_change":"2024-09-10T22:51:15.296922+0000","last_active":"2024-09-10T22:51:15.296977+0000","last_peered":"2024-09-10T22:51:15.296977+0000","last_clean":"2024-09-10T22:51:15.296977+0000","last_became_active":"2024-09-10T22:51:15.296658+0000","last_became_peered":"2024-09-10T22:51:15.296658+0000","last_unstale":"2024-09-10T22:51:15.296977+0000","last_undegraded":"2024-09-10T22:51:15.296977+0000","last_fullsized":"2024-09-10T22:51:15.296977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:18.932940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295896+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:51:15.295896+0000","last_peered":"2024-09-10T22:51:15.295896+0000","last_clean":"2024-09-10T22:51:15.295896+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:51:15.295896+0000","last_undegraded":"2024-09-10T22:51:15.295896+0000","last_fullsized":"2024-09-10T22:51:15.295896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295846+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:51:15.295846+0000","last_peered":"2024-09-10T22:51:15.295846+0000","last_clean":"2024-09-10T22:51:15.295846+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:51:15.295846+0000","last_undegraded":"2024-09-10T22:51:15.295846+0000","last_fullsized":"2024-09-10T22:51:15.295846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'29","reported_seq":246,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:52:06.196104+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:52:06.196104+0000","last_peered":"2024-09-10T22:52:06.196104+0000","last_clean":"2024-09-10T22:52:06.196104+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:52:06.196104+0000","last_undegraded":"2024-09-10T22:52:06.196104+0000","last_fullsized":"2024-09-10T22:52:06.196104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62058,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":194,"num_read_kb":157,"num_write":29,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162574+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:51:15.162574+0000","last_peered":"2024-09-10T22:51:15.162574+0000","last_clean":"2024-09-10T22:51:15.162574+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:51:15.162574+0000","last_undegraded":"2024-09-10T22:51:15.162574+0000","last_fullsized":"2024-09-10T22:51:15.162574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162287+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:51:15.162287+0000","last_peered":"2024-09-10T22:51:15.162287+0000","last_clean":"2024-09-10T22:51:15.162287+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:51:15.162287+0000","last_undegraded":"2024-09-10T22:51:15.162287+0000","last_fullsized":"2024-09-10T22:51:15.162287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154237+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:51:15.154237+0000","last_peered":"2024-09-10T22:51:15.154237+0000","last_clean":"2024-09-10T22:51:15.154237+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:51:15.154237+0000","last_undegraded":"2024-09-10T22:51:15.154237+0000","last_fullsized":"2024-09-10T22:51:15.154237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295672+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:51:15.295672+0000","last_peered":"2024-09-10T22:51:15.295672+0000","last_clean":"2024-09-10T22:51:15.295672+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:51:15.295672+0000","last_undegraded":"2024-09-10T22:51:15.295672+0000","last_fullsized":"2024-09-10T22:51:15.295672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156806+0000","last_change":"2024-09-10T22:51:15.156697+0000","last_active":"2024-09-10T22:51:15.156806+0000","last_peered":"2024-09-10T22:51:15.156806+0000","last_clean":"2024-09-10T22:51:15.156806+0000","last_became_active":"2024-09-10T22:51:15.156396+0000","last_became_peered":"2024-09-10T22:51:15.156396+0000","last_unstale":"2024-09-10T22:51:15.156806+0000","last_undegraded":"2024-09-10T22:51:15.156806+0000","last_fullsized":"2024-09-10T22:51:15.156806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:26.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.152948+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:51:15.152948+0000","last_peered":"2024-09-10T22:51:15.152948+0000","last_clean":"2024-09-10T22:51:15.152948+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:51:15.152948+0000","last_undegraded":"2024-09-10T22:51:15.152948+0000","last_fullsized":"2024-09-10T22:51:15.152948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154683+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:51:15.154683+0000","last_peered":"2024-09-10T22:51:15.154683+0000","last_clean":"2024-09-10T22:51:15.154683+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:51:15.154683+0000","last_undegraded":"2024-09-10T22:51:15.154683+0000","last_fullsized":"2024-09-10T22:51:15.154683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162012+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:51:15.162012+0000","last_peered":"2024-09-10T22:51:15.162012+0000","last_clean":"2024-09-10T22:51:15.162012+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:51:15.162012+0000","last_undegraded":"2024-09-10T22:51:15.162012+0000","last_fullsized":"2024-09-10T22:51:15.162012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295528+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:51:15.295528+0000","last_peered":"2024-09-10T22:51:15.295528+0000","last_clean":"2024-09-10T22:51:15.295528+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:51:15.295528+0000","last_undegraded":"2024-09-10T22:51:15.295528+0000","last_fullsized":"2024-09-10T22:51:15.295528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162997+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:51:15.162997+0000","last_peered":"2024-09-10T22:51:15.162997+0000","last_clean":"2024-09-10T22:51:15.162997+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:51:15.162997+0000","last_undegraded":"2024-09-10T22:51:15.162997+0000","last_fullsized":"2024-09-10T22:51:15.162997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155841+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:51:15.155841+0000","last_peered":"2024-09-10T22:51:15.155841+0000","last_clean":"2024-09-10T22:51:15.155841+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:51:15.155841+0000","last_undegraded":"2024-09-10T22:51:15.155841+0000","last_fullsized":"2024-09-10T22:51:15.155841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154030+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:51:15.154030+0000","last_peered":"2024-09-10T22:51:15.154030+0000","last_clean":"2024-09-10T22:51:15.154030+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:51:15.154030+0000","last_undegraded":"2024-09-10T22:51:15.154030+0000","last_fullsized":"2024-09-10T22:51:15.154030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155400+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:51:15.155400+0000","last_peered":"2024-09-10T22:51:15.155400+0000","last_clean":"2024-09-10T22:51:15.155400+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:51:15.155400+0000","last_undegraded":"2024-09-10T22:51:15.155400+0000","last_fullsized":"2024-09-10T22:51:15.155400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296015+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:51:15.296015+0000","last_peered":"2024-09-10T22:51:15.296015+0000","last_clean":"2024-09-10T22:51:15.296015+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:51:15.296015+0000","last_undegraded":"2024-09-10T22:51:15.296015+0000","last_fullsized":"2024-09-10T22:51:15.296015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155019+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:51:15.155019+0000","last_peered":"2024-09-10T22:51:15.155019+0000","last_clean":"2024-09-10T22:51:15.155019+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:51:15.155019+0000","last_undegraded":"2024-09-10T22:51:15.155019+0000","last_fullsized":"2024-09-10T22:51:15.155019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.158692+0000","last_change":"2024-09-10T22:51:15.158070+0000","last_active":"2024-09-10T22:51:15.158692+0000","last_peered":"2024-09-10T22:51:15.158692+0000","last_clean":"2024-09-10T22:51:15.158692+0000","last_became_active":"2024-09-10T22:51:15.157711+0000","last_became_peered":"2024-09-10T22:51:15.157711+0000","last_unstale":"2024-09-10T22:51:15.158692+0000","last_undegraded":"2024-09-10T22:51:15.158692+0000","last_fullsized":"2024-09-10T22:51:15.158692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:11.308860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155239+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.155239+0000","last_peered":"2024-09-10T22:51:15.155239+0000","last_clean":"2024-09-10T22:51:15.155239+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:51:15.155239+0000","last_undegraded":"2024-09-10T22:51:15.155239+0000","last_fullsized":"2024-09-10T22:51:15.155239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154891+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:51:15.154891+0000","last_peered":"2024-09-10T22:51:15.154891+0000","last_clean":"2024-09-10T22:51:15.154891+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:51:15.154891+0000","last_undegraded":"2024-09-10T22:51:15.154891+0000","last_fullsized":"2024-09-10T22:51:15.154891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153104+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:51:15.153104+0000","last_peered":"2024-09-10T22:51:15.153104+0000","last_clean":"2024-09-10T22:51:15.153104+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:51:15.153104+0000","last_undegraded":"2024-09-10T22:51:15.153104+0000","last_fullsized":"2024-09-10T22:51:15.153104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295351+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:51:15.295351+0000","last_peered":"2024-09-10T22:51:15.295351+0000","last_clean":"2024-09-10T22:51:15.295351+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:51:15.295351+0000","last_undegraded":"2024-09-10T22:51:15.295351+0000","last_fullsized":"2024-09-10T22:51:15.295351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154832+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:51:15.154832+0000","last_peered":"2024-09-10T22:51:15.154832+0000","last_clean":"2024-09-10T22:51:15.154832+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:51:15.154832+0000","last_undegraded":"2024-09-10T22:51:15.154832+0000","last_fullsized":"2024-09-10T22:51:15.154832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.165878+0000","last_change":"2024-09-10T22:51:15.165814+0000","last_active":"2024-09-10T22:51:15.165878+0000","last_peered":"2024-09-10T22:51:15.165878+0000","last_clean":"2024-09-10T22:51:15.165878+0000","last_became_active":"2024-09-10T22:51:15.165305+0000","last_became_peered":"2024-09-10T22:51:15.165305+0000","last_unstale":"2024-09-10T22:51:15.165878+0000","last_undegraded":"2024-09-10T22:51:15.165878+0000","last_fullsized":"2024-09-10T22:51:15.165878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:14.680823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154077+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:51:15.154077+0000","last_peered":"2024-09-10T22:51:15.154077+0000","last_clean":"2024-09-10T22:51:15.154077+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:51:15.154077+0000","last_undegraded":"2024-09-10T22:51:15.154077+0000","last_fullsized":"2024-09-10T22:51:15.154077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295816+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:51:15.295816+0000","last_peered":"2024-09-10T22:51:15.295816+0000","last_clean":"2024-09-10T22:51:15.295816+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:51:15.295816+0000","last_undegraded":"2024-09-10T22:51:15.295816+0000","last_fullsized":"2024-09-10T22:51:15.295816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155014+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:51:15.155014+0000","last_peered":"2024-09-10T22:51:15.155014+0000","last_clean":"2024-09-10T22:51:15.155014+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:51:15.155014+0000","last_undegraded":"2024-09-10T22:51:15.155014+0000","last_fullsized":"2024-09-10T22:51:15.155014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151167+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:51:15.151167+0000","last_peered":"2024-09-10T22:51:15.151167+0000","last_clean":"2024-09-10T22:51:15.151167+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:51:15.151167+0000","last_undegraded":"2024-09-10T22:51:15.151167+0000","last_fullsized":"2024-09-10T22:51:15.151167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150766+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:51:15.150766+0000","last_peered":"2024-09-10T22:51:15.150766+0000","last_clean":"2024-09-10T22:51:15.150766+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:51:15.150766+0000","last_undegraded":"2024-09-10T22:51:15.150766+0000","last_fullsized":"2024-09-10T22:51:15.150766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151181+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:51:15.151181+0000","last_peered":"2024-09-10T22:51:15.151181+0000","last_clean":"2024-09-10T22:51:15.151181+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:51:15.151181+0000","last_undegraded":"2024-09-10T22:51:15.151181+0000","last_fullsized":"2024-09-10T22:51:15.151181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162786+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:51:15.162786+0000","last_peered":"2024-09-10T22:51:15.162786+0000","last_clean":"2024-09-10T22:51:15.162786+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:51:15.162786+0000","last_undegraded":"2024-09-10T22:51:15.162786+0000","last_fullsized":"2024-09-10T22:51:15.162786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154876+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:51:15.154876+0000","last_peered":"2024-09-10T22:51:15.154876+0000","last_clean":"2024-09-10T22:51:15.154876+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:51:15.154876+0000","last_undegraded":"2024-09-10T22:51:15.154876+0000","last_fullsized":"2024-09-10T22:51:15.154876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150866+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:51:15.150866+0000","last_peered":"2024-09-10T22:51:15.150866+0000","last_clean":"2024-09-10T22:51:15.150866+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:51:15.150866+0000","last_undegraded":"2024-09-10T22:51:15.150866+0000","last_fullsized":"2024-09-10T22:51:15.150866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295230+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:51:15.295230+0000","last_peered":"2024-09-10T22:51:15.295230+0000","last_clean":"2024-09-10T22:51:15.295230+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:51:15.295230+0000","last_undegraded":"2024-09-10T22:51:15.295230+0000","last_fullsized":"2024-09-10T22:51:15.295230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153127+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:51:15.153127+0000","last_peered":"2024-09-10T22:51:15.153127+0000","last_clean":"2024-09-10T22:51:15.153127+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:51:15.153127+0000","last_undegraded":"2024-09-10T22:51:15.153127+0000","last_fullsized":"2024-09-10T22:51:15.153127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154511+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:51:15.154511+0000","last_peered":"2024-09-10T22:51:15.154511+0000","last_clean":"2024-09-10T22:51:15.154511+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:51:15.154511+0000","last_undegraded":"2024-09-10T22:51:15.154511+0000","last_fullsized":"2024-09-10T22:51:15.154511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295736+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:51:15.295736+0000","last_peered":"2024-09-10T22:51:15.295736+0000","last_clean":"2024-09-10T22:51:15.295736+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:51:15.295736+0000","last_undegraded":"2024-09-10T22:51:15.295736+0000","last_fullsized":"2024-09-10T22:51:15.295736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161616+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:51:15.161616+0000","last_peered":"2024-09-10T22:51:15.161616+0000","last_clean":"2024-09-10T22:51:15.161616+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:51:15.161616+0000","last_undegraded":"2024-09-10T22:51:15.161616+0000","last_fullsized":"2024-09-10T22:51:15.161616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295097+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:51:15.295097+0000","last_peered":"2024-09-10T22:51:15.295097+0000","last_clean":"2024-09-10T22:51:15.295097+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:51:15.295097+0000","last_undegraded":"2024-09-10T22:51:15.295097+0000","last_fullsized":"2024-09-10T22:51:15.295097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155731+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:51:15.155731+0000","last_peered":"2024-09-10T22:51:15.155731+0000","last_clean":"2024-09-10T22:51:15.155731+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:51:15.155731+0000","last_undegraded":"2024-09-10T22:51:15.155731+0000","last_fullsized":"2024-09-10T22:51:15.155731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.297065+0000","last_change":"2024-09-10T22:51:15.296938+0000","last_active":"2024-09-10T22:51:15.297065+0000","last_peered":"2024-09-10T22:51:15.297065+0000","last_clean":"2024-09-10T22:51:15.297065+0000","last_became_active":"2024-09-10T22:51:15.296291+0000","last_became_peered":"2024-09-10T22:51:15.296291+0000","last_unstale":"2024-09-10T22:51:15.297065+0000","last_undegraded":"2024-09-10T22:51:15.297065+0000","last_fullsized":"2024-09-10T22:51:15.297065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:26.752740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295243+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:51:15.295243+0000","last_peered":"2024-09-10T22:51:15.295243+0000","last_clean":"2024-09-10T22:51:15.295243+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:51:15.295243+0000","last_undegraded":"2024-09-10T22:51:15.295243+0000","last_fullsized":"2024-09-10T22:51:15.295243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161953+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:51:15.161953+0000","last_peered":"2024-09-10T22:51:15.161953+0000","last_clean":"2024-09-10T22:51:15.161953+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:51:15.161953+0000","last_undegraded":"2024-09-10T22:51:15.161953+0000","last_fullsized":"2024-09-10T22:51:15.161953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154833+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:51:15.154833+0000","last_peered":"2024-09-10T22:51:15.154833+0000","last_clean":"2024-09-10T22:51:15.154833+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:51:15.154833+0000","last_undegraded":"2024-09-10T22:51:15.154833+0000","last_fullsized":"2024-09-10T22:51:15.154833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295992+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:51:15.295992+0000","last_peered":"2024-09-10T22:51:15.295992+0000","last_clean":"2024-09-10T22:51:15.295992+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:51:15.295992+0000","last_undegraded":"2024-09-10T22:51:15.295992+0000","last_fullsized":"2024-09-10T22:51:15.295992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162544+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:51:15.162544+0000","last_peered":"2024-09-10T22:51:15.162544+0000","last_clean":"2024-09-10T22:51:15.162544+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:51:15.162544+0000","last_undegraded":"2024-09-10T22:51:15.162544+0000","last_fullsized":"2024-09-10T22:51:15.162544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295742+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:51:15.295742+0000","last_peered":"2024-09-10T22:51:15.295742+0000","last_clean":"2024-09-10T22:51:15.295742+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:51:15.295742+0000","last_undegraded":"2024-09-10T22:51:15.295742+0000","last_fullsized":"2024-09-10T22:51:15.295742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.173507+0000","last_change":"2024-09-10T22:51:15.173004+0000","last_active":"2024-09-10T22:51:15.173507+0000","last_peered":"2024-09-10T22:51:15.173507+0000","last_clean":"2024-09-10T22:51:15.173507+0000","last_became_active":"2024-09-10T22:51:15.165748+0000","last_became_peered":"2024-09-10T22:51:15.165748+0000","last_unstale":"2024-09-10T22:51:15.173507+0000","last_undegraded":"2024-09-10T22:51:15.173507+0000","last_fullsized":"2024-09-10T22:51:15.173507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:38.001944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153491+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:51:15.153491+0000","last_peered":"2024-09-10T22:51:15.153491+0000","last_clean":"2024-09-10T22:51:15.153491+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:51:15.153491+0000","last_undegraded":"2024-09-10T22:51:15.153491+0000","last_fullsized":"2024-09-10T22:51:15.153491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295425+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:51:15.295425+0000","last_peered":"2024-09-10T22:51:15.295425+0000","last_clean":"2024-09-10T22:51:15.295425+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:51:15.295425+0000","last_undegraded":"2024-09-10T22:51:15.295425+0000","last_fullsized":"2024-09-10T22:51:15.295425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295318+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:51:15.295318+0000","last_peered":"2024-09-10T22:51:15.295318+0000","last_clean":"2024-09-10T22:51:15.295318+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:51:15.295318+0000","last_undegraded":"2024-09-10T22:51:15.295318+0000","last_fullsized":"2024-09-10T22:51:15.295318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150890+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:51:15.150890+0000","last_peered":"2024-09-10T22:51:15.150890+0000","last_clean":"2024-09-10T22:51:15.150890+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:51:15.150890+0000","last_undegraded":"2024-09-10T22:51:15.150890+0000","last_fullsized":"2024-09-10T22:51:15.150890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62856,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":200,"num_read_kb":163,"num_write":84,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":188568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1507328,"data_stored":1057409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":831488,"data_stored":405254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":405276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1429504,"data_stored":995739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1437696,"data_stored":995339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":404694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":466764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":897024,"data_stored":466422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:52:08.900 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T22:52:08.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:52:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 -- 172.21.15.120:0/1689093998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b88104290 msgr2=0x7f4b8810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 --2- 172.21.15.120:0/1689093998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b88104290 0x7f4b8810ac10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009cd0 tx=0x7f4b8002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 -- 172.21.15.120:0/1689093998 shutdown_connections 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 --2- 172.21.15.120:0/1689093998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b88104290 0x7f4b8810ac10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 --2- 172.21.15.120:0/1689093998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b881038d0 0x7f4b88103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 --2- 172.21.15.120:0/1689093998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 0x7f4b88102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 -- 172.21.15.120:0/1689093998 >> 172.21.15.120:0/1689093998 conn(0x7f4b880fe040 msgr2=0x7f4b88100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 -- 172.21.15.120:0/1689093998 shutdown_connections 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.035+0000 7f4b8759e640 1 -- 172.21.15.120:0/1689093998 wait complete. 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 Processor -- start 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 -- start start 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 0x7f4b88198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b88104290 0x7f4b88199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810e3d0 con 0x7f4b881038d0 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810e250 con 0x7f4b88104290 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8759e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810e550 con 0x7f4b88102780 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57118/0 (socket says 172.21.15.120:57118) 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 -- 172.21.15.120:0/2812496725 learned_addr learned my addr 172.21.15.120:0/2812496725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b8659c640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 0x7f4b88198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b86d9d640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b88104290 0x7f4b88199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 -- 172.21.15.120:0/2812496725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 msgr2=0x7f4b88198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 0x7f4b88198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 -- 172.21.15.120:0/2812496725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b88104290 msgr2=0x7f4b88199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b88104290 0x7f4b88199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.036+0000 7f4b7ffff640 1 -- 172.21.15.120:0/2812496725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819a5a0 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b7ffff640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000ccc0 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b700108e0 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8819a890 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70018880 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.037+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a0f60 con 0x7f4b881038d0 2024-09-10T22:52:09.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.038+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88109a90 con 0x7f4b881038d0 2024-09-10T22:52:09.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.039+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f4b70010430 con 0x7f4b881038d0 2024-09-10T22:52:09.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.040+0000 7f4b7f7fe640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 0x7f4b740b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.040+0000 7f4b8659c640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 0x7f4b740b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.040+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f4b700e60a0 con 0x7f4b881038d0 2024-09-10T22:52:09.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.040+0000 7f4b8659c640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 0x7f4b740b78b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800abe0 tx=0x7f4b7800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.044+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b700ace30 con 0x7f4b881038d0 2024-09-10T22:52:09.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:08 smithi176 ceph-mon[34982]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T22:52:09.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:08 smithi153 ceph-mon[34951]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T22:52:09.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.215+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 --> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b88063590 con 0x7f4b740b53f0 2024-09-10T22:52:09.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.219+0000 7f4b7f7fe640 1 -- 172.21.15.120:0/2812496725 <== mgr.14150 v2:172.21.15.120:6800/578504839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258631 (secure 0 0 0) 0x7f4b88063590 con 0x7f4b740b53f0 2024-09-10T22:52:09.224 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:09.225 INFO:teuthology.orchestra.run.smithi120.stderr:dumped all 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 msgr2=0x7f4b740b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 0x7f4b740b78b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800abe0 tx=0x7f4b7800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 msgr2=0x7f4b881990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000edf0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 shutdown_connections 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b740b53f0 0x7f4b740b78b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b88104290 0x7f4b88199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b881038d0 0x7f4b881990b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 --2- 172.21.15.120:0/2812496725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b88102780 0x7f4b88198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 >> 172.21.15.120:0/2812496725 conn(0x7f4b880fe040 msgr2=0x7f4b88106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 shutdown_connections 2024-09-10T22:52:09.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.222+0000 7f4b8759e640 1 -- 172.21.15.120:0/2812496725 wait complete. 2024-09-10T22:52:09.236 INFO:teuthology.orchestra.run.smithi120.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-10T22:52:08.171758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653458,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":227,"num_write":209,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377684,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564140,"statfs":{"total":1151940427776,"available":1151553679360,"internally_reserved":0,"allocated":12005376,"data_stored":6816130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003084"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156012+0000","last_change":"2024-09-10T22:50:27.006403+0000","last_active":"2024-09-10T22:51:15.156012+0000","last_peered":"2024-09-10T22:51:15.156012+0000","last_clean":"2024-09-10T22:51:15.156012+0000","last_became_active":"2024-09-10T22:50:27.002594+0000","last_became_peered":"2024-09-10T22:50:27.002594+0000","last_unstale":"2024-09-10T22:51:15.156012+0000","last_undegraded":"2024-09-10T22:51:15.156012+0000","last_fullsized":"2024-09-10T22:51:15.156012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:14.440624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154505+0000","last_change":"2024-09-10T22:50:27.006582+0000","last_active":"2024-09-10T22:51:15.154505+0000","last_peered":"2024-09-10T22:51:15.154505+0000","last_clean":"2024-09-10T22:51:15.154505+0000","last_became_active":"2024-09-10T22:50:27.006243+0000","last_became_peered":"2024-09-10T22:50:27.006243+0000","last_unstale":"2024-09-10T22:51:15.154505+0000","last_undegraded":"2024-09-10T22:51:15.154505+0000","last_fullsized":"2024-09-10T22:51:15.154505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:01.175188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153680+0000","last_change":"2024-09-10T22:50:27.009307+0000","last_active":"2024-09-10T22:51:15.153680+0000","last_peered":"2024-09-10T22:51:15.153680+0000","last_clean":"2024-09-10T22:51:15.153680+0000","last_became_active":"2024-09-10T22:50:27.008986+0000","last_became_peered":"2024-09-10T22:50:27.008986+0000","last_unstale":"2024-09-10T22:51:15.153680+0000","last_undegraded":"2024-09-10T22:51:15.153680+0000","last_fullsized":"2024-09-10T22:51:15.153680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:34.491347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153157+0000","last_change":"2024-09-10T22:50:27.016105+0000","last_active":"2024-09-10T22:51:15.153157+0000","last_peered":"2024-09-10T22:51:15.153157+0000","last_clean":"2024-09-10T22:51:15.153157+0000","last_became_active":"2024-09-10T22:50:27.014989+0000","last_became_peered":"2024-09-10T22:50:27.014989+0000","last_unstale":"2024-09-10T22:51:15.153157+0000","last_undegraded":"2024-09-10T22:51:15.153157+0000","last_fullsized":"2024-09-10T22:51:15.153157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:20.315270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161803+0000","last_change":"2024-09-10T22:50:27.004118+0000","last_active":"2024-09-10T22:51:15.161803+0000","last_peered":"2024-09-10T22:51:15.161803+0000","last_clean":"2024-09-10T22:51:15.161803+0000","last_became_active":"2024-09-10T22:50:27.003831+0000","last_became_peered":"2024-09-10T22:50:27.003831+0000","last_unstale":"2024-09-10T22:51:15.161803+0000","last_undegraded":"2024-09-10T22:51:15.161803+0000","last_fullsized":"2024-09-10T22:51:15.161803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:58.206849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150981+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.150981+0000","last_peered":"2024-09-10T22:51:15.150981+0000","last_clean":"2024-09-10T22:51:15.150981+0000","last_became_active":"2024-09-10T22:50:27.012805+0000","last_became_peered":"2024-09-10T22:50:27.012805+0000","last_unstale":"2024-09-10T22:51:15.150981+0000","last_undegraded":"2024-09-10T22:51:15.150981+0000","last_fullsized":"2024-09-10T22:51:15.150981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:45.734415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154759+0000","last_change":"2024-09-10T22:50:27.015019+0000","last_active":"2024-09-10T22:51:15.154759+0000","last_peered":"2024-09-10T22:51:15.154759+0000","last_clean":"2024-09-10T22:51:15.154759+0000","last_became_active":"2024-09-10T22:50:27.014745+0000","last_became_peered":"2024-09-10T22:50:27.014745+0000","last_unstale":"2024-09-10T22:51:15.154759+0000","last_undegraded":"2024-09-10T22:51:15.154759+0000","last_fullsized":"2024-09-10T22:51:15.154759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.448077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294963+0000","last_change":"2024-09-10T22:50:27.010166+0000","last_active":"2024-09-10T22:51:15.294963+0000","last_peered":"2024-09-10T22:51:15.294963+0000","last_clean":"2024-09-10T22:51:15.294963+0000","last_became_active":"2024-09-10T22:50:27.009845+0000","last_became_peered":"2024-09-10T22:50:27.009845+0000","last_unstale":"2024-09-10T22:51:15.294963+0000","last_undegraded":"2024-09-10T22:51:15.294963+0000","last_fullsized":"2024-09-10T22:51:15.294963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:30.827510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296097+0000","last_change":"2024-09-10T22:50:27.015796+0000","last_active":"2024-09-10T22:51:15.296097+0000","last_peered":"2024-09-10T22:51:15.296097+0000","last_clean":"2024-09-10T22:51:15.296097+0000","last_became_active":"2024-09-10T22:50:27.015488+0000","last_became_peered":"2024-09-10T22:50:27.015488+0000","last_unstale":"2024-09-10T22:51:15.296097+0000","last_undegraded":"2024-09-10T22:51:15.296097+0000","last_fullsized":"2024-09-10T22:51:15.296097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151443+0000","last_change":"2024-09-10T22:50:27.010933+0000","last_active":"2024-09-10T22:51:15.151443+0000","last_peered":"2024-09-10T22:51:15.151443+0000","last_clean":"2024-09-10T22:51:15.151443+0000","last_became_active":"2024-09-10T22:50:27.010616+0000","last_became_peered":"2024-09-10T22:50:27.010616+0000","last_unstale":"2024-09-10T22:51:15.151443+0000","last_undegraded":"2024-09-10T22:51:15.151443+0000","last_fullsized":"2024-09-10T22:51:15.151443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:35.671824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155194+0000","last_change":"2024-09-10T22:50:27.018674+0000","last_active":"2024-09-10T22:51:15.155194+0000","last_peered":"2024-09-10T22:51:15.155194+0000","last_clean":"2024-09-10T22:51:15.155194+0000","last_became_active":"2024-09-10T22:50:27.018259+0000","last_became_peered":"2024-09-10T22:50:27.018259+0000","last_unstale":"2024-09-10T22:51:15.155194+0000","last_undegraded":"2024-09-10T22:51:15.155194+0000","last_fullsized":"2024-09-10T22:51:15.155194+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:11.173704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153263+0000","last_change":"2024-09-10T22:50:27.015477+0000","last_active":"2024-09-10T22:51:15.153263+0000","last_peered":"2024-09-10T22:51:15.153263+0000","last_clean":"2024-09-10T22:51:15.153263+0000","last_became_active":"2024-09-10T22:50:27.014621+0000","last_became_peered":"2024-09-10T22:50:27.014621+0000","last_unstale":"2024-09-10T22:51:15.153263+0000","last_undegraded":"2024-09-10T22:51:15.153263+0000","last_fullsized":"2024-09-10T22:51:15.153263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:00.587535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154928+0000","last_change":"2024-09-10T22:50:27.003943+0000","last_active":"2024-09-10T22:51:15.154928+0000","last_peered":"2024-09-10T22:51:15.154928+0000","last_clean":"2024-09-10T22:51:15.154928+0000","last_became_active":"2024-09-10T22:50:27.003641+0000","last_became_peered":"2024-09-10T22:50:27.003641+0000","last_unstale":"2024-09-10T22:51:15.154928+0000","last_undegraded":"2024-09-10T22:51:15.154928+0000","last_fullsized":"2024-09-10T22:51:15.154928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:20.809014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153640+0000","last_change":"2024-09-10T22:50:27.005537+0000","last_active":"2024-09-10T22:51:15.153640+0000","last_peered":"2024-09-10T22:51:15.153640+0000","last_clean":"2024-09-10T22:51:15.153640+0000","last_became_active":"2024-09-10T22:50:27.005233+0000","last_became_peered":"2024-09-10T22:50:27.005233+0000","last_unstale":"2024-09-10T22:51:15.153640+0000","last_undegraded":"2024-09-10T22:51:15.153640+0000","last_fullsized":"2024-09-10T22:51:15.153640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:37.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153570+0000","last_change":"2024-09-10T22:50:27.017743+0000","last_active":"2024-09-10T22:51:15.153570+0000","last_peered":"2024-09-10T22:51:15.153570+0000","last_clean":"2024-09-10T22:51:15.153570+0000","last_became_active":"2024-09-10T22:50:27.017438+0000","last_became_peered":"2024-09-10T22:50:27.017438+0000","last_unstale":"2024-09-10T22:51:15.153570+0000","last_undegraded":"2024-09-10T22:51:15.153570+0000","last_fullsized":"2024-09-10T22:51:15.153570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:46.409269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295653+0000","last_change":"2024-09-10T22:50:27.012210+0000","last_active":"2024-09-10T22:51:15.295653+0000","last_peered":"2024-09-10T22:51:15.295653+0000","last_clean":"2024-09-10T22:51:15.295653+0000","last_became_active":"2024-09-10T22:50:27.011763+0000","last_became_peered":"2024-09-10T22:50:27.011763+0000","last_unstale":"2024-09-10T22:51:15.295653+0000","last_undegraded":"2024-09-10T22:51:15.295653+0000","last_fullsized":"2024-09-10T22:51:15.295653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:57.364268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156076+0000","last_change":"2024-09-10T22:50:27.005305+0000","last_active":"2024-09-10T22:51:15.156076+0000","last_peered":"2024-09-10T22:51:15.156076+0000","last_clean":"2024-09-10T22:51:15.156076+0000","last_became_active":"2024-09-10T22:50:27.002134+0000","last_became_peered":"2024-09-10T22:50:27.002134+0000","last_unstale":"2024-09-10T22:51:15.156076+0000","last_undegraded":"2024-09-10T22:51:15.156076+0000","last_fullsized":"2024-09-10T22:51:15.156076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:47.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150803+0000","last_change":"2024-09-10T22:50:27.003810+0000","last_active":"2024-09-10T22:51:15.150803+0000","last_peered":"2024-09-10T22:51:15.150803+0000","last_clean":"2024-09-10T22:51:15.150803+0000","last_became_active":"2024-09-10T22:50:27.003493+0000","last_became_peered":"2024-09-10T22:50:27.003493+0000","last_unstale":"2024-09-10T22:51:15.150803+0000","last_undegraded":"2024-09-10T22:51:15.150803+0000","last_fullsized":"2024-09-10T22:51:15.150803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:18.923027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162560+0000","last_change":"2024-09-10T22:50:27.016061+0000","last_active":"2024-09-10T22:51:15.162560+0000","last_peered":"2024-09-10T22:51:15.162560+0000","last_clean":"2024-09-10T22:51:15.162560+0000","last_became_active":"2024-09-10T22:50:27.015742+0000","last_became_peered":"2024-09-10T22:50:27.015742+0000","last_unstale":"2024-09-10T22:51:15.162560+0000","last_undegraded":"2024-09-10T22:51:15.162560+0000","last_fullsized":"2024-09-10T22:51:15.162560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:42.709842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155723+0000","last_change":"2024-09-10T22:50:27.011322+0000","last_active":"2024-09-10T22:51:15.155723+0000","last_peered":"2024-09-10T22:51:15.155723+0000","last_clean":"2024-09-10T22:51:15.155723+0000","last_became_active":"2024-09-10T22:50:27.011009+0000","last_became_peered":"2024-09-10T22:50:27.011009+0000","last_unstale":"2024-09-10T22:51:15.155723+0000","last_undegraded":"2024-09-10T22:51:15.155723+0000","last_fullsized":"2024-09-10T22:51:15.155723+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:42.145685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155039+0000","last_change":"2024-09-10T22:50:27.007278+0000","last_active":"2024-09-10T22:51:15.155039+0000","last_peered":"2024-09-10T22:51:15.155039+0000","last_clean":"2024-09-10T22:51:15.155039+0000","last_became_active":"2024-09-10T22:50:27.005978+0000","last_became_peered":"2024-09-10T22:50:27.005978+0000","last_unstale":"2024-09-10T22:51:15.155039+0000","last_undegraded":"2024-09-10T22:51:15.155039+0000","last_fullsized":"2024-09-10T22:51:15.155039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:46.414471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295743+0000","last_change":"2024-09-10T22:50:27.011182+0000","last_active":"2024-09-10T22:51:15.295743+0000","last_peered":"2024-09-10T22:51:15.295743+0000","last_clean":"2024-09-10T22:51:15.295743+0000","last_became_active":"2024-09-10T22:50:27.010880+0000","last_became_peered":"2024-09-10T22:50:27.010880+0000","last_unstale":"2024-09-10T22:51:15.295743+0000","last_undegraded":"2024-09-10T22:51:15.295743+0000","last_fullsized":"2024-09-10T22:51:15.295743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:44.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.294977+0000","last_change":"2024-09-10T22:50:27.010091+0000","last_active":"2024-09-10T22:51:15.294977+0000","last_peered":"2024-09-10T22:51:15.294977+0000","last_clean":"2024-09-10T22:51:15.294977+0000","last_became_active":"2024-09-10T22:50:27.009096+0000","last_became_peered":"2024-09-10T22:50:27.009096+0000","last_unstale":"2024-09-10T22:51:15.294977+0000","last_undegraded":"2024-09-10T22:51:15.294977+0000","last_fullsized":"2024-09-10T22:51:15.294977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:49.122670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295461+0000","last_change":"2024-09-10T22:50:27.006508+0000","last_active":"2024-09-10T22:51:15.295461+0000","last_peered":"2024-09-10T22:51:15.295461+0000","last_clean":"2024-09-10T22:51:15.295461+0000","last_became_active":"2024-09-10T22:50:27.006153+0000","last_became_peered":"2024-09-10T22:50:27.006153+0000","last_unstale":"2024-09-10T22:51:15.295461+0000","last_undegraded":"2024-09-10T22:51:15.295461+0000","last_fullsized":"2024-09-10T22:51:15.295461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:44.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295171+0000","last_change":"2024-09-10T22:50:27.012318+0000","last_active":"2024-09-10T22:51:15.295171+0000","last_peered":"2024-09-10T22:51:15.295171+0000","last_clean":"2024-09-10T22:51:15.295171+0000","last_became_active":"2024-09-10T22:50:27.012100+0000","last_became_peered":"2024-09-10T22:50:27.012100+0000","last_unstale":"2024-09-10T22:51:15.295171+0000","last_undegraded":"2024-09-10T22:51:15.295171+0000","last_fullsized":"2024-09-10T22:51:15.295171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:03.077260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162515+0000","last_change":"2024-09-10T22:50:27.003188+0000","last_active":"2024-09-10T22:51:15.162515+0000","last_peered":"2024-09-10T22:51:15.162515+0000","last_clean":"2024-09-10T22:51:15.162515+0000","last_became_active":"2024-09-10T22:50:27.002900+0000","last_became_peered":"2024-09-10T22:50:27.002900+0000","last_unstale":"2024-09-10T22:51:15.162515+0000","last_undegraded":"2024-09-10T22:51:15.162515+0000","last_fullsized":"2024-09-10T22:51:15.162515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:57.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156495+0000","last_change":"2024-09-10T22:50:27.013607+0000","last_active":"2024-09-10T22:51:15.156495+0000","last_peered":"2024-09-10T22:51:15.156495+0000","last_clean":"2024-09-10T22:51:15.156495+0000","last_became_active":"2024-09-10T22:50:27.013302+0000","last_became_peered":"2024-09-10T22:50:27.013302+0000","last_unstale":"2024-09-10T22:51:15.156495+0000","last_undegraded":"2024-09-10T22:51:15.156495+0000","last_fullsized":"2024-09-10T22:51:15.156495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:18.607941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154828+0000","last_change":"2024-09-10T22:50:27.012018+0000","last_active":"2024-09-10T22:51:15.154828+0000","last_peered":"2024-09-10T22:51:15.154828+0000","last_clean":"2024-09-10T22:51:15.154828+0000","last_became_active":"2024-09-10T22:50:27.011724+0000","last_became_peered":"2024-09-10T22:50:27.011724+0000","last_unstale":"2024-09-10T22:51:15.154828+0000","last_undegraded":"2024-09-10T22:51:15.154828+0000","last_fullsized":"2024-09-10T22:51:15.154828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154404+0000","last_change":"2024-09-10T22:50:27.012392+0000","last_active":"2024-09-10T22:51:15.154404+0000","last_peered":"2024-09-10T22:51:15.154404+0000","last_clean":"2024-09-10T22:51:15.154404+0000","last_became_active":"2024-09-10T22:50:27.012101+0000","last_became_peered":"2024-09-10T22:50:27.012101+0000","last_unstale":"2024-09-10T22:51:15.154404+0000","last_undegraded":"2024-09-10T22:51:15.154404+0000","last_fullsized":"2024-09-10T22:51:15.154404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:36.046212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161965+0000","last_change":"2024-09-10T22:50:27.010556+0000","last_active":"2024-09-10T22:51:15.161965+0000","last_peered":"2024-09-10T22:51:15.161965+0000","last_clean":"2024-09-10T22:51:15.161965+0000","last_became_active":"2024-09-10T22:50:27.010254+0000","last_became_peered":"2024-09-10T22:50:27.010254+0000","last_unstale":"2024-09-10T22:51:15.161965+0000","last_undegraded":"2024-09-10T22:51:15.161965+0000","last_fullsized":"2024-09-10T22:51:15.161965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:47.236374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162068+0000","last_change":"2024-09-10T22:50:27.017688+0000","last_active":"2024-09-10T22:51:15.162068+0000","last_peered":"2024-09-10T22:51:15.162068+0000","last_clean":"2024-09-10T22:51:15.162068+0000","last_became_active":"2024-09-10T22:50:27.017273+0000","last_became_peered":"2024-09-10T22:50:27.017273+0000","last_unstale":"2024-09-10T22:51:15.162068+0000","last_undegraded":"2024-09-10T22:51:15.162068+0000","last_fullsized":"2024-09-10T22:51:15.162068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.161612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155610+0000","last_change":"2024-09-10T22:50:27.012020+0000","last_active":"2024-09-10T22:51:15.155610+0000","last_peered":"2024-09-10T22:51:15.155610+0000","last_clean":"2024-09-10T22:51:15.155610+0000","last_became_active":"2024-09-10T22:50:27.010179+0000","last_became_peered":"2024-09-10T22:50:27.010179+0000","last_unstale":"2024-09-10T22:51:15.155610+0000","last_undegraded":"2024-09-10T22:51:15.155610+0000","last_fullsized":"2024-09-10T22:51:15.155610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:10.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161948+0000","last_change":"2024-09-10T22:50:27.013052+0000","last_active":"2024-09-10T22:51:15.161948+0000","last_peered":"2024-09-10T22:51:15.161948+0000","last_clean":"2024-09-10T22:51:15.161948+0000","last_became_active":"2024-09-10T22:50:27.012743+0000","last_became_peered":"2024-09-10T22:50:27.012743+0000","last_unstale":"2024-09-10T22:51:15.161948+0000","last_undegraded":"2024-09-10T22:51:15.161948+0000","last_fullsized":"2024-09-10T22:51:15.161948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:16.258697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295741+0000","last_change":"2024-09-10T22:50:24.998938+0000","last_active":"2024-09-10T22:51:15.295741+0000","last_peered":"2024-09-10T22:51:15.295741+0000","last_clean":"2024-09-10T22:51:15.295741+0000","last_became_active":"2024-09-10T22:50:24.998628+0000","last_became_peered":"2024-09-10T22:50:24.998628+0000","last_unstale":"2024-09-10T22:51:15.295741+0000","last_undegraded":"2024-09-10T22:51:15.295741+0000","last_fullsized":"2024-09-10T22:51:15.295741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:23.351182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296700+0000","last_change":"2024-09-10T22:51:15.296217+0000","last_active":"2024-09-10T22:51:15.296700+0000","last_peered":"2024-09-10T22:51:15.296700+0000","last_clean":"2024-09-10T22:51:15.296700+0000","last_became_active":"2024-09-10T22:51:15.295842+0000","last_became_peered":"2024-09-10T22:51:15.295842+0000","last_unstale":"2024-09-10T22:51:15.296700+0000","last_undegraded":"2024-09-10T22:51:15.296700+0000","last_fullsized":"2024-09-10T22:51:15.296700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:32.464281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151051+0000","last_change":"2024-09-10T22:50:24.995400+0000","last_active":"2024-09-10T22:51:15.151051+0000","last_peered":"2024-09-10T22:51:15.151051+0000","last_clean":"2024-09-10T22:51:15.151051+0000","last_became_active":"2024-09-10T22:50:24.995060+0000","last_became_peered":"2024-09-10T22:50:24.995060+0000","last_unstale":"2024-09-10T22:51:15.151051+0000","last_undegraded":"2024-09-10T22:51:15.151051+0000","last_fullsized":"2024-09-10T22:51:15.151051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:38.096532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153986+0000","last_change":"2024-09-10T22:50:24.993798+0000","last_active":"2024-09-10T22:51:15.153986+0000","last_peered":"2024-09-10T22:51:15.153986+0000","last_clean":"2024-09-10T22:51:15.153986+0000","last_became_active":"2024-09-10T22:50:24.993418+0000","last_became_peered":"2024-09-10T22:50:24.993418+0000","last_unstale":"2024-09-10T22:51:15.153986+0000","last_undegraded":"2024-09-10T22:51:15.153986+0000","last_fullsized":"2024-09-10T22:51:15.153986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:37.939910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295017+0000","last_change":"2024-09-10T22:50:27.007715+0000","last_active":"2024-09-10T22:51:15.295017+0000","last_peered":"2024-09-10T22:51:15.295017+0000","last_clean":"2024-09-10T22:51:15.295017+0000","last_became_active":"2024-09-10T22:50:27.007359+0000","last_became_peered":"2024-09-10T22:50:27.007359+0000","last_unstale":"2024-09-10T22:51:15.295017+0000","last_undegraded":"2024-09-10T22:51:15.295017+0000","last_fullsized":"2024-09-10T22:51:15.295017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295730+0000","last_change":"2024-09-10T22:50:24.998697+0000","last_active":"2024-09-10T22:51:15.295730+0000","last_peered":"2024-09-10T22:51:15.295730+0000","last_clean":"2024-09-10T22:51:15.295730+0000","last_became_active":"2024-09-10T22:50:24.998357+0000","last_became_peered":"2024-09-10T22:50:24.998357+0000","last_unstale":"2024-09-10T22:51:15.295730+0000","last_undegraded":"2024-09-10T22:51:15.295730+0000","last_fullsized":"2024-09-10T22:51:15.295730+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150604+0000","last_change":"2024-09-10T22:50:27.003996+0000","last_active":"2024-09-10T22:51:15.150604+0000","last_peered":"2024-09-10T22:51:15.150604+0000","last_clean":"2024-09-10T22:51:15.150604+0000","last_became_active":"2024-09-10T22:50:27.003584+0000","last_became_peered":"2024-09-10T22:50:27.003584+0000","last_unstale":"2024-09-10T22:51:15.150604+0000","last_undegraded":"2024-09-10T22:51:15.150604+0000","last_fullsized":"2024-09-10T22:51:15.150604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:30.690491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151268+0000","last_change":"2024-09-10T22:50:24.995487+0000","last_active":"2024-09-10T22:51:15.151268+0000","last_peered":"2024-09-10T22:51:15.151268+0000","last_clean":"2024-09-10T22:51:15.151268+0000","last_became_active":"2024-09-10T22:50:24.995145+0000","last_became_peered":"2024-09-10T22:50:24.995145+0000","last_unstale":"2024-09-10T22:51:15.151268+0000","last_undegraded":"2024-09-10T22:51:15.151268+0000","last_fullsized":"2024-09-10T22:51:15.151268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:43.612471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295974+0000","last_change":"2024-09-10T22:50:27.003363+0000","last_active":"2024-09-10T22:51:15.295974+0000","last_peered":"2024-09-10T22:51:15.295974+0000","last_clean":"2024-09-10T22:51:15.295974+0000","last_became_active":"2024-09-10T22:50:27.003072+0000","last_became_peered":"2024-09-10T22:50:27.003072+0000","last_unstale":"2024-09-10T22:51:15.295974+0000","last_undegraded":"2024-09-10T22:51:15.295974+0000","last_fullsized":"2024-09-10T22:51:15.295974+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:56.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155301+0000","last_change":"2024-09-10T22:50:24.996967+0000","last_active":"2024-09-10T22:51:15.155301+0000","last_peered":"2024-09-10T22:51:15.155301+0000","last_clean":"2024-09-10T22:51:15.155301+0000","last_became_active":"2024-09-10T22:50:24.996423+0000","last_became_peered":"2024-09-10T22:50:24.996423+0000","last_unstale":"2024-09-10T22:51:15.155301+0000","last_undegraded":"2024-09-10T22:51:15.155301+0000","last_fullsized":"2024-09-10T22:51:15.155301+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:38.506935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296977+0000","last_change":"2024-09-10T22:51:15.296922+0000","last_active":"2024-09-10T22:51:15.296977+0000","last_peered":"2024-09-10T22:51:15.296977+0000","last_clean":"2024-09-10T22:51:15.296977+0000","last_became_active":"2024-09-10T22:51:15.296658+0000","last_became_peered":"2024-09-10T22:51:15.296658+0000","last_unstale":"2024-09-10T22:51:15.296977+0000","last_undegraded":"2024-09-10T22:51:15.296977+0000","last_fullsized":"2024-09-10T22:51:15.296977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:18.932940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295896+0000","last_change":"2024-09-10T22:50:25.292287+0000","last_active":"2024-09-10T22:51:15.295896+0000","last_peered":"2024-09-10T22:51:15.295896+0000","last_clean":"2024-09-10T22:51:15.295896+0000","last_became_active":"2024-09-10T22:50:25.291929+0000","last_became_peered":"2024-09-10T22:50:25.291929+0000","last_unstale":"2024-09-10T22:51:15.295896+0000","last_undegraded":"2024-09-10T22:51:15.295896+0000","last_fullsized":"2024-09-10T22:51:15.295896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:34.683878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295846+0000","last_change":"2024-09-10T22:50:27.014789+0000","last_active":"2024-09-10T22:51:15.295846+0000","last_peered":"2024-09-10T22:51:15.295846+0000","last_clean":"2024-09-10T22:51:15.295846+0000","last_became_active":"2024-09-10T22:50:27.014458+0000","last_became_peered":"2024-09-10T22:50:27.014458+0000","last_unstale":"2024-09-10T22:51:15.295846+0000","last_undegraded":"2024-09-10T22:51:15.295846+0000","last_fullsized":"2024-09-10T22:51:15.295846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:22.791059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'29","reported_seq":246,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:52:06.196104+0000","last_change":"2024-09-10T22:50:24.997077+0000","last_active":"2024-09-10T22:52:06.196104+0000","last_peered":"2024-09-10T22:52:06.196104+0000","last_clean":"2024-09-10T22:52:06.196104+0000","last_became_active":"2024-09-10T22:50:24.996403+0000","last_became_peered":"2024-09-10T22:50:24.996403+0000","last_unstale":"2024-09-10T22:52:06.196104+0000","last_undegraded":"2024-09-10T22:52:06.196104+0000","last_fullsized":"2024-09-10T22:52:06.196104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:37.475098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62058,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":194,"num_read_kb":157,"num_write":29,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162574+0000","last_change":"2024-09-10T22:50:27.011664+0000","last_active":"2024-09-10T22:51:15.162574+0000","last_peered":"2024-09-10T22:51:15.162574+0000","last_clean":"2024-09-10T22:51:15.162574+0000","last_became_active":"2024-09-10T22:50:27.011202+0000","last_became_peered":"2024-09-10T22:50:27.011202+0000","last_unstale":"2024-09-10T22:51:15.162574+0000","last_undegraded":"2024-09-10T22:51:15.162574+0000","last_fullsized":"2024-09-10T22:51:15.162574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:40.641284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162287+0000","last_change":"2024-09-10T22:50:24.997497+0000","last_active":"2024-09-10T22:51:15.162287+0000","last_peered":"2024-09-10T22:51:15.162287+0000","last_clean":"2024-09-10T22:51:15.162287+0000","last_became_active":"2024-09-10T22:50:24.997164+0000","last_became_peered":"2024-09-10T22:50:24.997164+0000","last_unstale":"2024-09-10T22:51:15.162287+0000","last_undegraded":"2024-09-10T22:51:15.162287+0000","last_fullsized":"2024-09-10T22:51:15.162287+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154237+0000","last_change":"2024-09-10T22:50:27.007835+0000","last_active":"2024-09-10T22:51:15.154237+0000","last_peered":"2024-09-10T22:51:15.154237+0000","last_clean":"2024-09-10T22:51:15.154237+0000","last_became_active":"2024-09-10T22:50:27.006125+0000","last_became_peered":"2024-09-10T22:50:27.006125+0000","last_unstale":"2024-09-10T22:51:15.154237+0000","last_undegraded":"2024-09-10T22:51:15.154237+0000","last_fullsized":"2024-09-10T22:51:15.154237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.838934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295672+0000","last_change":"2024-09-10T22:50:24.997016+0000","last_active":"2024-09-10T22:51:15.295672+0000","last_peered":"2024-09-10T22:51:15.295672+0000","last_clean":"2024-09-10T22:51:15.295672+0000","last_became_active":"2024-09-10T22:50:24.996704+0000","last_became_peered":"2024-09-10T22:50:24.996704+0000","last_unstale":"2024-09-10T22:51:15.295672+0000","last_undegraded":"2024-09-10T22:51:15.295672+0000","last_fullsized":"2024-09-10T22:51:15.295672+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:28.375165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.156806+0000","last_change":"2024-09-10T22:51:15.156697+0000","last_active":"2024-09-10T22:51:15.156806+0000","last_peered":"2024-09-10T22:51:15.156806+0000","last_clean":"2024-09-10T22:51:15.156806+0000","last_became_active":"2024-09-10T22:51:15.156396+0000","last_became_peered":"2024-09-10T22:51:15.156396+0000","last_unstale":"2024-09-10T22:51:15.156806+0000","last_undegraded":"2024-09-10T22:51:15.156806+0000","last_fullsized":"2024-09-10T22:51:15.156806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:26.820879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.152948+0000","last_change":"2024-09-10T22:50:24.992825+0000","last_active":"2024-09-10T22:51:15.152948+0000","last_peered":"2024-09-10T22:51:15.152948+0000","last_clean":"2024-09-10T22:51:15.152948+0000","last_became_active":"2024-09-10T22:50:24.992453+0000","last_became_peered":"2024-09-10T22:50:24.992453+0000","last_unstale":"2024-09-10T22:51:15.152948+0000","last_undegraded":"2024-09-10T22:51:15.152948+0000","last_fullsized":"2024-09-10T22:51:15.152948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:08.345805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154683+0000","last_change":"2024-09-10T22:50:27.016910+0000","last_active":"2024-09-10T22:51:15.154683+0000","last_peered":"2024-09-10T22:51:15.154683+0000","last_clean":"2024-09-10T22:51:15.154683+0000","last_became_active":"2024-09-10T22:50:27.016606+0000","last_became_peered":"2024-09-10T22:50:27.016606+0000","last_unstale":"2024-09-10T22:51:15.154683+0000","last_undegraded":"2024-09-10T22:51:15.154683+0000","last_fullsized":"2024-09-10T22:51:15.154683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:29.299551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162012+0000","last_change":"2024-09-10T22:50:25.292890+0000","last_active":"2024-09-10T22:51:15.162012+0000","last_peered":"2024-09-10T22:51:15.162012+0000","last_clean":"2024-09-10T22:51:15.162012+0000","last_became_active":"2024-09-10T22:50:25.292538+0000","last_became_peered":"2024-09-10T22:50:25.292538+0000","last_unstale":"2024-09-10T22:51:15.162012+0000","last_undegraded":"2024-09-10T22:51:15.162012+0000","last_fullsized":"2024-09-10T22:51:15.162012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:54.835208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295528+0000","last_change":"2024-09-10T22:50:27.014369+0000","last_active":"2024-09-10T22:51:15.295528+0000","last_peered":"2024-09-10T22:51:15.295528+0000","last_clean":"2024-09-10T22:51:15.295528+0000","last_became_active":"2024-09-10T22:50:27.013899+0000","last_became_peered":"2024-09-10T22:50:27.013899+0000","last_unstale":"2024-09-10T22:51:15.295528+0000","last_undegraded":"2024-09-10T22:51:15.295528+0000","last_fullsized":"2024-09-10T22:51:15.295528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:15.023853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162997+0000","last_change":"2024-09-10T22:50:24.996450+0000","last_active":"2024-09-10T22:51:15.162997+0000","last_peered":"2024-09-10T22:51:15.162997+0000","last_clean":"2024-09-10T22:51:15.162997+0000","last_became_active":"2024-09-10T22:50:24.996128+0000","last_became_peered":"2024-09-10T22:50:24.996128+0000","last_unstale":"2024-09-10T22:51:15.162997+0000","last_undegraded":"2024-09-10T22:51:15.162997+0000","last_fullsized":"2024-09-10T22:51:15.162997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:39.756894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155841+0000","last_change":"2024-09-10T22:50:27.011701+0000","last_active":"2024-09-10T22:51:15.155841+0000","last_peered":"2024-09-10T22:51:15.155841+0000","last_clean":"2024-09-10T22:51:15.155841+0000","last_became_active":"2024-09-10T22:50:27.010782+0000","last_became_peered":"2024-09-10T22:50:27.010782+0000","last_unstale":"2024-09-10T22:51:15.155841+0000","last_undegraded":"2024-09-10T22:51:15.155841+0000","last_fullsized":"2024-09-10T22:51:15.155841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:58.029321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154030+0000","last_change":"2024-09-10T22:50:25.000899+0000","last_active":"2024-09-10T22:51:15.154030+0000","last_peered":"2024-09-10T22:51:15.154030+0000","last_clean":"2024-09-10T22:51:15.154030+0000","last_became_active":"2024-09-10T22:50:25.000503+0000","last_became_peered":"2024-09-10T22:50:25.000503+0000","last_unstale":"2024-09-10T22:51:15.154030+0000","last_undegraded":"2024-09-10T22:51:15.154030+0000","last_fullsized":"2024-09-10T22:51:15.154030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:33.299132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155400+0000","last_change":"2024-09-10T22:50:27.013494+0000","last_active":"2024-09-10T22:51:15.155400+0000","last_peered":"2024-09-10T22:51:15.155400+0000","last_clean":"2024-09-10T22:51:15.155400+0000","last_became_active":"2024-09-10T22:50:27.009858+0000","last_became_peered":"2024-09-10T22:50:27.009858+0000","last_unstale":"2024-09-10T22:51:15.155400+0000","last_undegraded":"2024-09-10T22:51:15.155400+0000","last_fullsized":"2024-09-10T22:51:15.155400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.519541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.296015+0000","last_change":"2024-09-10T22:50:24.997748+0000","last_active":"2024-09-10T22:51:15.296015+0000","last_peered":"2024-09-10T22:51:15.296015+0000","last_clean":"2024-09-10T22:51:15.296015+0000","last_became_active":"2024-09-10T22:50:24.997420+0000","last_became_peered":"2024-09-10T22:50:24.997420+0000","last_unstale":"2024-09-10T22:51:15.296015+0000","last_undegraded":"2024-09-10T22:51:15.296015+0000","last_fullsized":"2024-09-10T22:51:15.296015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:39.297881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155019+0000","last_change":"2024-09-10T22:50:27.015844+0000","last_active":"2024-09-10T22:51:15.155019+0000","last_peered":"2024-09-10T22:51:15.155019+0000","last_clean":"2024-09-10T22:51:15.155019+0000","last_became_active":"2024-09-10T22:50:27.015568+0000","last_became_peered":"2024-09-10T22:50:27.015568+0000","last_unstale":"2024-09-10T22:51:15.155019+0000","last_undegraded":"2024-09-10T22:51:15.155019+0000","last_fullsized":"2024-09-10T22:51:15.155019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.561884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.158692+0000","last_change":"2024-09-10T22:51:15.158070+0000","last_active":"2024-09-10T22:51:15.158692+0000","last_peered":"2024-09-10T22:51:15.158692+0000","last_clean":"2024-09-10T22:51:15.158692+0000","last_became_active":"2024-09-10T22:51:15.157711+0000","last_became_peered":"2024-09-10T22:51:15.157711+0000","last_unstale":"2024-09-10T22:51:15.158692+0000","last_undegraded":"2024-09-10T22:51:15.158692+0000","last_fullsized":"2024-09-10T22:51:15.158692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:11.308860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155239+0000","last_change":"2024-09-10T22:50:27.013157+0000","last_active":"2024-09-10T22:51:15.155239+0000","last_peered":"2024-09-10T22:51:15.155239+0000","last_clean":"2024-09-10T22:51:15.155239+0000","last_became_active":"2024-09-10T22:50:27.012830+0000","last_became_peered":"2024-09-10T22:50:27.012830+0000","last_unstale":"2024-09-10T22:51:15.155239+0000","last_undegraded":"2024-09-10T22:51:15.155239+0000","last_fullsized":"2024-09-10T22:51:15.155239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:29.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154891+0000","last_change":"2024-09-10T22:49:16.873597+0000","last_active":"2024-09-10T22:51:15.154891+0000","last_peered":"2024-09-10T22:51:15.154891+0000","last_clean":"2024-09-10T22:51:15.154891+0000","last_became_active":"2024-09-10T22:49:16.867857+0000","last_became_peered":"2024-09-10T22:49:16.867857+0000","last_unstale":"2024-09-10T22:51:15.154891+0000","last_undegraded":"2024-09-10T22:51:15.154891+0000","last_fullsized":"2024-09-10T22:51:15.154891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:56.701675+0000","last_clean_scrub_stamp":"2024-09-10T22:45:56.701675+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:23.772227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153104+0000","last_change":"2024-09-10T22:50:24.992931+0000","last_active":"2024-09-10T22:51:15.153104+0000","last_peered":"2024-09-10T22:51:15.153104+0000","last_clean":"2024-09-10T22:51:15.153104+0000","last_became_active":"2024-09-10T22:50:24.992616+0000","last_became_peered":"2024-09-10T22:50:24.992616+0000","last_unstale":"2024-09-10T22:51:15.153104+0000","last_undegraded":"2024-09-10T22:51:15.153104+0000","last_fullsized":"2024-09-10T22:51:15.153104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:12.993256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295351+0000","last_change":"2024-09-10T22:50:27.010434+0000","last_active":"2024-09-10T22:51:15.295351+0000","last_peered":"2024-09-10T22:51:15.295351+0000","last_clean":"2024-09-10T22:51:15.295351+0000","last_became_active":"2024-09-10T22:50:27.009218+0000","last_became_peered":"2024-09-10T22:50:27.009218+0000","last_unstale":"2024-09-10T22:51:15.295351+0000","last_undegraded":"2024-09-10T22:51:15.295351+0000","last_fullsized":"2024-09-10T22:51:15.295351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:19.200339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154832+0000","last_change":"2024-09-10T22:50:24.998221+0000","last_active":"2024-09-10T22:51:15.154832+0000","last_peered":"2024-09-10T22:51:15.154832+0000","last_clean":"2024-09-10T22:51:15.154832+0000","last_became_active":"2024-09-10T22:50:24.997744+0000","last_became_peered":"2024-09-10T22:50:24.997744+0000","last_unstale":"2024-09-10T22:51:15.154832+0000","last_undegraded":"2024-09-10T22:51:15.154832+0000","last_fullsized":"2024-09-10T22:51:15.154832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:06.926154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.165878+0000","last_change":"2024-09-10T22:51:15.165814+0000","last_active":"2024-09-10T22:51:15.165878+0000","last_peered":"2024-09-10T22:51:15.165878+0000","last_clean":"2024-09-10T22:51:15.165878+0000","last_became_active":"2024-09-10T22:51:15.165305+0000","last_became_peered":"2024-09-10T22:51:15.165305+0000","last_unstale":"2024-09-10T22:51:15.165878+0000","last_undegraded":"2024-09-10T22:51:15.165878+0000","last_fullsized":"2024-09-10T22:51:15.165878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:14.680823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154077+0000","last_change":"2024-09-10T22:50:25.292262+0000","last_active":"2024-09-10T22:51:15.154077+0000","last_peered":"2024-09-10T22:51:15.154077+0000","last_clean":"2024-09-10T22:51:15.154077+0000","last_became_active":"2024-09-10T22:50:25.291695+0000","last_became_peered":"2024-09-10T22:50:25.291695+0000","last_unstale":"2024-09-10T22:51:15.154077+0000","last_undegraded":"2024-09-10T22:51:15.154077+0000","last_fullsized":"2024-09-10T22:51:15.154077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.672958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295816+0000","last_change":"2024-09-10T22:50:27.013197+0000","last_active":"2024-09-10T22:51:15.295816+0000","last_peered":"2024-09-10T22:51:15.295816+0000","last_clean":"2024-09-10T22:51:15.295816+0000","last_became_active":"2024-09-10T22:50:27.012914+0000","last_became_peered":"2024-09-10T22:50:27.012914+0000","last_unstale":"2024-09-10T22:51:15.295816+0000","last_undegraded":"2024-09-10T22:51:15.295816+0000","last_fullsized":"2024-09-10T22:51:15.295816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:56.406795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155014+0000","last_change":"2024-09-10T22:50:24.994202+0000","last_active":"2024-09-10T22:51:15.155014+0000","last_peered":"2024-09-10T22:51:15.155014+0000","last_clean":"2024-09-10T22:51:15.155014+0000","last_became_active":"2024-09-10T22:50:24.993866+0000","last_became_peered":"2024-09-10T22:50:24.993866+0000","last_unstale":"2024-09-10T22:51:15.155014+0000","last_undegraded":"2024-09-10T22:51:15.155014+0000","last_fullsized":"2024-09-10T22:51:15.155014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151167+0000","last_change":"2024-09-10T22:50:27.010120+0000","last_active":"2024-09-10T22:51:15.151167+0000","last_peered":"2024-09-10T22:51:15.151167+0000","last_clean":"2024-09-10T22:51:15.151167+0000","last_became_active":"2024-09-10T22:50:27.009814+0000","last_became_peered":"2024-09-10T22:50:27.009814+0000","last_unstale":"2024-09-10T22:51:15.151167+0000","last_undegraded":"2024-09-10T22:51:15.151167+0000","last_fullsized":"2024-09-10T22:51:15.151167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.245128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150766+0000","last_change":"2024-09-10T22:50:25.292217+0000","last_active":"2024-09-10T22:51:15.150766+0000","last_peered":"2024-09-10T22:51:15.150766+0000","last_clean":"2024-09-10T22:51:15.150766+0000","last_became_active":"2024-09-10T22:50:25.291865+0000","last_became_peered":"2024-09-10T22:50:25.291865+0000","last_unstale":"2024-09-10T22:51:15.150766+0000","last_undegraded":"2024-09-10T22:51:15.150766+0000","last_fullsized":"2024-09-10T22:51:15.150766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:02.851754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.151181+0000","last_change":"2024-09-10T22:50:27.002425+0000","last_active":"2024-09-10T22:51:15.151181+0000","last_peered":"2024-09-10T22:51:15.151181+0000","last_clean":"2024-09-10T22:51:15.151181+0000","last_became_active":"2024-09-10T22:50:26.999747+0000","last_became_peered":"2024-09-10T22:50:26.999747+0000","last_unstale":"2024-09-10T22:51:15.151181+0000","last_undegraded":"2024-09-10T22:51:15.151181+0000","last_fullsized":"2024-09-10T22:51:15.151181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:51.188044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162786+0000","last_change":"2024-09-10T22:50:25.292937+0000","last_active":"2024-09-10T22:51:15.162786+0000","last_peered":"2024-09-10T22:51:15.162786+0000","last_clean":"2024-09-10T22:51:15.162786+0000","last_became_active":"2024-09-10T22:50:25.292594+0000","last_became_peered":"2024-09-10T22:50:25.292594+0000","last_unstale":"2024-09-10T22:51:15.162786+0000","last_undegraded":"2024-09-10T22:51:15.162786+0000","last_fullsized":"2024-09-10T22:51:15.162786+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:53.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154876+0000","last_change":"2024-09-10T22:50:27.008172+0000","last_active":"2024-09-10T22:51:15.154876+0000","last_peered":"2024-09-10T22:51:15.154876+0000","last_clean":"2024-09-10T22:51:15.154876+0000","last_became_active":"2024-09-10T22:50:27.007652+0000","last_became_peered":"2024-09-10T22:50:27.007652+0000","last_unstale":"2024-09-10T22:51:15.154876+0000","last_undegraded":"2024-09-10T22:51:15.154876+0000","last_fullsized":"2024-09-10T22:51:15.154876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:52.466515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150866+0000","last_change":"2024-09-10T22:50:24.996330+0000","last_active":"2024-09-10T22:51:15.150866+0000","last_peered":"2024-09-10T22:51:15.150866+0000","last_clean":"2024-09-10T22:51:15.150866+0000","last_became_active":"2024-09-10T22:50:24.995935+0000","last_became_peered":"2024-09-10T22:50:24.995935+0000","last_unstale":"2024-09-10T22:51:15.150866+0000","last_undegraded":"2024-09-10T22:51:15.150866+0000","last_fullsized":"2024-09-10T22:51:15.150866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:43.268327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295230+0000","last_change":"2024-09-10T22:50:27.016700+0000","last_active":"2024-09-10T22:51:15.295230+0000","last_peered":"2024-09-10T22:51:15.295230+0000","last_clean":"2024-09-10T22:51:15.295230+0000","last_became_active":"2024-09-10T22:50:27.016352+0000","last_became_peered":"2024-09-10T22:50:27.016352+0000","last_unstale":"2024-09-10T22:51:15.295230+0000","last_undegraded":"2024-09-10T22:51:15.295230+0000","last_fullsized":"2024-09-10T22:51:15.295230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:16.000321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153127+0000","last_change":"2024-09-10T22:50:25.142699+0000","last_active":"2024-09-10T22:51:15.153127+0000","last_peered":"2024-09-10T22:51:15.153127+0000","last_clean":"2024-09-10T22:51:15.153127+0000","last_became_active":"2024-09-10T22:50:25.142406+0000","last_became_peered":"2024-09-10T22:50:25.142406+0000","last_unstale":"2024-09-10T22:51:15.153127+0000","last_undegraded":"2024-09-10T22:51:15.153127+0000","last_fullsized":"2024-09-10T22:51:15.153127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:46.049554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154511+0000","last_change":"2024-09-10T22:50:27.013755+0000","last_active":"2024-09-10T22:51:15.154511+0000","last_peered":"2024-09-10T22:51:15.154511+0000","last_clean":"2024-09-10T22:51:15.154511+0000","last_became_active":"2024-09-10T22:50:27.013339+0000","last_became_peered":"2024-09-10T22:50:27.013339+0000","last_unstale":"2024-09-10T22:51:15.154511+0000","last_undegraded":"2024-09-10T22:51:15.154511+0000","last_fullsized":"2024-09-10T22:51:15.154511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:49.578450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295736+0000","last_change":"2024-09-10T22:50:25.291903+0000","last_active":"2024-09-10T22:51:15.295736+0000","last_peered":"2024-09-10T22:51:15.295736+0000","last_clean":"2024-09-10T22:51:15.295736+0000","last_became_active":"2024-09-10T22:50:25.291653+0000","last_became_peered":"2024-09-10T22:50:25.291653+0000","last_unstale":"2024-09-10T22:51:15.295736+0000","last_undegraded":"2024-09-10T22:51:15.295736+0000","last_fullsized":"2024-09-10T22:51:15.295736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:07.066916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161616+0000","last_change":"2024-09-10T22:50:27.009944+0000","last_active":"2024-09-10T22:51:15.161616+0000","last_peered":"2024-09-10T22:51:15.161616+0000","last_clean":"2024-09-10T22:51:15.161616+0000","last_became_active":"2024-09-10T22:50:27.009611+0000","last_became_peered":"2024-09-10T22:50:27.009611+0000","last_unstale":"2024-09-10T22:51:15.161616+0000","last_undegraded":"2024-09-10T22:51:15.161616+0000","last_fullsized":"2024-09-10T22:51:15.161616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.960544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295097+0000","last_change":"2024-09-10T22:50:24.995211+0000","last_active":"2024-09-10T22:51:15.295097+0000","last_peered":"2024-09-10T22:51:15.295097+0000","last_clean":"2024-09-10T22:51:15.295097+0000","last_became_active":"2024-09-10T22:50:24.994875+0000","last_became_peered":"2024-09-10T22:50:24.994875+0000","last_unstale":"2024-09-10T22:51:15.295097+0000","last_undegraded":"2024-09-10T22:51:15.295097+0000","last_fullsized":"2024-09-10T22:51:15.295097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:54.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.155731+0000","last_change":"2024-09-10T22:50:27.009372+0000","last_active":"2024-09-10T22:51:15.155731+0000","last_peered":"2024-09-10T22:51:15.155731+0000","last_clean":"2024-09-10T22:51:15.155731+0000","last_became_active":"2024-09-10T22:50:27.008902+0000","last_became_peered":"2024-09-10T22:50:27.008902+0000","last_unstale":"2024-09-10T22:51:15.155731+0000","last_undegraded":"2024-09-10T22:51:15.155731+0000","last_fullsized":"2024-09-10T22:51:15.155731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:21.331887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.297065+0000","last_change":"2024-09-10T22:51:15.296938+0000","last_active":"2024-09-10T22:51:15.297065+0000","last_peered":"2024-09-10T22:51:15.297065+0000","last_clean":"2024-09-10T22:51:15.297065+0000","last_became_active":"2024-09-10T22:51:15.296291+0000","last_became_peered":"2024-09-10T22:51:15.296291+0000","last_unstale":"2024-09-10T22:51:15.297065+0000","last_undegraded":"2024-09-10T22:51:15.297065+0000","last_fullsized":"2024-09-10T22:51:15.297065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:26.752740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295243+0000","last_change":"2024-09-10T22:50:27.002896+0000","last_active":"2024-09-10T22:51:15.295243+0000","last_peered":"2024-09-10T22:51:15.295243+0000","last_clean":"2024-09-10T22:51:15.295243+0000","last_became_active":"2024-09-10T22:50:27.002144+0000","last_became_peered":"2024-09-10T22:50:27.002144+0000","last_unstale":"2024-09-10T22:51:15.295243+0000","last_undegraded":"2024-09-10T22:51:15.295243+0000","last_fullsized":"2024-09-10T22:51:15.295243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:47.644263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.161953+0000","last_change":"2024-09-10T22:50:24.999218+0000","last_active":"2024-09-10T22:51:15.161953+0000","last_peered":"2024-09-10T22:51:15.161953+0000","last_clean":"2024-09-10T22:51:15.161953+0000","last_became_active":"2024-09-10T22:50:24.998840+0000","last_became_peered":"2024-09-10T22:50:24.998840+0000","last_unstale":"2024-09-10T22:51:15.161953+0000","last_undegraded":"2024-09-10T22:51:15.161953+0000","last_fullsized":"2024-09-10T22:51:15.161953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:29.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.154833+0000","last_change":"2024-09-10T22:50:27.018766+0000","last_active":"2024-09-10T22:51:15.154833+0000","last_peered":"2024-09-10T22:51:15.154833+0000","last_clean":"2024-09-10T22:51:15.154833+0000","last_became_active":"2024-09-10T22:50:27.018254+0000","last_became_peered":"2024-09-10T22:50:27.018254+0000","last_unstale":"2024-09-10T22:51:15.154833+0000","last_undegraded":"2024-09-10T22:51:15.154833+0000","last_fullsized":"2024-09-10T22:51:15.154833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:39.694027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295992+0000","last_change":"2024-09-10T22:50:27.014615+0000","last_active":"2024-09-10T22:51:15.295992+0000","last_peered":"2024-09-10T22:51:15.295992+0000","last_clean":"2024-09-10T22:51:15.295992+0000","last_became_active":"2024-09-10T22:50:27.014312+0000","last_became_peered":"2024-09-10T22:50:27.014312+0000","last_unstale":"2024-09-10T22:51:15.295992+0000","last_undegraded":"2024-09-10T22:51:15.295992+0000","last_fullsized":"2024-09-10T22:51:15.295992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:40.671862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.162544+0000","last_change":"2024-09-10T22:50:24.996511+0000","last_active":"2024-09-10T22:51:15.162544+0000","last_peered":"2024-09-10T22:51:15.162544+0000","last_clean":"2024-09-10T22:51:15.162544+0000","last_became_active":"2024-09-10T22:50:24.996188+0000","last_became_peered":"2024-09-10T22:50:24.996188+0000","last_unstale":"2024-09-10T22:51:15.162544+0000","last_undegraded":"2024-09-10T22:51:15.162544+0000","last_fullsized":"2024-09-10T22:51:15.162544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:12.457955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295742+0000","last_change":"2024-09-10T22:50:27.002608+0000","last_active":"2024-09-10T22:51:15.295742+0000","last_peered":"2024-09-10T22:51:15.295742+0000","last_clean":"2024-09-10T22:51:15.295742+0000","last_became_active":"2024-09-10T22:50:27.002321+0000","last_became_peered":"2024-09-10T22:50:27.002321+0000","last_unstale":"2024-09-10T22:51:15.295742+0000","last_undegraded":"2024-09-10T22:51:15.295742+0000","last_fullsized":"2024-09-10T22:51:15.295742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:12.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.173507+0000","last_change":"2024-09-10T22:51:15.173004+0000","last_active":"2024-09-10T22:51:15.173507+0000","last_peered":"2024-09-10T22:51:15.173507+0000","last_clean":"2024-09-10T22:51:15.173507+0000","last_became_active":"2024-09-10T22:51:15.165748+0000","last_became_peered":"2024-09-10T22:51:15.165748+0000","last_unstale":"2024-09-10T22:51:15.173507+0000","last_undegraded":"2024-09-10T22:51:15.173507+0000","last_fullsized":"2024-09-10T22:51:15.173507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:38.001944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.153491+0000","last_change":"2024-09-10T22:50:27.004597+0000","last_active":"2024-09-10T22:51:15.153491+0000","last_peered":"2024-09-10T22:51:15.153491+0000","last_clean":"2024-09-10T22:51:15.153491+0000","last_became_active":"2024-09-10T22:50:27.004252+0000","last_became_peered":"2024-09-10T22:50:27.004252+0000","last_unstale":"2024-09-10T22:51:15.153491+0000","last_undegraded":"2024-09-10T22:51:15.153491+0000","last_fullsized":"2024-09-10T22:51:15.153491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:38.155961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295425+0000","last_change":"2024-09-10T22:50:25.291932+0000","last_active":"2024-09-10T22:51:15.295425+0000","last_peered":"2024-09-10T22:51:15.295425+0000","last_clean":"2024-09-10T22:51:15.295425+0000","last_became_active":"2024-09-10T22:50:25.291604+0000","last_became_peered":"2024-09-10T22:50:25.291604+0000","last_unstale":"2024-09-10T22:51:15.295425+0000","last_undegraded":"2024-09-10T22:51:15.295425+0000","last_fullsized":"2024-09-10T22:51:15.295425+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:50.045417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.295318+0000","last_change":"2024-09-10T22:50:27.010198+0000","last_active":"2024-09-10T22:51:15.295318+0000","last_peered":"2024-09-10T22:51:15.295318+0000","last_clean":"2024-09-10T22:51:15.295318+0000","last_became_active":"2024-09-10T22:50:27.009929+0000","last_became_peered":"2024-09-10T22:50:27.009929+0000","last_unstale":"2024-09-10T22:51:15.295318+0000","last_undegraded":"2024-09-10T22:51:15.295318+0000","last_fullsized":"2024-09-10T22:51:15.295318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:25.985438+0000","last_clean_scrub_stamp":"2024-09-10T22:50:25.985438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:54.118966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:51:15.150890+0000","last_change":"2024-09-10T22:50:24.998480+0000","last_active":"2024-09-10T22:51:15.150890+0000","last_peered":"2024-09-10T22:51:15.150890+0000","last_clean":"2024-09-10T22:51:15.150890+0000","last_became_active":"2024-09-10T22:50:24.998193+0000","last_became_peered":"2024-09-10T22:50:24.998193+0000","last_unstale":"2024-09-10T22:51:15.150890+0000","last_undegraded":"2024-09-10T22:51:15.150890+0000","last_fullsized":"2024-09-10T22:51:15.150890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:50:23.979293+0000","last_clean_scrub_stamp":"2024-09-10T22:50:23.979293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:48.753469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62856,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":200,"num_read_kb":163,"num_write":84,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":188568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1507328,"data_stored":1057409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":831488,"data_stored":405254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31914434},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":835584,"data_stored":405276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1429504,"data_stored":995739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1437696,"data_stored":995339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":404694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":466764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":897024,"data_stored":466422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":831488,"data_stored":404762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:52:09.241 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:09.896 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:09 smithi120 ceph-mon[31247]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:09.896 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:09 smithi120 ceph-mon[31247]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 -- 172.21.15.120:0/520260592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac98102dc0 msgr2=0x7fac981031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 --2- 172.21.15.120:0/520260592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac98102dc0 0x7fac981031a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac94009900 tx=0x7fac9402a410 comp rx=0 tx=0).stop 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 -- 172.21.15.120:0/520260592 shutdown_connections 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 --2- 172.21.15.120:0/520260592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98109150 0x7fac9810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 --2- 172.21.15.120:0/520260592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac981087d0 0x7fac98108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 --2- 172.21.15.120:0/520260592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac98102dc0 0x7fac981031a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.974+0000 7fac9fbad640 1 -- 172.21.15.120:0/520260592 >> 172.21.15.120:0/520260592 conn(0x7fac980fe830 msgr2=0x7fac98100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.975+0000 7fac9fbad640 1 -- 172.21.15.120:0/520260592 shutdown_connections 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.975+0000 7fac9fbad640 1 -- 172.21.15.120:0/520260592 wait complete. 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 Processor -- start 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 -- start start 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 0x7fac98199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 0x7fac9819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fac98112e50 con 0x7fac981087d0 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fac98112cd0 con 0x7fac98102dc0 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9fbad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fac98112fd0 con 0x7fac98109150 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57128/0 (socket says 172.21.15.120:57128) 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 -- 172.21.15.120:0/3623847533 learned_addr learned my addr 172.21.15.120:0/3623847533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d949640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 0x7fac98199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9e14a640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 0x7fac9819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 -- 172.21.15.120:0/3623847533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 msgr2=0x7fac9819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 0x7fac9819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 -- 172.21.15.120:0/3623847533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 msgr2=0x7fac98199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 0x7fac98199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 -- 172.21.15.120:0/3623847533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9819f4f0 con 0x7fac981087d0 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d949640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 0x7fac98199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9e14a640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 0x7fac9819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac9d148640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac8800b4a0 tx=0x7fac8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.976+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88014070 con 0x7fac981087d0 2024-09-10T22:52:09.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.977+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88010ac0 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.977+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9819f7e0 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.977+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88017c30 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.977+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac981a5ea0 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.978+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac98103220 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.979+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fac88010610 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.980+0000 7fac8effd640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 0x7fac840b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.980+0000 7fac9d949640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 0x7fac840b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.980+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fac880e5740 con 0x7fac981087d0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.980+0000 7fac9d949640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 0x7fac840b7830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fac94031440 tx=0x7fac940095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:09.983+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac880ac4d0 con 0x7fac981087d0 2024-09-10T22:52:10.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:09 smithi176 ceph-mon[34982]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:10.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:09 smithi176 ceph-mon[34982]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:09 smithi153 ceph-mon[34951]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:09 smithi153 ceph-mon[34951]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:10.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.197+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac9819a570 con 0x7fac981087d0 2024-09-10T22:52:10.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.197+0000 7fac8effd640 1 -- 172.21.15.120:0/3623847533 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7fac880abdd0 con 0x7fac981087d0 2024-09-10T22:52:10.200 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:10.200 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:52:10.200 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 msgr2=0x7fac840b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 0x7fac840b7830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fac94031440 tx=0x7fac940095f0 comp rx=0 tx=0).stop 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 msgr2=0x7fac98199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac8800b4a0 tx=0x7fac8800b960 comp rx=0 tx=0).stop 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 shutdown_connections 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fac840b5370 0x7fac840b7830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:10.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac98109150 0x7fac9819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:10.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fac981087d0 0x7fac98199980 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:10.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 --2- 172.21.15.120:0/3623847533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac98102dc0 0x7fac98199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:10.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 >> 172.21.15.120:0/3623847533 conn(0x7fac980fe830 msgr2=0x7fac98100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:10.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 shutdown_connections 2024-09-10T22:52:10.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:10.200+0000 7fac9fbad640 1 -- 172.21.15.120:0/3623847533 wait complete. 2024-09-10T22:52:10.948 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:52:11.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:10 smithi176 ceph-mon[34982]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:11.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:10 smithi176 ceph-mon[34982]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T22:52:11.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:10 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3623847533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:11.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:10 smithi120 ceph-mon[31247]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:10 smithi120 ceph-mon[31247]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T22:52:11.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3623847533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:10 smithi153 ceph-mon[34951]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:52:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:10 smithi153 ceph-mon[34951]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T22:52:11.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:10 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3623847533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:11.213 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- 172.21.15.120:0/2673246298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 msgr2=0x7fbd181020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/2673246298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd181020b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009900 tx=0x7fbd0802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- 172.21.15.120:0/2673246298 shutdown_connections 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/2673246298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18108060 0x7fbd1810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/2673246298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 0x7fbd18107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/2673246298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd181020b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- 172.21.15.120:0/2673246298 >> 172.21.15.120:0/2673246298 conn(0x7fbd180fd720 msgr2=0x7fbd180ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.748 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- 172.21.15.120:0/2673246298 shutdown_connections 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- 172.21.15.120:0/2673246298 wait complete. 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 Processor -- start 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 -- start start 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.718+0000 7fbd1dc50640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 0x7fbd18199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18108060 0x7fbd1819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112e40 con 0x7fbd18101cd0 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112cc0 con 0x7fbd18108060 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18112fc0 con 0x7fbd181076e0 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57146/0 (socket says 172.21.15.120:57146) 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 -- 172.21.15.120:0/4244929896 learned_addr learned my addr 172.21.15.120:0/4244929896 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:11.749 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd0ffff640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 0x7fbd18199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 -- 172.21.15.120:0/4244929896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 msgr2=0x7fbd18199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 0x7fbd18199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 -- 172.21.15.120:0/4244929896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18108060 msgr2=0x7fbd1819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18108060 0x7fbd1819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 -- 172.21.15.120:0/4244929896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1819f3e0 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd177fe640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006010 tx=0x7fbd080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0803f030 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08003800 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1819f670 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08035070 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.719+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a5d90 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.720+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd181020b0 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.721+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fbd08031b10 con 0x7fbd18101cd0 2024-09-10T22:52:11.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.723+0000 7fbd157fa640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 0x7fbd040b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.723+0000 7fbd0ffff640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 0x7fbd040b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.723+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fbd081069b0 con 0x7fbd18101cd0 2024-09-10T22:52:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.723+0000 7fbd0ffff640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 0x7fbd040b78b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd181011b0 tx=0x7fbd00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.726+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbd080cd6e0 con 0x7fbd18101cd0 2024-09-10T22:52:11.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.901+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd18109ea0 con 0x7fbd18101cd0 2024-09-10T22:52:11.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.902+0000 7fbd157fa640 1 -- 172.21.15.120:0/4244929896 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22746 (secure 0 0 0) 0x7fbd080ccfe0 con 0x7fbd18101cd0 2024-09-10T22:52:11.905 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:11.905 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":77,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:51:15.147106+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.904+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 msgr2=0x7fbd040b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.904+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 0x7fbd040b78b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd181011b0 tx=0x7fbd00008040 comp rx=0 tx=0).stop 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.904+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 msgr2=0x7fbd18199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.904+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd08006010 tx=0x7fbd080042f0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 shutdown_connections 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbd040b53f0 0x7fbd040b78b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd18108060 0x7fbd1819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd181076e0 0x7fbd18199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 --2- 172.21.15.120:0/4244929896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18101cd0 0x7fbd18199330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 >> 172.21.15.120:0/4244929896 conn(0x7fbd180fd720 msgr2=0x7fbd180ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 shutdown_connections 2024-09-10T22:52:11.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:11.905+0000 7fbd1dc50640 1 -- 172.21.15.120:0/4244929896 wait complete. 2024-09-10T22:52:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:11 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4244929896' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:11 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4244929896' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:11 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4244929896' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:52:12.598 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T22:52:12.598 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T22:52:12.609 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T22:52:12.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T22:52:12.610 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T22:52:12.610 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:52:12.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:52:12.612 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T22:52:12.627 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:52:12.628 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T22:52:12.628 INFO:tasks.workunit:timeout=3h 2024-09-10T22:52:12.628 INFO:tasks.workunit:cleanup=True 2024-09-10T22:52:12.628 DEBUG:teuthology.orchestra.run.smithi120:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 54h/84d Inode: 1099511628283 Links: 2 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-09-10 22:50:43.583652259 +0000 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-09-10 22:50:43.583652259 +0000 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-09-10 22:52:05.002795120 +0000 2024-09-10T22:52:12.646 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-09-10 22:50:43.583652259 +0000 2024-09-10T22:52:12.647 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T22:52:12.647 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T22:52:12.718 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:52:12.774 INFO:tasks.workunit.client.0.smithi120.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T22:52:12.868 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:13.307 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:12 smithi120 ceph-mon[31247]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:13.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:12 smithi176 ceph-mon[34982]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:13.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 -- 172.21.15.120:0/679810357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c1023f0 msgr2=0x7f186c1068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 --2- 172.21.15.120:0/679810357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c1023f0 0x7f186c1068c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 -- 172.21.15.120:0/679810357 shutdown_connections 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 --2- 172.21.15.120:0/679810357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c106f10 0x7f186c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 --2- 172.21.15.120:0/679810357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c1023f0 0x7f186c1068c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 --2- 172.21.15.120:0/679810357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 0x7f186c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 -- 172.21.15.120:0/679810357 >> 172.21.15.120:0/679810357 conn(0x7f186c0fd720 msgr2=0x7f186c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 -- 172.21.15.120:0/679810357 shutdown_connections 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.389+0000 7f18739ed640 1 -- 172.21.15.120:0/679810357 wait complete. 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 Processor -- start 2024-09-10T22:52:13.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 -- start start 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 0x7f186c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 0x7f186c19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1193f0 con 0x7f186c106f10 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f186c119270 con 0x7f186c1023f0 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f18739ed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f186c119570 con 0x7f186c101ad0 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.390+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60456/0 (socket says 172.21.15.120:60456) 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 -- 172.21.15.120:0/3297242403 learned_addr learned my addr 172.21.15.120:0/3297242403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871789640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 0x7f186c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1870f88640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 0x7f186c19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 -- 172.21.15.120:0/3297242403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 msgr2=0x7f186c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 0x7f186c19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 -- 172.21.15.120:0/3297242403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 msgr2=0x7f186c19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 0x7f186c19fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 -- 172.21.15.120:0/3297242403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1a59f0 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1870f88640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 0x7f186c19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f186400eaa0 tx=0x7f186400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f18627fc640 1 -- 172.21.15.120:0/3297242403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186400cdb0 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1a5ce0 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f18627fc640 1 -- 172.21.15.120:0/3297242403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1864005020 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f18627fc640 1 -- 172.21.15.120:0/3297242403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640116d0 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.391+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1ac420 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.394+0000 7f1875dc2740 1 Processor -- start 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.394+0000 7f18627fc640 1 -- 172.21.15.120:0/3297242403 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f186400f040 con 0x7f186c106f10 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.394+0000 7f1875dc2740 1 -- start start 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234eda8790 0x56234edaab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56234ed12e60 con 0x56234ed9bee0 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56234ed02290 con 0x56234ed9c800 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1875dc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56234ed060a0 con 0x56234eda8790 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60470/0 (socket says 172.21.15.120:60470) 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 -- 172.21.15.120:0/2609215534 learned_addr learned my addr 172.21.15.120:0/2609215534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1871789640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234eda8790 0x56234edaab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 -- 172.21.15.120:0/2609215534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234eda8790 msgr2=0x56234edaab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234eda8790 0x56234edaab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 -- 172.21.15.120:0/2609215534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 msgr2=0x56234eda8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 -- 172.21.15.120:0/2609215534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234ea60930 con 0x56234ed9bee0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1871789640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f18627fc640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 0x7f18580b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1870f88640 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda98ff997b35c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f1871789640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 0x7f18580b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.395+0000 7f18627fc640 1 -- 172.21.15.120:0/3297242403 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f18640e5c50 con 0x7f186c106f10 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f18577fe640 1 -- 172.21.15.120:0/2609215534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c043020 con 0x56234ed9bee0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f18577fe640 1 -- 172.21.15.120:0/2609215534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c004030 con 0x56234ed9bee0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f18577fe640 1 -- 172.21.15.120:0/2609215534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c035070 con 0x56234ed9bee0 2024-09-10T22:52:13.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 -- 172.21.15.120:0/2609215534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 msgr2=0x56234ed9c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1871789640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 0x7f18580b7830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56234ed9f060 tx=0x7f1868008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c0095e0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 -- 172.21.15.120:0/2609215534 shutdown_connections 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234eda8790 0x56234edaab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 --2- 172.21.15.120:0/2609215534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9bee0 0x56234ed9c2c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 -- 172.21.15.120:0/2609215534 >> 172.21.15.120:0/2609215534 conn(0x56234ed99d10 msgr2=0x56234ed9aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 -- 172.21.15.120:0/2609215534 shutdown_connections 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.396+0000 7f1875dc2740 1 -- 172.21.15.120:0/2609215534 wait complete. 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 -- start start 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 Processor -- start 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 0x56234eda5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9c800 0x56234eda5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56234eb63890 con 0x56234eda8790 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56234ec9cd40 con 0x56234ed9bee0 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56234edab3e0 con 0x56234ed9c800 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1875dc2740 10 client.0 _ll_register_callbacks cb 0x56234ed00c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.397+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60484/0 (socket says 172.21.15.120:60484) 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 -- 172.21.15.120:0/4198584840 learned_addr learned my addr 172.21.15.120:0/4198584840 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1870f88640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9c800 0x56234eda5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871789640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 0x56234eda5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 -- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9c800 msgr2=0x56234eda5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9c800 0x56234eda5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 -- 172.21.15.120:0/4198584840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 msgr2=0x56234eda5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 0x56234eda5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 -- 172.21.15.120:0/4198584840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234edb47f0 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871789640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 0x56234eda5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1871f8a640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f18640a9550 tx=0x7f18640ac9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd3c489d2cb25c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401f030 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f18640cf070 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c004030 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f18640b5870 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1875dc2740 10 client.15264 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56234edb7200 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.398+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c004030 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f18640a7c30 con 0x56234eda8790 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 10 client.15264 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:13.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 10 client.15264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 20 client.15264 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234edb86d0 0x56234edbab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 4 client.15264 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234ed14db0 con 0x56234edb86d0 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1871789640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234edb86d0 0x56234edbab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1871789640 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234edb86d0 0x56234edbab50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5f2f1d14a34856 server_cookie=bfd6d873e4e0bf80 in_seq=0 out_seq=0 2024-09-10T22:52:13.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.399+0000 7f1854ff9640 10 client.15264 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:13.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1854ff9640 1 -- 172.21.15.120:0/4198584840 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56234ed14db0 con 0x56234edb86d0 2024-09-10T22:52:13.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1854ff9640 10 client.15264 handle_command_reply: tid=0 2024-09-10T22:52:13.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 client.15264 shutdown 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 msgr2=0x56234eda61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f18640a9550 tx=0x7f18640ac9e0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 shutdown_connections 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234edb86d0 0x56234edbab50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda61d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9c800 0x56234eda5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.400+0000 7f1875dc2740 1 --2- 172.21.15.120:0/4198584840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9bee0 0x56234eda5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.401+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 >> 172.21.15.120:0/4198584840 conn(0x56234ed99d10 msgr2=0x56234ed98f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.401+0000 7f1854ff9640 0 client.15264 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.401+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 shutdown_connections 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.401+0000 7f1875dc2740 1 -- 172.21.15.120:0/4198584840 wait complete. 2024-09-10T22:52:13.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.401+0000 7f1875dc2740 20 client.15264 trim_cache size 0 max 16384 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 Processor -- start 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 -- start start 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234eda4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b28a0 con 0x56234eda8790 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b5120 con 0x56234ed9c800 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1875dc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b2f10 con 0x56234ed9bee0 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1871f8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60490/0 (socket says 172.21.15.120:60490) 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1871f8a640 1 -- 172.21.15.120:0/3750664923 learned_addr learned my addr 172.21.15.120:0/3750664923 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.418+0000 7f1870f88640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871789640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234eda4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 -- 172.21.15.120:0/3750664923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 msgr2=0x56234eda4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234eda4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 -- 172.21.15.120:0/3750664923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 msgr2=0x56234eda5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 -- 172.21.15.120:0/3750664923 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234ed9a790 con 0x56234eda8790 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871789640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234eda4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1871f8a640 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f18640a88b0 tx=0x7f18640a7e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf059afb1607231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1856ffd640 1 -- 172.21.15.120:0/3750664923 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640c1e60 con 0x56234eda8790 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1856ffd640 1 -- 172.21.15.120:0/3750664923 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640cf070 con 0x56234eda8790 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.419+0000 7f1856ffd640 1 -- 172.21.15.120:0/3750664923 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401f030 con 0x56234eda8790 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- 172.21.15.120:0/3750664923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 msgr2=0x56234eda2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f18640a88b0 tx=0x7f18640a7e00 comp rx=0 tx=0).stop 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- 172.21.15.120:0/3750664923 shutdown_connections 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234eda8790 0x56234eda2960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234ed9c800 0x56234eda5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- 172.21.15.120:0/3750664923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234eda4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- 172.21.15.120:0/3750664923 >> 172.21.15.120:0/3750664923 conn(0x56234ed99d10 msgr2=0x56234eda56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- 172.21.15.120:0/3750664923 shutdown_connections 2024-09-10T22:52:13.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- 172.21.15.120:0/3750664923 wait complete. 2024-09-10T22:52:13.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- start start 2024-09-10T22:52:13.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 Processor -- start 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234edad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234eda8790 0x56234ee1e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b35e0 con 0x56234ed9c800 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b4a50 con 0x56234eda8790 2024-09-10T22:52:13.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b3cb0 con 0x56234ed9bee0 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.420+0000 7f1875dc2740 10 client.0 _ll_register_callbacks cb 0x7f18640a89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60502/0 (socket says 172.21.15.120:60502) 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 -- 172.21.15.120:0/1168621764 learned_addr learned my addr 172.21.15.120:0/1168621764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1871f8a640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234eda8790 0x56234ee1e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1871789640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234edad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 -- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 msgr2=0x56234edad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234edad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 -- 172.21.15.120:0/1168621764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234eda8790 msgr2=0x56234ee1e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234eda8790 0x56234ee1e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 -- 172.21.15.120:0/1168621764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234ed9b8c0 con 0x56234ed9c800 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1871789640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234edad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:13.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1870f88640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5497db3c291670b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c035070 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f185c005450 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18640cf070 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f185c043040 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1875dc2740 10 client.15270 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.421+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56234ee26080 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640cf070 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f185c032040 con 0x56234ed9c800 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 10 client.15270 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 10 client.15270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 20 client.15270 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234ee26310 0x56234ee28700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 4 client.15270 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56234ed10860 con 0x56234ee26310 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1871789640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234ee26310 0x56234ee28700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1871789640 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234ee26310 0x56234ee28700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a405e96d702b3797 server_cookie=454a88482b498a7c in_seq=0 out_seq=0 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.422+0000 7f1854ff9640 10 client.15270 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1854ff9640 1 -- 172.21.15.120:0/1168621764 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56234ed10860 con 0x56234ee26310 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1854ff9640 10 client.15270 handle_command_reply: tid=0 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 client.15270 shutdown 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 msgr2=0x56234edad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 shutdown_connections 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56234ee26310 0x56234ee28700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56234eda8790 0x56234ee1e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1854ff9640 0 client.15270 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:13.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56234ed9c800 0x56234edad6d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 --2- 172.21.15.120:0/1168621764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56234ed9bee0 0x56234edad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 >> 172.21.15.120:0/1168621764 conn(0x56234ed99d10 msgr2=0x56234eda5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 shutdown_connections 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 1 -- 172.21.15.120:0/1168621764 wait complete. 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.424+0000 7f1875dc2740 20 client.15270 trim_cache size 0 max 16384 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "c6719e3d-696f-4c8f-92f0-e7df480d8da9", 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:52:13.432 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 msgr2=0x7f18580b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 0x7f18580b7830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56234ed9f060 tx=0x7f1868008040 comp rx=0 tx=0).stop 2024-09-10T22:52:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 msgr2=0x7f186c1a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f186400eaa0 tx=0x7f186400ef60 comp rx=0 tx=0).stop 2024-09-10T22:52:13.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 shutdown_connections 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18580b5370 0x7f18580b7830 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f186c106f10 0x7f186c1a5220 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f186c1023f0 0x7f186c19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 --2- 172.21.15.120:0/3297242403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f186c101ad0 0x7f186c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 >> 172.21.15.120:0/3297242403 conn(0x7f186c0fd720 msgr2=0x7f186c104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 shutdown_connections 2024-09-10T22:52:13.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:13.425+0000 7f18739ed640 1 -- 172.21.15.120:0/3297242403 wait complete. 2024-09-10T22:52:13.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:12 smithi153 ceph-mon[34951]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:14.175 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:14 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:52:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:14 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:14 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:52:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:14 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:14 smithi120 ceph-mon[31247]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:14.488 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:14 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:52:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:14 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:14 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:52:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:14 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:14 smithi176 ceph-mon[34982]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:14 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:52:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:14 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:14 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:52:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:14 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:14.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:14 smithi153 ceph-mon[34951]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:14.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 -- 172.21.15.120:0/3084535498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8109150 msgr2=0x7f00e810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 --2- 172.21.15.120:0/3084535498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8109150 0x7f00e810f950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 -- 172.21.15.120:0/3084535498 shutdown_connections 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 --2- 172.21.15.120:0/3084535498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8109150 0x7f00e810f950 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 --2- 172.21.15.120:0/3084535498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 --2- 172.21.15.120:0/3084535498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00e8102dc0 0x7f00e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 -- 172.21.15.120:0/3084535498 >> 172.21.15.120:0/3084535498 conn(0x7f00e80fe830 msgr2=0x7f00e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 -- 172.21.15.120:0/3084535498 shutdown_connections 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.929+0000 7f00eda35640 1 -- 172.21.15.120:0/3084535498 wait complete. 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 Processor -- start 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 -- start start 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00e8109150 0x7f00e8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112e20 con 0x7f00e8102dc0 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112ca0 con 0x7f00e81087d0 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00eda35640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112fa0 con 0x7f00e8109150 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60526/0 (socket says 172.21.15.120:60526) 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 -- 172.21.15.120:0/3219228349 learned_addr learned my addr 172.21.15.120:0/3219228349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e67fc640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 -- 172.21.15.120:0/3219228349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00e8109150 msgr2=0x7f00e8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00e8109150 0x7f00e8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 -- 172.21.15.120:0/3219228349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 msgr2=0x7f00e80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e6ffd640 1 -- 172.21.15.120:0/3219228349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81a7670 con 0x7f00e8102dc0 2024-09-10T22:52:14.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.930+0000 7f00e67fc640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eab0 tx=0x7f00e000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00eca33640 1 -- 172.21.15.120:0/3219228349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e000ce40 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00eca33640 1 -- 172.21.15.120:0/3219228349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e0011920 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00eca33640 1 -- 172.21.15.120:0/3219228349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0010a30 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a7960 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.931+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a7ea0 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.933+0000 7f00eca33640 1 -- 172.21.15.120:0/3219228349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f00e000f040 con 0x7f00e8102dc0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 Processor -- start 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 -- start start 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04f96480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc04fa4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc04f0ce50 con 0x55bc04f969c0 2024-09-10T22:52:14.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc04efc280 con 0x55bc04fa2950 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00efe0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bc04f00090 con 0x55bc04f960a0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00eca33640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 0x7f00d40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60530/0 (socket says 172.21.15.120:60530) 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e6ffd640 1 -- 172.21.15.120:0/1356715053 learned_addr learned my addr 172.21.15.120:0/1356715053 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e67fc640 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc04fa4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e77fe640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 0x7f00d40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.934+0000 7f00e77fe640 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04f96480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00eca33640 1 -- 172.21.15.120:0/3219228349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f00e00e60e0 con 0x7f00e8102dc0 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 -- 172.21.15.120:0/1356715053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 msgr2=0x55bc04f96480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04f96480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 -- 172.21.15.120:0/1356715053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 msgr2=0x55bc04fa4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc04fa4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 -- 172.21.15.120:0/1356715053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc04c5a650 con 0x55bc04f969c0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00e00e3a00 tx=0x7f00e00ac670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba319f99371e546 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00e77fe640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 0x7f00d40b7830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00d80057c0 tx=0x7f00d8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00ddffb640 1 -- 172.21.15.120:0/1356715053 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e00ac770 con 0x55bc04f969c0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00ddffb640 1 -- 172.21.15.120:0/1356715053 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e00b1250 con 0x55bc04f969c0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.935+0000 7f00ddffb640 1 -- 172.21.15.120:0/1356715053 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e00b5870 con 0x55bc04f969c0 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- 172.21.15.120:0/1356715053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 msgr2=0x55bc04fa2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00e00e3a00 tx=0x7f00e00ac670 comp rx=0 tx=0).stop 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- 172.21.15.120:0/1356715053 shutdown_connections 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc04fa4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa2410 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 --2- 172.21.15.120:0/1356715053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04f96480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- 172.21.15.120:0/1356715053 >> 172.21.15.120:0/1356715053 conn(0x55bc04f93ed0 msgr2=0x55bc04f95070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- 172.21.15.120:0/1356715053 shutdown_connections 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- 172.21.15.120:0/1356715053 wait complete. 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.936+0000 7f00efe0a740 1 -- start start 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 Processor -- start 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f960a0 0x55bc04fa2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 0x55bc04f9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc04d5d700 con 0x55bc04fa2950 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bc04e97110 con 0x55bc04f960a0 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bc04fa5570 con 0x55bc04f969c0 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00efe0a740 10 client.0 _ll_register_callbacks cb 0x55bc04eb0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60536/0 (socket says 172.21.15.120:60536) 2024-09-10T22:52:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 -- 172.21.15.120:0/510469622 learned_addr learned my addr 172.21.15.120:0/510469622 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e67fc640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 0x55bc04f9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f960a0 0x55bc04fa2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 -- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 msgr2=0x55bc04f9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 0x55bc04f9f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 -- 172.21.15.120:0/510469622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f960a0 msgr2=0x55bc04fa2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f960a0 0x55bc04fa2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 -- 172.21.15.120:0/510469622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc04ec4c40 con 0x55bc04fa2950 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e67fc640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 0x55bc04f9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00e77fe640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00d8033000 tx=0x7f00d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59ffa06bd33570d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.937+0000 7f00c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:14.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8035070 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f00d804a610 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e00b1250 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f00d804a810 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 10 client.15282 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bc04fb0eb0 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e00b1250 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f00d803a040 con 0x55bc04fa2950 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 10 client.15282 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 20 client.15282 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc04fb2460 0x55bc04fb48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 4 client.15282 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc04f0eda0 con 0x55bc04fb2460 2024-09-10T22:52:14.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.938+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc04fb2460 0x55bc04fb48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.939+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc04fb2460 0x55bc04fb48c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772f1e9f104bf975 server_cookie=bc703f5ce8e1b65d in_seq=0 out_seq=0 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.939+0000 7f00c6ffd640 10 client.15282 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.939+0000 7f00c6ffd640 1 -- 172.21.15.120:0/510469622 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc04f0eda0 con 0x55bc04fb2460 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.939+0000 7f00c6ffd640 10 client.15282 handle_command_reply: tid=0 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 client.15282 shutdown 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 msgr2=0x55bc04f9fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00d8033000 tx=0x7f00d8009260 comp rx=0 tx=0).stop 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 shutdown_connections 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc04fb2460 0x55bc04fb48c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9fc50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f969c0 0x55bc04f9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 --2- 172.21.15.120:0/510469622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f960a0 0x55bc04fa2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 >> 172.21.15.120:0/510469622 conn(0x55bc04f93ed0 msgr2=0x55bc04f99a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00c6ffd640 0 client.15282 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:14.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 shutdown_connections 2024-09-10T22:52:14.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 1 -- 172.21.15.120:0/510469622 wait complete. 2024-09-10T22:52:14.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.940+0000 7f00efe0a740 20 client.15282 trim_cache size 0 max 16384 2024-09-10T22:52:14.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 Processor -- start 2024-09-10T22:52:14.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 -- start start 2024-09-10T22:52:14.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f969c0 0x55bc04f92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ae4d0 con 0x55bc04fa2950 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81b0d50 con 0x55bc04f969c0 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00efe0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81aeb40 con 0x55bc04f960a0 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60542/0 (socket says 172.21.15.120:60542) 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.958+0000 7f00e77fe640 1 -- 172.21.15.120:0/2078983023 learned_addr learned my addr 172.21.15.120:0/2078983023 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e67fc640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f969c0 0x55bc04f92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 -- 172.21.15.120:0/2078983023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 msgr2=0x55bc04fae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 -- 172.21.15.120:0/2078983023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f969c0 msgr2=0x55bc04f92e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f969c0 0x55bc04f92e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 -- 172.21.15.120:0/2078983023 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc04f9f420 con 0x55bc04fa2950 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00e77fe640 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d8031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c522e20555c13f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00dd7fa640 1 -- 172.21.15.120:0/2078983023 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d803a1a0 con 0x55bc04fa2950 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00dd7fa640 1 -- 172.21.15.120:0/2078983023 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d804f070 con 0x55bc04fa2950 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.959+0000 7f00dd7fa640 1 -- 172.21.15.120:0/2078983023 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d804ac10 con 0x55bc04fa2950 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- 172.21.15.120:0/2078983023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 msgr2=0x55bc04f9c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d8031b70 comp rx=0 tx=0).stop 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- 172.21.15.120:0/2078983023 shutdown_connections 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04fa2950 0x55bc04f9c880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04f969c0 0x55bc04f92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 --2- 172.21.15.120:0/2078983023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- 172.21.15.120:0/2078983023 >> 172.21.15.120:0/2078983023 conn(0x55bc04f93ed0 msgr2=0x55bc04f92700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- 172.21.15.120:0/2078983023 shutdown_connections 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- 172.21.15.120:0/2078983023 wait complete. 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.960+0000 7f00efe0a740 1 -- start start 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 Processor -- start 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fa7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc050183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81af210 con 0x55bc04f969c0 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81b0680 con 0x55bc04fa2950 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81af8e0 con 0x55bc04f960a0 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00efe0a740 10 client.0 _ll_register_callbacks cb 0x7f00d8005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00e67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00e67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60550/0 (socket says 172.21.15.120:60550) 2024-09-10T22:52:14.975 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00e67fc640 1 -- 172.21.15.120:0/3479483214 learned_addr learned my addr 172.21.15.120:0/3479483214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00e77fe640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc050183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.961+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fa7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 -- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 msgr2=0x55bc04fa7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fa7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 -- 172.21.15.120:0/3479483214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 msgr2=0x55bc050183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc050183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 -- 172.21.15.120:0/3479483214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc05018ae0 con 0x55bc04f969c0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e77fe640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc050183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00e67fc640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00d000f770 tx=0x7f00d000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21b7334e7e6e4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0020060 con 0x55bc04f969c0 2024-09-10T22:52:14.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f00d000eca0 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d804f070 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f00d000eea0 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00efe0a740 10 client.15288 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bc0501ff20 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.962+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d804f070 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f00d001a070 con 0x55bc04f969c0 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 10 client.15288 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 20 client.15288 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc05020700 0x55bc05022b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 4 client.15288 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc04f0a850 con 0x55bc05020700 2024-09-10T22:52:14.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc05020700 0x55bc05022b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00e6ffd640 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc05020700 0x55bc05022b20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1347de7f59c8df6 server_cookie=7844590abd5421ca in_seq=0 out_seq=0 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00c6ffd640 10 client.15288 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00c6ffd640 1 -- 172.21.15.120:0/3479483214 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55bc04f0a850 con 0x55bc05020700 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00c6ffd640 10 client.15288 handle_command_reply: tid=0 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.963+0000 7f00efe0a740 1 client.15288 shutdown 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 msgr2=0x55bc04fa7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00d000f770 tx=0x7f00d000fc30 comp rx=0 tx=0).stop 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 shutdown_connections 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bc05020700 0x55bc05022b20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bc04fa2950 0x55bc050183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc04f969c0 0x55bc04fa7690 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 --2- 172.21.15.120:0/3479483214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc04f960a0 0x55bc04fa7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 >> 172.21.15.120:0/3479483214 conn(0x55bc04f93ed0 msgr2=0x55bc04f92f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00c6ffd640 0 client.15288 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 shutdown_connections 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 1 -- 172.21.15.120:0/3479483214 wait complete. 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00efe0a740 20 client.15288 trim_cache size 0 max 16384 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:52:14.979 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:14.980 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 msgr2=0x7f00d40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 0x7f00d40b7830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00d80057c0 tx=0x7f00d8005fd0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 msgr2=0x7f00e80754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.964+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eab0 tx=0x7f00e000ef70 comp rx=0 tx=0).stop 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 shutdown_connections 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f00d40b5370 0x7f00d40b7830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00e8109150 0x7f00e8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00e81087d0 0x7f00e80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 --2- 172.21.15.120:0/3219228349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f00e8102dc0 0x7f00e80754a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:14.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 >> 172.21.15.120:0/3219228349 conn(0x7f00e80fe830 msgr2=0x7f00e80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:14.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 shutdown_connections 2024-09-10T22:52:14.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:14.965+0000 7f00eda35640 1 -- 172.21.15.120:0/3219228349 wait complete. 2024-09-10T22:52:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:52:15.624 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:52:15.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:52:15.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:52:15.918 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:16.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3768684934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94108060 msgr2=0x7f5d9410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3768684934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94108060 0x7f5d9410e860 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009ca0 tx=0x7f5d8c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3768684934 shutdown_connections 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3768684934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94108060 0x7f5d9410e860 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3768684934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d941076e0 0x7f5d94107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3768684934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d94101cd0 0x7f5d941020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3768684934 >> 172.21.15.120:0/3768684934 conn(0x7f5d940fd700 msgr2=0x7f5d940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3768684934 shutdown_connections 2024-09-10T22:52:16.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.402+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3768684934 wait complete. 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 Processor -- start 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 -- start start 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 0x7f5d9419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d94108060 0x7f5d9419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94111d20 con 0x7f5d94101cd0 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94111ba0 con 0x7f5d941076e0 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.403+0000 7f5d98a5e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94111ea0 con 0x7f5d94108060 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60570/0 (socket says 172.21.15.120:60570) 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 -- 172.21.15.120:0/3306741640 learned_addr learned my addr 172.21.15.120:0/3306741640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d92d9d640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d94108060 0x7f5d9419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d8bfff640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 0x7f5d9419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 -- 172.21.15.120:0/3306741640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d94108060 msgr2=0x7f5d9419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d94108060 0x7f5d9419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 -- 172.21.15.120:0/3306741640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 msgr2=0x7f5d9419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 0x7f5d9419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 -- 172.21.15.120:0/3306741640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d941a5470 con 0x7f5d94101cd0 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d8bfff640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 0x7f5d9419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d9259c640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e9f0 tx=0x7f5d8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400cd80 con 0x7f5d94101cd0 2024-09-10T22:52:16.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d941a5760 con 0x7f5d94101cd0 2024-09-10T22:52:16.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d840108e0 con 0x7f5d94101cd0 2024-09-10T22:52:16.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84018980 con 0x7f5d94101cd0 2024-09-10T22:52:16.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.404+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941a5ca0 con 0x7f5d94101cd0 2024-09-10T22:52:16.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.405+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d941020b0 con 0x7f5d94101cd0 2024-09-10T22:52:16.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.407+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5d84010430 con 0x7f5d94101cd0 2024-09-10T22:52:16.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.408+0000 7f5d8b7fe640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 0x7f5d800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.408+0000 7f5d8bfff640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 0x7f5d800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.408+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f5d840e6450 con 0x7f5d94101cd0 2024-09-10T22:52:16.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.409+0000 7f5d8bfff640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 0x7f5d800b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0058c0 tx=0x7f5d7c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.411+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5d840ad340 con 0x7f5d94101cd0 2024-09-10T22:52:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:16 smithi176 ceph-mon[34982]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:16.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.623+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d9406b8a0 con 0x7f5d94101cd0 2024-09-10T22:52:16.624 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:16 smithi120 ceph-mon[31247]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:16.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.623+0000 7f5d8b7fe640 1 -- 172.21.15.120:0/3306741640 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7f5d9406b8a0 con 0x7f5d94101cd0 2024-09-10T22:52:16.625 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:16.625 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:52:16.625 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:52:16.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 msgr2=0x7f5d800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 0x7f5d800b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0058c0 tx=0x7f5d7c005850 comp rx=0 tx=0).stop 2024-09-10T22:52:16.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 msgr2=0x7f5d9419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e9f0 tx=0x7f5d8400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 shutdown_connections 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5d800b52e0 0x7f5d800b77a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d94108060 0x7f5d9419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d941076e0 0x7f5d9419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 --2- 172.21.15.120:0/3306741640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d94101cd0 0x7f5d9419a090 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 >> 172.21.15.120:0/3306741640 conn(0x7f5d940fd700 msgr2=0x7f5d940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 shutdown_connections 2024-09-10T22:52:16.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:16.626+0000 7f5d98a5e640 1 -- 172.21.15.120:0/3306741640 wait complete. 2024-09-10T22:52:16.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:16 smithi153 ceph-mon[34951]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-09-10T22:52:17.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:52:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:17 smithi176 ceph-mon[34982]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:52:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:17 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3306741640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:17.656 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:17 smithi120 ceph-mon[31247]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:52:17.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:17 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3306741640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:17 smithi153 ceph-mon[34951]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:52:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:17 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3306741640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:18.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 -- 172.21.15.120:0/352231851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 msgr2=0x7f3cd810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 --2- 172.21.15.120:0/352231851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd810cb70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009ca0 tx=0x7f3cc802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 -- 172.21.15.120:0/352231851 shutdown_connections 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 --2- 172.21.15.120:0/352231851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd810d140 0x7f3cd810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 --2- 172.21.15.120:0/352231851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd810cb70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 --2- 172.21.15.120:0/352231851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd8103b90 0x7f3cd8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 -- 172.21.15.120:0/352231851 >> 172.21.15.120:0/352231851 conn(0x7f3cd80fd700 msgr2=0x7f3cd80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 -- 172.21.15.120:0/352231851 shutdown_connections 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.147+0000 7f3cdef80640 1 -- 172.21.15.120:0/352231851 wait complete. 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 Processor -- start 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 -- start start 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 0x7f3cd8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd810d140 0x7f3cd819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8112d60 con 0x7f3cd8104590 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8112be0 con 0x7f3cd8103b90 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdef80640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8112ee0 con 0x7f3cd810d140 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60592/0 (socket says 172.21.15.120:60592) 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 -- 172.21.15.120:0/1087920835 learned_addr learned my addr 172.21.15.120:0/1087920835 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd810d140 0x7f3cd819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 0x7f3cd8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 -- 172.21.15.120:0/1087920835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd810d140 msgr2=0x7f3cd819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd810d140 0x7f3cd819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 -- 172.21.15.120:0/1087920835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 msgr2=0x7f3cd8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 0x7f3cd8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cd7fff640 1 -- 172.21.15.120:0/1087920835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd819f3e0 con 0x7f3cd8104590 2024-09-10T22:52:18.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.148+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 0x7f3cd8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802ade0 tx=0x7f3cc8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cd5ffb640 1 -- 172.21.15.120:0/1087920835 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003d60 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd819f670 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cd5ffb640 1 -- 172.21.15.120:0/1087920835 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc80044f0 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cd5ffb640 1 -- 172.21.15.120:0/1087920835 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc802fd10 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.149+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81a5d90 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.151+0000 7f3cd5ffb640 1 -- 172.21.15.120:0/1087920835 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3cc8004040 con 0x7f3cd8104590 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 Processor -- start 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 -- start start 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b49c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557978aafea0 con 0x557978b4a190 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557978a9f2d0 con 0x557978b394d0 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3ce1355740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557978aa30e0 con 0x557978b39df0 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60594/0 (socket says 172.21.15.120:60594) 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2503762015 learned_addr learned my addr 172.21.15.120:0/2503762015 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cdd51d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b49c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2503762015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 msgr2=0x557978b49c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b49c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2503762015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 msgr2=0x557978b398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2503762015 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579787fd650 con 0x557978b4a190 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b49c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:18.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.152+0000 7f3cd5ffb640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 0x7f3cc40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 0x7f3cc40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3cc810f1b0 tx=0x7f3cc810f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccc6278897be55d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cd5ffb640 1 -- 172.21.15.120:0/1087920835 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f3cc8107110 con 0x7f3cd8104590 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cceffd640 1 -- 172.21.15.120:0/2503762015 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80c9f50 con 0x557978b4a190 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cceffd640 1 -- 172.21.15.120:0/2503762015 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc802feb0 con 0x557978b4a190 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.153+0000 7f3cceffd640 1 -- 172.21.15.120:0/2503762015 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80cd070 con 0x557978b4a190 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- 172.21.15.120:0/2503762015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 msgr2=0x557978b4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 0x7f3cc40b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc0005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3cc810f1b0 tx=0x7f3cc810f2a0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- 172.21.15.120:0/2503762015 shutdown_connections 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b4c580 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b49c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2503762015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- 172.21.15.120:0/2503762015 >> 172.21.15.120:0/2503762015 conn(0x557978b37300 msgr2=0x557978b384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- 172.21.15.120:0/2503762015 shutdown_connections 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- 172.21.15.120:0/2503762015 wait complete. 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 -- start start 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 Processor -- start 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.154+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b46ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557978900700 con 0x557978b4a190 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557978a3a160 con 0x557978b394d0 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557978aafea0 con 0x557978b39df0 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 10 client.0 _ll_register_callbacks cb 0x557978a539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60600/0 (socket says 172.21.15.120:60600) 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 -- 172.21.15.120:0/3671511526 learned_addr learned my addr 172.21.15.120:0/3671511526 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b46ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 -- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 msgr2=0x557978b46ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b46ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 -- 172.21.15.120:0/3671511526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 msgr2=0x557978b49a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b49a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 -- 172.21.15.120:0/3671511526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557978a67c90 con 0x557978b4a190 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3cc001c7a0 tx=0x7f3cc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3a94432eced993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc002c060 con 0x557978b4a190 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557978ab1df0 con 0x557978b4a190 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f3cc0016030 con 0x557978b4a190 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3cc001ce50 con 0x557978b4a190 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 10 client.15306 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:18.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.155+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557978b58350 con 0x557978b4a190 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557978ab1df0 con 0x557978b4a190 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3cc000f680 con 0x557978b4a190 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 10 client.15306 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 10 client.15306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 20 client.15306 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978b5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 4 client.15306 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557978aad8a0 con 0x557978b59860 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978b5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978b5bcc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5a7d6ea05b9204 server_cookie=714ae55813040fe4 in_seq=0 out_seq=0 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.156+0000 7f3cb7fff640 10 client.15306 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3671511526 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557978aad8a0 con 0x557978b59860 2024-09-10T22:52:18.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3cb7fff640 10 client.15306 handle_command_reply: tid=0 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 client.15306 shutdown 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 msgr2=0x557978b474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3cc001c7a0 tx=0x7f3cc0004730 comp rx=0 tx=0).stop 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 shutdown_connections 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978b5bcc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b4a190 0x557978b474f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3cb7fff640 0 client.15306 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b39df0 0x557978b46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3671511526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b49a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.157+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 >> 172.21.15.120:0/3671511526 conn(0x557978b37300 msgr2=0x557978b3ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.158+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 shutdown_connections 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.158+0000 7f3ce1355740 1 -- 172.21.15.120:0/3671511526 wait complete. 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.158+0000 7f3ce1355740 20 client.15306 trim_cache size 0 max 16384 2024-09-10T22:52:18.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.170+0000 7f3ce1355740 1 Processor -- start 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.170+0000 7f3ce1355740 1 -- start start 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b55c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81aa1d0 con 0x557978b39df0 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81aca50 con 0x557978b394d0 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3ce1355740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81aa840 con 0x557978b4a190 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60608/0 (socket says 172.21.15.120:60608) 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2747044229 learned_addr learned my addr 172.21.15.120:0/2747044229 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b55c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2747044229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 msgr2=0x557978b4b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2747044229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 msgr2=0x557978b55c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b55c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 -- 172.21.15.120:0/2747044229 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557978b49de0 con 0x557978b39df0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8104fb0 tx=0x7f3cc8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d659a247d3f72d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cce7fc640 1 -- 172.21.15.120:0/2747044229 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80cd250 con 0x557978b39df0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cce7fc640 1 -- 172.21.15.120:0/2747044229 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8036070 con 0x557978b39df0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.171+0000 7f3cce7fc640 1 -- 172.21.15.120:0/2747044229 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80cd6d0 con 0x557978b39df0 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- 172.21.15.120:0/2747044229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 msgr2=0x557978b4afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8104fb0 tx=0x7f3cc8005c30 comp rx=0 tx=0).stop 2024-09-10T22:52:18.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- 172.21.15.120:0/2747044229 shutdown_connections 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b4afd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 --2- 172.21.15.120:0/2747044229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b55c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- 172.21.15.120:0/2747044229 >> 172.21.15.120:0/2747044229 conn(0x557978b37300 msgr2=0x557978b4bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- 172.21.15.120:0/2747044229 shutdown_connections 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- 172.21.15.120:0/2747044229 wait complete. 2024-09-10T22:52:18.213 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 -- start start 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.172+0000 7f3ce1355740 1 Processor -- start 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81aaf10 con 0x557978b39df0 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81ac380 con 0x557978b394d0 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81ab5e0 con 0x557978b4a190 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 10 client.0 _ll_register_callbacks cb 0x7f3cc000a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:18.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60612/0 (socket says 172.21.15.120:60612) 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 -- 172.21.15.120:0/3586915688 learned_addr learned my addr 172.21.15.120:0/3586915688 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cdd51d640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 -- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 msgr2=0x557978b4ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 -- 172.21.15.120:0/3586915688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 msgr2=0x557978b51ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b51ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 -- 172.21.15.120:0/3586915688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557978b528d0 con 0x557978b39df0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b51ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cd7fff640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8104c90 tx=0x7f3cc810ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f97436e29681ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80c97a0 con 0x557978b39df0 2024-09-10T22:52:18.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f3cc80d2040 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.173+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc8036070 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3cc80c9f50 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 10 client.15312 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557978bc6bd0 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8036070 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3cc80cdb60 con 0x557978b39df0 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 10 client.15312 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 20 client.15312 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978bc8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 4 client.15312 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557978aaacd0 con 0x557978b59860 2024-09-10T22:52:18.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.174+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978bc8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3cb7fff640 10 client.15312 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3cdcd1c640 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978bc8fc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c1810578f87874 server_cookie=53e175c91e7343bd in_seq=0 out_seq=0 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3cb7fff640 1 -- 172.21.15.120:0/3586915688 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557978aaacd0 con 0x557978b59860 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3cb7fff640 10 client.15312 handle_command_reply: tid=0 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 client.15312 shutdown 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 msgr2=0x557978b521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8104c90 tx=0x7f3cc810ada0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 shutdown_connections 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557978b59860 0x557978bc8fc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557978b4a190 0x557978b4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3cb7fff640 0 client.15312 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557978b39df0 0x557978b521e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 --2- 172.21.15.120:0/3586915688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557978b394d0 0x557978b51ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.175+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 >> 172.21.15.120:0/3586915688 conn(0x557978b37300 msgr2=0x557978b4bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 shutdown_connections 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3ce1355740 1 -- 172.21.15.120:0/3586915688 wait complete. 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3ce1355740 20 client.15312 trim_cache size 0 max 16384 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 msgr2=0x7f3cc40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 0x7f3cc40b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc0005f60 comp rx=0 tx=0).stop 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 msgr2=0x7f3cd8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3cc802ade0 tx=0x7f3cc8003970 comp rx=0 tx=0).stop 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 shutdown_connections 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3cc40b52e0 0x7f3cc40b77a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cd810d140 0x7f3cd819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd8104590 0x7f3cd8199870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 --2- 172.21.15.120:0/1087920835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8103b90 0x7f3cd8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 >> 172.21.15.120:0/1087920835 conn(0x7f3cd80fd700 msgr2=0x7f3cd80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 shutdown_connections 2024-09-10T22:52:18.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:18.176+0000 7f3cdef80640 1 -- 172.21.15.120:0/1087920835 wait complete. 2024-09-10T22:52:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:18 smithi120 ceph-mon[31247]: pgmap v339: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-10T22:52:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:18 smithi176 ceph-mon[34982]: pgmap v339: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-10T22:52:18.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:18 smithi153 ceph-mon[34951]: pgmap v339: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-09-10T22:52:20.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:52:20.103 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:52:20.368 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:20 smithi120 ceph-mon[31247]: pgmap v340: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T22:52:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:20 smithi176 ceph-mon[34982]: pgmap v340: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T22:52:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:20 smithi153 ceph-mon[34951]: pgmap v340: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 -- 172.21.15.120:0/622584000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd228101d10 msgr2=0x7fd2281020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 --2- 172.21.15.120:0/622584000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd228101d10 0x7fd2281020f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 -- 172.21.15.120:0/622584000 shutdown_connections 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 --2- 172.21.15.120:0/622584000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2281080a0 0x7fd22810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 --2- 172.21.15.120:0/622584000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 0x7fd228107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 --2- 172.21.15.120:0/622584000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd228101d10 0x7fd2281020f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 -- 172.21.15.120:0/622584000 >> 172.21.15.120:0/622584000 conn(0x7fd2280fd7c0 msgr2=0x7fd2280ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 -- 172.21.15.120:0/622584000 shutdown_connections 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.895+0000 7fd22ee95640 1 -- 172.21.15.120:0/622584000 wait complete. 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 Processor -- start 2024-09-10T22:52:20.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 -- start start 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 0x7fd2281993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 0x7fd228199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111d60 con 0x7fd2281080a0 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111be0 con 0x7fd228101d10 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22ee95640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111ee0 con 0x7fd228107720 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60628/0 (socket says 172.21.15.120:60628) 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd22d432640 1 -- 172.21.15.120:0/2866886509 learned_addr learned my addr 172.21.15.120:0/2866886509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.896+0000 7fd227fff640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 0x7fd228199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 0x7fd2281993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 -- 172.21.15.120:0/2866886509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 msgr2=0x7fd228199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 0x7fd228199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 -- 172.21.15.120:0/2866886509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 msgr2=0x7fd2281993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 0x7fd2281993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 -- 172.21.15.120:0/2866886509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22819f480 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 0x7fd2281993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22d432640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd220009fd0 tx=0x7fd22000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd225ffb640 1 -- 172.21.15.120:0/2866886509 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220017070 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd225ffb640 1 -- 172.21.15.120:0/2866886509 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220003e20 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22819f770 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd225ffb640 1 -- 172.21.15.120:0/2866886509 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220010460 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.897+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281a5e30 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.900+0000 7fd225ffb640 1 -- 172.21.15.120:0/2866886509 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd220003970 con 0x7fd2281080a0 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.900+0000 7fd225ffb640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 0x7fd2180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 0x7fd2180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd225ffb640 1 -- 172.21.15.120:0/2866886509 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fd2200e5820 con 0x7fd2281080a0 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 Processor -- start 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 -- start start 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 0x7fd2180b7820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd21c031440 tx=0x7fd21c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a4a790 0x558a09a4cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a099b4e60 con 0x558a09a3dee0 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a099a4290 con 0x558a09a3e800 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd23126a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a099a80a0 con 0x558a09a4a790 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd22cc31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd22cc31640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60638/0 (socket says 172.21.15.120:60638) 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd22cc31640 1 -- 172.21.15.120:0/947458327 learned_addr learned my addr 172.21.15.120:0/947458327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:20.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.901+0000 7fd227fff640 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22d432640 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a4a790 0x558a09a4cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 -- 172.21.15.120:0/947458327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a4a790 msgr2=0x558a09a4cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a4a790 0x558a09a4cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 -- 172.21.15.120:0/947458327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 msgr2=0x558a09a4a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a4a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 -- 172.21.15.120:0/947458327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a09702930 con 0x558a09a3dee0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd22cc31640 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518a6f2768e6b228 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd212ffd640 1 -- 172.21.15.120:0/947458327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c035050 con 0x558a09a3dee0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd212ffd640 1 -- 172.21.15.120:0/947458327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd21c04c070 con 0x558a09a3dee0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.902+0000 7fd212ffd640 1 -- 172.21.15.120:0/947458327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0046c0 con 0x558a09a3dee0 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- 172.21.15.120:0/947458327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 msgr2=0x558a09a3e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c03c000 comp rx=0 tx=0).stop 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- 172.21.15.120:0/947458327 shutdown_connections 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a4a790 0x558a09a4cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a4a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 --2- 172.21.15.120:0/947458327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3dee0 0x558a09a3e2c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- 172.21.15.120:0/947458327 >> 172.21.15.120:0/947458327 conn(0x558a09a3bd10 msgr2=0x558a09a3ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- 172.21.15.120:0/947458327 shutdown_connections 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- 172.21.15.120:0/947458327 wait complete. 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 -- start start 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.903+0000 7fd23126a740 1 Processor -- start 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a47730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a4a790 0x558a09a481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a09805890 con 0x558a09a3e800 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a0993ed40 con 0x558a09a4a790 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a09a4d3e0 con 0x558a09a3dee0 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd23126a740 10 client.0 _ll_register_callbacks cb 0x558a099a2c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60648/0 (socket says 172.21.15.120:60648) 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 -- 172.21.15.120:0/801894415 learned_addr learned my addr 172.21.15.120:0/801894415 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd22cc31640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a47730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd22d432640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a4a790 0x558a09a481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 -- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 msgr2=0x558a09a47730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a47730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 -- 172.21.15.120:0/801894415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a4a790 msgr2=0x558a09a481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a4a790 0x558a09a481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.904+0000 7fd227fff640 1 -- 172.21.15.120:0/801894415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a09a567c0 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd227fff640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd214010820 tx=0x7fd214010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3173ac65fca8bd82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21400fb90 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fd21400fd30 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c04c070 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd214005020 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 10 client.15324 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558a09a591d0 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd21c04c070 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fd214025480 con 0x558a09a3e800 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 10 client.15324 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 20 client.15324 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:20.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09a5a630 0x558a09a5ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 4 client.15324 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.905+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a099b6db0 con 0x558a09a5a630 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.906+0000 7fd22cc31640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09a5a630 0x558a09a5ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.906+0000 7fd207fff640 10 client.15324 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.906+0000 7fd22cc31640 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09a5a630 0x558a09a5ca80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da3d59747a958e2 server_cookie=283d49397e3043b2 in_seq=0 out_seq=0 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd207fff640 1 -- 172.21.15.120:0/801894415 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a099b6db0 con 0x558a09a5a630 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd207fff640 10 client.15324 handle_command_reply: tid=0 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 client.15324 shutdown 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 msgr2=0x558a09a47c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd214010820 tx=0x7fd214010ce0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 shutdown_connections 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09a5a630 0x558a09a5ca80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a4a790 0x558a09a481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a3e800 0x558a09a47c70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 --2- 172.21.15.120:0/801894415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a47730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 >> 172.21.15.120:0/801894415 conn(0x558a09a3bd10 msgr2=0x558a09a3af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd207fff640 0 client.15324 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 shutdown_connections 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 1 -- 172.21.15.120:0/801894415 wait complete. 2024-09-10T22:52:20.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.907+0000 7fd23126a740 20 client.15324 trim_cache size 0 max 16384 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 Processor -- start 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 -- start start 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a44b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281aa160 con 0x558a09a4a790 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ac9e0 con 0x558a09a3e800 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd23126a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281aa7d0 con 0x558a09a3dee0 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.925+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60656/0 (socket says 172.21.15.120:60656) 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 -- 172.21.15.120:0/1171788891 learned_addr learned my addr 172.21.15.120:0/1171788891 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd227fff640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22cc31640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a44b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 -- 172.21.15.120:0/1171788891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 msgr2=0x558a09a44b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a44b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 -- 172.21.15.120:0/1171788891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 msgr2=0x558a09a45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 -- 172.21.15.120:0/1171788891 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a09a489f0 con 0x558a09a4a790 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd227fff640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd22d432640 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b760 tx=0x7fd2200cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91bf41bd001af29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd2127fc640 1 -- 172.21.15.120:0/1171788891 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220020040 con 0x558a09a4a790 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd2127fc640 1 -- 172.21.15.120:0/1171788891 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200abb20 con 0x558a09a4a790 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.926+0000 7fd2127fc640 1 -- 172.21.15.120:0/1171788891 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b55a0 con 0x558a09a4a790 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 -- 172.21.15.120:0/1171788891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 msgr2=0x558a09a62390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b760 tx=0x7fd2200cf040 comp rx=0 tx=0).stop 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 -- 172.21.15.120:0/1171788891 shutdown_connections 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a62390 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 --2- 172.21.15.120:0/1171788891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 -- 172.21.15.120:0/1171788891 >> 172.21.15.120:0/1171788891 conn(0x558a09a3bd10 msgr2=0x558a09a45fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 -- 172.21.15.120:0/1171788891 shutdown_connections 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.927+0000 7fd23126a740 1 -- 172.21.15.120:0/1171788891 wait complete. 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 -- start start 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 Processor -- start 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a52bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281aaea0 con 0x558a09a4a790 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ac310 con 0x558a09a3e800 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ab570 con 0x558a09a3dee0 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.928+0000 7fd23126a740 10 client.0 _ll_register_callbacks cb 0x7fd21400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60668/0 (socket says 172.21.15.120:60668) 2024-09-10T22:52:20.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 -- 172.21.15.120:0/2088561969 learned_addr learned my addr 172.21.15.120:0/2088561969 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd227fff640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a52bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 -- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 msgr2=0x558a09a526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 -- 172.21.15.120:0/2088561969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 msgr2=0x558a09a52bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a52bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 -- 172.21.15.120:0/2088561969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a09a4dba0 con 0x558a09a4a790 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd22d432640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd220012040 tx=0x7fd2200ab780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d02001948a227d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd207fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200ab470 con 0x558a09a4a790 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fd2200a7960 con 0x558a09a4a790 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2200abb20 con 0x558a09a4a790 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd2200ac710 con 0x558a09a4a790 2024-09-10T22:52:20.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd23126a740 10 client.15330 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.929+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558a09ac7dd0 con 0x558a09a4a790 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200b55a0 con 0x558a09a4a790 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fd2200970b0 con 0x558a09a4a790 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 10 client.15330 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 20 client.15330 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09ac8410 0x558a09aca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 4 client.15330 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fd2200abb20 con 0x558a09ac8410 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.930+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09ac8410 0x558a09aca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.931+0000 7fd22cc31640 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09ac8410 0x558a09aca830 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a24f752397e89d server_cookie=f05f9570651d8cb8 in_seq=0 out_seq=0 2024-09-10T22:52:20.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.931+0000 7fd207fff640 10 client.15330 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd207fff640 1 -- 172.21.15.120:0/2088561969 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fd2200abb20 con 0x558a09ac8410 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd207fff640 10 client.15330 handle_command_reply: tid=0 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 client.15330 shutdown 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 msgr2=0x558a09a4d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd220012040 tx=0x7fd2200ab780 comp rx=0 tx=0).stop 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 shutdown_connections 2024-09-10T22:52:20.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a09ac8410 0x558a09aca830 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a09a4a790 0x558a09a4d660 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a09a3e800 0x558a09a52bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 --2- 172.21.15.120:0/2088561969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a09a3dee0 0x558a09a526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 >> 172.21.15.120:0/2088561969 conn(0x558a09a3bd10 msgr2=0x558a09a55630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.932+0000 7fd207fff640 0 client.15330 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 shutdown_connections 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd23126a740 1 -- 172.21.15.120:0/2088561969 wait complete. 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd23126a740 20 client.15330 trim_cache size 0 max 16384 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "7cd29d46-2281-4b94-a993-80e3b8f9b9bd", 2024-09-10T22:52:20.939 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 msgr2=0x7fd2180b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 0x7fd2180b7820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd21c031440 tx=0x7fd21c005a20 comp rx=0 tx=0).stop 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 msgr2=0x7fd22819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.933+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd220009fd0 tx=0x7fd22000b260 comp rx=0 tx=0).stop 2024-09-10T22:52:20.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 shutdown_connections 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd2180b5360 0x7fd2180b7820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2281080a0 0x7fd22819ecb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd228107720 0x7fd228199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 --2- 172.21.15.120:0/2866886509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228101d10 0x7fd2281993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 >> 172.21.15.120:0/2866886509 conn(0x7fd2280fd7c0 msgr2=0x7fd22806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 shutdown_connections 2024-09-10T22:52:20.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:20.934+0000 7fd22ee95640 1 -- 172.21.15.120:0/2866886509 wait complete. 2024-09-10T22:52:21.528 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:21 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:52:21.528 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:21 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.528 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:21 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:52:21.528 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:21 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:21 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:52:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:21 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:21 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:52:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:21 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:21 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:52:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:21 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:21 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:52:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:21 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:21.837 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:22.110 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:22.572 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:22 smithi120 ceph-mon[31247]: pgmap v341: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:22 smithi176 ceph-mon[34982]: pgmap v341: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:22.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 -- 172.21.15.120:0/2737741533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c108020 msgr2=0x7efd9c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 --2- 172.21.15.120:0/2737741533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c108020 0x7efd9c10e820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efd94009ca0 tx=0x7efd9402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 -- 172.21.15.120:0/2737741533 shutdown_connections 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 --2- 172.21.15.120:0/2737741533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c108020 0x7efd9c10e820 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 --2- 172.21.15.120:0/2737741533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1076a0 0x7efd9c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 --2- 172.21.15.120:0/2737741533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c101c90 0x7efd9c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 -- 172.21.15.120:0/2737741533 >> 172.21.15.120:0/2737741533 conn(0x7efd9c0fd700 msgr2=0x7efd9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 -- 172.21.15.120:0/2737741533 shutdown_connections 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.657+0000 7efda30de640 1 -- 172.21.15.120:0/2737741533 wait complete. 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 Processor -- start 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 -- start start 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c101c90 0x7efd9c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 0x7efd9c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c111ce0 con 0x7efd9c1076a0 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c111b60 con 0x7efd9c108020 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efda30de640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c111e60 con 0x7efd9c101c90 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.658+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60696/0 (socket says 172.21.15.120:60696) 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 -- 172.21.15.120:0/3854968009 learned_addr learned my addr 172.21.15.120:0/3854968009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efda0e7a640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c101c90 0x7efd9c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efda167b640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 0x7efd9c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 -- 172.21.15.120:0/3854968009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c101c90 msgr2=0x7efd9c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c101c90 0x7efd9c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 -- 172.21.15.120:0/3854968009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 msgr2=0x7efd9c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 0x7efd9c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 -- 172.21.15.120:0/3854968009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c1a54d0 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efda167b640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 0x7efd9c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b4a0 tx=0x7efd8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd99ffb640 1 -- 172.21.15.120:0/3854968009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88014070 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd99ffb640 1 -- 172.21.15.120:0/3854968009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd88010ac0 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c1a57c0 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efd99ffb640 1 -- 172.21.15.120:0/3854968009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88017c30 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.659+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c1a5d30 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.662+0000 7efd99ffb640 1 -- 172.21.15.120:0/3854968009 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efd88010610 con 0x7efd9c1076a0 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.662+0000 7efda54b3740 1 Processor -- start 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.662+0000 7efda54b3740 1 -- start start 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.662+0000 7efda54b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 0x557c91d6c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda54b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 0x557c91d7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda54b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda54b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c91ce2e50 con 0x557c91d7d140 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda54b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557c91cd2280 con 0x557c91d6cda0 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda54b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c91cd6090 con 0x557c91d6c480 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 0x557c91d6c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60710/0 (socket says 172.21.15.120:60710) 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd9bfff640 1 -- 172.21.15.120:0/2970501713 learned_addr learned my addr 172.21.15.120:0/2970501713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda0e7a640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 0x557c91d7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd99ffb640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 0x7efd8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 -- 172.21.15.120:0/2970501713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 msgr2=0x557c91d7cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 0x557c91d7cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 -- 172.21.15.120:0/2970501713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 msgr2=0x557c91d7f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd99ffb640 1 -- 172.21.15.120:0/3854968009 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7efd880e5270 con 0x7efd9c1076a0 2024-09-10T22:52:22.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 -- 172.21.15.120:0/2970501713 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c91a30650 con 0x557c91d6c480 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda0e7a640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 0x557c91d7cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efd9bfff640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.663+0000 7efda167b640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 0x7efd8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.664+0000 7efda167b640 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 0x557c91d6c860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd94035000 tx=0x7efd94031b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa3895b029284e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.664+0000 7efd86ffd640 1 -- 172.21.15.120:0/2970501713 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94004040 con 0x557c91d6c480 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.664+0000 7efda167b640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 0x7efd8c0b77a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd9404b000 tx=0x7efd94048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.664+0000 7efd86ffd640 1 -- 172.21.15.120:0/2970501713 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd94036070 con 0x557c91d6c480 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.664+0000 7efd86ffd640 1 -- 172.21.15.120:0/2970501713 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9402fbc0 con 0x557c91d6c480 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- 172.21.15.120:0/2970501713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 msgr2=0x557c91d6c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 0x557c91d6c860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd94035000 tx=0x7efd94031b70 comp rx=0 tx=0).stop 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- 172.21.15.120:0/2970501713 shutdown_connections 2024-09-10T22:52:22.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d7f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6cda0 0x557c91d7cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 --2- 172.21.15.120:0/2970501713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6c480 0x557c91d6c860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- 172.21.15.120:0/2970501713 >> 172.21.15.120:0/2970501713 conn(0x557c91d6a2b0 msgr2=0x557c91d6b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- 172.21.15.120:0/2970501713 shutdown_connections 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- 172.21.15.120:0/2970501713 wait complete. 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 -- start start 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.665+0000 7efda54b3740 1 Processor -- start 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d7a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557c91b33700 con 0x557c91d6cda0 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557c91c6d110 con 0x557c91d6c480 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557c91d7fc70 con 0x557c91d7d140 2024-09-10T22:52:22.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 10 client.0 _ll_register_callbacks cb 0x557c91c86990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60720/0 (socket says 172.21.15.120:60720) 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 -- 172.21.15.120:0/1905721120 learned_addr learned my addr 172.21.15.120:0/1905721120 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda167b640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d7a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda0e7a640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 -- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 msgr2=0x557c91d7a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d7a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 -- 172.21.15.120:0/1905721120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 msgr2=0x557c91d7cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d7cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 -- 172.21.15.120:0/1905721120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c91c9ac40 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd9bfff640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd88004700 tx=0x7efd88010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970088dfec34972f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8801e070 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7efd880aac60 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.666+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94036070 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efd880ab350 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 10 client.15342 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557c91d8b640 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd94036070 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efd880aa6d0 con 0x557c91d6cda0 2024-09-10T22:52:22.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 10 client.15342 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 20 client.15342 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91d8cbe0 0x557c91d8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 4 client.15342 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c91ce4da0 con 0x557c91d8cbe0 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.667+0000 7efda0e7a640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91d8cbe0 0x557c91d8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.668+0000 7efda0e7a640 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91d8cbe0 0x557c91d8efd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a281143786aaf26c server_cookie=866b965ae90c5cbb in_seq=0 out_seq=0 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.668+0000 7efd7bfff640 10 client.15342 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efd7bfff640 1 -- 172.21.15.120:0/1905721120 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c91ce4da0 con 0x557c91d8cbe0 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efd7bfff640 10 client.15342 handle_command_reply: tid=0 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 client.15342 shutdown 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 msgr2=0x557c91d79f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd88004700 tx=0x7efd88010ef0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 shutdown_connections 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91d8cbe0 0x557c91d8efd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d7a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d79f50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efd7bfff640 0 client.15342 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 --2- 172.21.15.120:0/1905721120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d7cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 >> 172.21.15.120:0/1905721120 conn(0x557c91d6a2b0 msgr2=0x557c91d6fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 shutdown_connections 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 1 -- 172.21.15.120:0/1905721120 wait complete. 2024-09-10T22:52:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.669+0000 7efda54b3740 20 client.15342 trim_cache size 0 max 16384 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 Processor -- start 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 -- start start 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6cda0 0x557c91d7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d94710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1aa130 con 0x557c91d7d140 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ac9b0 con 0x557c91d6c480 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda54b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1aa7a0 con 0x557c91d6cda0 2024-09-10T22:52:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda0e7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda167b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d94710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda0e7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49314/0 (socket says 172.21.15.120:49314) 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 -- 172.21.15.120:0/2819656929 learned_addr learned my addr 172.21.15.120:0/2819656929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6cda0 0x557c91d7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.687+0000 7efda167b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d94710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60730/0 (socket says 172.21.15.120:60730) 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 -- 172.21.15.120:0/2819656929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6cda0 msgr2=0x557c91d7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6cda0 0x557c91d7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 -- 172.21.15.120:0/2819656929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 msgr2=0x557c91d94710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d94710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 -- 172.21.15.120:0/2819656929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c91d65900 con 0x557c91d6c480 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efda0e7a640 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd9000eb40 tx=0x7efd900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0c69b413afb63e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efd867fc640 1 -- 172.21.15.120:0/2819656929 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90028070 con 0x557c91d6c480 2024-09-10T22:52:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efd867fc640 1 -- 172.21.15.120:0/2819656929 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001c980 con 0x557c91d6c480 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.688+0000 7efd867fc640 1 -- 172.21.15.120:0/2819656929 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90023a10 con 0x557c91d6c480 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 -- 172.21.15.120:0/2819656929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 msgr2=0x557c91d88e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd9000eb40 tx=0x7efd900118d0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 -- 172.21.15.120:0/2819656929 shutdown_connections 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d7d140 0x557c91d94710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d6cda0 0x557c91d7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 --2- 172.21.15.120:0/2819656929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d88e60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 -- 172.21.15.120:0/2819656929 >> 172.21.15.120:0/2819656929 conn(0x557c91d6a2b0 msgr2=0x557c91d7b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 -- 172.21.15.120:0/2819656929 shutdown_connections 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.689+0000 7efda54b3740 1 -- 172.21.15.120:0/2819656929 wait complete. 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 -- start start 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 Processor -- start 2024-09-10T22:52:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1aae70 con 0x557c91d6cda0 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ac2e0 con 0x557c91d6c480 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c1ab540 con 0x557c91d7d140 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda54b3740 10 client.0 _ll_register_callbacks cb 0x7efd8800b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efd9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60734/0 (socket says 172.21.15.120:60734) 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efd9bfff640 1 -- 172.21.15.120:0/3732387615 learned_addr learned my addr 172.21.15.120:0/3732387615 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda0e7a640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.690+0000 7efda167b640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 -- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 msgr2=0x557c91d81e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d81e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 -- 172.21.15.120:0/3732387615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 msgr2=0x557c91d85620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 -- 172.21.15.120:0/3732387615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c91d826c0 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda0e7a640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d850a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd90026040 tx=0x7efd9000e7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95efe7911b13b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efd7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90028040 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9001c980 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7efd9001f050 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efd9001cb30 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda54b3740 10 client.24839 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.691+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557c91dfa3d0 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001c980 con 0x557c91d6c480 2024-09-10T22:52:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efd90004790 con 0x557c91d6c480 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 10 client.24839 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 10 client.24839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 20 client.24839 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91dfaae0 0x557c91dfcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 4 client.24839 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c91ce0850 con 0x557c91dfaae0 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.692+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91dfaae0 0x557c91dfcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efd9bfff640 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91dfaae0 0x557c91dfcf00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cb56c4523013ce server_cookie=4d590834eaf0d442 in_seq=0 out_seq=0 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efd7bfff640 10 client.24839 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efd7bfff640 1 -- 172.21.15.120:0/3732387615 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x557c91ce0850 con 0x557c91dfaae0 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efd7bfff640 10 client.24839 handle_command_reply: tid=0 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 client.24839 shutdown 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 msgr2=0x557c91d850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d850a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd90026040 tx=0x7efd9000e7f0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 shutdown_connections 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557c91dfaae0 0x557c91dfcf00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c91d7d140 0x557c91d81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557c91d6cda0 0x557c91d85620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 --2- 172.21.15.120:0/3732387615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c91d6c480 0x557c91d850a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 >> 172.21.15.120:0/3732387615 conn(0x557c91d6a2b0 msgr2=0x557c91d7bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efd7bfff640 0 client.24839 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 shutdown_connections 2024-09-10T22:52:22.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 1 -- 172.21.15.120:0/3732387615 wait complete. 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.693+0000 7efda54b3740 20 client.24839 trim_cache size 0 max 16384 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "7cd29d46-2281-4b94-a993-80e3b8f9b9bd", 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:52:22.702 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:52:22.703 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:52:22.704 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:52:22.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:22.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:22.704 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:22.705 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:22 smithi153 ceph-mon[34951]: pgmap v341: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 msgr2=0x7efd8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 0x7efd8c0b77a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd9404b000 tx=0x7efd94048000 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 msgr2=0x7efd9c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b4a0 tx=0x7efd8800b960 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 shutdown_connections 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efd8c0b52e0 0x7efd8c0b77a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd9c108020 0x7efd9c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd9c1076a0 0x7efd9c19a610 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 --2- 172.21.15.120:0/3854968009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c101c90 0x7efd9c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 >> 172.21.15.120:0/3854968009 conn(0x7efd9c0fd700 msgr2=0x7efd9c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 shutdown_connections 2024-09-10T22:52:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:22.694+0000 7efda30de640 1 -- 172.21.15.120:0/3854968009 wait complete. 2024-09-10T22:52:23.563 INFO:tasks.cephfs.filesystem:scrub status for tag:7cd29d46-2281-4b94-a993-80e3b8f9b9bd - {'path': '/', 'tag': '7cd29d46-2281-4b94-a993-80e3b8f9b9bd', 'options': 'recursive,force'} 2024-09-10T22:52:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:24 smithi176 ceph-mon[34982]: pgmap v342: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-09-10T22:52:24.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:24 smithi120 ceph-mon[31247]: pgmap v342: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-09-10T22:52:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:24 smithi153 ceph-mon[34951]: pgmap v342: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 853 B/s wr, 2 op/s 2024-09-10T22:52:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:25 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:52:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:25 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:52:25.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:25 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:52:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 -- 172.21.15.120:0/1784077762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 msgr2=0x7ff1dc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/1784077762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc107d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009900 tx=0x7ff1cc02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 -- 172.21.15.120:0/1784077762 shutdown_connections 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/1784077762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 0x7ff1dc10a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/1784077762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc107d70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/1784077762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 0x7ff1dc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 -- 172.21.15.120:0/1784077762 >> 172.21.15.120:0/1784077762 conn(0x7ff1dc0fcf30 msgr2=0x7ff1dc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 -- 172.21.15.120:0/1784077762 shutdown_connections 2024-09-10T22:52:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.743+0000 7ff1e27a1640 1 -- 172.21.15.120:0/1784077762 wait complete. 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 Processor -- start 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 -- start start 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 0x7ff1dc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 0x7ff1dc199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc10e3f0 con 0x7ff1dc1058e0 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc10e270 con 0x7ff1dc06b430 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e27a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc10e570 con 0x7ff1dc108500 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54466/0 (socket says 172.21.15.120:54466) 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.744+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/4102097154 learned_addr learned my addr 172.21.15.120:0/4102097154 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e179f640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 0x7ff1dc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 0x7ff1dc199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/4102097154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 msgr2=0x7ff1dc199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 0x7ff1dc199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/4102097154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 msgr2=0x7ff1dc198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 0x7ff1dc198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/4102097154 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc19a690 con 0x7ff1dc1058e0 2024-09-10T22:52:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc005940 tx=0x7ff1cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1d27fc640 1 -- 172.21.15.120:0/4102097154 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc03f030 con 0x7ff1dc1058e0 2024-09-10T22:52:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc19a920 con 0x7ff1dc1058e0 2024-09-10T22:52:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1d27fc640 1 -- 172.21.15.120:0/4102097154 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc003aa0 con 0x7ff1dc1058e0 2024-09-10T22:52:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1d27fc640 1 -- 172.21.15.120:0/4102097154 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc035070 con 0x7ff1dc1058e0 2024-09-10T22:52:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.745+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc1a1050 con 0x7ff1dc1058e0 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1d27fc640 1 -- 172.21.15.120:0/4102097154 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff1cc03bbb0 con 0x7ff1dc1058e0 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 Processor -- start 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 -- start start 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab950a60 0x5576ab952e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab953390 0x5576ab955780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576ab8c26f0 con 0x5576ab955dd0 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576ab8b1b20 con 0x5576ab953390 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5576ab8b5930 con 0x5576ab950a60 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54470/0 (socket says 172.21.15.120:54470) 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.748+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3039501297 learned_addr learned my addr 172.21.15.120:0/3039501297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab950a60 0x5576ab952e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e179f640 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab953390 0x5576ab955780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1d27fc640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 0x7ff1c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3039501297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab950a60 msgr2=0x5576ab952e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab950a60 0x5576ab952e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3039501297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab953390 msgr2=0x5576ab955780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab953390 0x5576ab955780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3039501297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ab877530 con 0x5576ab955dd0 2024-09-10T22:52:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 0x7ff1c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1d27fc640 1 -- 172.21.15.120:0/4102097154 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff1cc1064d0 con 0x7ff1dc1058e0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0a4c20 tx=0x7ff1cc091430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a9857d7655fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 0x7ff1c80b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005c40 tx=0x7ff1d4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1c77fe640 1 -- 172.21.15.120:0/3039501297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0cc610 con 0x5576ab955dd0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1c77fe640 1 -- 172.21.15.120:0/3039501297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc09bb70 con 0x5576ab955dd0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.749+0000 7ff1c77fe640 1 -- 172.21.15.120:0/3039501297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0c8430 con 0x5576ab955dd0 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3039501297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 msgr2=0x5576ab958290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0a4c20 tx=0x7ff1cc091430 comp rx=0 tx=0).stop 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3039501297 shutdown_connections 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab955dd0 0x5576ab958290 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab953390 0x5576ab955780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3039501297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab950a60 0x5576ab952e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3039501297 >> 172.21.15.120:0/3039501297 conn(0x5576ab94bbb0 msgr2=0x5576ab94c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3039501297 shutdown_connections 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3039501297 wait complete. 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.750+0000 7ff1e67d8740 1 -- start start 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 Processor -- start 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 0x5576ab95a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab955dd0 0x5576ab95aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5576ab7e9ab0 con 0x5576ab950a60 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5576ab857730 con 0x5576ab955dd0 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5576ab958b70 con 0x5576ab953390 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e67d8740 10 client.0 _ll_register_callbacks cb 0x5576ab8ab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54472/0 (socket says 172.21.15.120:54472) 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 -- 172.21.15.120:0/1593907947 learned_addr learned my addr 172.21.15.120:0/1593907947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab955dd0 0x5576ab95aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 0x5576ab95a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 -- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 msgr2=0x5576ab95a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 0x5576ab95a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 -- 172.21.15.120:0/1593907947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab955dd0 msgr2=0x5576ab95aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab955dd0 0x5576ab95aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e179f640 1 -- 172.21.15.120:0/1593907947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ab613790 con 0x5576ab950a60 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.751+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 0x5576ab95a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e179f640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80114c0 tx=0x7ff1d8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b932d0a094810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8005020 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff1d80051c0 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576ab8c4640 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff1d8005400 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 10 client.15360 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5576ab68c620 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5576ab8c4640 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff1d8010b20 con 0x5576ab950a60 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 10 client.15360 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 20 client.15360 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab965690 0x5576ab967b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 4 client.15360 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.752+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ab8c00f0 con 0x5576ab965690 2024-09-10T22:52:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.753+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab965690 0x5576ab967b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.753+0000 7ff1c4ff9640 10 client.15360 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.753+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab965690 0x5576ab967b10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f7f35fdd39e2a4 server_cookie=5bbfae821bb69def in_seq=0 out_seq=0 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/1593907947 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576ab8c00f0 con 0x5576ab965690 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1c4ff9640 10 client.15360 handle_command_reply: tid=0 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 client.15360 shutdown 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 msgr2=0x5576ab959d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80114c0 tx=0x7ff1d8011980 comp rx=0 tx=0).stop 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 shutdown_connections 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab965690 0x5576ab967b10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab955dd0 0x5576ab95aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1c4ff9640 0 client.15360 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab953390 0x5576ab95a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/1593907947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab950a60 0x5576ab959d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 >> 172.21.15.120:0/1593907947 conn(0x5576ab94bbb0 msgr2=0x5576ab94ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 shutdown_connections 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 1 -- 172.21.15.120:0/1593907947 wait complete. 2024-09-10T22:52:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.754+0000 7ff1e67d8740 20 client.15360 trim_cache size 0 max 16384 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 Processor -- start 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 -- start start 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab94ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab94ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1a7c20 con 0x5576ab953390 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1aa4a0 con 0x5576ab950a60 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1a8290 con 0x5576ab955dd0 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54484/0 (socket says 172.21.15.120:54484) 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/710706068 learned_addr learned my addr 172.21.15.120:0/710706068 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e179f640 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab94ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab94ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/710706068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 msgr2=0x5576ab94ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab94ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/710706068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 msgr2=0x5576ab94ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab94ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/710706068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ab9472c0 con 0x5576ab953390 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.772+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc104370 tx=0x7ff1cc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02322ca04f8c1f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1c6ffd640 1 -- 172.21.15.120:0/710706068 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc049040 con 0x5576ab953390 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1c6ffd640 1 -- 172.21.15.120:0/710706068 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc0cc7d0 con 0x5576ab953390 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1c6ffd640 1 -- 172.21.15.120:0/710706068 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc09bd60 con 0x5576ab953390 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 -- 172.21.15.120:0/710706068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 msgr2=0x5576ab94e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc104370 tx=0x7ff1cc005c90 comp rx=0 tx=0).stop 2024-09-10T22:52:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 -- 172.21.15.120:0/710706068 shutdown_connections 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab94ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab94e9f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/710706068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab94ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 -- 172.21.15.120:0/710706068 >> 172.21.15.120:0/710706068 conn(0x5576ab94bbb0 msgr2=0x5576ab94a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 -- 172.21.15.120:0/710706068 shutdown_connections 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.773+0000 7ff1e67d8740 1 -- 172.21.15.120:0/710706068 wait complete. 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 -- start start 2024-09-10T22:52:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 Processor -- start 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab961420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab960210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1a8960 con 0x5576ab953390 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1a9dd0 con 0x5576ab950a60 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1a9030 con 0x5576ab955dd0 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e67d8740 10 client.0 _ll_register_callbacks cb 0x7ff1d800bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e0f9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54492/0 (socket says 172.21.15.120:54492) 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3691693153 learned_addr learned my addr 172.21.15.120:0/3691693153 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.774+0000 7ff1e179f640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab961420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e1fa0640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab960210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 msgr2=0x5576ab960210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab960210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3691693153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 msgr2=0x5576ab961420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab961420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 -- 172.21.15.120:0/3691693153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ab95b360 con 0x5576ab953390 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e0f9e640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc10c0e0 tx=0x7ff1cc0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb0399b6c6e9df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc049040 con 0x5576ab953390 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff1cc0980d0 con 0x5576ab953390 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc0cc7d0 con 0x5576ab953390 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc0cc430 con 0x5576ab953390 2024-09-10T22:52:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 10 client.15366 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5576ab95f6e0 con 0x5576ab953390 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc0cc7d0 con 0x5576ab953390 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff1cc0c86d0 con 0x5576ab953390 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 10 client.15366 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 20 client.15366 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab9d2650 0x5576ab9d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 4 client.15366 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.775+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576ab8bd520 con 0x5576ab9d2650 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.776+0000 7ff1e179f640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab9d2650 0x5576ab9d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.776+0000 7ff1c4ff9640 10 client.15366 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.776+0000 7ff1e179f640 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab9d2650 0x5576ab9d4a40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e863778212f23b5 server_cookie=4c6df24a5ac749a in_seq=0 out_seq=0 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1c4ff9640 1 -- 172.21.15.120:0/3691693153 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576ab8bd520 con 0x5576ab9d2650 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1c4ff9640 10 client.15366 handle_command_reply: tid=0 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 client.15366 shutdown 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 msgr2=0x5576ab961ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc10c0e0 tx=0x7ff1cc0314a0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 shutdown_connections 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5576ab9d2650 0x5576ab9d4a40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5576ab955dd0 0x5576ab960210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1c4ff9640 0 client.15366 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5576ab953390 0x5576ab961ac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 --2- 172.21.15.120:0/3691693153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5576ab950a60 0x5576ab961420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 >> 172.21.15.120:0/3691693153 conn(0x5576ab94bbb0 msgr2=0x5576ab94a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 shutdown_connections 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 1 -- 172.21.15.120:0/3691693153 wait complete. 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.787+0000 7ff1e67d8740 20 client.15366 trim_cache size 0 max 16384 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:52:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 msgr2=0x7ff1c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 0x7ff1c80b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4005c40 tx=0x7ff1d4004a40 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 msgr2=0x7ff1dc1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc005940 tx=0x7ff1cc031a50 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 shutdown_connections 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1c80b52e0 0x7ff1c80b77a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1dc108500 0x7ff1dc199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1dc1058e0 0x7ff1dc1991a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 --2- 172.21.15.120:0/4102097154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc06b430 0x7ff1dc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 >> 172.21.15.120:0/4102097154 conn(0x7ff1dc0fcf30 msgr2=0x7ff1dc103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 shutdown_connections 2024-09-10T22:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:25.788+0000 7ff1e27a1640 1 -- 172.21.15.120:0/4102097154 wait complete. 2024-09-10T22:52:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:26 smithi176 ceph-mon[34982]: pgmap v343: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T22:52:26.600 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:26 smithi153 ceph-mon[34951]: pgmap v343: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T22:52:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:26 smithi120 ceph-mon[31247]: pgmap v343: 97 pgs: 97 active+clean; 648 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T22:52:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:28 smithi176 ceph-mon[34982]: pgmap v344: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T22:52:28.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:28 smithi120 ceph-mon[31247]: pgmap v344: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T22:52:28.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:28 smithi153 ceph-mon[34951]: pgmap v344: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T22:52:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:30 smithi176 ceph-mon[34982]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:30.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:30 smithi120 ceph-mon[31247]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:30.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:30 smithi153 ceph-mon[34951]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:32 smithi176 ceph-mon[34982]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T22:52:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:32 smithi120 ceph-mon[31247]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T22:52:32.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:32 smithi153 ceph-mon[34951]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T22:52:33.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:33 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:33.672 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:33 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:33 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:34 smithi176 ceph-mon[34982]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:34.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:34 smithi120 ceph-mon[31247]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:34.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:34 smithi153 ceph-mon[34951]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:35.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:35.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:35 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:35.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:35.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:35 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:35.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:35.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:35 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:52:36.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:36 smithi176 ceph-mon[34982]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:36.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:36 smithi120 ceph-mon[31247]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:36.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:36 smithi153 ceph-mon[34951]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T22:52:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:38 smithi176 ceph-mon[34982]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T22:52:38.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:38 smithi120 ceph-mon[31247]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T22:52:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:38 smithi153 ceph-mon[34951]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 -- 172.21.15.120:0/3986905598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38102990 msgr2=0x7f6e38102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 --2- 172.21.15.120:0/3986905598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38102990 0x7f6e38102d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009930 tx=0x7f6e3402a420 comp rx=0 tx=0).stop 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 -- 172.21.15.120:0/3986905598 shutdown_connections 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 --2- 172.21.15.120:0/3986905598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e381044a0 0x7f6e3810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 --2- 172.21.15.120:0/3986905598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e38103ae0 0x7f6e38103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 --2- 172.21.15.120:0/3986905598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38102990 0x7f6e38102d90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 -- 172.21.15.120:0/3986905598 >> 172.21.15.120:0/3986905598 conn(0x7f6e380fe250 msgr2=0x7f6e38100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 -- 172.21.15.120:0/3986905598 shutdown_connections 2024-09-10T22:52:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.923+0000 7f6e4052b640 1 -- 172.21.15.120:0/3986905598 wait complete. 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.924+0000 7f6e4052b640 1 Processor -- start 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 -- start start 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e38102990 0x7f6e38198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e381044a0 0x7f6e3819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3810e5e0 con 0x7f6e38103ae0 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3810e460 con 0x7f6e381044a0 2024-09-10T22:52:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e4052b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3810e760 con 0x7f6e38102990 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e3dac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e3dac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55678/0 (socket says 172.21.15.120:55678) 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e3dac6640 1 -- 172.21.15.120:0/4065463670 learned_addr learned my addr 172.21.15.120:0/4065463670 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e38102990 0x7f6e38198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.925+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e381044a0 0x7f6e3819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 -- 172.21.15.120:0/4065463670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e38102990 msgr2=0x7f6e38198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e38102990 0x7f6e38198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 -- 172.21.15.120:0/4065463670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e381044a0 msgr2=0x7f6e3819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e381044a0 0x7f6e3819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 -- 172.21.15.120:0/4065463670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3819a8c0 con 0x7f6e38103ae0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d950 tx=0x7f6e2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e2f7fe640 1 -- 172.21.15.120:0/4065463670 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e38103ae0 2024-09-10T22:52:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3819abb0 con 0x7f6e38103ae0 2024-09-10T22:52:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e38075560 con 0x7f6e38103ae0 2024-09-10T22:52:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.926+0000 7f6e2f7fe640 1 -- 172.21.15.120:0/4065463670 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e28009c30 con 0x7f6e38103ae0 2024-09-10T22:52:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.927+0000 7f6e2f7fe640 1 -- 172.21.15.120:0/4065463670 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800f620 con 0x7f6e38103ae0 2024-09-10T22:52:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 Processor -- start 2024-09-10T22:52:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 -- start start 2024-09-10T22:52:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 0x5583e9f2ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2dbd0 0x5583e9f30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5583e9e9a6f0 con 0x5583e9f2dbd0 2024-09-10T22:52:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5583e9e89b20 con 0x5583e9f28860 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.930+0000 7f6e433d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5583e9e8d930 con 0x5583e9f2b190 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60324/0 (socket says 172.21.15.120:60324) 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/235694320 learned_addr learned my addr 172.21.15.120:0/235694320 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 0x5583e9f2ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/235694320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 msgr2=0x5583e9f2ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 0x5583e9f2ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/235694320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2dbd0 msgr2=0x5583e9f30090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2dbd0 0x5583e9f30090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/235694320 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e9e4f530 con 0x5583e9f2b190 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 0x5583e9f2ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.931+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402a940 tx=0x7f6e34031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64bac0be2e9ea07d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e2cbfe640 1 -- 172.21.15.120:0/235694320 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34035030 con 0x5583e9f2b190 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e2cbfe640 1 -- 172.21.15.120:0/235694320 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e34040070 con 0x5583e9f2b190 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e2cbfe640 1 -- 172.21.15.120:0/235694320 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34003980 con 0x5583e9f2b190 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e2f7fe640 1 -- 172.21.15.120:0/4065463670 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6e28010460 con 0x7f6e38103ae0 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e433d3740 1 -- 172.21.15.120:0/235694320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 msgr2=0x5583e9f2d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.932+0000 7f6e433d3740 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402a940 tx=0x7f6e34031b10 comp rx=0 tx=0).stop 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 -- 172.21.15.120:0/235694320 shutdown_connections 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2dbd0 0x5583e9f30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f2d580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 --2- 172.21.15.120:0/235694320 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f28860 0x5583e9f2ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 -- 172.21.15.120:0/235694320 >> 172.21.15.120:0/235694320 conn(0x5583e9f239b0 msgr2=0x5583e9f24740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 -- 172.21.15.120:0/235694320 shutdown_connections 2024-09-10T22:52:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 -- 172.21.15.120:0/235694320 wait complete. 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e2f7fe640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 0x7f6e240b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 -- start start 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e433d3740 1 Processor -- start 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 0x7f6e240b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.933+0000 7f6e2f7fe640 1 -- 172.21.15.120:0/4065463670 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6e280e56e0 con 0x7f6e38103ae0 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60326/0 (socket says 172.21.15.120:60326) 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/2248600160 learned_addr learned my addr 172.21.15.120:0/2248600160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5583e9dc1ab0 con 0x5583e9f2b190 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5583e9e2f730 con 0x5583e9f2dbd0 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5583e9f30b50 con 0x5583e9f28860 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 0x7f6e240b7750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e38199b70 tx=0x7f6e30012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e433d3740 10 client.0 _ll_register_callbacks cb 0x5583e9e836b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/2248600160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 msgr2=0x5583e9f33170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f33170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/2248600160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 msgr2=0x5583e9f325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e9beb790 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009a50 tx=0x7f6e340042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29bc275b58b0a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.934+0000 7f6e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34035030 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6e34003b70 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e34040070 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e34031910 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 10 client.24856 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5583e9c64620 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e34003d70 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e3403c440 con 0x5583e9f28860 2024-09-10T22:52:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 10 client.24856 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 10 client.24856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 20 client.24856 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.935+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9f3f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.936+0000 7f6e433d3740 4 client.24856 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.936+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6e34040070 con 0x5583e9f3d360 2024-09-10T22:52:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.936+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9f3f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.936+0000 7f6e21ffb640 10 client.24856 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.936+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9f3f750 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d50ed0742dc48d server_cookie=d18f047842a45bc7 in_seq=0 out_seq=0 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e21ffb640 1 -- 172.21.15.120:0/2248600160 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6e34040070 con 0x5583e9f3d360 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e21ffb640 10 client.24856 handle_command_reply: tid=0 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 client.24856 shutdown 2024-09-10T22:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 msgr2=0x5583e9f31f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009a50 tx=0x7f6e340042f0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 shutdown_connections 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9f3f750 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e21ffb640 0 client.24856 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 --2- 172.21.15.120:0/2248600160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f31f00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 >> 172.21.15.120:0/2248600160 conn(0x5583e9f239b0 msgr2=0x5583e9f22bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 shutdown_connections 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 1 -- 172.21.15.120:0/2248600160 wait complete. 2024-09-10T22:52:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.937+0000 7f6e433d3740 20 client.24856 trim_cache size 0 max 16384 2024-09-10T22:52:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.965+0000 7f6e433d3740 1 Processor -- start 2024-09-10T22:52:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.965+0000 7f6e433d3740 1 -- start start 2024-09-10T22:52:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f3a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f31160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a5b90 con 0x5583e9f28860 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a8410 con 0x5583e9f2dbd0 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e433d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a6200 con 0x5583e9f2b190 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55706/0 (socket says 172.21.15.120:55706) 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/729630531 learned_addr learned my addr 172.21.15.120:0/729630531 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f3a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/729630531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 msgr2=0x5583e9f3a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f3a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/729630531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 msgr2=0x5583e9f31160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/729630531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e9f2afc0 con 0x5583e9f28860 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.966+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6e3403c8b0 tx=0x7f6e34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bf2db1e5a5f993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.967+0000 7f6e23fff640 1 -- 172.21.15.120:0/729630531 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34009520 con 0x5583e9f28860 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.967+0000 7f6e23fff640 1 -- 172.21.15.120:0/729630531 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e34040070 con 0x5583e9f28860 2024-09-10T22:52:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.967+0000 7f6e23fff640 1 -- 172.21.15.120:0/729630531 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3403c6a0 con 0x5583e9f28860 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- 172.21.15.120:0/729630531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 msgr2=0x5583e9f33150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6e3403c8b0 tx=0x7f6e34005fd0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- 172.21.15.120:0/729630531 shutdown_connections 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f31160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f2b190 0x5583e9f3a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 --2- 172.21.15.120:0/729630531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f28860 0x5583e9f33150 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- 172.21.15.120:0/729630531 >> 172.21.15.120:0/729630531 conn(0x5583e9f239b0 msgr2=0x5583e9f24450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- 172.21.15.120:0/729630531 shutdown_connections 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- 172.21.15.120:0/729630531 wait complete. 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 -- start start 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.968+0000 7f6e433d3740 1 Processor -- start 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3dac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60330/0 (socket says 172.21.15.120:60330) 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3dac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55710/0 (socket says 172.21.15.120:55710) 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a68d0 con 0x5583e9f2b190 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/3562976621 learned_addr learned my addr 172.21.15.120:0/3562976621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a7d40 con 0x5583e9f2dbd0 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381a6fa0 con 0x5583e9f28860 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e433d3740 10 client.0 _ll_register_callbacks cb 0x7f6e34005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3eac8640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/3562976621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 msgr2=0x5583e9f39070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f39070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/3562976621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 msgr2=0x5583e9f38b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e9f39850 con 0x5583e9f28860 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e3e2c7640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e3403c9a0 tx=0x7f6e3403caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c0c8943c600978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.969+0000 7f6e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:52:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3403ccc0 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6e34003b80 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e34040070 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e34009520 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 10 client.24862 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5583e9fb2a40 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e34067070 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e34045470 con 0x5583e9f28860 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 10 client.24862 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 10 client.24862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 20 client.24862 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9fb5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 4 client.24862 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.971+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6e34040070 con 0x5583e9f3d360 2024-09-10T22:52:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.972+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9fb5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.972+0000 7f6e21ffb640 10 client.24862 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.972+0000 7f6e3dac6640 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9fb5290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54880c288d8d9a83 server_cookie=9d4d992fadd444c6 in_seq=0 out_seq=0 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e21ffb640 1 -- 172.21.15.120:0/3562976621 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6e34040070 con 0x5583e9f3d360 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e21ffb640 10 client.24862 handle_command_reply: tid=0 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 client.24862 shutdown 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 msgr2=0x5583e9f39e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e3403c9a0 tx=0x7f6e3403caa0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 shutdown_connections 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5583e9f3d360 0x5583e9fb5290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5583e9f2dbd0 0x5583e9f39070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e21ffb640 0 client.24862 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5583e9f2b190 0x5583e9f38b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 --2- 172.21.15.120:0/3562976621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5583e9f28860 0x5583e9f39e30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 >> 172.21.15.120:0/3562976621 conn(0x5583e9f239b0 msgr2=0x5583e9f2dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 shutdown_connections 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.976+0000 7f6e433d3740 1 -- 172.21.15.120:0/3562976621 wait complete. 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e433d3740 20 client.24862 trim_cache size 0 max 16384 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:52:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:52:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 msgr2=0x7f6e240b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 0x7f6e240b7750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e38199b70 tx=0x7f6e30012040 comp rx=0 tx=0).stop 2024-09-10T22:52:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 msgr2=0x7f6e38199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d950 tx=0x7f6e2800de10 comp rx=0 tx=0).stop 2024-09-10T22:52:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 shutdown_connections 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6e240b5290 0x7f6e240b7750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e381044a0 0x7f6e3819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e38103ae0 0x7f6e38199440 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 --2- 172.21.15.120:0/4065463670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e38102990 0x7f6e38198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.977+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 >> 172.21.15.120:0/4065463670 conn(0x7f6e380fe250 msgr2=0x7f6e38106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.978+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 shutdown_connections 2024-09-10T22:52:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:38.978+0000 7f6e4052b640 1 -- 172.21.15.120:0/4065463670 wait complete. 2024-09-10T22:52:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:40 smithi176 ceph-mon[34982]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 3 op/s 2024-09-10T22:52:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:40 smithi120 ceph-mon[31247]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 3 op/s 2024-09-10T22:52:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:40 smithi153 ceph-mon[34951]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 3 op/s 2024-09-10T22:52:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:42 smithi176 ceph-mon[34982]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:42.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:42 smithi120 ceph-mon[31247]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:42.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:42 smithi153 ceph-mon[34951]: pgmap v351: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:44 smithi176 ceph-mon[34982]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:44.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:44 smithi120 ceph-mon[31247]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:44 smithi153 ceph-mon[34951]: pgmap v352: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:46 smithi176 ceph-mon[34982]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:46 smithi120 ceph-mon[31247]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:46 smithi153 ceph-mon[34951]: pgmap v353: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:48.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:48 smithi120 ceph-mon[31247]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:48 smithi176 ceph-mon[34982]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:48.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:48 smithi153 ceph-mon[34951]: pgmap v354: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:50.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:50 smithi120 ceph-mon[31247]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:50 smithi176 ceph-mon[34982]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:50 smithi153 ceph-mon[34951]: pgmap v355: 97 pgs: 97 active+clean; 650 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T22:52:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 -- 172.21.15.120:0/3888330973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 msgr2=0x7f7d2c10ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 --2- 172.21.15.120:0/3888330973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c10ab80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009ca0 tx=0x7f7d2402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 -- 172.21.15.120:0/3888330973 shutdown_connections 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 --2- 172.21.15.120:0/3888330973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c10ab80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 --2- 172.21.15.120:0/3888330973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c105b10 0x7f7d2c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 --2- 172.21.15.120:0/3888330973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 -- 172.21.15.120:0/3888330973 >> 172.21.15.120:0/3888330973 conn(0x7f7d2c0fd150 msgr2=0x7f7d2c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 -- 172.21.15.120:0/3888330973 shutdown_connections 2024-09-10T22:52:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.103+0000 7f7d30917640 1 -- 172.21.15.120:0/3888330973 wait complete. 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 Processor -- start 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 -- start start 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 0x7f7d2c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10e610 con 0x7f7d2c1086c0 2024-09-10T22:52:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10e490 con 0x7f7d2c06bfb0 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d30917640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10e790 con 0x7f7d2c105b10 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40204/0 (socket says 172.21.15.120:40204) 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1309427018 learned_addr learned my addr 172.21.15.120:0/1309427018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 0x7f7d2c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1309427018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 msgr2=0x7f7d2c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 0x7f7d2c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1309427018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 msgr2=0x7f7d2c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.104+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1309427018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c19a8d0 con 0x7f7d2c1086c0 2024-09-10T22:52:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 0x7f7d2c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097b0 tx=0x7f7d24030810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d1b7fe640 1 -- 172.21.15.120:0/1309427018 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24030e40 con 0x7f7d2c1086c0 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c19ab60 con 0x7f7d2c1086c0 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d1b7fe640 1 -- 172.21.15.120:0/1309427018 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d24041070 con 0x7f7d2c1086c0 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d1b7fe640 1 -- 172.21.15.120:0/1309427018 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2404b060 con 0x7f7d2c1086c0 2024-09-10T22:52:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.105+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c101540 con 0x7f7d2c1086c0 2024-09-10T22:52:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.108+0000 7f7d337c1740 1 Processor -- start 2024-09-10T22:52:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d1b7fe640 1 -- 172.21.15.120:0/1309427018 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7d2404e060 con 0x7f7d2c1086c0 2024-09-10T22:52:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 -- start start 2024-09-10T22:52:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965df2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f965d596f0 con 0x55f965df53e0 2024-09-10T22:52:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f965d48b20 con 0x55f965df0070 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d337c1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f965d4c930 con 0x55f965df29a0 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d29d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d29d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40216/0 (socket says 172.21.15.120:40216) 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.109+0000 7f7d29d9b640 1 -- 172.21.15.120:0/3535362176 learned_addr learned my addr 172.21.15.120:0/3535362176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965df2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 -- 172.21.15.120:0/3535362176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 msgr2=0x55f965df2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965df2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 -- 172.21.15.120:0/3535362176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 msgr2=0x55f965df78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 -- 172.21.15.120:0/3535362176 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f965d0e530 con 0x55f965df29a0 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d1b7fe640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 0x7f7d140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 0x7f7d140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df4d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009fd0 tx=0x7f7d1c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=970a5d7c76d821f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.110+0000 7f7d1b7fe640 1 -- 172.21.15.120:0/1309427018 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f7d241065a0 con 0x7f7d2c1086c0 2024-09-10T22:52:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d18ff9640 1 -- 172.21.15.120:0/3535362176 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004310 con 0x55f965df29a0 2024-09-10T22:52:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d18ff9640 1 -- 172.21.15.120:0/3535362176 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c0044b0 con 0x55f965df29a0 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d18ff9640 1 -- 172.21.15.120:0/3535362176 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0035f0 con 0x55f965df29a0 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 -- 172.21.15.120:0/3535362176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 msgr2=0x55f965df4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 0x7f7d140b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009fd0 tx=0x7f7d20009790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df4d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009fd0 tx=0x7f7d1c0118d0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 -- 172.21.15.120:0/3535362176 shutdown_connections 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df4d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 --2- 172.21.15.120:0/3535362176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965df2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.111+0000 7f7d337c1740 1 -- 172.21.15.120:0/3535362176 >> 172.21.15.120:0/3535362176 conn(0x55f965deb1c0 msgr2=0x55f965debf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 -- 172.21.15.120:0/3535362176 shutdown_connections 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 -- 172.21.15.120:0/3535362176 wait complete. 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 -- start start 2024-09-10T22:52:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 Processor -- start 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df0070 0x55f965df96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df29a0 0x55f965df9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f965c80ab0 con 0x55f965df53e0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.112+0000 7f7d337c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f965cee730 con 0x55f965df29a0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d337c1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f965df83c0 con 0x55f965df0070 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40232/0 (socket says 172.21.15.120:40232) 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1634138911 learned_addr learned my addr 172.21.15.120:0/1634138911 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df0070 0x55f965df96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d337c1740 10 client.0 _ll_register_callbacks cb 0x55f965d426b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df29a0 0x55f965df9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df0070 msgr2=0x55f965df96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df0070 0x55f965df96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1634138911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df29a0 msgr2=0x55f965df9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df29a0 0x55f965df9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/1634138911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f965aaa790 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d240ccdd0 tx=0x7f7d240cc670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797acb88f3a7b574 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24004d40 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c0044b0 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f7d241060d0 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7d240b42f0 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d337c1740 10 client.15393 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.113+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f965b23620 con 0x55f965df53e0 2024-09-10T22:52:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d240f2020 con 0x55f965df53e0 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f7d240c8790 con 0x55f965df53e0 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 10 client.15393 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 20 client.15393 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e04f80 0x55f965e073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 4 client.15393 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f965d5b640 con 0x55f965e04f80 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.114+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e04f80 0x55f965e073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.115+0000 7f7d11ffb640 10 client.15393 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.115+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e04f80 0x55f965e073d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ca6716ff44424e server_cookie=461e2eadb69dd6e in_seq=0 out_seq=0 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.115+0000 7f7d11ffb640 1 -- 172.21.15.120:0/1634138911 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f965d5b640 con 0x55f965e04f80 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.115+0000 7f7d11ffb640 10 client.15393 handle_command_reply: tid=0 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.115+0000 7f7d337c1740 1 client.15393 shutdown 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 msgr2=0x55f965dfa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d240ccdd0 tx=0x7f7d240cc670 comp rx=0 tx=0).stop 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 shutdown_connections 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e04f80 0x55f965e073d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d11ffb640 0 client.15393 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965dfa940 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df29a0 0x55f965df9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 --2- 172.21.15.120:0/1634138911 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df0070 0x55f965df96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 >> 172.21.15.120:0/1634138911 conn(0x55f965deb1c0 msgr2=0x55f965df2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 shutdown_connections 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 1 -- 172.21.15.120:0/1634138911 wait complete. 2024-09-10T22:52:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.116+0000 7f7d337c1740 20 client.15393 trim_cache size 0 max 16384 2024-09-10T22:52:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 Processor -- start 2024-09-10T22:52:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 -- start start 2024-09-10T22:52:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a5bc0 con 0x55f965df53e0 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a8420 con 0x55f965df0070 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d337c1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a6210 con 0x55f965df29a0 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.144+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54986/0 (socket says 172.21.15.120:54986) 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 -- 172.21.15.120:0/50500012 learned_addr learned my addr 172.21.15.120:0/50500012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 -- 172.21.15.120:0/50500012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 msgr2=0x55f965df3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 -- 172.21.15.120:0/50500012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 msgr2=0x55f965df44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 -- 172.21.15.120:0/50500012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f965def180 con 0x55f965df0070 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c014040 tx=0x7f7d1c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ccefceedf4521f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d13fff640 1 -- 172.21.15.120:0/50500012 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c032030 con 0x55f965df0070 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d13fff640 1 -- 172.21.15.120:0/50500012 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c009b30 con 0x55f965df0070 2024-09-10T22:52:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.145+0000 7f7d13fff640 1 -- 172.21.15.120:0/50500012 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00f4e0 con 0x55f965df0070 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- 172.21.15.120:0/50500012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 msgr2=0x55f965dfb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c014040 tx=0x7f7d1c005d30 comp rx=0 tx=0).stop 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- 172.21.15.120:0/50500012 shutdown_connections 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df53e0 0x55f965df44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 --2- 172.21.15.120:0/50500012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df0070 0x55f965dfb210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- 172.21.15.120:0/50500012 >> 172.21.15.120:0/50500012 conn(0x55f965deb1c0 msgr2=0x55f965df4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- 172.21.15.120:0/50500012 shutdown_connections 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- 172.21.15.120:0/50500012 wait complete. 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 -- start start 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.146+0000 7f7d337c1740 1 Processor -- start 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 0x55f965df5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a68e0 con 0x55f965df0070 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a7d50 con 0x55f965df53e0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1a6fb0 con 0x55f965df29a0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d337c1740 10 client.0 _ll_register_callbacks cb 0x7f7d24005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 0x55f965df5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55002/0 (socket says 172.21.15.120:55002) 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/32735945 learned_addr learned my addr 172.21.15.120:0/32735945 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d29d9b640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 msgr2=0x55f965df6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/32735945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 msgr2=0x55f965df5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 0x55f965df5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 -- 172.21.15.120:0/32735945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f965e701a0 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 0x55f965df5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d2ad9d640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d240c8bd0 tx=0x7f7d24004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d722a7aeee7652 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.147+0000 7f7d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2407c020 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c009b30 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f7d241062e0 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7d240ccc10 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 10 client.24887 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f965e71520 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d240f2070 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f7d240d2a00 con 0x55f965df53e0 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 10 client.24887 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 10 client.24887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 20 client.24887 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e71d40 0x55f965e74160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 4 client.24887 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.148+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7d1c009b30 con 0x55f965e71d40 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.149+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e71d40 0x55f965e74160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.149+0000 7f7d11ffb640 10 client.24887 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.149+0000 7f7d2a59c640 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e71d40 0x55f965e74160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34975eae2935218a server_cookie=d4bbf13a2ac7d377 in_seq=0 out_seq=0 2024-09-10T22:52:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d11ffb640 1 -- 172.21.15.120:0/32735945 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7d1c009b30 con 0x55f965e71d40 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d11ffb640 10 client.24887 handle_command_reply: tid=0 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 client.24887 shutdown 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 msgr2=0x55f965df6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d240c8bd0 tx=0x7f7d24004d70 comp rx=0 tx=0).stop 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 shutdown_connections 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f965e71d40 0x55f965e74160 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f965df53e0 0x55f965df6890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d11ffb640 0 client.24887 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f965df29a0 0x55f965df6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 --2- 172.21.15.120:0/32735945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965df0070 0x55f965df5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 >> 172.21.15.120:0/32735945 conn(0x55f965deb1c0 msgr2=0x55f965df3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 shutdown_connections 2024-09-10T22:52:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 1 -- 172.21.15.120:0/32735945 wait complete. 2024-09-10T22:52:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.157+0000 7f7d337c1740 20 client.24887 trim_cache size 0 max 16384 2024-09-10T22:52:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:52:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:52:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:52:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 msgr2=0x7f7d140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 0x7f7d140b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009fd0 tx=0x7f7d20009790 comp rx=0 tx=0).stop 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 msgr2=0x7f7d2c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097b0 tx=0x7f7d24030810 comp rx=0 tx=0).stop 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 shutdown_connections 2024-09-10T22:52:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d140b52e0 0x7f7d140b77a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d2c1086c0 0x7f7d2c19a010 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d2c105b10 0x7f7d2c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 --2- 172.21.15.120:0/1309427018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d2c06bfb0 0x7f7d2c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 >> 172.21.15.120:0/1309427018 conn(0x7f7d2c0fd150 msgr2=0x7f7d2c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 shutdown_connections 2024-09-10T22:52:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:52:52.158+0000 7f7d30917640 1 -- 172.21.15.120:0/1309427018 wait complete. 2024-09-10T22:52:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:52 smithi176 ceph-mon[34982]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:52.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:52 smithi120 ceph-mon[31247]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:52.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:52 smithi153 ceph-mon[34951]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T22:52:53.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:53.892 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:54.320 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:54 smithi120 ceph-mon[31247]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 -- 172.21.15.120:0/3472353919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f01076a0 msgr2=0x7f35f0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 --2- 172.21.15.120:0/3472353919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f01076a0 0x7f35f0107ae0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009900 tx=0x7f35e402a410 comp rx=0 tx=0).stop 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 -- 172.21.15.120:0/3472353919 shutdown_connections 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 --2- 172.21.15.120:0/3472353919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f0108020 0x7f35f010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 --2- 172.21.15.120:0/3472353919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f01076a0 0x7f35f0107ae0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 --2- 172.21.15.120:0/3472353919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 -- 172.21.15.120:0/3472353919 >> 172.21.15.120:0/3472353919 conn(0x7f35f00fd700 msgr2=0x7f35f00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 -- 172.21.15.120:0/3472353919 shutdown_connections 2024-09-10T22:52:54.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.429+0000 7f35f48fb640 1 -- 172.21.15.120:0/3472353919 wait complete. 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.430+0000 7f35f48fb640 1 Processor -- start 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.430+0000 7f35f48fb640 1 -- start start 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.430+0000 7f35f48fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35f48fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 0x7f35f0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35f48fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f0108020 0x7f35f019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35f48fb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112d40 con 0x7f35f0108020 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35f48fb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112bc0 con 0x7f35f01076a0 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35f48fb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112ec0 con 0x7f35f0101c90 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38834/0 (socket says 172.21.15.120:38834) 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 -- 172.21.15.120:0/874243768 learned_addr learned my addr 172.21.15.120:0/874243768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:54.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35eed9d640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f0108020 0x7f35f019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35edd9b640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 0x7f35f0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 -- 172.21.15.120:0/874243768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 msgr2=0x7f35f0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 0x7f35f0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 -- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f0108020 msgr2=0x7f35f019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f0108020 0x7f35f019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35ee59c640 1 -- 172.21.15.120:0/874243768 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019f400 con 0x7f35f0101c90 2024-09-10T22:52:54.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.431+0000 7f35edd9b640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 0x7f35f0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35ee59c640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d950 tx=0x7f35e000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35df7fe640 1 -- 172.21.15.120:0/874243768 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0017070 con 0x7f35f0101c90 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35df7fe640 1 -- 172.21.15.120:0/874243768 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e0009c30 con 0x7f35f0101c90 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35df7fe640 1 -- 172.21.15.120:0/874243768 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000f620 con 0x7f35f0101c90 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f019f6f0 con 0x7f35f0101c90 2024-09-10T22:52:54.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.433+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a5db0 con 0x7f35f0101c90 2024-09-10T22:52:54.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 Processor -- start 2024-09-10T22:52:54.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 -- start start 2024-09-10T22:52:54.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35df7fe640 1 -- 172.21.15.120:0/874243768 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f35e0010460 con 0x7f35f0101c90 2024-09-10T22:52:54.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261480 0x55d26b261860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 0x55d26b271c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b274530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d26b1d7e50 con 0x55d26b272140 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d26b1c7280 con 0x55d26b261480 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.437+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d26b1cb090 con 0x55d26b261da0 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 0x55d26b271c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b274530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b274530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51432/0 (socket says 172.21.15.120:51432) 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35edd9b640 1 -- 172.21.15.120:0/1442088640 learned_addr learned my addr 172.21.15.120:0/1442088640 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 -- 172.21.15.120:0/1442088640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261480 msgr2=0x55d26b261860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261480 0x55d26b261860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 -- 172.21.15.120:0/1442088640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 msgr2=0x55d26b274530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b274530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.438+0000 7f35ee59c640 1 -- 172.21.15.120:0/1442088640 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26af25650 con 0x55d26b261da0 2024-09-10T22:52:54.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 0x55d26b271c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35e00e3310 tx=0x7f35e00ed670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9fbed518d5d1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35dcff9640 1 -- 172.21.15.120:0/1442088640 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00f4070 con 0x55d26b261da0 2024-09-10T22:52:54.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35df7fe640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 0x7f35d80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35dcff9640 1 -- 172.21.15.120:0/1442088640 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e00e7780 con 0x55d26b261da0 2024-09-10T22:52:54.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35dcff9640 1 -- 172.21.15.120:0/1442088640 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00f0910 con 0x55d26b261da0 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35eed9d640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 0x7f35d80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.439+0000 7f35df7fe640 1 -- 172.21.15.120:0/874243768 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f35e00e58d0 con 0x7f35f0101c90 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.440+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1442088640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 msgr2=0x55d26b271c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.440+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 0x55d26b271c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35e00e3310 tx=0x7f35e00ed670 comp rx=0 tx=0).stop 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35eed9d640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 0x7f35d80b78a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009370 tx=0x7f35e80092a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1442088640 shutdown_connections 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b274530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261da0 0x55d26b271c00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1442088640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261480 0x55d26b261860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1442088640 >> 172.21.15.120:0/1442088640 conn(0x55d26b25f2b0 msgr2=0x55d26b260450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1442088640 shutdown_connections 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1442088640 wait complete. 2024-09-10T22:52:54.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 -- start start 2024-09-10T22:52:54.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.441+0000 7f35f6cd0740 1 Processor -- start 2024-09-10T22:52:54.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b26af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b26b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38850/0 (socket says 172.21.15.120:38850) 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b26af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b26af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35052/0 (socket says 172.21.15.120:35052) 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b26b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35ee59c640 1 -- 172.21.15.120:0/1895203233 learned_addr learned my addr 172.21.15.120:0/1895203233 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d26b028700 con 0x55d26b272140 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d26b162110 con 0x55d26b261da0 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d26b2679f0 con 0x55d26b261480 2024-09-10T22:52:54.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.442+0000 7f35f6cd0740 10 client.0 _ll_register_callbacks cb 0x55d26b17b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 -- 172.21.15.120:0/1895203233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 msgr2=0x55d26b26af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b26af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 -- 172.21.15.120:0/1895203233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 msgr2=0x55d26b26b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b26b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26b18fc40 con 0x55d26b261480 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35ee59c640 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35e00a8630 tx=0x7f35e000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc07881334d21e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0020020 con 0x55d26b261480 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f35e00ac4c0 con 0x55d26b261480 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e00e7780 con 0x55d26b261480 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.443+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f35e00f4070 con 0x55d26b261480 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.444+0000 7f35f6cd0740 10 client.24892 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:54.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.444+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d26b27c2f0 con 0x55d26b261480 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.444+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e00cf0d0 con 0x55d26b261480 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f35e00b5160 con 0x55d26b261480 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 10 client.24892 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 10 client.24892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 20 client.24892 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b27d640 0x55d26b27fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 4 client.24892 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35e00e7780 con 0x55d26b27d640 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.445+0000 7f35edd9b640 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b27d640 0x55d26b27fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.446+0000 7f35d5ffb640 10 client.24892 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:54.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.446+0000 7f35edd9b640 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b27d640 0x55d26b27fa30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391529fd549b99a1 server_cookie=4e2d0b4d25cf020b in_seq=0 out_seq=0 2024-09-10T22:52:54.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.446+0000 7f35d5ffb640 1 -- 172.21.15.120:0/1895203233 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f35e00e7780 con 0x55d26b27d640 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.446+0000 7f35d5ffb640 10 client.24892 handle_command_reply: tid=0 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.446+0000 7f35f6cd0740 1 client.24892 shutdown 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 msgr2=0x55d26b271b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35e00a8630 tx=0x7f35e000bf60 comp rx=0 tx=0).stop 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 shutdown_connections 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b27d640 0x55d26b27fa30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b272140 0x55d26b26b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b26af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/1895203233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b261480 0x55d26b271b20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 >> 172.21.15.120:0/1895203233 conn(0x55d26b25f2b0 msgr2=0x55d26b25e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35d5ffb640 0 client.24892 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:54.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 shutdown_connections 2024-09-10T22:52:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 1 -- 172.21.15.120:0/1895203233 wait complete. 2024-09-10T22:52:54.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.447+0000 7f35f6cd0740 20 client.24892 trim_cache size 0 max 16384 2024-09-10T22:52:54.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 Processor -- start 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 -- start start 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b279bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b27a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01aa200 con 0x55d26b261480 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01aca80 con 0x55d26b261da0 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35f6cd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01aa870 con 0x55d26b272140 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51446/0 (socket says 172.21.15.120:51446) 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35ee59c640 1 -- 172.21.15.120:0/2495276782 learned_addr learned my addr 172.21.15.120:0/2495276782 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.482+0000 7f35edd9b640 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b279bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 -- 172.21.15.120:0/2495276782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 msgr2=0x55d26b27a130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b27a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 -- 172.21.15.120:0/2495276782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 msgr2=0x55d26b25a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 -- 172.21.15.120:0/2495276782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26b269450 con 0x55d26b261da0 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35ee59c640 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35edd9b640 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b279bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009230 tx=0x7f35e4040c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc2378d72944636 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.483+0000 7f35d7fff640 1 -- 172.21.15.120:0/2495276782 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4035040 con 0x55d26b261da0 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.484+0000 7f35d7fff640 1 -- 172.21.15.120:0/2495276782 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e4040d80 con 0x55d26b261da0 2024-09-10T22:52:54.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.484+0000 7f35d7fff640 1 -- 172.21.15.120:0/2495276782 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40046c0 con 0x55d26b261da0 2024-09-10T22:52:54.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 -- 172.21.15.120:0/2495276782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 msgr2=0x55d26b279bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b279bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009230 tx=0x7f35e4040c10 comp rx=0 tx=0).stop 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 -- 172.21.15.120:0/2495276782 shutdown_connections 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b27a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b279bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/2495276782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b25a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 -- 172.21.15.120:0/2495276782 >> 172.21.15.120:0/2495276782 conn(0x55d26b25f2b0 msgr2=0x55d26b273f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 -- 172.21.15.120:0/2495276782 shutdown_connections 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.485+0000 7f35f6cd0740 1 -- 172.21.15.120:0/2495276782 wait complete. 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.486+0000 7f35f6cd0740 1 -- start start 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.486+0000 7f35f6cd0740 1 Processor -- start 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.486+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.486+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b2de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35f6cd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b2e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35ee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51456/0 (socket says 172.21.15.120:51456) 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35ee59c640 1 -- 172.21.15.120:0/3306391319 learned_addr learned my addr 172.21.15.120:0/3306391319 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01aaf40 con 0x55d26b261480 2024-09-10T22:52:54.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01ac3b0 con 0x55d26b261da0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01ab610 con 0x55d26b272140 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35f6cd0740 10 client.0 _ll_register_callbacks cb 0x7f35e00eba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b2de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35eed9d640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b2e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 -- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 msgr2=0x55d26b2e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b2e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 -- 172.21.15.120:0/3306391319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 msgr2=0x55d26b2de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35edd9b640 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26b2e9700 con 0x55d26b261da0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.487+0000 7f35ee59c640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35edd9b640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b2de680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55d26b26aaf0 tx=0x7f35e40042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbcae88a9da92700 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:52:54.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4035040 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f35e4005440 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e4040d80 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.488+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f35e4060050 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 10 client.24902 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d26b2ea8b0 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e4055030 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f35e4003710 con 0x55d26b261da0 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 10 client.24902 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 10 client.24902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 20 client.24902 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:54.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b2eaf30 0x55d26b2ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:54.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 4 client.24902 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:54.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.489+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f35e4040d80 con 0x55d26b2eaf30 2024-09-10T22:52:54.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35ee59c640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b2eaf30 0x55d26b2ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:54.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35ee59c640 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b2eaf30 0x55d26b2ed350 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7f0cdcb64f8f36 server_cookie=c87fde2963a8d2a6 in_seq=0 out_seq=0 2024-09-10T22:52:54.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35d5ffb640 10 client.24902 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35d5ffb640 1 -- 172.21.15.120:0/3306391319 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x7f35e4040d80 con 0x55d26b2eaf30 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35d5ffb640 10 client.24902 handle_command_reply: tid=0 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.490+0000 7f35f6cd0740 1 client.24902 shutdown 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 msgr2=0x55d26b2de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b2de680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55d26b26aaf0 tx=0x7f35e40042a0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 shutdown_connections 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d26b2eaf30 0x55d26b2ed350 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d26b272140 0x55d26b2e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d26b261da0 0x55d26b2de680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 --2- 172.21.15.120:0/3306391319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d26b261480 0x55d26b2de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:52:54.497 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 >> 172.21.15.120:0/3306391319 conn(0x55d26b25f2b0 msgr2=0x55d26b266330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35d5ffb640 0 client.24902 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 shutdown_connections 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 1 -- 172.21.15.120:0/3306391319 wait complete. 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.491+0000 7f35f6cd0740 20 client.24902 trim_cache size 0 max 16384 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 msgr2=0x7f35d80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 0x7f35d80b78a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009370 tx=0x7f35e80092a0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 msgr2=0x7f35f0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d950 tx=0x7f35e000de10 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 shutdown_connections 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f35d80b53e0 0x7f35d80b78a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35f0108020 0x7f35f019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35f01076a0 0x7f35f0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 --2- 172.21.15.120:0/874243768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35f0101c90 0x7f35f0199350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.492+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 >> 172.21.15.120:0/874243768 conn(0x7f35f00fd700 msgr2=0x7f35f00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:54.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.494+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 shutdown_connections 2024-09-10T22:52:54.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:54.494+0000 7f35f48fb640 1 -- 172.21.15.120:0/874243768 wait complete. 2024-09-10T22:52:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:54 smithi176 ceph-mon[34982]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T22:52:54.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:54 smithi153 ceph-mon[34951]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T22:52:55.290 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:52:55.290 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:55.574 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 -- 172.21.15.120:0/138720328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4641045c0 msgr2=0x7ff46410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 --2- 172.21.15.120:0/138720328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4641045c0 0x7ff46410cab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff454009900 tx=0x7ff45402a410 comp rx=0 tx=0).stop 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 -- 172.21.15.120:0/138720328 shutdown_connections 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 --2- 172.21.15.120:0/138720328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff46410d100 0x7ff46410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 --2- 172.21.15.120:0/138720328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4641045c0 0x7ff46410cab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 --2- 172.21.15.120:0/138720328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff464103ca0 0x7ff464104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 -- 172.21.15.120:0/138720328 >> 172.21.15.120:0/138720328 conn(0x7ff4640fd700 msgr2=0x7ff4640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 -- 172.21.15.120:0/138720328 shutdown_connections 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.095+0000 7ff46c724640 1 -- 172.21.15.120:0/138720328 wait complete. 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 Processor -- start 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 -- start start 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff464103ca0 0x7ff464199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4641045c0 0x7ff464199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff464112d70 con 0x7ff46410d100 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff464112bf0 con 0x7ff464103ca0 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46c724640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff464112ef0 con 0x7ff4641045c0 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46acc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46acc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51472/0 (socket says 172.21.15.120:51472) 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46acc1640 1 -- 172.21.15.120:0/3960396732 learned_addr learned my addr 172.21.15.120:0/3960396732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff46a4c0640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff464103ca0 0x7ff464199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.096+0000 7ff469cbf640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4641045c0 0x7ff464199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 -- 172.21.15.120:0/3960396732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4641045c0 msgr2=0x7ff464199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4641045c0 0x7ff464199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 -- 172.21.15.120:0/3960396732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff464103ca0 msgr2=0x7ff464199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff464103ca0 0x7ff464199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 -- 172.21.15.120:0/3960396732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419f3b0 con 0x7ff46410d100 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46acc1640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017070 con 0x7ff46410d100 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45c003e20 con 0x7ff46410d100 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c010460 con 0x7ff46410d100 2024-09-10T22:52:56.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.097+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419f6a0 con 0x7ff46410d100 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.098+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a5d60 con 0x7ff46410d100 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.099+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff464104080 con 0x7ff46410d100 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.101+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff45c003970 con 0x7ff46410d100 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.102+0000 7ff45b7fe640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 0x7ff4500b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.102+0000 7ff46a4c0640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 0x7ff4500b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.102+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7ff45c0e57a0 con 0x7ff46410d100 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.102+0000 7ff46a4c0640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 0x7ff4500b7820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff460009bd0 tx=0x7ff460005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.110+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff45c0ac040 con 0x7ff46410d100 2024-09-10T22:52:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:56 smithi120 ceph-mon[31247]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T22:52:56.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.471+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff46402e5c0 con 0x7ff46410d100 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":19,"btime":"2024-09-10T22:50:37:038717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:50:36.741193+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1046536521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1046536521},{"type":"v1","addr":"172.21.15.176:6837","nonce":1046536521}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:52:56.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.472+0000 7ff45b7fe640 1 -- 172.21.15.120:0/3960396732 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11868 (secure 0 0 0) 0x7ff45c0b0a70 con 0x7ff46410d100 2024-09-10T22:52:56.475 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 19 2024-09-10T22:52:56.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 msgr2=0x7ff4500b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 0x7ff4500b7820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff460009bd0 tx=0x7ff460005f90 comp rx=0 tx=0).stop 2024-09-10T22:52:56.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 msgr2=0x7ff46419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00b260 comp rx=0 tx=0).stop 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 shutdown_connections 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4500b5360 0x7ff4500b7820 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff46410d100 0x7ff46419ebe0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4641045c0 0x7ff464199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 --2- 172.21.15.120:0/3960396732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff464103ca0 0x7ff464199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 >> 172.21.15.120:0/3960396732 conn(0x7ff4640fd700 msgr2=0x7ff4640fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 shutdown_connections 2024-09-10T22:52:56.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:56.475+0000 7ff46c724640 1 -- 172.21.15.120:0/3960396732 wait complete. 2024-09-10T22:52:56.501 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:56 smithi153 ceph-mon[34951]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T22:52:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:56 smithi176 ceph-mon[34982]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T22:52:57.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:52:57.433 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:57.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3960396732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3960396732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3960396732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:57.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.981+0000 7fbfd0810640 1 -- 172.21.15.120:0/2803947724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 msgr2=0x7fbfcc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:57.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.981+0000 7fbfd0810640 1 --2- 172.21.15.120:0/2803947724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc10e820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ca0 tx=0x7fbfc402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 -- 172.21.15.120:0/2803947724 shutdown_connections 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 --2- 172.21.15.120:0/2803947724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc10e820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 --2- 172.21.15.120:0/2803947724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfcc1076a0 0x7fbfcc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 --2- 172.21.15.120:0/2803947724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 -- 172.21.15.120:0/2803947724 >> 172.21.15.120:0/2803947724 conn(0x7fbfcc0fd700 msgr2=0x7fbfcc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:57.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 -- 172.21.15.120:0/2803947724 shutdown_connections 2024-09-10T22:52:57.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.982+0000 7fbfd0810640 1 -- 172.21.15.120:0/2803947724 wait complete. 2024-09-10T22:52:57.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 Processor -- start 2024-09-10T22:52:57.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 -- start start 2024-09-10T22:52:57.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfcc1076a0 0x7fbfcc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111ca0 con 0x7fbfcc108020 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111b20 con 0x7fbfcc101c90 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.983+0000 7fbfd0810640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111e20 con 0x7fbfcc1076a0 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35110/0 (socket says 172.21.15.120:35110) 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 -- 172.21.15.120:0/1836091832 learned_addr learned my addr 172.21.15.120:0/1836091832 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 -- 172.21.15.120:0/1836091832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfcc1076a0 msgr2=0x7fbfcc198360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfcc1076a0 0x7fbfcc198360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 -- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 msgr2=0x7fbfcc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.984+0000 7fbfca59c640 1 -- 172.21.15.120:0/1836091832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc19d3b0 con 0x7fbfcc101c90 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfca59c640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d950 tx=0x7fbfbc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfbb7fe640 1 -- 172.21.15.120:0/1836091832 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc017070 con 0x7fbfcc101c90 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc19d6a0 con 0x7fbfcc101c90 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc19dc30 con 0x7fbfcc101c90 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfbb7fe640 1 -- 172.21.15.120:0/1836091832 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc009c30 con 0x7fbfcc101c90 2024-09-10T22:52:57.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.985+0000 7fbfbb7fe640 1 -- 172.21.15.120:0/1836091832 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00f620 con 0x7fbfcc101c90 2024-09-10T22:52:57.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.988+0000 7fbfbb7fe640 1 -- 172.21.15.120:0/1836091832 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbfbc010460 con 0x7fbfcc101c90 2024-09-10T22:52:57.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.988+0000 7fbfd2be5740 1 Processor -- start 2024-09-10T22:52:57.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.988+0000 7fbfbb7fe640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 0x7fbfb40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.989+0000 7fbfbb7fe640 1 -- 172.21.15.120:0/1836091832 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fbfbc0e58a0 con 0x7fbfcc101c90 2024-09-10T22:52:57.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.989+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 0x7fbfb40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.989+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 0x7fbfb40b78a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ca0 tx=0x7fbfc4005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:57.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 -- start start 2024-09-10T22:52:57.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c2468a10 0x5574c2474460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2476d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574c23deea0 con 0x5574c24680f0 2024-09-10T22:52:57.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574c23ce2d0 con 0x5574c24749a0 2024-09-10T22:52:57.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfd2be5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574c23d20e0 con 0x5574c2468a10 2024-09-10T22:52:57.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51502/0 (socket says 172.21.15.120:51502) 2024-09-10T22:52:57.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 -- 172.21.15.120:0/2851605838 learned_addr learned my addr 172.21.15.120:0/2851605838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:57.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2476d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c2468a10 0x5574c2474460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 -- 172.21.15.120:0/2851605838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c2468a10 msgr2=0x5574c2474460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c2468a10 0x5574c2474460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 -- 172.21.15.120:0/2851605838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 msgr2=0x5574c2476d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2476d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfca59c640 1 -- 172.21.15.120:0/2851605838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c212c650 con 0x5574c24680f0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.990+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2476d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.991+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0a8600 tx=0x7fbfbc0ab8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e39190df455d5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.991+0000 7fbfb8ff9640 1 -- 172.21.15.120:0/2851605838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc020070 con 0x5574c24680f0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.991+0000 7fbfb8ff9640 1 -- 172.21.15.120:0/2851605838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc091e50 con 0x5574c24680f0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.991+0000 7fbfb8ff9640 1 -- 172.21.15.120:0/2851605838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc010980 con 0x5574c24680f0 2024-09-10T22:52:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2851605838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 msgr2=0x5574c24684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0a8600 tx=0x7fbfbc0ab8d0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2851605838 shutdown_connections 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2476d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c2468a10 0x5574c2474460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2851605838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c24680f0 0x5574c24684d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2851605838 >> 172.21.15.120:0/2851605838 conn(0x5574c2465f20 msgr2=0x5574c24670c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2851605838 shutdown_connections 2024-09-10T22:52:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2851605838 wait complete. 2024-09-10T22:52:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.992+0000 7fbfd2be5740 1 -- start start 2024-09-10T22:52:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 Processor -- start 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24680f0 0x5574c2474250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2471c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51516/0 (socket says 172.21.15.120:51516) 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 -- 172.21.15.120:0/3615437293 learned_addr learned my addr 172.21.15.120:0/3615437293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574c222f700 con 0x5574c2468a10 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574c2369160 con 0x5574c24749a0 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574c2477560 con 0x5574c24680f0 2024-09-10T22:52:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24680f0 0x5574c2474250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfd2be5740 10 client.0 _ll_register_callbacks cb 0x5574c23829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2471c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 -- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24680f0 msgr2=0x5574c2474250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24680f0 0x5574c2474250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 -- 172.21.15.120:0/3615437293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 msgr2=0x5574c2471c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2471c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.993+0000 7fbfc9d9b640 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c2396c90 con 0x5574c2468a10 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000bd70 tx=0x7fbfc000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea430dd0078f09e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0015070 con 0x5574c2468a10 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fbfc0011650 con 0x5574c2468a10 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc091e50 con 0x5574c2468a10 2024-09-10T22:52:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0011850 con 0x5574c2468a10 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfd2be5740 10 client.15426 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5574c2482ef0 con 0x5574c2468a10 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.994+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc001a460 con 0x5574c2468a10 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbfc0024420 con 0x5574c2468a10 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 10 client.15426 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 10 client.15426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 20 client.15426 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c2486800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 4 client.15426 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbfbc091e50 con 0x5574c24843a0 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c2486800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c2486800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df02733eb7b20274 server_cookie=847a5f87ea444719 in_seq=0 out_seq=0 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.995+0000 7fbfb1ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/3615437293 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbfbc091e50 con 0x5574c24843a0 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfb1ffb640 10 client.15426 handle_command_reply: tid=0 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 client.15426 shutdown 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 msgr2=0x5574c24716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000bd70 tx=0x7fbfc000f730 comp rx=0 tx=0).stop 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 shutdown_connections 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c2486800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfb1ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24749a0 0x5574c2471c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24716a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3615437293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24680f0 0x5574c2474250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 >> 172.21.15.120:0/3615437293 conn(0x5574c2465f20 msgr2=0x5574c246ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 shutdown_connections 2024-09-10T22:52:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.996+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3615437293 wait complete. 2024-09-10T22:52:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:57.997+0000 7fbfd2be5740 20 client.15426 trim_cache size 0 max 16384 2024-09-10T22:52:58.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.022+0000 7fbfd2be5740 1 Processor -- start 2024-09-10T22:52:58.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.022+0000 7fbfd2be5740 1 -- start start 2024-09-10T22:52:58.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.022+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2465760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.022+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51530/0 (socket says 172.21.15.120:51530) 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c246e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfcad9d640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c246e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2465760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfc9d9b640 1 -- 172.21.15.120:0/3453929325 learned_addr learned my addr 172.21.15.120:0/3453929325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc072dc0 con 0x5574c2468a10 2024-09-10T22:52:58.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc19e1e0 con 0x5574c24680f0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc073430 con 0x5574c24749a0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfca59c640 1 -- 172.21.15.120:0/3453929325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 msgr2=0x5574c246e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c246e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.023+0000 7fbfca59c640 1 -- 172.21.15.120:0/3453929325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 msgr2=0x5574c24880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfca59c640 1 -- 172.21.15.120:0/3453929325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c2465ca0 con 0x5574c24680f0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfca59c640 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2465760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b5780 tx=0x7fbfbc010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b472aebcdc2fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfb3fff640 1 -- 172.21.15.120:0/3453929325 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc020040 con 0x5574c24680f0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfb3fff640 1 -- 172.21.15.120:0/3453929325 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc091e50 con 0x5574c24680f0 2024-09-10T22:52:58.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.024+0000 7fbfb3fff640 1 -- 172.21.15.120:0/3453929325 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0109e0 con 0x5574c24680f0 2024-09-10T22:52:58.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 msgr2=0x5574c2465760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2465760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b5780 tx=0x7fbfbc010cb0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 shutdown_connections 2024-09-10T22:52:58.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c246e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c24880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/3453929325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2465760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 >> 172.21.15.120:0/3453929325 conn(0x5574c2465f20 msgr2=0x5574c24734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 shutdown_connections 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- 172.21.15.120:0/3453929325 wait complete. 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.025+0000 7fbfd2be5740 1 -- start start 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 Processor -- start 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.034 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c2512af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfca59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35158/0 (socket says 172.21.15.120:35158) 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfc9d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51542/0 (socket says 172.21.15.120:51542) 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfca59c640 1 -- 172.21.15.120:0/2456377893 learned_addr learned my addr 172.21.15.120:0/2456377893 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc073b00 con 0x5574c2468a10 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc074f70 con 0x5574c24680f0 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.026+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc0741d0 con 0x5574c24749a0 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 -- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 msgr2=0x5574c2512af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c2512af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 -- 172.21.15.120:0/2456377893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 msgr2=0x5574c2478380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:58.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c2513030 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfd2be5740 10 client.0 _ll_register_callbacks cb 0x7fbfc000b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfca59c640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0abfc0 tx=0x7fbfbc00b530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12aaac60bbb7227 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfb1ffb640 10 client.24923 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc091e50 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/2456377893 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc020070 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/2456377893 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fbfbc0c1bb0 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/2456377893 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc0b0b10 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.027+0000 7fbfd2be5740 10 client.24923 fetch_fsmap learned FSMap version 19 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5574c2514090 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/2456377893 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbfbc0ab290 con 0x5574c24680f0 2024-09-10T22:52:58.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 10 client.24923 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 10 client.24923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 20 client.24923 populate_metadata read hostname 'smithi120' 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c25163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 4 client.24923 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.028+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574c23e0df0 con 0x5574c24843a0 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c25163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfb1ffb640 10 client.24923 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfc9d9b640 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c25163a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e6043494a4bccc server_cookie=54634252afa30227 in_seq=0 out_seq=0 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfb1ffb640 1 -- 172.21.15.120:0/2456377893 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574c23e0df0 con 0x5574c24843a0 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfb1ffb640 10 client.24923 handle_command_reply: tid=0 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfd2be5740 1 client.24923 shutdown 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 msgr2=0x5574c2477e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.029+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0abfc0 tx=0x7fbfbc00b530 comp rx=0 tx=0).stop 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 shutdown_connections 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574c24843a0 0x5574c25163a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfb1ffb640 0 client.24923 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574c24749a0 0x5574c2512af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574c2468a10 0x5574c2478380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 --2- 172.21.15.120:0/2456377893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574c24680f0 0x5574c2477e40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 >> 172.21.15.120:0/2456377893 conn(0x5574c2465f20 msgr2=0x5574c246fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 shutdown_connections 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 1 -- 172.21.15.120:0/2456377893 wait complete. 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.030+0000 7fbfd2be5740 20 client.24923 trim_cache size 0 max 16384 2024-09-10T22:52:58.038 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 msgr2=0x7fbfb40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 0x7fbfb40b78a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ca0 tx=0x7fbfc4005f80 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 msgr2=0x7fbfcc197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d950 tx=0x7fbfbc00de10 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 shutdown_connections 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbfb40b53e0 0x7fbfb40b78a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbfcc108020 0x7fbfcc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfcc1076a0 0x7fbfcc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 --2- 172.21.15.120:0/1836091832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfcc101c90 0x7fbfcc197e20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 >> 172.21.15.120:0/1836091832 conn(0x7fbfcc0fd700 msgr2=0x7fbfcc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 shutdown_connections 2024-09-10T22:52:58.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:52:58.031+0000 7fbfd0810640 1 -- 172.21.15.120:0/1836091832 wait complete. 2024-09-10T22:52:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:52:58 smithi176 ceph-mon[34982]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:52:58.612 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:52:58 smithi120 ceph-mon[31247]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:52:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:52:58 smithi153 ceph-mon[34951]: pgmap v359: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:52:59.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:52:59.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:53:00.153 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:00.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:00 smithi120 ceph-mon[31247]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T22:53:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:00 smithi176 ceph-mon[34982]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T22:53:00.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 -- 172.21.15.120:0/2285196474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 msgr2=0x7fc08410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 --2- 172.21.15.120:0/2285196474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08410ef20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc078009900 tx=0x7fc07802a480 comp rx=0 tx=0).stop 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 -- 172.21.15.120:0/2285196474 shutdown_connections 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 --2- 172.21.15.120:0/2285196474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc084111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 --2- 172.21.15.120:0/2285196474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08410ef20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 --2- 172.21.15.120:0/2285196474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 0x7fc084101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 -- 172.21.15.120:0/2285196474 >> 172.21.15.120:0/2285196474 conn(0x7fc0840fd700 msgr2=0x7fc0840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 -- 172.21.15.120:0/2285196474 shutdown_connections 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 -- 172.21.15.120:0/2285196474 wait complete. 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.621+0000 7fc089135640 1 Processor -- start 2024-09-10T22:53:00.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 -- start start 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 0x7fc08419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc0841a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc084114f50 con 0x7fc0841021d0 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc084114dd0 con 0x7fc08410f460 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc089135640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841150d0 con 0x7fc0841018b0 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc08259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc08359e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc0841a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc08259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51556/0 (socket says 172.21.15.120:51556) 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc08259c640 1 -- 172.21.15.120:0/1463819439 learned_addr learned my addr 172.21.15.120:0/1463819439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.622+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 0x7fc08419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 -- 172.21.15.120:0/1463819439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 msgr2=0x7fc08419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 0x7fc08419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 -- 172.21.15.120:0/1463819439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 msgr2=0x7fc0841a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc0841a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 -- 172.21.15.120:0/1463819439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0841a1590 con 0x7fc0841021d0 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08359e640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc0841a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc08259c640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc078005940 tx=0x7fc0780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc06bfff640 1 -- 172.21.15.120:0/1463819439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07803f030 con 0x7fc0841021d0 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0841a1820 con 0x7fc0841021d0 2024-09-10T22:53:00.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc06bfff640 1 -- 172.21.15.120:0/1463819439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc078003cb0 con 0x7fc0841021d0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc06bfff640 1 -- 172.21.15.120:0/1463819439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078035070 con 0x7fc0841021d0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.623+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0841a7f40 con 0x7fc0841021d0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.625+0000 7fc06bfff640 1 -- 172.21.15.120:0/1463819439 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc07803bbf0 con 0x7fc0841021d0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.626+0000 7fc06bfff640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 0x7fc06c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.626+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 0x7fc06c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.626+0000 7fc08b50a740 1 Processor -- start 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.626+0000 7fc06bfff640 1 -- 172.21.15.120:0/1463819439 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fc0781066f0 con 0x7fc0841021d0 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.626+0000 7fc08b50a740 1 -- start start 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782e0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 0x5625782e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56257824ae60 con 0x5625782d3ee0 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56257823a290 con 0x5625782e0790 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08b50a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56257823e0a0 con 0x5625782d4800 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 0x7fc06c0b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc074005fd0 tx=0x7fc07400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51564/0 (socket says 172.21.15.120:51564) 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 -- 172.21.15.120:0/204360135 learned_addr learned my addr 172.21.15.120:0/204360135 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08259c640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08359e640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 0x5625782e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 -- 172.21.15.120:0/204360135 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 msgr2=0x5625782e0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782e0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 -- 172.21.15.120:0/204360135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 msgr2=0x5625782e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 0x5625782e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 -- 172.21.15.120:0/204360135 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562577f98930 con 0x5625782d3ee0 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc08359e640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 0x5625782e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.627+0000 7fc082d9d640 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc074003c90 tx=0x7fc074013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8749d41ee2e58447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc0697fa640 1 -- 172.21.15.120:0/204360135 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07401e420 con 0x5625782d3ee0 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc0697fa640 1 -- 172.21.15.120:0/204360135 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07401ea70 con 0x5625782d3ee0 2024-09-10T22:53:00.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc0697fa640 1 -- 172.21.15.120:0/204360135 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07401d9e0 con 0x5625782d3ee0 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 -- 172.21.15.120:0/204360135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 msgr2=0x5625782d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc074003c90 tx=0x7fc074013000 comp rx=0 tx=0).stop 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 -- 172.21.15.120:0/204360135 shutdown_connections 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782e0790 0x5625782e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782e0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 --2- 172.21.15.120:0/204360135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d3ee0 0x5625782d42c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.628+0000 7fc08b50a740 1 -- 172.21.15.120:0/204360135 >> 172.21.15.120:0/204360135 conn(0x5625782d1d10 msgr2=0x5625782d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- 172.21.15.120:0/204360135 shutdown_connections 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- 172.21.15.120:0/204360135 wait complete. 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- start start 2024-09-10T22:53:00.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 Processor -- start 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782dd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782ddc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56257809b890 con 0x5625782e0790 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5625781d4d40 con 0x5625782d3ee0 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625782e3440 con 0x5625782d4800 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08b50a740 10 client.0 _ll_register_callbacks cb 0x562578238c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51576/0 (socket says 172.21.15.120:51576) 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08359e640 1 -- 172.21.15.120:0/2350296872 learned_addr learned my addr 172.21.15.120:0/2350296872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc082d9d640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.629+0000 7fc08259c640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782ddc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 -- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 msgr2=0x5625782ddc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782ddc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 -- 172.21.15.120:0/2350296872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 msgr2=0x5625782dd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782dd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 -- 172.21.15.120:0/2350296872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625782ec820 con 0x5625782e0790 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08259c640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782ddc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08359e640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc07c011a20 tx=0x7fc07c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf1bae81ab0d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc066ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c01b470 con 0x5625782e0790 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fc07c01b610 con 0x5625782e0790 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07401ea70 con 0x5625782e0790 2024-09-10T22:53:00.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc07c01b810 con 0x5625782e0790 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08b50a740 10 client.15444 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5625782ef1a0 con 0x5625782e0790 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.630+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07401ea70 con 0x5625782e0790 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc07c00f040 con 0x5625782e0790 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 10 client.15444 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 20 client.15444 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x5625782f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 4 client.15444 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257824cdb0 con 0x5625782f0660 2024-09-10T22:53:00.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc082d9d640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x5625782f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc066ff5640 10 client.15444 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.631+0000 7fc082d9d640 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x5625782f2a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc1810d74f9e79c server_cookie=420cf46038cd0e19 in_seq=0 out_seq=0 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc066ff5640 1 -- 172.21.15.120:0/2350296872 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56257824cdb0 con 0x5625782f0660 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc066ff5640 10 client.15444 handle_command_reply: tid=0 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 client.15444 shutdown 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 msgr2=0x5625782de1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc07c011a20 tx=0x7fc07c011ee0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 shutdown_connections 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x5625782f2a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc066ff5640 0 client.15444 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782de1a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782ddc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 --2- 172.21.15.120:0/2350296872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 >> 172.21.15.120:0/2350296872 conn(0x5625782d1d10 msgr2=0x5625782d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 shutdown_connections 2024-09-10T22:53:00.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.632+0000 7fc08b50a740 1 -- 172.21.15.120:0/2350296872 wait complete. 2024-09-10T22:53:00.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.633+0000 7fc08b50a740 20 client.15444 trim_cache size 0 max 16384 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 Processor -- start 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 -- start start 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782da930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841ae510 con 0x5625782e0790 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841b0d90 con 0x5625782d3ee0 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08b50a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841aeb80 con 0x5625782d4800 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51590/0 (socket says 172.21.15.120:51590) 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc08359e640 1 -- 172.21.15.120:0/853193694 learned_addr learned my addr 172.21.15.120:0/853193694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.664+0000 7fc082d9d640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08259c640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 -- 172.21.15.120:0/853193694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 msgr2=0x5625782dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782dae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 -- 172.21.15.120:0/853193694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 msgr2=0x5625782da930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782da930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 -- 172.21.15.120:0/853193694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625782d4630 con 0x5625782e0790 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08259c640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc08359e640 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc07c002230 tx=0x7fc07c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b694dd8df46a7d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc068ff9640 1 -- 172.21.15.120:0/853193694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c005bd0 con 0x5625782e0790 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc068ff9640 1 -- 172.21.15.120:0/853193694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c016070 con 0x5625782e0790 2024-09-10T22:53:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.665+0000 7fc068ff9640 1 -- 172.21.15.120:0/853193694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c012450 con 0x5625782e0790 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- 172.21.15.120:0/853193694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 msgr2=0x5625782f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc07c002230 tx=0x7fc07c011690 comp rx=0 tx=0).stop 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- 172.21.15.120:0/853193694 shutdown_connections 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782e0790 0x5625782f8310 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782d4800 0x5625782dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 --2- 172.21.15.120:0/853193694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- 172.21.15.120:0/853193694 >> 172.21.15.120:0/853193694 conn(0x5625782d1d10 msgr2=0x5625782df570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- 172.21.15.120:0/853193694 shutdown_connections 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- 172.21.15.120:0/853193694 wait complete. 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 -- start start 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.666+0000 7fc08b50a740 1 Processor -- start 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782e0790 0x5625782e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841af250 con 0x5625782d4800 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841b06c0 con 0x5625782d3ee0 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841af920 con 0x5625782e0790 2024-09-10T22:53:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08b50a740 10 client.0 _ll_register_callbacks cb 0x7fc07c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc082d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc082d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35206/0 (socket says 172.21.15.120:35206) 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51592/0 (socket says 172.21.15.120:51592) 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc082d9d640 1 -- 172.21.15.120:0/1059970798 learned_addr learned my addr 172.21.15.120:0/1059970798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08359e640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782e0790 0x5625782e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 -- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782e0790 msgr2=0x5625782e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782e0790 0x5625782e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 -- 172.21.15.120:0/1059970798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 msgr2=0x5625782e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc08259c640 1 -- 172.21.15.120:0/1059970798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257835d510 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.667+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08259c640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0781041c0 tx=0x7fc07802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46173f1ca805223b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078049040 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07c016070 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7fc0780cd190 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc078005d50 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 10 client.15450 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56257835e7f0 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c016070 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc0780d4ce0 con 0x5625782d4800 2024-09-10T22:53:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 10 client.15450 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 20 client.15450 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x562578360fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 4 client.15450 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.668+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562578248860 con 0x5625782f0660 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.669+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x562578360fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.669+0000 7fc082d9d640 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x562578360fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb80fadd33aaaa0d server_cookie=55237e0fc5c5fe8e in_seq=0 out_seq=0 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.669+0000 7fc066ff5640 10 client.15450 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.670+0000 7fc066ff5640 1 -- 172.21.15.120:0/1059970798 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562578248860 con 0x5625782f0660 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.670+0000 7fc066ff5640 10 client.15450 handle_command_reply: tid=0 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.670+0000 7fc08b50a740 1 client.15450 shutdown 2024-09-10T22:53:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 msgr2=0x5625782e36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0781041c0 tx=0x7fc07802a480 comp rx=0 tx=0).stop 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 shutdown_connections 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625782f0660 0x562578360fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625782e0790 0x5625782e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc066ff5640 0 client.15450 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625782d4800 0x5625782e36e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 --2- 172.21.15.120:0/1059970798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625782d3ee0 0x5625782e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 >> 172.21.15.120:0/1059970798 conn(0x5625782d1d10 msgr2=0x5625782eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 shutdown_connections 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 1 -- 172.21.15.120:0/1059970798 wait complete. 2024-09-10T22:53:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.671+0000 7fc08b50a740 20 client.15450 trim_cache size 0 max 16384 2024-09-10T22:53:00.678 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:53:00.678 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:53:00.678 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "9ef6bbf7-89a6-4969-ac4a-672295c39b5e", 2024-09-10T22:53:00.678 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:53:00.678 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 msgr2=0x7fc06c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 0x7fc06c0b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc074005fd0 tx=0x7fc07400b390 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 msgr2=0x7fc08419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc078005940 tx=0x7fc0780042f0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 shutdown_connections 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc06c0b52e0 0x7fc06c0b77a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc08410f460 0x7fc0841a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0841021d0 0x7fc08419ba20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 --2- 172.21.15.120:0/1463819439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0841018b0 0x7fc08419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 >> 172.21.15.120:0/1463819439 conn(0x7fc0840fd700 msgr2=0x7fc084110970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 shutdown_connections 2024-09-10T22:53:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:00.672+0000 7fc089135640 1 -- 172.21.15.120:0/1463819439 wait complete. 2024-09-10T22:53:00.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:00 smithi153 ceph-mon[34951]: pgmap v360: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T22:53:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:01 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:53:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:01 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:01 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:53:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:01 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:01 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:53:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:01 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:01 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:53:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:01 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:01 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:53:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:01 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:01 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:53:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:01 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:01.817 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:53:02.083 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:02.458 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:02 smithi120 ceph-mon[31247]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:53:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:02 smithi176 ceph-mon[34982]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:53:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 -- 172.21.15.120:0/298983785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 msgr2=0x7f1f28102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 --2- 172.21.15.120:0/298983785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28102070 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009900 tx=0x7f1f2402a410 comp rx=0 tx=0).stop 2024-09-10T22:53:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 -- 172.21.15.120:0/298983785 shutdown_connections 2024-09-10T22:53:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 --2- 172.21.15.120:0/298983785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 0x7f1f2810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 --2- 172.21.15.120:0/298983785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 0x7f1f28107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 --2- 172.21.15.120:0/298983785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28102070 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 -- 172.21.15.120:0/298983785 >> 172.21.15.120:0/298983785 conn(0x7f1f280fd700 msgr2=0x7f1f280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.678+0000 7f1f3050c640 1 -- 172.21.15.120:0/298983785 shutdown_connections 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.679+0000 7f1f3050c640 1 -- 172.21.15.120:0/298983785 wait complete. 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.679+0000 7f1f3050c640 1 Processor -- start 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.679+0000 7f1f3050c640 1 -- start start 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 0x7f1f281997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 0x7f1f2819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28111ce0 con 0x7f1f28101c90 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28111b60 con 0x7f1f281076a0 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28111e60 con 0x7f1f28108020 2024-09-10T22:53:02.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51616/0 (socket says 172.21.15.120:51616) 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 -- 172.21.15.120:0/3618090971 learned_addr learned my addr 172.21.15.120:0/3618090971 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 0x7f1f2819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 0x7f1f281997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 -- 172.21.15.120:0/3618090971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 msgr2=0x7f1f2819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 0x7f1f2819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 -- 172.21.15.120:0/3618090971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 msgr2=0x7f1f281997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 0x7f1f281997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 -- 172.21.15.120:0/3618090971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2819f310 con 0x7f1f28101c90 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1f24005940 tx=0x7f1f240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f1b7fe640 1 -- 172.21.15.120:0/3618090971 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2403f030 con 0x7f1f28101c90 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2819f5a0 con 0x7f1f28101c90 2024-09-10T22:53:02.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f1b7fe640 1 -- 172.21.15.120:0/3618090971 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f24003cb0 con 0x7f1f28101c90 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.680+0000 7f1f1b7fe640 1 -- 172.21.15.120:0/3618090971 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24035070 con 0x7f1f28101c90 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.681+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f281a5cc0 con 0x7f1f28101c90 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 Processor -- start 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 -- start start 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f1b7fe640 1 -- 172.21.15.120:0/3618090971 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1f2403bbf0 con 0x7f1f28101c90 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc45860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567cdbbbe50 con 0x5567cdc56140 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567cdbab280 con 0x5567cdc45da0 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.683+0000 7f1f328e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5567cdbaf090 con 0x5567cdc45480 2024-09-10T22:53:02.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51622/0 (socket says 172.21.15.120:51622) 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 -- 172.21.15.120:0/51031276 learned_addr learned my addr 172.21.15.120:0/51031276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc45860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 -- 172.21.15.120:0/51031276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 msgr2=0x5567cdc45860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc45860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 -- 172.21.15.120:0/51031276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 msgr2=0x5567cdc55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc55c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 -- 172.21.15.120:0/51031276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cd909650 con 0x5567cdc56140 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00bcb0 tx=0x7f1f1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f5215390ec1829 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f1b7fe640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 0x7f1f140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 0x7f1f140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f18ff9640 1 -- 172.21.15.120:0/51031276 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c015070 con 0x5567cdc56140 2024-09-10T22:53:02.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f18ff9640 1 -- 172.21.15.120:0/51031276 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c01a910 con 0x5567cdc56140 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f1b7fe640 1 -- 172.21.15.120:0/3618090971 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f1f24106810 con 0x7f1f28101c90 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.684+0000 7f1f18ff9640 1 -- 172.21.15.120:0/51031276 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c011980 con 0x5567cdc56140 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 0x7f1f140b77a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f20007ed0 tx=0x7f1f20007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 -- 172.21.15.120:0/51031276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 msgr2=0x5567cdc58530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00bcb0 tx=0x7f1f1c00f730 comp rx=0 tx=0).stop 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 -- 172.21.15.120:0/51031276 shutdown_connections 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc58530 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 --2- 172.21.15.120:0/51031276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc45860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 -- 172.21.15.120:0/51031276 >> 172.21.15.120:0/51031276 conn(0x5567cdc432b0 msgr2=0x5567cdc44450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 -- 172.21.15.120:0/51031276 shutdown_connections 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.685+0000 7f1f328e1740 1 -- 172.21.15.120:0/51031276 wait complete. 2024-09-10T22:53:02.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 -- start start 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 Processor -- start 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc55a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc52f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567cda0c700 con 0x5567cdc56140 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567cdb46110 con 0x5567cdc45da0 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5567cdc58df0 con 0x5567cdc45480 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2eaa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f328e1740 10 client.0 _ll_register_callbacks cb 0x5567cdb5f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2eaa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51636/0 (socket says 172.21.15.120:51636) 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2eaa9640 1 -- 172.21.15.120:0/2169310348 learned_addr learned my addr 172.21.15.120:0/2169310348 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc52f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc55a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.686+0000 7f1f2eaa9640 1 -- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 msgr2=0x5567cdc55a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc55a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f2eaa9640 1 -- 172.21.15.120:0/2169310348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 msgr2=0x5567cdc52f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc52f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f2eaa9640 1 -- 172.21.15.120:0/2169310348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cdb73c40 con 0x5567cdc56140 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000ed30 tx=0x7f1f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4635aee5c45dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2001cd30 con 0x5567cdc56140 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f1c01a910 con 0x5567cdc56140 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f1f2001d410 con 0x5567cdc56140 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f2001d610 con 0x5567cdc56140 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f328e1740 10 client.15462 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:02.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5567cdc64830 con 0x5567cdc56140 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.687+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c01a910 con 0x5567cdc56140 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f200287d0 con 0x5567cdc56140 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 10 client.15462 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 20 client.15462 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdc65c90 0x5567cdc680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 4 client.15462 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567cdbbdda0 con 0x5567cdc65c90 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdc65c90 0x5567cdc680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f11ffb640 10 client.15462 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.688+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdc65c90 0x5567cdc680a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3d36653c897186 server_cookie=3d5ae639c016fd00 in_seq=0 out_seq=0 2024-09-10T22:53:02.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f11ffb640 1 -- 172.21.15.120:0/2169310348 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567cdbbdda0 con 0x5567cdc65c90 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f11ffb640 10 client.15462 handle_command_reply: tid=0 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 client.15462 shutdown 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 msgr2=0x5567cdc535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000ed30 tx=0x7f1f20004a40 comp rx=0 tx=0).stop 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 shutdown_connections 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdc65c90 0x5567cdc680a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc535a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45da0 0x5567cdc52f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f11ffb640 0 client.15462 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 --2- 172.21.15.120:0/2169310348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc55a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 >> 172.21.15.120:0/2169310348 conn(0x5567cdc432b0 msgr2=0x5567cdc423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 shutdown_connections 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 1 -- 172.21.15.120:0/2169310348 wait complete. 2024-09-10T22:53:02.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.689+0000 7f1f328e1740 20 client.15462 trim_cache size 0 max 16384 2024-09-10T22:53:02.709 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:02 smithi153 ceph-mon[34951]: pgmap v361: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T22:53:02.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 Processor -- start 2024-09-10T22:53:02.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 -- start start 2024-09-10T22:53:02.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 0x5567cdc3e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281aa090 con 0x5567cdc56140 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281ac8f0 con 0x5567cdc45480 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f328e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281aa6e0 con 0x5567cdc45da0 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f2e2a8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f2e2a8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35268/0 (socket says 172.21.15.120:35268) 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f2e2a8640 1 -- 172.21.15.120:0/4097243588 learned_addr learned my addr 172.21.15.120:0/4097243588 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.707+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 0x5567cdc3e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 -- 172.21.15.120:0/4097243588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 msgr2=0x5567cdc62050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 -- 172.21.15.120:0/4097243588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 msgr2=0x5567cdc43f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc43f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 -- 172.21.15.120:0/4097243588 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cdc51420 con 0x5567cdc45da0 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 0x5567cdc3e2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00bc80 tx=0x7f1f1c011580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938a977211917fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f13fff640 1 -- 172.21.15.120:0/4097243588 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c015050 con 0x5567cdc45da0 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f13fff640 1 -- 172.21.15.120:0/4097243588 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c029070 con 0x5567cdc45da0 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.708+0000 7f1f13fff640 1 -- 172.21.15.120:0/4097243588 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c011b60 con 0x5567cdc45da0 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- 172.21.15.120:0/4097243588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 msgr2=0x5567cdc3e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 0x5567cdc3e2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00bc80 tx=0x7f1f1c011580 comp rx=0 tx=0).stop 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- 172.21.15.120:0/4097243588 shutdown_connections 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc56140 0x5567cdc43f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45da0 0x5567cdc3e2d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 --2- 172.21.15.120:0/4097243588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc45480 0x5567cdc62050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- 172.21.15.120:0/4097243588 >> 172.21.15.120:0/4097243588 conn(0x5567cdc432b0 msgr2=0x5567cdc517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- 172.21.15.120:0/4097243588 shutdown_connections 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- 172.21.15.120:0/4097243588 wait complete. 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.709+0000 7f1f328e1740 1 -- start start 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 Processor -- start 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 0x5567cdc5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281aadb0 con 0x5567cdc45da0 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281ac220 con 0x5567cdc56140 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281ab480 con 0x5567cdc45480 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f328e1740 10 client.0 _ll_register_callbacks cb 0x7f1f20007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51648/0 (socket says 172.21.15.120:51648) 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 -- 172.21.15.120:0/810527761 learned_addr learned my addr 172.21.15.120:0/810527761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 0x5567cdc5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 -- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 msgr2=0x5567cdc5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 -- 172.21.15.120:0/810527761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 msgr2=0x5567cdc5b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 0x5567cdc5b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2daa7640 1 -- 172.21.15.120:0/810527761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cdc5bb50 con 0x5567cdc45da0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.710+0000 7f1f2eaa9640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 0x5567cdc5b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f2daa7640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c026040 tx=0x7f1f1c00b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104f9a9af3205430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:02.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c015050 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f1f1c004450 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f1c029070 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f1c03e070 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f328e1740 10 client.15468 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5567cdcd3990 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.711+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c029070 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f1c0037b0 con 0x5567cdc45da0 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 10 client.15468 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 20 client.15468 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdcd3e60 0x5567cdcd6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 4 client.15468 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:53:02.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567cdbb9850 con 0x5567cdcd3e60 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdcd3e60 0x5567cdcd6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f2e2a8640 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdcd3e60 0x5567cdcd6280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cb1c5a543b4539 server_cookie=3746037d4e06f6c6 in_seq=0 out_seq=0 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.712+0000 7f1f11ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f11ffb640 1 -- 172.21.15.120:0/810527761 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x5567cdbb9850 con 0x5567cdcd3e60 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f11ffb640 10 client.15468 handle_command_reply: tid=0 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 client.15468 shutdown 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 msgr2=0x5567cdc5b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c026040 tx=0x7f1f1c00b2c0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 shutdown_connections 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567cdcd3e60 0x5567cdcd6280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567cdc56140 0x5567cdc5b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567cdc45da0 0x5567cdc5b080 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 --2- 172.21.15.120:0/810527761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567cdc45480 0x5567cdc5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f11ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 >> 172.21.15.120:0/810527761 conn(0x5567cdc432b0 msgr2=0x5567cdc43f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 shutdown_connections 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 1 -- 172.21.15.120:0/810527761 wait complete. 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.713+0000 7f1f328e1740 20 client.15468 trim_cache size 0 max 16384 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:53:02.721 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "9ef6bbf7-89a6-4969-ac4a-672295c39b5e", 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:02.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:02.723 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:02.724 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:02.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:02.724 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:02.724 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:53:02.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 msgr2=0x7f1f140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 0x7f1f140b77a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f20007ed0 tx=0x7f1f20007680 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 msgr2=0x7f1f28199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1f24005940 tx=0x7f1f240042f0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 shutdown_connections 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1f140b52e0 0x7f1f140b77a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f28108020 0x7f1f2819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f281076a0 0x7f1f281997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 --2- 172.21.15.120:0/3618090971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f28101c90 0x7f1f28199260 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 >> 172.21.15.120:0/3618090971 conn(0x7f1f280fd700 msgr2=0x7f1f280ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 shutdown_connections 2024-09-10T22:53:02.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:02.714+0000 7f1f3050c640 1 -- 172.21.15.120:0/3618090971 wait complete. 2024-09-10T22:53:03.645 INFO:tasks.cephfs.filesystem:scrub status for tag:9ef6bbf7-89a6-4969-ac4a-672295c39b5e - {'path': '/', 'tag': '9ef6bbf7-89a6-4969-ac4a-672295c39b5e', 'options': 'recursive,force'} 2024-09-10T22:53:04.495 INFO:tasks.workunit.client.0.smithi120.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-10T22:53:04.495 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.495 INFO:tasks.workunit.client.0.smithi120.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T22:53:04.495 INFO:tasks.workunit.client.0.smithi120.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: git switch -c 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:Or undo this operation with: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: git switch - 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr: 2024-09-10T22:53:04.496 INFO:tasks.workunit.client.0.smithi120.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:53:04.500 DEBUG:teuthology.orchestra.run.smithi120:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T22:53:04.526 INFO:tasks.workunit.client.0.smithi120.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T22:53:04.528 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T22:53:04.528 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T22:53:05.224 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T22:53:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 -- 172.21.15.120:0/520771484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d1006b470 msgr2=0x7f6d10105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 --2- 172.21.15.120:0/520771484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d1006b470 0x7f6d10105530 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009ca0 tx=0x7f6d0c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 -- 172.21.15.120:0/520771484 shutdown_connections 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 --2- 172.21.15.120:0/520771484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10108670 0x7f6d1010ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 --2- 172.21.15.120:0/520771484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 0x7f6d10107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 --2- 172.21.15.120:0/520771484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d1006b470 0x7f6d10105530 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 -- 172.21.15.120:0/520771484 >> 172.21.15.120:0/520771484 conn(0x7f6d100fd140 msgr2=0x7f6d100ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 -- 172.21.15.120:0/520771484 shutdown_connections 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 -- 172.21.15.120:0/520771484 wait complete. 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.255+0000 7f6d182a9640 1 Processor -- start 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 -- start start 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d1006b470 0x7f6d10102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 0x7f6d101014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010e5a0 con 0x7f6d10108670 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010e420 con 0x7f6d10105ac0 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d182a9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010e720 con 0x7f6d1006b470 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45878/0 (socket says 172.21.15.120:45878) 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 -- 172.21.15.120:0/2485406726 learned_addr learned my addr 172.21.15.120:0/2485406726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d15844640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 0x7f6d101014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16045640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d1006b470 0x7f6d10102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 -- 172.21.15.120:0/2485406726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d1006b470 msgr2=0x7f6d10102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d1006b470 0x7f6d10102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 -- 172.21.15.120:0/2485406726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 msgr2=0x7f6d101014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 0x7f6d101014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 -- 172.21.15.120:0/2485406726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d10102280 con 0x7f6d10108670 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d16846640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c970 tx=0x7f6d0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.256+0000 7f6d077fe640 1 -- 172.21.15.120:0/2485406726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08007bb0 con 0x7f6d10108670 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.257+0000 7f6d077fe640 1 -- 172.21.15.120:0/2485406726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d08007d50 con 0x7f6d10108670 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.257+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101a2de0 con 0x7f6d10108670 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.257+0000 7f6d077fe640 1 -- 172.21.15.120:0/2485406726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08018770 con 0x7f6d10108670 2024-09-10T22:53:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.257+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101a32a0 con 0x7f6d10108670 2024-09-10T22:53:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.259+0000 7f6d1b074740 1 Processor -- start 2024-09-10T22:53:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.259+0000 7f6d077fe640 1 -- 172.21.15.120:0/2485406726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6d0800f990 con 0x7f6d10108670 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.259+0000 7f6d1b074740 1 -- start start 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f35440c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f35446090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f353b06f0 con 0x559f35441190 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f3539fb20 con 0x559f35443bd0 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d1b074740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559f353a3930 con 0x559f3543e860 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45888/0 (socket says 172.21.15.120:45888) 2024-09-10T22:53:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 -- 172.21.15.120:0/1416661390 learned_addr learned my addr 172.21.15.120:0/1416661390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d15844640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f35446090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d15844640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f35440c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d077fe640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 0x7f6cfc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 -- 172.21.15.120:0/1416661390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 msgr2=0x559f35440c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f35440c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 -- 172.21.15.120:0/1416661390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 msgr2=0x559f35446090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f35446090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 -- 172.21.15.120:0/1416661390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f35365530 con 0x559f35441190 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d15844640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f35446090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16846640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 0x7f6cfc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d16045640 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c035000 tx=0x7f6d0c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779e14f7786e78fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.260+0000 7f6d077fe640 1 -- 172.21.15.120:0/2485406726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6d080e5be0 con 0x7f6d10108670 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d04bfe640 1 -- 172.21.15.120:0/1416661390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c003970 con 0x559f35441190 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d04bfe640 1 -- 172.21.15.120:0/1416661390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c036040 con 0x559f35441190 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d04bfe640 1 -- 172.21.15.120:0/1416661390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c02f660 con 0x559f35441190 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 -- 172.21.15.120:0/1416661390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 msgr2=0x559f35443580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c035000 tx=0x7f6d0c0099b0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d16846640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 0x7f6cfc0b7820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d080a89a0 tx=0x7f6d080a0c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 -- 172.21.15.120:0/1416661390 shutdown_connections 2024-09-10T22:53:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f35446090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f35443580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1416661390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f35440c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.261+0000 7f6d1b074740 1 -- 172.21.15.120:0/1416661390 >> 172.21.15.120:0/1416661390 conn(0x559f354399b0 msgr2=0x559f3543a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- 172.21.15.120:0/1416661390 shutdown_connections 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- 172.21.15.120:0/1416661390 wait complete. 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- start start 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 Processor -- start 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f3543e860 0x559f35447e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f35441190 0x559f35448530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f352d7ab0 con 0x559f35443bd0 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f35345730 con 0x559f3543e860 2024-09-10T22:53:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559f35446b50 con 0x559f35441190 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d1b074740 10 client.0 _ll_register_callbacks cb 0x559f353996b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d15844640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d15844640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45898/0 (socket says 172.21.15.120:45898) 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d15844640 1 -- 172.21.15.120:0/1499230898 learned_addr learned my addr 172.21.15.120:0/1499230898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.262+0000 7f6d15844640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f3543e860 0x559f35447e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d16045640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f35441190 0x559f35448530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 -- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f35441190 msgr2=0x559f35448530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f35441190 0x559f35448530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 -- 172.21.15.120:0/1499230898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f3543e860 msgr2=0x559f35447e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f3543e860 0x559f35447e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 -- 172.21.15.120:0/1499230898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f35101790 con 0x559f35443bd0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d15844640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6d00012d40 tx=0x7f6d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf96ca54cb58c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004030 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6d000041d0 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0c036040 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6d000043d0 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d1b074740 10 client.15480 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559f3517a620 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.263+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c036040 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6d000104c0 con 0x559f35443bd0 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 10 client.15480 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 20 client.15480 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f35455c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 4 client.15480 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f353b2640 con 0x559f35453780 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d16045640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f35455c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6d16045640 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f35455c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42ff52a20542228 server_cookie=3de34794b154b2da in_seq=0 out_seq=0 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.264+0000 7f6cf9ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/1499230898 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f353b2640 con 0x559f35453780 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6cf9ffb640 10 client.15480 handle_command_reply: tid=0 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 client.15480 shutdown 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 msgr2=0x559f35449100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6d00012d40 tx=0x7f6d000046c0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 shutdown_connections 2024-09-10T22:53:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f35455c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f35449100 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6cf9ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f35441190 0x559f35448530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 --2- 172.21.15.120:0/1499230898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f3543e860 0x559f35447e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 >> 172.21.15.120:0/1499230898 conn(0x559f354399b0 msgr2=0x559f35438bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 shutdown_connections 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 1 -- 172.21.15.120:0/1499230898 wait complete. 2024-09-10T22:53:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.265+0000 7f6d1b074740 20 client.15480 trim_cache size 0 max 16384 2024-09-10T22:53:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 Processor -- start 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 -- start start 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f354499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f3543e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101a9f40 con 0x559f35441190 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ac7c0 con 0x559f35443bd0 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d1b074740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101aa5b0 con 0x559f3543e860 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.280+0000 7f6d16045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d15844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f354499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45906/0 (socket says 172.21.15.120:45906) 2024-09-10T22:53:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d15844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f354499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40428/0 (socket says 172.21.15.120:40428) 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 -- 172.21.15.120:0/2088285424 learned_addr learned my addr 172.21.15.120:0/2088285424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d15844640 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f3543e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 -- 172.21.15.120:0/2088285424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 msgr2=0x559f354499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f354499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 -- 172.21.15.120:0/2088285424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 msgr2=0x559f3543e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f3543e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 -- 172.21.15.120:0/2088285424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f35446cd0 con 0x559f35441190 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6d16045640 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009430 tx=0x7f6d0c004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac8dbca66897b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6cfbfff640 1 -- 172.21.15.120:0/2088285424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c05c070 con 0x559f35441190 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6cfbfff640 1 -- 172.21.15.120:0/2088285424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c036040 con 0x559f35441190 2024-09-10T22:53:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.281+0000 7f6cfbfff640 1 -- 172.21.15.120:0/2088285424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c04a040 con 0x559f35441190 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- 172.21.15.120:0/2088285424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 msgr2=0x559f3543dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009430 tx=0x7f6d0c004910 comp rx=0 tx=0).stop 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- 172.21.15.120:0/2088285424 shutdown_connections 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35443bd0 0x559f3543e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35441190 0x559f3543dcf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 --2- 172.21.15.120:0/2088285424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f354499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- 172.21.15.120:0/2088285424 >> 172.21.15.120:0/2088285424 conn(0x559f354399b0 msgr2=0x559f3543a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- 172.21.15.120:0/2088285424 shutdown_connections 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- 172.21.15.120:0/2088285424 wait complete. 2024-09-10T22:53:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 -- start start 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.282+0000 7f6d1b074740 1 Processor -- start 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f3544cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 0x559f3544d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101aac80 con 0x559f35443bd0 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ac0f0 con 0x559f35441190 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ab350 con 0x559f3543e860 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 10 client.0 _ll_register_callbacks cb 0x7f6d00012950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45916/0 (socket says 172.21.15.120:45916) 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 -- 172.21.15.120:0/698046072 learned_addr learned my addr 172.21.15.120:0/698046072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d16045640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 0x559f3544d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f3544cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 -- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 msgr2=0x559f3544cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f3544cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 -- 172.21.15.120:0/698046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 msgr2=0x559f3544d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 0x559f3544d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 -- 172.21.15.120:0/698046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f3544e850 con 0x559f35443bd0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d16045640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 0x559f3544d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d15844640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6d00014010 tx=0x7f6d0000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42b18dfa2335bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00035620 con 0x559f35443bd0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f6d00027400 con 0x559f35443bd0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.283+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0c036040 con 0x559f35443bd0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6d000357c0 con 0x559f35443bd0 2024-09-10T22:53:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 10 client.15486 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559f3544fc30 con 0x559f35443bd0 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c036040 con 0x559f35443bd0 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6d0001c040 con 0x559f35443bd0 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 10 client.15486 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 20 client.15486 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f354c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 4 client.15486 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f353ae0f0 con 0x559f35453780 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.284+0000 7f6d16045640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f354c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.285+0000 7f6cf9ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.285+0000 7f6d16045640 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f354c2b90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1b0e362e19634e server_cookie=4bdba58f533af3e in_seq=0 out_seq=0 2024-09-10T22:53:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6cf9ffb640 1 -- 172.21.15.120:0/698046072 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f353ae0f0 con 0x559f35453780 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6cf9ffb640 10 client.15486 handle_command_reply: tid=0 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 client.15486 shutdown 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 msgr2=0x559f3544e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6d00014010 tx=0x7f6d0000b410 comp rx=0 tx=0).stop 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 shutdown_connections 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f35453780 0x559f354c2b90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f35443bd0 0x559f3544e020 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f35441190 0x559f3544d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6cf9ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 --2- 172.21.15.120:0/698046072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f3543e860 0x559f3544cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.288+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 >> 172.21.15.120:0/698046072 conn(0x559f354399b0 msgr2=0x559f3543a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 shutdown_connections 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d1b074740 1 -- 172.21.15.120:0/698046072 wait complete. 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d1b074740 20 client.15486 trim_cache size 0 max 16384 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:53:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 msgr2=0x7f6cfc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 0x7f6cfc0b7820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d080a89a0 tx=0x7f6d080a0c80 comp rx=0 tx=0).stop 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 msgr2=0x7f6d10101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c970 tx=0x7f6d0800ce30 comp rx=0 tx=0).stop 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 shutdown_connections 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6cfc0b5360 0x7f6cfc0b7820 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d10108670 0x7f6d10101a10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d10105ac0 0x7f6d101014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 --2- 172.21.15.120:0/2485406726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d1006b470 0x7f6d10102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 >> 172.21.15.120:0/2485406726 conn(0x7f6d100fd140 msgr2=0x7f6d10103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 shutdown_connections 2024-09-10T22:53:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:05.289+0000 7f6d182a9640 1 -- 172.21.15.120:0/2485406726 wait complete. 2024-09-10T22:53:05.297 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T22:53:05.334 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T22:53:05.337 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T22:53:05.337 INFO:tasks.workunit.client.0.smithi120.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T22:53:05.375 INFO:tasks.workunit.client.0.smithi120.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T22:53:05.379 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T22:53:05.379 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T22:53:05.435 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-10T22:53:05.437 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-10T22:53:05.437 DEBUG:teuthology.orchestra.run.smithi120:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-10T22:53:05.497 INFO:tasks.workunit.client.0.smithi120.stderr:+ dbench 1 2024-09-10T22:53:05.503 INFO:tasks.workunit.client.0.smithi120.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T22:53:05.503 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T22:53:05.503 INFO:tasks.workunit.client.0.smithi120.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T22:53:05.512 INFO:tasks.workunit.client.0.smithi120.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-10T22:53:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:05 smithi176 ceph-mon[34982]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T22:53:05.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:05 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:53:05.612 INFO:tasks.workunit.client.0.smithi120.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-10T22:53:05.612 INFO:tasks.workunit.client.0.smithi120.stdout:releasing clients 2024-09-10T22:53:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:05 smithi120 ceph-mon[31247]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T22:53:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:05 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:53:05.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:05 smithi153 ceph-mon[34951]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T22:53:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:05 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:53:06.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 3871 39.77 MB/sec warmup 1 sec latency 6.389 ms 2024-09-10T22:53:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:07 smithi176 ceph-mon[34982]: pgmap v363: 97 pgs: 97 active+clean; 733 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-10T22:53:07.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 7387 29.44 MB/sec warmup 2 sec latency 11.954 ms 2024-09-10T22:53:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:07 smithi120 ceph-mon[31247]: pgmap v363: 97 pgs: 97 active+clean; 733 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-10T22:53:07.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:07 smithi153 ceph-mon[34951]: pgmap v363: 97 pgs: 97 active+clean; 733 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.2 KiB/s wr, 4 op/s 2024-09-10T22:53:08.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 10930 26.00 MB/sec warmup 3 sec latency 12.955 ms 2024-09-10T22:53:09.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:09 smithi176 ceph-mon[34982]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 78 KiB/s wr, 18 op/s 2024-09-10T22:53:09.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 14523 24.28 MB/sec warmup 4 sec latency 13.764 ms 2024-09-10T22:53:09.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:09 smithi120 ceph-mon[31247]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 78 KiB/s wr, 18 op/s 2024-09-10T22:53:09.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:09 smithi153 ceph-mon[34951]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 78 KiB/s wr, 18 op/s 2024-09-10T22:53:10.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 18157 23.31 MB/sec warmup 5 sec latency 13.394 ms 2024-09-10T22:53:11.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:11 smithi176 ceph-mon[34982]: pgmap v365: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 202 KiB/s wr, 23 op/s 2024-09-10T22:53:11.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 21783 22.62 MB/sec warmup 6 sec latency 17.471 ms 2024-09-10T22:53:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:11 smithi120 ceph-mon[31247]: pgmap v365: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 202 KiB/s wr, 23 op/s 2024-09-10T22:53:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:11 smithi153 ceph-mon[34951]: pgmap v365: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 202 KiB/s wr, 23 op/s 2024-09-10T22:53:12.612 INFO:tasks.workunit.client.0.smithi120.stdout: 1 25444 22.12 MB/sec warmup 7 sec latency 13.236 ms 2024-09-10T22:53:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:13 smithi176 ceph-mon[34982]: pgmap v366: 97 pgs: 97 active+clean; 8.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 811 KiB/s wr, 93 op/s 2024-09-10T22:53:13.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 29012 21.75 MB/sec warmup 8 sec latency 20.362 ms 2024-09-10T22:53:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:13 smithi120 ceph-mon[31247]: pgmap v366: 97 pgs: 97 active+clean; 8.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 811 KiB/s wr, 93 op/s 2024-09-10T22:53:13.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:13 smithi153 ceph-mon[34951]: pgmap v366: 97 pgs: 97 active+clean; 8.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 811 KiB/s wr, 93 op/s 2024-09-10T22:53:14.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 32513 21.46 MB/sec warmup 9 sec latency 78.361 ms 2024-09-10T22:53:15.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:15 smithi176 ceph-mon[34982]: pgmap v367: 97 pgs: 97 active+clean; 13 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-10T22:53:15.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 36045 21.22 MB/sec warmup 10 sec latency 20.157 ms 2024-09-10T22:53:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:15 smithi120 ceph-mon[31247]: pgmap v367: 97 pgs: 97 active+clean; 13 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-10T22:53:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:15 smithi153 ceph-mon[34951]: pgmap v367: 97 pgs: 97 active+clean; 13 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-09-10T22:53:16.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 39588 21.03 MB/sec warmup 11 sec latency 14.841 ms 2024-09-10T22:53:17.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:17 smithi176 ceph-mon[34982]: pgmap v368: 97 pgs: 97 active+clean; 16 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T22:53:17.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 43105 20.87 MB/sec warmup 12 sec latency 21.460 ms 2024-09-10T22:53:17.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:17 smithi120 ceph-mon[31247]: pgmap v368: 97 pgs: 97 active+clean; 16 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T22:53:17.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:17 smithi153 ceph-mon[34951]: pgmap v368: 97 pgs: 97 active+clean; 16 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.381+0000 7f84181ec640 1 -- 172.21.15.120:0/1918971913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410101860 msgr2=0x7f8410101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.381+0000 7f84181ec640 1 --2- 172.21.15.120:0/1918971913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410101860 0x7f8410101c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f840c009930 tx=0x7f840c02a420 comp rx=0 tx=0).stop 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- 172.21.15.120:0/1918971913 shutdown_connections 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- 172.21.15.120:0/1918971913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8410103370 0x7f8410109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- 172.21.15.120:0/1918971913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84101029b0 0x7f8410102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- 172.21.15.120:0/1918971913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410101860 0x7f8410101c60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- 172.21.15.120:0/1918971913 >> 172.21.15.120:0/1918971913 conn(0x7f84100fd140 msgr2=0x7f84100ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- 172.21.15.120:0/1918971913 shutdown_connections 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- 172.21.15.120:0/1918971913 wait complete. 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 Processor -- start 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- start start 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 0x7f841006b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84101029b0 0x7f841019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f841010d4d0 con 0x7f8410103370 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f841010d350 con 0x7f8410101860 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f84181ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f841010d650 con 0x7f84101029b0 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.382+0000 7f8416789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8415f88640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 0x7f841006b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43290/0 (socket says 172.21.15.120:43290) 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 -- 172.21.15.120:0/2778684345 learned_addr learned my addr 172.21.15.120:0/2778684345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8415787640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84101029b0 0x7f841019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 -- 172.21.15.120:0/2778684345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84101029b0 msgr2=0x7f841019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84101029b0 0x7f841019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 -- 172.21.15.120:0/2778684345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 msgr2=0x7f841006b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 0x7f841006b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 -- 172.21.15.120:0/2778684345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84101a0950 con 0x7f8410103370 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8415f88640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 0x7f841006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8416789640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f840800eb10 tx=0x7f840800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8406ffd640 1 -- 172.21.15.120:0/2778684345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408004b20 con 0x7f8410103370 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8406ffd640 1 -- 172.21.15.120:0/2778684345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8408004cc0 con 0x7f8410103370 2024-09-10T22:53:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84101a0c40 con 0x7f8410103370 2024-09-10T22:53:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f8406ffd640 1 -- 172.21.15.120:0/2778684345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408010670 con 0x7f8410103370 2024-09-10T22:53:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.383+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a1190 con 0x7f8410103370 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.385+0000 7f8406ffd640 1 -- 172.21.15.120:0/2778684345 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8408005020 con 0x7f8410103370 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 Processor -- start 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 -- start start 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55626b21b6f0 con 0x55626b2b05a0 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55626b20ab20 con 0x55626b2adc70 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f841afbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55626b20e930 con 0x55626b2b2fe0 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8415f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35406/0 (socket says 172.21.15.120:35406) 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8415787640 1 -- 172.21.15.120:0/255518373 learned_addr learned my addr 172.21.15.120:0/255518373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8415787640 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.386+0000 7f8406ffd640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 0x7f83fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8416789640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 0x7f83fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 -- 172.21.15.120:0/255518373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 msgr2=0x55626b2b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 -- 172.21.15.120:0/255518373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 msgr2=0x55626b2b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 -- 172.21.15.120:0/255518373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626b1d0530 con 0x55626b2adc70 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8406ffd640 1 -- 172.21.15.120:0/2778684345 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f84080e6100 con 0x7f8410103370 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f8415787640 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8400012820 tx=0x7f840000b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4101a4cea2fbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f83fbfff640 1 -- 172.21.15.120:0/255518373 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400012da0 con 0x55626b2adc70 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f83fbfff640 1 -- 172.21.15.120:0/255518373 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400010c20 con 0x55626b2adc70 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.387+0000 7f83fbfff640 1 -- 172.21.15.120:0/255518373 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840001eba0 con 0x55626b2adc70 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- 172.21.15.120:0/255518373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 msgr2=0x55626b2b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f8416789640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 0x7f83fc0b77a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84080a9b60 tx=0x7f84080dcd60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8400012820 tx=0x7f840000b7e0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- 172.21.15.120:0/255518373 shutdown_connections 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 --2- 172.21.15.120:0/255518373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b0060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- 172.21.15.120:0/255518373 >> 172.21.15.120:0/255518373 conn(0x55626b2a49b0 msgr2=0x55626b2a5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- 172.21.15.120:0/255518373 shutdown_connections 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- 172.21.15.120:0/255518373 wait complete. 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 -- start start 2024-09-10T22:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.388+0000 7f841afbe740 1 Processor -- start 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55626b142ab0 con 0x55626b2b05a0 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55626b1b0730 con 0x55626b2adc70 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55626b2b5fc0 con 0x55626b2b2fe0 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f841afbe740 10 client.0 _ll_register_callbacks cb 0x55626b2046b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35416/0 (socket says 172.21.15.120:35416) 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415787640 1 -- 172.21.15.120:0/1924912166 learned_addr learned my addr 172.21.15.120:0/1924912166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415787640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 -- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 msgr2=0x55626b2b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 -- 172.21.15.120:0/1924912166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 msgr2=0x55626b2b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 -- 172.21.15.120:0/1924912166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626af6c790 con 0x55626b2b05a0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415787640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f8415f88640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b79a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f840c005ab0 tx=0x7f840c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738d2c7da494d38c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.389+0000 7f83f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c035040 con 0x55626b2b05a0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f840c004450 con 0x55626b2b05a0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400010c20 con 0x55626b2b05a0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f840c0035f0 con 0x55626b2b05a0 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 10 client.15498 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55626afe5620 con 0x55626b2b05a0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400010c20 con 0x55626b2b05a0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f840c044070 con 0x55626b2b05a0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 10 client.15498 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 20 client.15498 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b2c2be0 0x55626b2c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 4 client.15498 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55626b21d640 con 0x55626b2c2be0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.390+0000 7f8415787640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b2c2be0 0x55626b2c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.391+0000 7f83f97fa640 10 client.15498 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.391+0000 7f8415787640 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b2c2be0 0x55626b2c4fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58290cbdfd6449f3 server_cookie=4796b145ce471e9e in_seq=0 out_seq=0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.391+0000 7f83f97fa640 1 -- 172.21.15.120:0/1924912166 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55626b21d640 con 0x55626b2c2be0 2024-09-10T22:53:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.391+0000 7f83f97fa640 10 client.15498 handle_command_reply: tid=0 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.391+0000 7f841afbe740 1 client.15498 shutdown 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 msgr2=0x55626b2b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b79a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f840c005ab0 tx=0x7f840c031840 comp rx=0 tx=0).stop 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 shutdown_connections 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b2c2be0 0x55626b2c4fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2b79a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f83f97fa640 0 client.15498 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 --2- 172.21.15.120:0/1924912166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 >> 172.21.15.120:0/1924912166 conn(0x55626b2a49b0 msgr2=0x55626b2a3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 shutdown_connections 2024-09-10T22:53:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 1 -- 172.21.15.120:0/1924912166 wait complete. 2024-09-10T22:53:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.392+0000 7f841afbe740 20 client.15498 trim_cache size 0 max 16384 2024-09-10T22:53:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.406+0000 7f841afbe740 1 Processor -- start 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.406+0000 7f841afbe740 1 -- start start 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2b05a0 0x55626b2b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a5c50 con 0x55626b2adc70 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a84b0 con 0x55626b2b05a0 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f841afbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a62a0 con 0x55626b2b2fe0 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43312/0 (socket says 172.21.15.120:43312) 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 -- 172.21.15.120:0/1785667155 learned_addr learned my addr 172.21.15.120:0/1785667155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415f88640 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2b05a0 0x55626b2b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 -- 172.21.15.120:0/1785667155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 msgr2=0x55626b2b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 -- 172.21.15.120:0/1785667155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2b05a0 msgr2=0x55626b2b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2b05a0 0x55626b2b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 -- 172.21.15.120:0/1785667155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626b2b88e0 con 0x55626b2adc70 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f8415787640 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f840001e510 tx=0x7f840000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc6bb5dad566517 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f83fb7fe640 1 -- 172.21.15.120:0/1785667155 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400024040 con 0x55626b2adc70 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f83fb7fe640 1 -- 172.21.15.120:0/1785667155 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400016040 con 0x55626b2adc70 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.407+0000 7f83fb7fe640 1 -- 172.21.15.120:0/1785667155 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840001ea50 con 0x55626b2adc70 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- 172.21.15.120:0/1785667155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 msgr2=0x55626b2b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f840001e510 tx=0x7f840000bc60 comp rx=0 tx=0).stop 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- 172.21.15.120:0/1785667155 shutdown_connections 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2b05a0 0x55626b2b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 --2- 172.21.15.120:0/1785667155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2adc70 0x55626b2b3ad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- 172.21.15.120:0/1785667155 >> 172.21.15.120:0/1785667155 conn(0x55626b2a49b0 msgr2=0x55626b2b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- 172.21.15.120:0/1785667155 shutdown_connections 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- 172.21.15.120:0/1785667155 wait complete. 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 -- start start 2024-09-10T22:53:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.408+0000 7f841afbe740 1 Processor -- start 2024-09-10T22:53:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a6970 con 0x55626b2b05a0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a7de0 con 0x55626b2adc70 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a7040 con 0x55626b2b2fe0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f841afbe740 10 client.0 _ll_register_callbacks cb 0x7f840c032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43320/0 (socket says 172.21.15.120:43320) 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35434/0 (socket says 172.21.15.120:35434) 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 -- 172.21.15.120:0/925157692 learned_addr learned my addr 172.21.15.120:0/925157692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415787640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 -- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 msgr2=0x55626b2bde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2bde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 -- 172.21.15.120:0/925157692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 msgr2=0x55626b2bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 -- 172.21.15.120:0/925157692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626b2b2d00 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f8415f88640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f840c03ca20 tx=0x7f840c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27cd5d6598505b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.409+0000 7f83f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c035030 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8182+0+0 (secure 0 0 0) 0x7f840c03c440 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400016040 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f840c061050 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 10 client.15504 fetch_fsmap learned FSMap version 19 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55626b2bd1e0 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400016040 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f840c044040 con 0x55626b2b05a0 2024-09-10T22:53:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 10 client.15504 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 20 client.15504 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b3301b0 0x55626b3325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 4 client.15504 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55626b2190f0 con 0x55626b3301b0 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.410+0000 7f8415787640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b3301b0 0x55626b3325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.411+0000 7f8415787640 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b3301b0 0x55626b3325d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b2196a4680470d server_cookie=243ee9692ba0dbd4 in_seq=0 out_seq=0 2024-09-10T22:53:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.411+0000 7f83f97fa640 10 client.15504 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f83f97fa640 1 -- 172.21.15.120:0/925157692 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55626b2190f0 con 0x55626b3301b0 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f83f97fa640 10 client.15504 handle_command_reply: tid=0 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 client.15504 shutdown 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 msgr2=0x55626b2bd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f840c03ca20 tx=0x7f840c0095f0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 shutdown_connections 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55626b3301b0 0x55626b3325d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55626b2b2fe0 0x55626b2bde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55626b2b05a0 0x55626b2bd950 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 --2- 172.21.15.120:0/925157692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55626b2adc70 0x55626b2bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 >> 172.21.15.120:0/925157692 conn(0x55626b2a49b0 msgr2=0x55626b2a90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f83f97fa640 0 client.15504 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 shutdown_connections 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 1 -- 172.21.15.120:0/925157692 wait complete. 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.560+0000 7f841afbe740 20 client.15504 trim_cache size 0 max 16384 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:53:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:53:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 msgr2=0x7f83fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 0x7f83fc0b77a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84080a9b60 tx=0x7f84080dcd60 comp rx=0 tx=0).stop 2024-09-10T22:53:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 msgr2=0x7f84101a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f840800eb10 tx=0x7f840800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 shutdown_connections 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83fc0b52e0 0x7f83fc0b77a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8410103370 0x7f84101a0090 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84101029b0 0x7f841019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 --2- 172.21.15.120:0/2778684345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8410101860 0x7f841006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 >> 172.21.15.120:0/2778684345 conn(0x7f84100fd140 msgr2=0x7f8410105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 shutdown_connections 2024-09-10T22:53:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:18.561+0000 7f84181ec640 1 -- 172.21.15.120:0/2778684345 wait complete. 2024-09-10T22:53:18.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 46485 20.73 MB/sec warmup 13 sec latency 76.943 ms 2024-09-10T22:53:19.254 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:53:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setuser ceph since I am not root 2024-09-10T22:53:19.254 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:53:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setgroup ceph since I am not root 2024-09-10T22:53:19.254 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:53:18 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/2749704550 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e63783860 con 0x561e63761400 2024-09-10T22:53:19.254 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:53:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T22:53:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:19 smithi176 ceph-mon[34982]: pgmap v369: 97 pgs: 97 active+clean; 16 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-10T22:53:19.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:19 smithi176 ceph-mon[34982]: daemon mds.c restarted 2024-09-10T22:53:19.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 49834 20.53 MB/sec warmup 14 sec latency 16.419 ms 2024-09-10T22:53:19.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:19 smithi120 ceph-mon[31247]: pgmap v369: 97 pgs: 97 active+clean; 16 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-10T22:53:19.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:19 smithi120 ceph-mon[31247]: daemon mds.c restarted 2024-09-10T22:53:19.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:19 smithi153 ceph-mon[34951]: pgmap v369: 97 pgs: 97 active+clean; 16 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-10T22:53:19.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:19 smithi153 ceph-mon[34951]: daemon mds.c restarted 2024-09-10T22:53:20.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:20 smithi120 ceph-mon[31247]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:53:20.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:20 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/4005968272,v1:172.21.15.176:6837/4005968272] up:boot 2024-09-10T22:53:20.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:20 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:53:20.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:20 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:53:20.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:20 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:53:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:20 smithi176 ceph-mon[34982]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:53:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:20 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/4005968272,v1:172.21.15.176:6837/4005968272] up:boot 2024-09-10T22:53:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:20 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:53:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:20 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:53:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:20 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:53:20.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 53470 20.50 MB/sec warmup 15 sec latency 12.989 ms 2024-09-10T22:53:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:20 smithi153 ceph-mon[34951]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:53:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:20 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/4005968272,v1:172.21.15.176:6837/4005968272] up:boot 2024-09-10T22:53:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:20 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:53:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:20 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:53:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:20 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:53:21.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:21 smithi176 ceph-mon[34982]: pgmap v371: 97 pgs: 97 active+clean; 12 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2024-09-10T22:53:21.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 57009 20.41 MB/sec warmup 16 sec latency 12.537 ms 2024-09-10T22:53:21.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:21 smithi120 ceph-mon[31247]: pgmap v371: 97 pgs: 97 active+clean; 12 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2024-09-10T22:53:21.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:21 smithi153 ceph-mon[34951]: pgmap v371: 97 pgs: 97 active+clean; 12 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 366 op/s 2024-09-10T22:53:22.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 60153 20.08 MB/sec warmup 17 sec latency 15.826 ms 2024-09-10T22:53:23.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:23 smithi176 ceph-mon[34982]: pgmap v372: 97 pgs: 97 active+clean; 13 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 471 op/s 2024-09-10T22:53:23.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 63950 20.21 MB/sec warmup 18 sec latency 12.191 ms 2024-09-10T22:53:23.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:23 smithi120 ceph-mon[31247]: pgmap v372: 97 pgs: 97 active+clean; 13 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 471 op/s 2024-09-10T22:53:23.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:23 smithi153 ceph-mon[34951]: pgmap v372: 97 pgs: 97 active+clean; 13 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 471 op/s 2024-09-10T22:53:24.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 67496 20.15 MB/sec warmup 19 sec latency 19.360 ms 2024-09-10T22:53:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:25 smithi176 ceph-mon[34982]: pgmap v373: 97 pgs: 97 active+clean; 13 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 391 op/s 2024-09-10T22:53:25.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 71084 20.10 MB/sec warmup 20 sec latency 15.544 ms 2024-09-10T22:53:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:25 smithi120 ceph-mon[31247]: pgmap v373: 97 pgs: 97 active+clean; 13 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 391 op/s 2024-09-10T22:53:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:25 smithi153 ceph-mon[34951]: pgmap v373: 97 pgs: 97 active+clean; 13 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 391 op/s 2024-09-10T22:53:26.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 74755 20.11 MB/sec warmup 21 sec latency 12.272 ms 2024-09-10T22:53:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:27 smithi176 ceph-mon[34982]: pgmap v374: 97 pgs: 97 active+clean; 16 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-09-10T22:53:27.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 78322 20.06 MB/sec warmup 22 sec latency 16.168 ms 2024-09-10T22:53:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:27 smithi120 ceph-mon[31247]: pgmap v374: 97 pgs: 97 active+clean; 16 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-09-10T22:53:27.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:27 smithi153 ceph-mon[34951]: pgmap v374: 97 pgs: 97 active+clean; 16 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 368 op/s 2024-09-10T22:53:28.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 81757 19.98 MB/sec warmup 23 sec latency 12.553 ms 2024-09-10T22:53:29.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:29 smithi176 ceph-mon[34982]: pgmap v375: 97 pgs: 97 active+clean; 21 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 461 op/s 2024-09-10T22:53:29.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 85296 19.94 MB/sec warmup 24 sec latency 22.297 ms 2024-09-10T22:53:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:29 smithi120 ceph-mon[31247]: pgmap v375: 97 pgs: 97 active+clean; 21 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 461 op/s 2024-09-10T22:53:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:29 smithi153 ceph-mon[34951]: pgmap v375: 97 pgs: 97 active+clean; 21 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 461 op/s 2024-09-10T22:53:30.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 88715 19.90 MB/sec warmup 25 sec latency 21.521 ms 2024-09-10T22:53:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:31 smithi176 ceph-mon[34982]: pgmap v376: 97 pgs: 97 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-10T22:53:31.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 92386 19.88 MB/sec warmup 26 sec latency 12.506 ms 2024-09-10T22:53:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2026971510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 msgr2=0x7f8ca0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2026971510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0101c60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009930 tx=0x7f8c9c02a420 comp rx=0 tx=0).stop 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2026971510 shutdown_connections 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2026971510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 0x7f8ca0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2026971510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 0x7f8ca0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2026971510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0101c60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2026971510 >> 172.21.15.120:0/2026971510 conn(0x7f8ca00fd140 msgr2=0x7f8ca00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2026971510 shutdown_connections 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2026971510 wait complete. 2024-09-10T22:53:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 Processor -- start 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.650+0000 7f8ca7f3d640 1 -- start start 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 0x7f8ca0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 0x7f8ca0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010d4d0 con 0x7f8ca0101860 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010d350 con 0x7f8ca0103370 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010d650 con 0x7f8ca01029b0 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38674/0 (socket says 172.21.15.120:38674) 2024-09-10T22:53:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 -- 172.21.15.120:0/2075548271 learned_addr learned my addr 172.21.15.120:0/2075548271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 0x7f8ca0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 0x7f8ca0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 -- 172.21.15.120:0/2075548271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 msgr2=0x7f8ca0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 0x7f8ca0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 -- 172.21.15.120:0/2075548271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 msgr2=0x7f8ca0198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 0x7f8ca0198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 -- 172.21.15.120:0/2075548271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca01997f0 con 0x7f8ca0101860 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009a50 tx=0x7f8c9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8c96ffd640 1 -- 172.21.15.120:0/2075548271 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c03f030 con 0x7f8ca0101860 2024-09-10T22:53:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8c96ffd640 1 -- 172.21.15.120:0/2075548271 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c031910 con 0x7f8ca0101860 2024-09-10T22:53:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca0199a80 con 0x7f8ca0101860 2024-09-10T22:53:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8c96ffd640 1 -- 172.21.15.120:0/2075548271 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c035070 con 0x7f8ca0101860 2024-09-10T22:53:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.651+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01a01b0 con 0x7f8ca0101860 2024-09-10T22:53:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8c96ffd640 1 -- 172.21.15.120:0/2075548271 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8c9c03ba60 con 0x7f8ca0101860 2024-09-10T22:53:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 Processor -- start 2024-09-10T22:53:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 -- start start 2024-09-10T22:53:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4694c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4697bd0 0x5617b469a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617b46046f0 con 0x5617b4695190 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617b45f3b20 con 0x5617b4697bd0 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.654+0000 7f8caade5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5617b45f7930 con 0x5617b4692860 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca5cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4694c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca5cd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38676/0 (socket says 172.21.15.120:38676) 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca5cd9640 1 -- 172.21.15.120:0/2679342466 learned_addr learned my addr 172.21.15.120:0/2679342466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8c96ffd640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 0x7f8c8c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4697bd0 0x5617b469a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 0x7f8c8c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8c96ffd640 1 -- 172.21.15.120:0/2075548271 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8c9c106380 con 0x7f8ca0101860 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 -- 172.21.15.120:0/2679342466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4697bd0 msgr2=0x5617b469a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4697bd0 0x5617b469a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 -- 172.21.15.120:0/2679342466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 msgr2=0x5617b4697580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 -- 172.21.15.120:0/2679342466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b45b9530 con 0x5617b4692860 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4694c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0198a30 tx=0x7f8c98013ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a9519445b2fa90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.655+0000 7f8c8bfff640 1 -- 172.21.15.120:0/2679342466 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98023030 con 0x5617b4692860 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8c8bfff640 1 -- 172.21.15.120:0/2679342466 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9800fdf0 con 0x5617b4692860 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8c8bfff640 1 -- 172.21.15.120:0/2679342466 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98010780 con 0x5617b4692860 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 0x7f8c8c0b7820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c98023000 tx=0x7f8c98022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 -- 172.21.15.120:0/2679342466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 msgr2=0x5617b4694c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4694c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0198a30 tx=0x7f8c98013ef0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 -- 172.21.15.120:0/2679342466 shutdown_connections 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4697bd0 0x5617b469a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b4697580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 --2- 172.21.15.120:0/2679342466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4694c50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 -- 172.21.15.120:0/2679342466 >> 172.21.15.120:0/2679342466 conn(0x5617b468d9b0 msgr2=0x5617b468e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 -- 172.21.15.120:0/2679342466 shutdown_connections 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.656+0000 7f8caade5740 1 -- 172.21.15.120:0/2679342466 wait complete. 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 -- start start 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 Processor -- start 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b469bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4695190 0x5617b469c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617b452bab0 con 0x5617b4697bd0 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617b4599730 con 0x5617b4692860 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5617b469abb0 con 0x5617b4695190 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8caade5740 10 client.0 _ll_register_callbacks cb 0x5617b45ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8ca64da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8ca5cd9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b469bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8ca64da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38680/0 (socket says 172.21.15.120:38680) 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8ca54d8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4695190 0x5617b469c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.657+0000 7f8ca64da640 1 -- 172.21.15.120:0/3651624959 learned_addr learned my addr 172.21.15.120:0/3651624959 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 -- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4695190 msgr2=0x5617b469c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4695190 0x5617b469c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 -- 172.21.15.120:0/3651624959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 msgr2=0x5617b469bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b469bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 -- 172.21.15.120:0/3651624959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b4355790 con 0x5617b4697bd0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b469bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8ca64da640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c98013490 tx=0x7f8c9800f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec065735789dc45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98010be0 con 0x5617b4697bd0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9800fdf0 con 0x5617b4697bd0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8c98010d80 con 0x5617b4697bd0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8c98003970 con 0x5617b4697bd0 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 10 client.15522 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5617b43ce620 con 0x5617b4697bd0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9800fdf0 con 0x5617b4697bd0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8c98005040 con 0x5617b4697bd0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 10 client.15522 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 20 client.15522 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.658+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b46a74d0 0x5617b46a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.659+0000 7f8caade5740 4 client.15522 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.659+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b4606640 con 0x5617b46a74d0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.659+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b46a74d0 0x5617b46a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.659+0000 7f8c897fa640 10 client.15522 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.659+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b46a74d0 0x5617b46a9900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d8216f17982943 server_cookie=b45044c7b268fff6 in_seq=0 out_seq=0 2024-09-10T22:53:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.660+0000 7f8c897fa640 1 -- 172.21.15.120:0/3651624959 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617b4606640 con 0x5617b46a74d0 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.660+0000 7f8c897fa640 10 client.15522 handle_command_reply: tid=0 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.660+0000 7f8caade5740 1 client.15522 shutdown 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 msgr2=0x5617b469d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c98013490 tx=0x7f8c9800f940 comp rx=0 tx=0).stop 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 shutdown_connections 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b46a74d0 0x5617b46a9900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b469d130 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4695190 0x5617b469c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8c897fa640 0 client.15522 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 --2- 172.21.15.120:0/3651624959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b469bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 >> 172.21.15.120:0/3651624959 conn(0x5617b468d9b0 msgr2=0x5617b468cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 shutdown_connections 2024-09-10T22:53:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 1 -- 172.21.15.120:0/3651624959 wait complete. 2024-09-10T22:53:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.661+0000 7f8caade5740 20 client.15522 trim_cache size 0 max 16384 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 Processor -- start 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 -- start start 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4691cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4695190 0x5617b4692200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a6d10 con 0x5617b4697bd0 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a9590 con 0x5617b4695190 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8caade5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a7380 con 0x5617b4692860 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8ca64da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8ca64da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38684/0 (socket says 172.21.15.120:38684) 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8ca64da640 1 -- 172.21.15.120:0/840488047 learned_addr learned my addr 172.21.15.120:0/840488047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.675+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4691cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4695190 0x5617b4692200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 -- 172.21.15.120:0/840488047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 msgr2=0x5617b4691cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4691cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 -- 172.21.15.120:0/840488047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4695190 msgr2=0x5617b4692200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4695190 0x5617b4692200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 -- 172.21.15.120:0/840488047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b4694fc0 con 0x5617b4697bd0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8ca64da640 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c98013f30 tx=0x7f8c98009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24433185ddc44995 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8c8b7fe640 1 -- 172.21.15.120:0/840488047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98013570 con 0x5617b4697bd0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8c8b7fe640 1 -- 172.21.15.120:0/840488047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c980050d0 con 0x5617b4697bd0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.676+0000 7f8c8b7fe640 1 -- 172.21.15.120:0/840488047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98010c40 con 0x5617b4697bd0 2024-09-10T22:53:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- 172.21.15.120:0/840488047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 msgr2=0x5617b4697fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c98013f30 tx=0x7f8c98009650 comp rx=0 tx=0).stop 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- 172.21.15.120:0/840488047 shutdown_connections 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4697bd0 0x5617b4697fb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4695190 0x5617b4692200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- 172.21.15.120:0/840488047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4692860 0x5617b4691cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- 172.21.15.120:0/840488047 >> 172.21.15.120:0/840488047 conn(0x5617b468d9b0 msgr2=0x5617b4688af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- 172.21.15.120:0/840488047 shutdown_connections 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- 172.21.15.120:0/840488047 wait complete. 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- start start 2024-09-10T22:53:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 Processor -- start 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b46a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 0x5617b46a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a7a50 con 0x5617b4695190 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a8ec0 con 0x5617b4692860 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01a8120 con 0x5617b4697bd0 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8caade5740 10 client.0 _ll_register_callbacks cb 0x7f8c9c0c8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8ca54d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.677+0000 7f8ca54d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38694/0 (socket says 172.21.15.120:38694) 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 -- 172.21.15.120:0/2717150881 learned_addr learned my addr 172.21.15.120:0/2717150881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 0x5617b46a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b46a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 -- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 msgr2=0x5617b46a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 0x5617b46a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 -- 172.21.15.120:0/2717150881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 msgr2=0x5617b46a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b46a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 -- 172.21.15.120:0/2717150881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b46a29a0 con 0x5617b4695190 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca64da640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 0x5617b46a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8ca54d8640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c900128c0 tx=0x7f8c90012d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9c2e47e4d7ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000fb30 con 0x5617b4695190 2024-09-10T22:53:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8c9000fcd0 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c980050d0 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8c90004030 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8caade5740 10 client.15528 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5617b46a3de0 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.678+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c980050d0 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8c90028440 con 0x5617b4695190 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 10 client.15528 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 20 client.15528 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b4714a60 0x5617b4716e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 4 client.15528 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617b46020f0 con 0x5617b4714a60 2024-09-10T22:53:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b4714a60 0x5617b4716e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8c897fa640 10 client.15528 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.679+0000 7f8ca5cd9640 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b4714a60 0x5617b4716e80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8ae4ffff56f7f1 server_cookie=3c5652c41d4f3b7e in_seq=0 out_seq=0 2024-09-10T22:53:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:31 smithi120 ceph-mon[31247]: pgmap v376: 97 pgs: 97 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-10T22:53:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:31 smithi153 ceph-mon[34951]: pgmap v376: 97 pgs: 97 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2024-09-10T22:53:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.829+0000 7f8c897fa640 1 -- 172.21.15.120:0/2717150881 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617b46020f0 con 0x5617b4714a60 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.829+0000 7f8c897fa640 10 client.15528 handle_command_reply: tid=0 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.829+0000 7f8caade5740 1 client.15528 shutdown 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 msgr2=0x5617b46a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c900128c0 tx=0x7f8c90012d80 comp rx=0 tx=0).stop 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 shutdown_connections 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617b4714a60 0x5617b4716e80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617b4697bd0 0x5617b46a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8c897fa640 0 client.15528 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617b4695190 0x5617b46a15f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 --2- 172.21.15.120:0/2717150881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617b4692860 0x5617b46a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 >> 172.21.15.120:0/2717150881 conn(0x5617b468d9b0 msgr2=0x5617b4697fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 shutdown_connections 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 1 -- 172.21.15.120:0/2717150881 wait complete. 2024-09-10T22:53:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.830+0000 7f8caade5740 20 client.15528 trim_cache size 0 max 16384 2024-09-10T22:53:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:53:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:53:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:53:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 msgr2=0x7f8c8c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 0x7f8c8c0b7820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c98023000 tx=0x7f8c98022000 comp rx=0 tx=0).stop 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 msgr2=0x7f8ca0197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009a50 tx=0x7f8c9c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 shutdown_connections 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8c8c0b5360 0x7f8c8c0b7820 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ca0103370 0x7f8ca0198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ca01029b0 0x7f8ca0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 --2- 172.21.15.120:0/2075548271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ca0101860 0x7f8ca0197cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 >> 172.21.15.120:0/2075548271 conn(0x7f8ca00fd140 msgr2=0x7f8ca0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 shutdown_connections 2024-09-10T22:53:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:31.831+0000 7f8ca7f3d640 1 -- 172.21.15.120:0/2075548271 wait complete. 2024-09-10T22:53:32.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 95192 19.66 MB/sec warmup 27 sec latency 75.686 ms 2024-09-10T22:53:33.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:33 smithi176 ceph-mon[34982]: pgmap v377: 97 pgs: 97 active+clean; 27 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-09-10T22:53:33.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 98425 19.48 MB/sec warmup 28 sec latency 14.035 ms 2024-09-10T22:53:33.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:53:33.670 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:33 smithi120 ceph-mon[31247]: pgmap v377: 97 pgs: 97 active+clean; 27 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-09-10T22:53:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:33 smithi153 ceph-mon[34951]: pgmap v377: 97 pgs: 97 active+clean; 27 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-09-10T22:53:33.903 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:34.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 -- 172.21.15.120:0/344356330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 msgr2=0x7f3c401020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 --2- 172.21.15.120:0/344356330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401020c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009900 tx=0x7f3c3002a410 comp rx=0 tx=0).stop 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 -- 172.21.15.120:0/344356330 shutdown_connections 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 --2- 172.21.15.120:0/344356330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 0x7f3c4010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 --2- 172.21.15.120:0/344356330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 0x7f3c40107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 --2- 172.21.15.120:0/344356330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401020c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 -- 172.21.15.120:0/344356330 >> 172.21.15.120:0/344356330 conn(0x7f3c400fd730 msgr2=0x7f3c400ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 -- 172.21.15.120:0/344356330 shutdown_connections 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 -- 172.21.15.120:0/344356330 wait complete. 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.413+0000 7f3c44e35640 1 Processor -- start 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 -- start start 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 0x7f3c40198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 0x7f3c4019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40111d80 con 0x7f3c40101ce0 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40111c00 con 0x7f3c401076f0 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c44e35640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40111f00 con 0x7f3c40108070 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45954/0 (socket says 172.21.15.120:45954) 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 -- 172.21.15.120:0/838188495 learned_addr learned my addr 172.21.15.120:0/838188495 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 0x7f3c4019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c37fff640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 0x7f3c40198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 -- 172.21.15.120:0/838188495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 msgr2=0x7f3c4019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 0x7f3c4019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 -- 172.21.15.120:0/838188495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 msgr2=0x7f3c40198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 0x7f3c40198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 -- 172.21.15.120:0/838188495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4019d770 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.414+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005ab0 tx=0x7f3c30003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.415+0000 7f3c377fe640 1 -- 172.21.15.120:0/838188495 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003f070 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.415+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4019da00 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.415+0000 7f3c377fe640 1 -- 172.21.15.120:0/838188495 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c30031d60 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.415+0000 7f3c377fe640 1 -- 172.21.15.120:0/838188495 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035030 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.415+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4019df30 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.417+0000 7f3c4720a740 1 Processor -- start 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.417+0000 7f3c4720a740 1 -- start start 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.417+0000 7f3c377fe640 1 -- 172.21.15.120:0/838188495 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3c3003a3c0 con 0x7f3c40101ce0 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x5605291fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529207410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529209d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560529171e50 con 0x5605291fb0a0 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560529161280 con 0x560529207950 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c4720a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560529165090 con 0x5605291fb9c0 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x5605291fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529209d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x5605291fb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45956/0 (socket says 172.21.15.120:45956) 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c3ed9d640 1 -- 172.21.15.120:0/940976525 learned_addr learned my addr 172.21.15.120:0/940976525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 -- 172.21.15.120:0/940976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 msgr2=0x560529207410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529207410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 -- 172.21.15.120:0/940976525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 msgr2=0x5605291fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x5605291fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 -- 172.21.15.120:0/940976525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560528ebf650 con 0x560529207950 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c37fff640 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529209d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c28005d00 tx=0x7f3c280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7a025c947dc9f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.418+0000 7f3c377fe640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 0x7f3c2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.419+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 0x7f3c2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.419+0000 7f3c34ff9640 1 -- 172.21.15.120:0/940976525 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2801c480 con 0x560529207950 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.419+0000 7f3c377fe640 1 -- 172.21.15.120:0/838188495 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f3c30106640 con 0x7f3c40101ce0 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.419+0000 7f3c34ff9640 1 -- 172.21.15.120:0/940976525 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2801cad0 con 0x560529207950 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.419+0000 7f3c34ff9640 1 -- 172.21.15.120:0/940976525 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28011ab0 con 0x560529207950 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- 172.21.15.120:0/940976525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 msgr2=0x560529209d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 0x7f3c2c0b77a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c38011630 tx=0x7f3c38009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529209d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c28005d00 tx=0x7f3c280107a0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- 172.21.15.120:0/940976525 shutdown_connections 2024-09-10T22:53:34.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529209d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 --2- 172.21.15.120:0/940976525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x5605291fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- 172.21.15.120:0/940976525 >> 172.21.15.120:0/940976525 conn(0x5605291f8ed0 msgr2=0x5605291fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- 172.21.15.120:0/940976525 shutdown_connections 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- 172.21.15.120:0/940976525 wait complete. 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.420+0000 7f3c4720a740 1 -- start start 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 Processor -- start 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 0x560529207260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529204ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560528fc2700 con 0x5605291fb9c0 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605290fc110 con 0x560529207950 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56052920a5d0 con 0x5605291fb0a0 2024-09-10T22:53:34.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c4720a740 10 client.0 _ll_register_callbacks cb 0x560529115990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45962/0 (socket says 172.21.15.120:45962) 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 -- 172.21.15.120:0/2652596852 learned_addr learned my addr 172.21.15.120:0/2652596852 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 0x560529207260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529204ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 -- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 msgr2=0x560529207260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 0x560529207260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 -- 172.21.15.120:0/2652596852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 msgr2=0x560529204ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529204ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c37fff640 1 -- 172.21.15.120:0/2652596852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560529129c40 con 0x5605291fb9c0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.421+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 0x560529207260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c28000c00 tx=0x7f3c2800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a4b8b3c021ea59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:34.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28004020 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f3c280041c0 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2801cad0 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c280043c0 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 10 client.15540 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560529215f30 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2801cad0 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3c28011cb0 con 0x5605291fb9c0 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 10 client.15540 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 20 client.15540 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.422+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529217280 0x560529219670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.423+0000 7f3c4720a740 4 client.15540 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.423+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560529173da0 con 0x560529217280 2024-09-10T22:53:34.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.423+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529217280 0x560529219670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.423+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529217280 0x560529219670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc19ec7ba39b2f1 server_cookie=cd8eae5bee7addef in_seq=0 out_seq=0 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.423+0000 7f3c25ffb640 10 client.15540 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2652596852 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560529173da0 con 0x560529217280 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c25ffb640 10 client.15540 handle_command_reply: tid=0 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 client.15540 shutdown 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 msgr2=0x5605292046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c28000c00 tx=0x7f3c2800bf50 comp rx=0 tx=0).stop 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 shutdown_connections 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529217280 0x560529219670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529204ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb9c0 0x5605292046b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2652596852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb0a0 0x560529207260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 >> 172.21.15.120:0/2652596852 conn(0x5605291f8ed0 msgr2=0x5605291fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c25ffb640 0 client.15540 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 shutdown_connections 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 1 -- 172.21.15.120:0/2652596852 wait complete. 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.424+0000 7f3c4720a740 20 client.15540 trim_cache size 0 max 16384 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 Processor -- start 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 -- start start 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529203470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529208d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b29a0 con 0x5605291fb0a0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b5220 con 0x560529207950 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c4720a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b3010 con 0x5605291fb9c0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45964/0 (socket says 172.21.15.120:45964) 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 -- 172.21.15.120:0/3693142768 learned_addr learned my addr 172.21.15.120:0/3693142768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c37fff640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529203470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529208d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 -- 172.21.15.120:0/3693142768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 msgr2=0x560529203470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529203470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 -- 172.21.15.120:0/3693142768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 msgr2=0x560529208d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529208d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 -- 172.21.15.120:0/3693142768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605291ff3c0 con 0x5605291fb0a0 2024-09-10T22:53:34.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3ed9d640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529208d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.442+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c300cce70 tx=0x7f3c30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de517d47f6ad8358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c27fff640 1 -- 172.21.15.120:0/3693142768 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30049040 con 0x5605291fb0a0 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c27fff640 1 -- 172.21.15.120:0/3693142768 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c300cc9f0 con 0x5605291fb0a0 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c27fff640 1 -- 172.21.15.120:0/3693142768 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300cd140 con 0x5605291fb0a0 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 -- 172.21.15.120:0/3693142768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 msgr2=0x560529202f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c300cce70 tx=0x7f3c30009260 comp rx=0 tx=0).stop 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 -- 172.21.15.120:0/3693142768 shutdown_connections 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560529207950 0x560529208d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x560529203470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.443+0000 7f3c4720a740 1 --2- 172.21.15.120:0/3693142768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605291fb0a0 0x560529202f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 -- 172.21.15.120:0/3693142768 >> 172.21.15.120:0/3693142768 conn(0x5605291f8ed0 msgr2=0x5605291ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 -- 172.21.15.120:0/3693142768 shutdown_connections 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 -- 172.21.15.120:0/3693142768 wait complete. 2024-09-10T22:53:34.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 -- start start 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 Processor -- start 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.444+0000 7f3c4720a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560529207950 0x56052920a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c4720a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b36e0 con 0x560529207950 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c4720a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b4b50 con 0x5605291fb0a0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c4720a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401b3db0 con 0x5605291fb9c0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560529207950 0x56052920a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c4720a740 10 client.0 _ll_register_callbacks cb 0x7f3c38009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58396/0 (socket says 172.21.15.120:58396) 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36228/0 (socket says 172.21.15.120:36228) 2024-09-10T22:53:34.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 -- 172.21.15.120:0/2462564886 learned_addr learned my addr 172.21.15.120:0/2462564886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 -- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 msgr2=0x56052920fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 -- 172.21.15.120:0/2462564886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560529207950 msgr2=0x56052920a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560529207950 0x56052920a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 -- 172.21.15.120:0/2462564886 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605292104f0 con 0x5605291fb0a0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c3e59c640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c301041c0 tx=0x7f3c3003b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d93160230d0abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30049070 con 0x5605291fb0a0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.445+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f3c300cd140 con 0x5605291fb0a0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c300cc9f0 con 0x5605291fb0a0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c300ccee0 con 0x5605291fb0a0 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 10 client.25034 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:34.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5605292843c0 con 0x5605291fb0a0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c300cc9f0 con 0x5605291fb0a0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3c300b40c0 con 0x5605291fb0a0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 10 client.25034 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 10 client.25034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 20 client.25034 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529284b40 0x560529286f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 4 client.25034 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56052916f850 con 0x560529284b40 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.446+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529284b40 0x560529286f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c37fff640 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529284b40 0x560529286f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f48f82d8218c00c server_cookie=3dc3f692a1d34226 in_seq=0 out_seq=0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c25ffb640 10 client.25034 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c25ffb640 1 -- 172.21.15.120:0/2462564886 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x56052916f850 con 0x560529284b40 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c25ffb640 10 client.25034 handle_command_reply: tid=0 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 client.25034 shutdown 2024-09-10T22:53:34.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 msgr2=0x56052920f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c301041c0 tx=0x7f3c3003b460 comp rx=0 tx=0).stop 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 shutdown_connections 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560529284b40 0x560529286f30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560529207950 0x56052920a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605291fb9c0 0x56052920fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c25ffb640 0 client.25034 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 --2- 172.21.15.120:0/2462564886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605291fb0a0 0x56052920f870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 >> 172.21.15.120:0/2462564886 conn(0x5605291f8ed0 msgr2=0x560529202b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 shutdown_connections 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 1 -- 172.21.15.120:0/2462564886 wait complete. 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.447+0000 7f3c4720a740 20 client.25034 trim_cache size 0 max 16384 2024-09-10T22:53:34.467 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:53:34.468 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:34.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 msgr2=0x7f3c2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 0x7f3c2c0b77a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c38011630 tx=0x7f3c38009500 comp rx=0 tx=0).stop 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 msgr2=0x7f3c401981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005ab0 tx=0x7f3c30003c80 comp rx=0 tx=0).stop 2024-09-10T22:53:34.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 shutdown_connections 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c2c0b52e0 0x7f3c2c0b77a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c40108070 0x7f3c4019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c401076f0 0x7f3c40198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 --2- 172.21.15.120:0/838188495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c40101ce0 0x7f3c401981e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 >> 172.21.15.120:0/838188495 conn(0x7f3c400fd730 msgr2=0x7f3c400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 shutdown_connections 2024-09-10T22:53:34.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:34.448+0000 7f3c44e35640 1 -- 172.21.15.120:0/838188495 wait complete. 2024-09-10T22:53:34.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 101815 19.44 MB/sec warmup 29 sec latency 37.994 ms 2024-09-10T22:53:35.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:53:35.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:53:35.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:35 smithi176 ceph-mon[34982]: pgmap v378: 97 pgs: 97 active+clean; 27 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T22:53:35.585 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:35.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 105300 19.42 MB/sec warmup 30 sec latency 19.613 ms 2024-09-10T22:53:35.660 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:35 smithi120 ceph-mon[31247]: pgmap v378: 97 pgs: 97 active+clean; 27 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T22:53:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:35 smithi153 ceph-mon[34951]: pgmap v378: 97 pgs: 97 active+clean; 27 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 -- 172.21.15.120:0/910636074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 msgr2=0x7f69b810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 --2- 172.21.15.120:0/910636074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b810f970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009ca0 tx=0x7f69b002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 -- 172.21.15.120:0/910636074 shutdown_connections 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 --2- 172.21.15.120:0/910636074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b810f970 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 --2- 172.21.15.120:0/910636074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 --2- 172.21.15.120:0/910636074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 -- 172.21.15.120:0/910636074 >> 172.21.15.120:0/910636074 conn(0x7f69b80fe830 msgr2=0x7f69b8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 -- 172.21.15.120:0/910636074 shutdown_connections 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.122+0000 7f69bef6c640 1 -- 172.21.15.120:0/910636074 wait complete. 2024-09-10T22:53:36.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 Processor -- start 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 -- start start 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112e30 con 0x7f69b8109170 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112cb0 con 0x7f69b81087f0 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bef6c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112fb0 con 0x7f69b8102de0 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bcd08640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bd509640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bcd08640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36236/0 (socket says 172.21.15.120:36236) 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69bcd08640 1 -- 172.21.15.120:0/3930698569 learned_addr learned my addr 172.21.15.120:0/3930698569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:36.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.123+0000 7f69b7fff640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 -- 172.21.15.120:0/3930698569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 msgr2=0x7f69b8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 -- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 msgr2=0x7f69b819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 -- 172.21.15.120:0/3930698569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b819f2d0 con 0x7f69b8102de0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bd509640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69b7fff640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bcd08640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00d950 tx=0x7f69ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac017030 con 0x7f69b8102de0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ac004020 con 0x7f69b8102de0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b819f5c0 con 0x7f69b8102de0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0047a0 con 0x7f69b8102de0 2024-09-10T22:53:36.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.124+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a5c80 con 0x7f69b8102de0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.125+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b81031c0 con 0x7f69b8102de0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.127+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f69ac009a30 con 0x7f69b8102de0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.128+0000 7f69b5ffb640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 0x7f69a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.128+0000 7f69b7fff640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 0x7f69a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.128+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f69ac0e5610 con 0x7f69b8102de0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.129+0000 7f69b7fff640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 0x7f69a80b77a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69b8102040 tx=0x7f69a400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:36.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.133+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f69ac0ac500 con 0x7f69b8102de0 2024-09-10T22:53:36.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.361+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69b802e5c0 con 0x7f69b8102de0 2024-09-10T22:53:36.364 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.364 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.364 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.364 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.364 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:36 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.362+0000 7f69b5ffb640 1 -- 172.21.15.120:0/3930698569 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11868 (secure 0 0 0) 0x7f69ac0abe00 con 0x7f69b8102de0 2024-09-10T22:53:36.365 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:53:36.366 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":21,"btime":"2024-09-10T22:53:19:261525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:53:19.261509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24998":{"gid":24998,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/4005968272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":4005968272},{"type":"v1","addr":"172.21.15.176:6837","nonce":4005968272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:53:36.366 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 21 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 msgr2=0x7f69a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 0x7f69a80b77a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69b8102040 tx=0x7f69a400b410 comp rx=0 tx=0).stop 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 msgr2=0x7f69b8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00d950 tx=0x7f69ac00de10 comp rx=0 tx=0).stop 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 shutdown_connections 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69a80b52e0 0x7f69a80b77a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69b8109170 0x7f69b819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69b81087f0 0x7f69b8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 --2- 172.21.15.120:0/3930698569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69b8102de0 0x7f69b8199220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 >> 172.21.15.120:0/3930698569 conn(0x7f69b80fe830 msgr2=0x7f69b8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 shutdown_connections 2024-09-10T22:53:36.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:36.364+0000 7f69bef6c640 1 -- 172.21.15.120:0/3930698569 wait complete. 2024-09-10T22:53:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:36 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 108072 19.35 MB/sec warmup 31 sec latency 29.473 ms 2024-09-10T22:53:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:36 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:37.218 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:53:37.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:37 smithi120 ceph-mon[31247]: pgmap v379: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-09-10T22:53:37.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:37.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:37 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3930698569' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:37.474 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:37 smithi176 ceph-mon[34982]: pgmap v379: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-09-10T22:53:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:37 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3930698569' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:37.613 INFO:tasks.workunit.client.0.smithi120.stdout: 1 111209 19.25 MB/sec warmup 32 sec latency 18.945 ms 2024-09-10T22:53:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:37 smithi153 ceph-mon[34951]: pgmap v379: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-09-10T22:53:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:53:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:37 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3930698569' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3473999757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee881021b0 msgr2=0x7fee8810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3473999757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee881021b0 0x7fee8810ef00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009900 tx=0x7fee7c02a480 comp rx=0 tx=0).stop 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3473999757 shutdown_connections 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3473999757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 0x7fee88111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3473999757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee881021b0 0x7fee8810ef00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3473999757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee88101890 0x7fee88101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3473999757 >> 172.21.15.120:0/3473999757 conn(0x7fee880fd700 msgr2=0x7fee880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3473999757 shutdown_connections 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3473999757 wait complete. 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 Processor -- start 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 -- start start 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee881021b0 0x7fee8819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.972+0000 7fee8fd5a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 0x7fee881a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8fd5a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee88114f40 con 0x7fee88101890 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8fd5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee88114dc0 con 0x7fee881021b0 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8fd5a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee881150c0 con 0x7fee8810f440 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46008/0 (socket says 172.21.15.120:46008) 2024-09-10T22:53:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 -- 172.21.15.120:0/3492212726 learned_addr learned my addr 172.21.15.120:0/3492212726 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8d2f5640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee881021b0 0x7fee8819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 0x7fee881a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 -- 172.21.15.120:0/3492212726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 msgr2=0x7fee881a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 0x7fee881a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 -- 172.21.15.120:0/3492212726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee881021b0 msgr2=0x7fee8819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee881021b0 0x7fee8819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 -- 172.21.15.120:0/3492212726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee881a1580 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee7800ebc0 tx=0x7fee7800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee86ffd640 1 -- 172.21.15.120:0/3492212726 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78010630 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee881a1870 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee86ffd640 1 -- 172.21.15.120:0/3492212726 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78010c80 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee86ffd640 1 -- 172.21.15.120:0/3492212726 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78018b60 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.973+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881a8100 con 0x7fee88101890 2024-09-10T22:53:37.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 Processor -- start 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 -- start start 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee86ffd640 1 -- 172.21.15.120:0/3492212726 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fee780107d0 con 0x7fee88101890 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563435634ea0 con 0x5634356bea10 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634356242d0 con 0x5634356be0f0 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee9212f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5634356280e0 con 0x5634356ca9a0 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee8e2f7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46022/0 (socket says 172.21.15.120:46022) 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee8daf6640 1 -- 172.21.15.120:0/3205968208 learned_addr learned my addr 172.21.15.120:0/3205968208 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:37.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.976+0000 7fee8d2f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 -- 172.21.15.120:0/3205968208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 msgr2=0x5634356ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 -- 172.21.15.120:0/3205968208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 msgr2=0x5634356be4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356be4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 -- 172.21.15.120:0/3205968208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563435382650 con 0x5634356bea10 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee780ee480 tx=0x7fee780ee940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3207742a46ce9c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee6ffff640 1 -- 172.21.15.120:0/3205968208 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780f9070 con 0x5634356bea10 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee6ffff640 1 -- 172.21.15.120:0/3205968208 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee780e6ad0 con 0x5634356bea10 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee6ffff640 1 -- 172.21.15.120:0/3205968208 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780f49e0 con 0x5634356bea10 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee86ffd640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 0x7fee700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 0x7fee700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.977+0000 7fee86ffd640 1 -- 172.21.15.120:0/3492212726 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fee780e6170 con 0x7fee88101890 2024-09-10T22:53:37.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- 172.21.15.120:0/3205968208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 msgr2=0x5634356ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee780ee480 tx=0x7fee780ee940 comp rx=0 tx=0).stop 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 0x7fee700b77a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee80005d00 tx=0x7fee8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- 172.21.15.120:0/3205968208 shutdown_connections 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356ca460 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 --2- 172.21.15.120:0/3205968208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356be4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- 172.21.15.120:0/3205968208 >> 172.21.15.120:0/3205968208 conn(0x5634356bbf20 msgr2=0x5634356bd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- 172.21.15.120:0/3205968208 shutdown_connections 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- 172.21.15.120:0/3205968208 wait complete. 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 -- start start 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.978+0000 7fee9212f740 1 Processor -- start 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563435485700 con 0x5634356ca9a0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5634355bf160 con 0x5634356be0f0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5634356cd620 con 0x5634356bea10 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee9212f740 10 client.0 _ll_register_callbacks cb 0x5634355d89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8d2f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46032/0 (socket says 172.21.15.120:46032) 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 -- 172.21.15.120:0/3046835785 learned_addr learned my addr 172.21.15.120:0/3046835785 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 -- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 msgr2=0x5634356c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 -- 172.21.15.120:0/3046835785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 msgr2=0x5634356ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 -- 172.21.15.120:0/3046835785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634355ecc90 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fee80003c10 tx=0x7fee80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ba45a3bb7c68dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.979+0000 7fee6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8001fd10 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee780e6ad0 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fee80017030 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee800047a0 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 10 client.15564 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5634356d90e0 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee780e6ad0 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee800055c0 con 0x5634356ca9a0 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 10 client.15564 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:37.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 20 client.15564 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5634356da590 0x5634356dc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 4 client.15564 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563435636df0 con 0x5634356da590 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.980+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5634356da590 0x5634356dc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.981+0000 7fee8daf6640 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5634356da590 0x5634356dc980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d77b4fd6670e4c server_cookie=f73f0dee3c5ee231 in_seq=0 out_seq=0 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.981+0000 7fee6d7fa640 10 client.15564 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.981+0000 7fee6d7fa640 1 -- 172.21.15.120:0/3046835785 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563435636df0 con 0x5634356da590 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.981+0000 7fee6d7fa640 10 client.15564 handle_command_reply: tid=0 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.981+0000 7fee9212f740 1 client.15564 shutdown 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 msgr2=0x5634356c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fee80003c10 tx=0x7fee80004730 comp rx=0 tx=0).stop 2024-09-10T22:53:37.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 shutdown_connections 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5634356da590 0x5634356dc980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356ca9a0 0x5634356c7e40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee6d7fa640 0 client.15564 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 --2- 172.21.15.120:0/3046835785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 >> 172.21.15.120:0/3046835785 conn(0x5634356bbf20 msgr2=0x5634356c1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 shutdown_connections 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 1 -- 172.21.15.120:0/3046835785 wait complete. 2024-09-10T22:53:37.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.982+0000 7fee9212f740 20 client.15564 trim_cache size 0 max 16384 2024-09-10T22:53:38.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 Processor -- start 2024-09-10T22:53:38.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 -- start start 2024-09-10T22:53:38.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 0x5634356c6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ae590 con 0x5634356be0f0 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b0e10 con 0x5634356ca9a0 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee9212f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee881aec00 con 0x5634356bea10 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46046/0 (socket says 172.21.15.120:46046) 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 -- 172.21.15.120:0/2003641161 learned_addr learned my addr 172.21.15.120:0/2003641161 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8d2f5640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 0x5634356c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 -- 172.21.15.120:0/2003641161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 msgr2=0x5634356c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 -- 172.21.15.120:0/2003641161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 msgr2=0x5634356c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 0x5634356c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 -- 172.21.15.120:0/2003641161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634356b77e0 con 0x5634356be0f0 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 0x5634356c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.996+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fee780e9c90 tx=0x7fee780f44a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9908053156fc3d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee6f7fe640 1 -- 172.21.15.120:0/2003641161 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780a9060 con 0x5634356be0f0 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee6f7fe640 1 -- 172.21.15.120:0/2003641161 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee780e9a20 con 0x5634356be0f0 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee6f7fe640 1 -- 172.21.15.120:0/2003641161 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780a8180 con 0x5634356be0f0 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 -- 172.21.15.120:0/2003641161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 msgr2=0x5634356bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fee780e9c90 tx=0x7fee780f44a0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 -- 172.21.15.120:0/2003641161 shutdown_connections 2024-09-10T22:53:38.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356ca9a0 0x5634356c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356bea10 0x5634356c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.997+0000 7fee9212f740 1 --2- 172.21.15.120:0/2003641161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356be0f0 0x5634356bb760 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 -- 172.21.15.120:0/2003641161 >> 172.21.15.120:0/2003641161 conn(0x5634356bbf20 msgr2=0x5634356c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 -- 172.21.15.120:0/2003641161 shutdown_connections 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 -- 172.21.15.120:0/2003641161 wait complete. 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 -- start start 2024-09-10T22:53:38.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 Processor -- start 2024-09-10T22:53:38.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.998+0000 7fee9212f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee9212f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee9212f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee881af2d0 con 0x5634356bea10 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee9212f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b0740 con 0x5634356be0f0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee9212f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee881af9a0 con 0x5634356ca9a0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee9212f740 10 client.0 _ll_register_callbacks cb 0x7fee8000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36270/0 (socket says 172.21.15.120:36270) 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 -- 172.21.15.120:0/2967630659 learned_addr learned my addr 172.21.15.120:0/2967630659 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8daf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58472/0 (socket says 172.21.15.120:58472) 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8d2f5640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 -- 172.21.15.120:0/2967630659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 msgr2=0x5634356d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 -- 172.21.15.120:0/2967630659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 msgr2=0x5634356d2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356d2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 -- 172.21.15.120:0/2967630659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634356bc9a0 con 0x5634356ca9a0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8d2f5640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee8e2f7640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee80033df0 tx=0x7fee80033e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaeb1f30b8ef9207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:37.999+0000 7fee6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80004b50 con 0x5634356ca9a0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fee80004cf0 con 0x5634356ca9a0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee780e9a20 con 0x5634356ca9a0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee80005040 con 0x5634356ca9a0 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 10 client.25042 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:38.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5634357477b0 con 0x5634356ca9a0 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee780e9a20 con 0x5634356ca9a0 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee800117b0 con 0x5634356ca9a0 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 10 client.25042 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 10 client.25042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 20 client.25042 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563435747f60 0x56343574a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 4 client.25042 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.000+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634356328a0 con 0x563435747f60 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563435747f60 0x56343574a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee6d7fa640 10 client.25042 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee8daf6640 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563435747f60 0x56343574a380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f732679a3fb062f server_cookie=6610d84f1f90a4c6 in_seq=0 out_seq=0 2024-09-10T22:53:38.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee6d7fa640 1 -- 172.21.15.120:0/2967630659 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634356328a0 con 0x563435747f60 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee6d7fa640 10 client.25042 handle_command_reply: tid=0 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 client.25042 shutdown 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 msgr2=0x5634356cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee80033df0 tx=0x7fee80033e20 comp rx=0 tx=0).stop 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 shutdown_connections 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563435747f60 0x56343574a380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5634356ca9a0 0x5634356cd8a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5634356bea10 0x5634356d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 --2- 172.21.15.120:0/2967630659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5634356be0f0 0x5634356d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee6d7fa640 0 client.25042 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 >> 172.21.15.120:0/2967630659 conn(0x5634356bbf20 msgr2=0x5634356c5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.001+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 shutdown_connections 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee9212f740 1 -- 172.21.15.120:0/2967630659 wait complete. 2024-09-10T22:53:38.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee9212f740 20 client.25042 trim_cache size 0 max 16384 2024-09-10T22:53:38.011 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 msgr2=0x7fee700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 0x7fee700b77a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee80005d00 tx=0x7fee8000b390 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 msgr2=0x7fee8819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee7800ebc0 tx=0x7fee7800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 shutdown_connections 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee700b52e0 0x7fee700b77a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee8810f440 0x7fee881a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee881021b0 0x7fee8819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 --2- 172.21.15.120:0/3492212726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee88101890 0x7fee8819b4d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 >> 172.21.15.120:0/3492212726 conn(0x7fee880fd700 msgr2=0x7fee880ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 shutdown_connections 2024-09-10T22:53:38.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:38.002+0000 7fee8fd5a640 1 -- 172.21.15.120:0/3492212726 wait complete. 2024-09-10T22:53:38.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 114728 19.24 MB/sec warmup 33 sec latency 12.628 ms 2024-09-10T22:53:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:39 smithi176 ceph-mon[34982]: pgmap v380: 97 pgs: 97 active+clean; 23 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 432 op/s 2024-09-10T22:53:39.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 118023 19.20 MB/sec warmup 34 sec latency 14.299 ms 2024-09-10T22:53:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:39 smithi120 ceph-mon[31247]: pgmap v380: 97 pgs: 97 active+clean; 23 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 432 op/s 2024-09-10T22:53:39.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:39 smithi153 ceph-mon[34951]: pgmap v380: 97 pgs: 97 active+clean; 23 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 432 op/s 2024-09-10T22:53:39.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:53:39.827 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:53:40.084 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:40.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 121355 19.18 MB/sec warmup 35 sec latency 15.759 ms 2024-09-10T22:53:40.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 -- 172.21.15.120:0/2756416990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2001076a0 msgr2=0x7fe200107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 --2- 172.21.15.120:0/2756416990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2001076a0 0x7fe200107ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 -- 172.21.15.120:0/2756416990 shutdown_connections 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 --2- 172.21.15.120:0/2756416990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 0x7fe20010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 --2- 172.21.15.120:0/2756416990 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2001076a0 0x7fe200107ae0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 --2- 172.21.15.120:0/2756416990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe200101c90 0x7fe200102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.612+0000 7fe207025640 1 -- 172.21.15.120:0/2756416990 >> 172.21.15.120:0/2756416990 conn(0x7fe2000fd700 msgr2=0x7fe2000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- 172.21.15.120:0/2756416990 shutdown_connections 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- 172.21.15.120:0/2756416990 wait complete. 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 Processor -- start 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- start start 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2001076a0 0x7fe2000fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 0x7fe2000fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe200111ce0 con 0x7fe200101c90 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe200111b60 con 0x7fe200108020 2024-09-10T22:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe207025640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe200111e60 con 0x7fe2001076a0 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46082/0 (socket says 172.21.15.120:46082) 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe204dc1640 1 -- 172.21.15.120:0/2834603610 learned_addr learned my addr 172.21.15.120:0/2834603610 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 0x7fe2000fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.613+0000 7fe1fffff640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2001076a0 0x7fe2000fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 -- 172.21.15.120:0/2834603610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2001076a0 msgr2=0x7fe2000fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2001076a0 0x7fe2000fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 -- 172.21.15.120:0/2834603610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 msgr2=0x7fe2000fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 0x7fe2000fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 -- 172.21.15.120:0/2834603610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2001a7690 con 0x7fe200101c90 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe204dc1640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e9f0 tx=0x7fe1f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe1fdffb640 1 -- 172.21.15.120:0/2834603610 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000cd80 con 0x7fe200101c90 2024-09-10T22:53:40.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe1fdffb640 1 -- 172.21.15.120:0/2834603610 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f00054d0 con 0x7fe200101c90 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2001a7980 con 0x7fe200101c90 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe1fdffb640 1 -- 172.21.15.120:0/2834603610 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010690 con 0x7fe200101c90 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.614+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2001a7e40 con 0x7fe200101c90 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe1fdffb640 1 -- 172.21.15.120:0/2834603610 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe1f0004e90 con 0x7fe200101c90 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe2093fa740 1 Processor -- start 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe2093fa740 1 -- start start 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 0x55c9fdafc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.617+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2093fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9fda72e60 con 0x55c9fdafc800 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2093fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9fda62290 con 0x55c9fdb08790 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2093fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c9fda660a0 con 0x55c9fdafbee0 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 0x55c9fdafc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46092/0 (socket says 172.21.15.120:46092) 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe204dc1640 1 -- 172.21.15.120:0/2250289221 learned_addr learned my addr 172.21.15.120:0/2250289221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe1fffff640 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe1fdffb640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 0x7fe1e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 -- 172.21.15.120:0/2250289221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 msgr2=0x55c9fdb0ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb0ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 -- 172.21.15.120:0/2250289221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 msgr2=0x55c9fdb08250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 -- 172.21.15.120:0/2250289221 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fd7c0930 con 0x55c9fdafbee0 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe204dc1640 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 0x7fe1e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe1fdffb640 1 -- 172.21.15.120:0/2834603610 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe1f00e5fe0 con 0x7fe200101c90 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.618+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 0x55c9fdafc2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8011490 tx=0x7fe1f8011890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53dc7c74116cb226 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe1e6ffd640 1 -- 172.21.15.120:0/2250289221 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8017070 con 0x55c9fdafbee0 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe1e6ffd640 1 -- 172.21.15.120:0/2250289221 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f800fb30 con 0x55c9fdafbee0 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe1e6ffd640 1 -- 172.21.15.120:0/2250289221 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f801ba10 con 0x55c9fdafbee0 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2055c2640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 0x7fe1e80b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8022000 tx=0x7fe1f8023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 -- 172.21.15.120:0/2250289221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 msgr2=0x55c9fdafc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 0x55c9fdafc2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8011490 tx=0x7fe1f8011890 comp rx=0 tx=0).stop 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 -- 172.21.15.120:0/2250289221 shutdown_connections 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb0ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafc800 0x55c9fdb08250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 --2- 172.21.15.120:0/2250289221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafbee0 0x55c9fdafc2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 -- 172.21.15.120:0/2250289221 >> 172.21.15.120:0/2250289221 conn(0x55c9fdaf9d10 msgr2=0x55c9fdafaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 -- 172.21.15.120:0/2250289221 shutdown_connections 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.619+0000 7fe2093fa740 1 -- 172.21.15.120:0/2250289221 wait complete. 2024-09-10T22:53:40.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 -- start start 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 Processor -- start 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb05c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb06160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd8c3890 con 0x55c9fdafbee0 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd9fcd40 con 0x55c9fdb08790 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c9fdb0b440 con 0x55c9fdafc800 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2093fa740 10 client.0 _ll_register_callbacks cb 0x55c9fda60c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46102/0 (socket says 172.21.15.120:46102) 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe204dc1640 1 -- 172.21.15.120:0/3385460899 learned_addr learned my addr 172.21.15.120:0/3385460899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe1fffff640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb05c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.620+0000 7fe2055c2640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb06160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 -- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 msgr2=0x55c9fdb05c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb05c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 -- 172.21.15.120:0/3385460899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 msgr2=0x55c9fdb06160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb06160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 -- 172.21.15.120:0/3385460899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fdb147a0 con 0x55c9fdafbee0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097d0 tx=0x7fe1f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533eaa8a216345e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00abee0 con 0x55c9fdafbee0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f800fb30 con 0x55c9fdafbee0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe1f00acbd0 con 0x55c9fdafbee0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe1f00a8610 con 0x55c9fdafbee0 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe2093fa740 10 client.15582 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:40.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c9fdb17150 con 0x55c9fdafbee0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f800fb30 con 0x55c9fdafbee0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.621+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe1f00c3080 con 0x55c9fdafbee0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 10 client.15582 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 20 client.15582 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb186b0 0x55c9fdb1aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 4 client.15582 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9fda74db0 con 0x55c9fdb186b0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe1fffff640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb186b0 0x55c9fdb1aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe1dffff640 10 client.15582 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.622+0000 7fe1fffff640 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb186b0 0x55c9fdb1aac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe7226f4f6f278f4 server_cookie=94ffb7376caec0ec in_seq=0 out_seq=0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe1dffff640 1 -- 172.21.15.120:0/3385460899 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9fda74db0 con 0x55c9fdb186b0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe1dffff640 10 client.15582 handle_command_reply: tid=0 2024-09-10T22:53:40.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 client.15582 shutdown 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 msgr2=0x55c9fdb056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097d0 tx=0x7fe1f00048f0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 shutdown_connections 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb186b0 0x55c9fdb1aac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdb08790 0x55c9fdb06160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb05c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3385460899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb056e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 >> 172.21.15.120:0/3385460899 conn(0x55c9fdaf9d10 msgr2=0x55c9fdaf8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe1dffff640 0 client.15582 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 shutdown_connections 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.623+0000 7fe2093fa740 1 -- 172.21.15.120:0/3385460899 wait complete. 2024-09-10T22:53:40.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.624+0000 7fe2093fa740 20 client.15582 trim_cache size 0 max 16384 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 Processor -- start 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 -- start start 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 0x55c9fdb028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdb08790 0x55c9fdb02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001ae3d0 con 0x55c9fdafbee0 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001b0c50 con 0x55c9fdafc800 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2093fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001aea40 con 0x55c9fdb08790 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe204dc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb07420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46108/0 (socket says 172.21.15.120:46108) 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe204dc1640 1 -- 172.21.15.120:0/296500453 learned_addr learned my addr 172.21.15.120:0/296500453 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe1fffff640 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 0x55c9fdb028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.641+0000 7fe2055c2640 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdb08790 0x55c9fdb02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 -- 172.21.15.120:0/296500453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdb08790 msgr2=0x55c9fdb02e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdb08790 0x55c9fdb02e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 -- 172.21.15.120:0/296500453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 msgr2=0x55c9fdb07420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb07420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 -- 172.21.15.120:0/296500453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fdafa790 con 0x55c9fdafc800 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1fffff640 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 0x55c9fdb028f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009230 tx=0x7fe1f4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f55d7ac7c40344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1e67fc640 1 -- 172.21.15.120:0/296500453 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4035040 con 0x55c9fdafc800 2024-09-10T22:53:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1e67fc640 1 -- 172.21.15.120:0/296500453 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f40054f0 con 0x55c9fdafc800 2024-09-10T22:53:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.642+0000 7fe1e67fc640 1 -- 172.21.15.120:0/296500453 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4004d90 con 0x55c9fdafc800 2024-09-10T22:53:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- 172.21.15.120:0/296500453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 msgr2=0x55c9fdb028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 0x55c9fdb028f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009230 tx=0x7fe1f4005db0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- 172.21.15.120:0/296500453 shutdown_connections 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdb08790 0x55c9fdb02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafc800 0x55c9fdb028f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 --2- 172.21.15.120:0/296500453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdafbee0 0x55c9fdb07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- 172.21.15.120:0/296500453 >> 172.21.15.120:0/296500453 conn(0x55c9fdaf9d10 msgr2=0x55c9fdb06d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- 172.21.15.120:0/296500453 shutdown_connections 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- 172.21.15.120:0/296500453 wait complete. 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 -- start start 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.643+0000 7fe2093fa740 1 Processor -- start 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafbee0 0x55c9fdb0cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb0d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001af110 con 0x55c9fdb08790 2024-09-10T22:53:40.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001b0580 con 0x55c9fdafbee0 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe2001af7e0 con 0x55c9fdafc800 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2093fa740 10 client.0 _ll_register_callbacks cb 0x7fe1f000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46118/0 (socket says 172.21.15.120:46118) 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb0d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 -- 172.21.15.120:0/3359594266 learned_addr learned my addr 172.21.15.120:0/3359594266 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafbee0 0x55c9fdb0cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 -- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 msgr2=0x55c9fdb0d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb0d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 -- 172.21.15.120:0/3359594266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafbee0 msgr2=0x55c9fdb0cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafbee0 0x55c9fdb0cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 -- 172.21.15.120:0/3359594266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fdb84ad0 con 0x55c9fdb08790 2024-09-10T22:53:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe2055c2640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8011d90 tx=0x7fe1f801b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6181075d3639a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.644+0000 7fe1dffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8017070 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe1f800f680 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f40054f0 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe1f8009c30 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 10 client.15588 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c9fdb85df0 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f40054f0 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe1f8040460 con 0x55c9fdb08790 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 10 client.15588 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 20 client.15588 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb865c0 0x55c9fdb889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 4 client.15588 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9fda70860 con 0x55c9fdb865c0 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.645+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb865c0 0x55c9fdb889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.646+0000 7fe204dc1640 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb865c0 0x55c9fdb889e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2698e9976a6fd2 server_cookie=a7a08d371caeb008 in_seq=0 out_seq=0 2024-09-10T22:53:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.646+0000 7fe1dffff640 10 client.15588 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe1dffff640 1 -- 172.21.15.120:0/3359594266 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9fda70860 con 0x55c9fdb865c0 2024-09-10T22:53:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe1dffff640 10 client.15588 handle_command_reply: tid=0 2024-09-10T22:53:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 client.15588 shutdown 2024-09-10T22:53:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 msgr2=0x55c9fdb84590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8011d90 tx=0x7fe1f801b610 comp rx=0 tx=0).stop 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 shutdown_connections 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fdb865c0 0x55c9fdb889e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fdb08790 0x55c9fdb84590 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe1dffff640 0 client.15588 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fdafc800 0x55c9fdb0d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 --2- 172.21.15.120:0/3359594266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fdafbee0 0x55c9fdb0cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 >> 172.21.15.120:0/3359594266 conn(0x55c9fdaf9d10 msgr2=0x55c9fdb05e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 shutdown_connections 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 1 -- 172.21.15.120:0/3359594266 wait complete. 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.649+0000 7fe2093fa740 20 client.15588 trim_cache size 0 max 16384 2024-09-10T22:53:40.654 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "363bfa6b-82dd-4066-8117-7d10a3d87602", 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 msgr2=0x7fe1e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 0x7fe1e80b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8022000 tx=0x7fe1f8023000 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 msgr2=0x7fe2000fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e9f0 tx=0x7fe1f000edf0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 shutdown_connections 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1e80b52e0 0x7fe1e80b77a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe200108020 0x7fe2000fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2001076a0 0x7fe2000fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 --2- 172.21.15.120:0/2834603610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe200101c90 0x7fe2000fdc00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 >> 172.21.15.120:0/2834603610 conn(0x7fe2000fd700 msgr2=0x7fe20006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 shutdown_connections 2024-09-10T22:53:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:40.650+0000 7fe207025640 1 -- 172.21.15.120:0/2834603610 wait complete. 2024-09-10T22:53:41.533 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:41 smithi120 ceph-mon[31247]: pgmap v381: 97 pgs: 97 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.5 MiB/s wr, 375 op/s 2024-09-10T22:53:41.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:41 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:53:41.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:41 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:53:41.534 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.561 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:53:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:41 smithi176 ceph-mon[34982]: pgmap v381: 97 pgs: 97 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.5 MiB/s wr, 375 op/s 2024-09-10T22:53:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:41 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:53:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:41 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:53:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 124635 19.17 MB/sec warmup 36 sec latency 20.391 ms 2024-09-10T22:53:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:41 smithi153 ceph-mon[34951]: pgmap v381: 97 pgs: 97 active+clean; 32 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.5 MiB/s wr, 375 op/s 2024-09-10T22:53:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:41 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:53:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:41 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:53:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:41.835 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:42.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 -- 172.21.15.120:0/2302206100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee4101cb0 msgr2=0x7feee4102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/2302206100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee4101cb0 0x7feee4102090 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 -- 172.21.15.120:0/2302206100 shutdown_connections 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/2302206100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4108040 0x7feee410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/2302206100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee41076c0 0x7feee4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/2302206100 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee4101cb0 0x7feee4102090 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.372+0000 7feeeb7f4640 1 -- 172.21.15.120:0/2302206100 >> 172.21.15.120:0/2302206100 conn(0x7feee40fd700 msgr2=0x7feee40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- 172.21.15.120:0/2302206100 shutdown_connections 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- 172.21.15.120:0/2302206100 wait complete. 2024-09-10T22:53:42.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 Processor -- start 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- start start 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 0x7feee40fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee4108040 0x7feee40fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111d00 con 0x7feee41076c0 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111b80 con 0x7feee4101cb0 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feeeb7f4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111e80 con 0x7feee4108040 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.373+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46138/0 (socket says 172.21.15.120:46138) 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 -- 172.21.15.120:0/3518034215 learned_addr learned my addr 172.21.15.120:0/3518034215 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee9d91640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee4108040 0x7feee40fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee9590640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 0x7feee40fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 -- 172.21.15.120:0/3518034215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee4108040 msgr2=0x7feee40fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee4108040 0x7feee40fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 -- 172.21.15.120:0/3518034215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 msgr2=0x7feee40fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 0x7feee40fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 -- 172.21.15.120:0/3518034215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee41a76b0 con 0x7feee41076c0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee9590640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 0x7feee40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feee8d8f640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feeda7fc640 1 -- 172.21.15.120:0/3518034215 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed400cd80 con 0x7feee41076c0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee41a79a0 con 0x7feee41076c0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feeda7fc640 1 -- 172.21.15.120:0/3518034215 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed40054d0 con 0x7feee41076c0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feeda7fc640 1 -- 172.21.15.120:0/3518034215 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4010690 con 0x7feee41076c0 2024-09-10T22:53:42.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.374+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a7e90 con 0x7feee41076c0 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeda7fc640 1 -- 172.21.15.120:0/3518034215 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7feed4004e90 con 0x7feee41076c0 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 Processor -- start 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 -- start start 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 0x55d40205c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d401fc6e50 con 0x55d402050480 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d401fb6280 con 0x55d402050da0 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feeedbc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d401fba090 con 0x55d40205cd30 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feee9d91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feee9d91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46142/0 (socket says 172.21.15.120:46142) 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feee9590640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 0x55d40205c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feee9d91640 1 -- 172.21.15.120:0/3059176581 learned_addr learned my addr 172.21.15.120:0/3059176581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.377+0000 7feee8d8f640 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 -- 172.21.15.120:0/3059176581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 msgr2=0x55d40205f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 -- 172.21.15.120:0/3059176581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 msgr2=0x55d40205c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 0x55d40205c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 -- 172.21.15.120:0/3059176581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d401d14650 con 0x55d402050480 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9590640 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 0x55d40205c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feeda7fc640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 0x7feed00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9590640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 0x7feed00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feee9d91640 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feedc003970 tx=0x7feedc003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e7c65d4d8770b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feeda7fc640 1 -- 172.21.15.120:0/3518034215 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7feed40e5f90 con 0x7feee41076c0 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feecf7fe640 1 -- 172.21.15.120:0/3059176581 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc01a470 con 0x55d402050480 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feecf7fe640 1 -- 172.21.15.120:0/3059176581 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc015070 con 0x55d402050480 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.378+0000 7feecf7fe640 1 -- 172.21.15.120:0/3059176581 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0118e0 con 0x55d402050480 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- 172.21.15.120:0/3059176581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 msgr2=0x55d402050860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feee9590640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 0x7feed00b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feee0031440 tx=0x7feee0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feedc003970 tx=0x7feedc003e30 comp rx=0 tx=0).stop 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- 172.21.15.120:0/3059176581 shutdown_connections 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050da0 0x55d40205c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 --2- 172.21.15.120:0/3059176581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402050860 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- 172.21.15.120:0/3059176581 >> 172.21.15.120:0/3059176581 conn(0x55d40204e2b0 msgr2=0x55d40204f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- 172.21.15.120:0/3059176581 shutdown_connections 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- 172.21.15.120:0/3059176581 wait complete. 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 -- start start 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.379+0000 7feeedbc9740 1 Processor -- start 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050480 0x55d40205c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d401e17700 con 0x55d402050da0 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d401f51110 con 0x55d402050480 2024-09-10T22:53:42.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d40205f9b0 con 0x55d40205cd30 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeedbc9740 10 client.0 _ll_register_callbacks cb 0x55d401f6a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee9d91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46148/0 (socket says 172.21.15.120:46148) 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 -- 172.21.15.120:0/625131055 learned_addr learned my addr 172.21.15.120:0/625131055 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee9590640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050480 0x55d40205c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 -- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 msgr2=0x55d40205a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 -- 172.21.15.120:0/625131055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050480 msgr2=0x55d40205c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050480 0x55d40205c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 -- 172.21.15.120:0/625131055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d401f7ec40 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee9d91640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feee8d8f640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feed400c6a0 tx=0x7feed40e9630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aadeef437909a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.380+0000 7feeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40ac5f0 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7feed40e9c70 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feedc015070 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feed40abf90 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 10 client.15600 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d40206b370 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc015070 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feed40e80d0 con 0x55d402050da0 2024-09-10T22:53:42.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 10 client.15600 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 20 client.15600 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d40206c8d0 0x55d40206ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 4 client.15600 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d401fc8da0 con 0x55d40206c8d0 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.381+0000 7feee9590640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d40206c8d0 0x55d40206ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.382+0000 7feee9590640 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d40206c8d0 0x55d40206ecc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3231995ae3d09c17 server_cookie=9d14ed571d855f0 in_seq=0 out_seq=0 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.382+0000 7feeccff9640 10 client.15600 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.382+0000 7feeccff9640 1 -- 172.21.15.120:0/625131055 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d401fc8da0 con 0x55d40206c8d0 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.382+0000 7feeccff9640 10 client.15600 handle_command_reply: tid=0 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.382+0000 7feeedbc9740 1 client.15600 shutdown 2024-09-10T22:53:42.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 msgr2=0x55d402059aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feed400c6a0 tx=0x7feed40e9630 comp rx=0 tx=0).stop 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 shutdown_connections 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d40206c8d0 0x55d40206ecc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d40205cd30 0x55d40205a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeccff9640 0 client.15600 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050da0 0x55d402059aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 --2- 172.21.15.120:0/625131055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d402050480 0x55d40205c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 >> 172.21.15.120:0/625131055 conn(0x55d40204e2b0 msgr2=0x55d402053e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 shutdown_connections 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 1 -- 172.21.15.120:0/625131055 wait complete. 2024-09-10T22:53:42.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.383+0000 7feeedbc9740 20 client.15600 trim_cache size 0 max 16384 2024-09-10T22:53:42.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 Processor -- start 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 -- start start 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d40204d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ae450 con 0x55d402050480 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feee41b0cd0 con 0x55d40205cd30 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feeedbc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feee41aeac0 con 0x55d402050da0 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feee9590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.400+0000 7feee9590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46156/0 (socket says 172.21.15.120:46156) 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 -- 172.21.15.120:0/1393548373 learned_addr learned my addr 172.21.15.120:0/1393548373 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:42.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee8d8f640 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d40204d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9d91640 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 -- 172.21.15.120:0/1393548373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 msgr2=0x55d40204d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d40204d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 -- 172.21.15.120:0/1393548373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 msgr2=0x55d40205e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 -- 172.21.15.120:0/1393548373 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d402059850 con 0x55d402050480 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feee9590640 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feee002a910 tx=0x7feee003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f988f9213949bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feeceffd640 1 -- 172.21.15.120:0/1393548373 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0035040 con 0x55d402050480 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feeceffd640 1 -- 172.21.15.120:0/1393548373 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee0055070 con 0x55d402050480 2024-09-10T22:53:42.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.401+0000 7feeceffd640 1 -- 172.21.15.120:0/1393548373 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00035f0 con 0x55d402050480 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- 172.21.15.120:0/1393548373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 msgr2=0x55d402068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feee002a910 tx=0x7feee003e000 comp rx=0 tx=0).stop 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- 172.21.15.120:0/1393548373 shutdown_connections 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d40204d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 --2- 172.21.15.120:0/1393548373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402068be0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- 172.21.15.120:0/1393548373 >> 172.21.15.120:0/1393548373 conn(0x55d40204e2b0 msgr2=0x55d40204cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- 172.21.15.120:0/1393548373 shutdown_connections 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- 172.21.15.120:0/1393548373 wait complete. 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 -- start start 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.402+0000 7feeedbc9740 1 Processor -- start 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feee41af190 con 0x55d402050480 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feee41b0600 con 0x55d40205cd30 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feee41af860 con 0x55d402050da0 2024-09-10T22:53:42.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feeedbc9740 10 client.0 _ll_register_callbacks cb 0x7feed40e97f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee8d8f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36390/0 (socket says 172.21.15.120:36390) 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee8d8f640 1 -- 172.21.15.120:0/184237114 learned_addr learned my addr 172.21.15.120:0/184237114 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9d91640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 -- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 msgr2=0x55d402065190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 -- 172.21.15.120:0/184237114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 msgr2=0x55d40205fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee9590640 1 -- 172.21.15.120:0/184237114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4020659a0 con 0x55d402050480 2024-09-10T22:53:42.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.403+0000 7feee8d8f640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feee9590640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402064c50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feee0004a60 tx=0x7feee00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e287540ebb257230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeccff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee005d070 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7feee0004dd0 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee0055070 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feee0069050 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeedbc9740 10 client.15606 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d4020d9d70 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.404+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee0055070 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feed40051b0 con 0x55d402050480 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 10 client.15606 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:42.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 20 client.15606 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4020da460 0x55d4020dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 4 client.15606 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d401fc4850 con 0x55d4020da460 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feee8d8f640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4020da460 0x55d4020dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feeccff9640 10 client.15606 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.405+0000 7feee8d8f640 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4020da460 0x55d4020dc880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78dfcec477821688 server_cookie=d5c8ad007c8caf6c in_seq=0 out_seq=0 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeccff9640 1 -- 172.21.15.120:0/184237114 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x55d401fc4850 con 0x55d4020da460 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeccff9640 10 client.15606 handle_command_reply: tid=0 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 client.15606 shutdown 2024-09-10T22:53:42.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 msgr2=0x55d402064c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402064c50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feee0004a60 tx=0x7feee00095b0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 shutdown_connections 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4020da460 0x55d4020dc880 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d40205cd30 0x55d40205fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d402050da0 0x55d402065190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeccff9640 0 client.15606 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 --2- 172.21.15.120:0/184237114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d402050480 0x55d402064c50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 >> 172.21.15.120:0/184237114 conn(0x55d40204e2b0 msgr2=0x55d40204d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 shutdown_connections 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 1 -- 172.21.15.120:0/184237114 wait complete. 2024-09-10T22:53:42.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.406+0000 7feeedbc9740 20 client.15606 trim_cache size 0 max 16384 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "363bfa6b-82dd-4066-8117-7d10a3d87602", 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:42.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:53:42.417 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:42.418 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:53:42.419 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:53:42.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 msgr2=0x7feed00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 0x7feed00b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feee0031440 tx=0x7feee0005b50 comp rx=0 tx=0).stop 2024-09-10T22:53:42.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 msgr2=0x7feee40fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:42.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400edf0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 shutdown_connections 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feed00b52e0 0x7feed00b77a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feee4108040 0x7feee40fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feee41076c0 0x7feee40fe140 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 --2- 172.21.15.120:0/3518034215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee4101cb0 0x7feee40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 >> 172.21.15.120:0/3518034215 conn(0x7feee40fd700 msgr2=0x7feee406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 shutdown_connections 2024-09-10T22:53:42.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:53:42.407+0000 7feeeb7f4640 1 -- 172.21.15.120:0/3518034215 wait complete. 2024-09-10T22:53:42.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 127533 19.05 MB/sec warmup 37 sec latency 74.469 ms 2024-09-10T22:53:43.246 INFO:tasks.cephfs.filesystem:scrub status for tag:363bfa6b-82dd-4066-8117-7d10a3d87602 - {'path': '/', 'tag': '363bfa6b-82dd-4066-8117-7d10a3d87602', 'options': 'recursive,force'} 2024-09-10T22:53:43.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:43 smithi176 ceph-mon[34982]: pgmap v382: 97 pgs: 97 active+clean; 49 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-10T22:53:43.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 130733 19.00 MB/sec warmup 38 sec latency 19.565 ms 2024-09-10T22:53:43.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:43 smithi120 ceph-mon[31247]: pgmap v382: 97 pgs: 97 active+clean; 49 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-10T22:53:43.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:43 smithi153 ceph-mon[34951]: pgmap v382: 97 pgs: 97 active+clean; 49 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-09-10T22:53:44.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 134059 18.96 MB/sec warmup 39 sec latency 12.262 ms 2024-09-10T22:53:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 -- 172.21.15.120:0/588938037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de806b3b0 msgr2=0x7f1de81055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 --2- 172.21.15.120:0/588938037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de806b3b0 0x7f1de81055f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009900 tx=0x7f1dd802a410 comp rx=0 tx=0).stop 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 -- 172.21.15.120:0/588938037 shutdown_connections 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 --2- 172.21.15.120:0/588938037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de8108680 0x7f1de810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 --2- 172.21.15.120:0/588938037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 --2- 172.21.15.120:0/588938037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de806b3b0 0x7f1de81055f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.930+0000 7f1decccd640 1 -- 172.21.15.120:0/588938037 >> 172.21.15.120:0/588938037 conn(0x7f1de80fd140 msgr2=0x7f1de80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- 172.21.15.120:0/588938037 shutdown_connections 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- 172.21.15.120:0/588938037 wait complete. 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 Processor -- start 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- start start 2024-09-10T22:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810e600 con 0x7f1de8108680 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810e480 con 0x7f1de8105b30 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.931+0000 7f1decccd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810e780 con 0x7f1de806b3b0 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35848/0 (socket says 172.21.15.120:35848) 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58864/0 (socket says 172.21.15.120:58864) 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50806/0 (socket says 172.21.15.120:50806) 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 -- 172.21.15.120:0/3482833850 learned_addr learned my addr 172.21.15.120:0/3482833850 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 -- 172.21.15.120:0/3482833850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 msgr2=0x7f1de8102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 -- 172.21.15.120:0/3482833850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 msgr2=0x7f1de8101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 -- 172.21.15.120:0/3482833850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de8102340 con 0x7f1de8108680 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de659c640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1de6d9d640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1de000eb10 tx=0x7f1de000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1ddf7fe640 1 -- 172.21.15.120:0/3482833850 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0004b20 con 0x7f1de8108680 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de81a2e70 con 0x7f1de8108680 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1ddf7fe640 1 -- 172.21.15.120:0/3482833850 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1de0004cc0 con 0x7f1de8108680 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1ddf7fe640 1 -- 172.21.15.120:0/3482833850 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0010690 con 0x7f1de8108680 2024-09-10T22:53:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.932+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de81a3380 con 0x7f1de8108680 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1ddf7fe640 1 -- 172.21.15.120:0/3482833850 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1de0005020 con 0x7f1de8108680 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 Processor -- start 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 -- start start 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd81190 0x5579bcd83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd83bd0 0x5579bcd86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5579bccf06f0 con 0x5579bcd83bd0 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5579bccdfb20 con 0x5579bcd81190 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1defa9f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5579bcce3930 con 0x5579bcd7e860 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.935+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd83bd0 0x5579bcd86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50816/0 (socket says 172.21.15.120:50816) 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 -- 172.21.15.120:0/2706899963 learned_addr learned my addr 172.21.15.120:0/2706899963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1de659c640 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd81190 0x5579bcd83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddf7fe640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 0x7f1dd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 -- 172.21.15.120:0/2706899963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd81190 msgr2=0x5579bcd83580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd81190 0x5579bcd83580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 -- 172.21.15.120:0/2706899963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd83bd0 msgr2=0x5579bcd86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd83bd0 0x5579bcd86090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 -- 172.21.15.120:0/2706899963 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bcca5530 con 0x5579bcd7e860 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1de6d9d640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 0x7f1dd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddf7fe640 1 -- 172.21.15.120:0/3482833850 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f1de00e6120 con 0x7f1de8108680 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddffff640 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0012790 tx=0x7f1dd000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3d1ef87a87f33a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddcff9640 1 -- 172.21.15.120:0/2706899963 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0004050 con 0x5579bcd7e860 2024-09-10T22:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddcff9640 1 -- 172.21.15.120:0/2706899963 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd00054d0 con 0x5579bcd7e860 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1de6d9d640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 0x7f1dd40b77a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1de00e3fc0 tx=0x7f1de00dcd80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.936+0000 7f1ddcff9640 1 -- 172.21.15.120:0/2706899963 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0004c00 con 0x5579bcd7e860 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- 172.21.15.120:0/2706899963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 msgr2=0x5579bcd80c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0012790 tx=0x7f1dd000b390 comp rx=0 tx=0).stop 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- 172.21.15.120:0/2706899963 shutdown_connections 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd83bd0 0x5579bcd86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd81190 0x5579bcd83580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 --2- 172.21.15.120:0/2706899963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd80c50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- 172.21.15.120:0/2706899963 >> 172.21.15.120:0/2706899963 conn(0x5579bcd799b0 msgr2=0x5579bcd7a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- 172.21.15.120:0/2706899963 shutdown_connections 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- 172.21.15.120:0/2706899963 wait complete. 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.937+0000 7f1defa9f740 1 -- start start 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 Processor -- start 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5579bcc17ab0 con 0x5579bcd81190 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5579bcc85730 con 0x5579bcd7e860 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5579bcd86bb0 con 0x5579bcd83bd0 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1defa9f740 10 client.0 _ll_register_callbacks cb 0x5579bccd96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35870/0 (socket says 172.21.15.120:35870) 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1de659c640 1 -- 172.21.15.120:0/1342337607 learned_addr learned my addr 172.21.15.120:0/1342337607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 -- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 msgr2=0x5579bcd891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 -- 172.21.15.120:0/1342337607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 msgr2=0x5579bcd885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 -- 172.21.15.120:0/1342337607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bca41790 con 0x5579bcd7e860 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1de659c640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1ddffff640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd87f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b6b0 tx=0x7f1dd0017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4f8fa7b1859ced server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.938+0000 7f1dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00052c0 con 0x5579bcd7e860 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f1dd00049a0 con 0x5579bcd7e860 2024-09-10T22:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd00054d0 con 0x5579bcd7e860 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1dd0036050 con 0x5579bcd7e860 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 10 client.25103 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5579bcaba620 con 0x5579bcd7e860 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd00054d0 con 0x5579bcd7e860 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1dd001f460 con 0x5579bcd7e860 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 10 client.25103 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 10 client.25103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 20 client.25103 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bcd934a0 0x5579bcd958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 4 client.25103 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579bccf2640 con 0x5579bcd934a0 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.939+0000 7f1de659c640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bcd934a0 0x5579bcd958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.940+0000 7f1dcdffb640 10 client.25103 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.940+0000 7f1de659c640 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bcd934a0 0x5579bcd958d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857e44e133c16083 server_cookie=6bd2230332aa5eb5 in_seq=0 out_seq=0 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.940+0000 7f1dcdffb640 1 -- 172.21.15.120:0/1342337607 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5579bccf2640 con 0x5579bcd934a0 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.940+0000 7f1dcdffb640 10 client.25103 handle_command_reply: tid=0 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 client.25103 shutdown 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 msgr2=0x5579bcd87f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd87f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b6b0 tx=0x7f1dd0017040 comp rx=0 tx=0).stop 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 shutdown_connections 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bcd934a0 0x5579bcd958d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1dcdffb640 0 client.25103 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 --2- 172.21.15.120:0/1342337607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd87f30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 >> 172.21.15.120:0/1342337607 conn(0x5579bcd799b0 msgr2=0x5579bcd78bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 shutdown_connections 2024-09-10T22:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 1 -- 172.21.15.120:0/1342337607 wait complete. 2024-09-10T22:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.941+0000 7f1defa9f740 20 client.25103 trim_cache size 0 max 16384 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.956+0000 7f1defa9f740 1 Processor -- start 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.956+0000 7f1defa9f740 1 -- start start 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd83fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81aa050 con 0x5579bcd81190 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81ac8d0 con 0x5579bcd7e860 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1defa9f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81aa6c0 con 0x5579bcd83bd0 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35882/0 (socket says 172.21.15.120:35882) 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 -- 172.21.15.120:0/3742006656 learned_addr learned my addr 172.21.15.120:0/3742006656 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd83fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 -- 172.21.15.120:0/3742006656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 msgr2=0x5579bcd83fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd83fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 -- 172.21.15.120:0/3742006656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 msgr2=0x5579bcd7dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd7dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 -- 172.21.15.120:0/3742006656 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bcd80fc0 con 0x5579bcd81190 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1de659c640 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009230 tx=0x7f1dd80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ba97cd3ba181ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1dcffff640 1 -- 172.21.15.120:0/3742006656 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd804a020 con 0x5579bcd81190 2024-09-10T22:53:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1dcffff640 1 -- 172.21.15.120:0/3742006656 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd80054f0 con 0x5579bcd81190 2024-09-10T22:53:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.957+0000 7f1dcffff640 1 -- 172.21.15.120:0/3742006656 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8035040 con 0x5579bcd81190 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- 172.21.15.120:0/3742006656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 msgr2=0x5579bcd7e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009230 tx=0x7f1dd80314a0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- 172.21.15.120:0/3742006656 shutdown_connections 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd83bd0 0x5579bcd83fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd7e290 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3742006656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd7e860 0x5579bcd7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- 172.21.15.120:0/3742006656 >> 172.21.15.120:0/3742006656 conn(0x5579bcd799b0 msgr2=0x5579bcd87ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- 172.21.15.120:0/3742006656 shutdown_connections 2024-09-10T22:53:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- 172.21.15.120:0/3742006656 wait complete. 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.958+0000 7f1defa9f740 1 -- start start 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 Processor -- start 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd8c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd83bd0 0x5579bcd8daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81aad90 con 0x5579bcd81190 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81ac200 con 0x5579bcd83bd0 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81ab460 con 0x5579bcd7e860 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1defa9f740 10 client.0 _ll_register_callbacks cb 0x7f1dd000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35890/0 (socket says 172.21.15.120:35890) 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 -- 172.21.15.120:0/3594032098 learned_addr learned my addr 172.21.15.120:0/3594032098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd8c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd83bd0 0x5579bcd8daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 -- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 msgr2=0x5579bcd8c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd8c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 -- 172.21.15.120:0/3594032098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd83bd0 msgr2=0x5579bcd8daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd83bd0 0x5579bcd8daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 -- 172.21.15.120:0/3594032098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bcd8e320 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1de659c640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80096c0 tx=0x7f1dd8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed5ac1e80d82011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.959+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd804a030 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f1dd8031b20 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd80054f0 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1dd805e050 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 10 client.15624 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5579bcd8f690 con 0x5579bcd81190 2024-09-10T22:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd80054f0 con 0x5579bcd81190 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1dd8032040 con 0x5579bcd81190 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 10 client.15624 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 20 client.15624 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bce003e0 0x5579bce027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 4 client.15624 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579bccee0f0 con 0x5579bce003e0 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.960+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bce003e0 0x5579bce027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.961+0000 7f1ddffff640 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bce003e0 0x5579bce027d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bb68548690c05c server_cookie=3f0e33f3feb1a482 in_seq=0 out_seq=0 2024-09-10T22:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:44.961+0000 7f1dcdffb640 10 client.15624 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1dcdffb640 1 -- 172.21.15.120:0/3594032098 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579bccee0f0 con 0x5579bce003e0 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1dcdffb640 10 client.15624 handle_command_reply: tid=0 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 client.15624 shutdown 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 msgr2=0x5579bcd8cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80096c0 tx=0x7f1dd8004740 comp rx=0 tx=0).stop 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 shutdown_connections 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5579bce003e0 0x5579bce027d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1dcdffb640 0 client.15624 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5579bcd83bd0 0x5579bcd8daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5579bcd81190 0x5579bcd8cf70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 --2- 172.21.15.120:0/3594032098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5579bcd7e860 0x5579bcd8c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 >> 172.21.15.120:0/3594032098 conn(0x5579bcd799b0 msgr2=0x5579bcd7dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 shutdown_connections 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 1 -- 172.21.15.120:0/3594032098 wait complete. 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.096+0000 7f1defa9f740 20 client.15624 trim_cache size 0 max 16384 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:53:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:53:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:53:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 msgr2=0x7f1dd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 0x7f1dd40b77a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1de00e3fc0 tx=0x7f1de00dcd80 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 msgr2=0x7f1de8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1de000eb10 tx=0x7f1de000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 shutdown_connections 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1dd40b52e0 0x7f1dd40b77a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1de8108680 0x7f1de8101ad0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1de8105b30 0x7f1de8101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 --2- 172.21.15.120:0/3482833850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1de806b3b0 0x7f1de8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 >> 172.21.15.120:0/3482833850 conn(0x7f1de80fd140 msgr2=0x7f1de8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 shutdown_connections 2024-09-10T22:53:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:45.097+0000 7f1decccd640 1 -- 172.21.15.120:0/3482833850 wait complete. 2024-09-10T22:53:45.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:45 smithi120 ceph-mon[31247]: pgmap v383: 97 pgs: 97 active+clean; 49 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T22:53:45.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:45 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:53:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:45 smithi176 ceph-mon[34982]: pgmap v383: 97 pgs: 97 active+clean; 49 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T22:53:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:45 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:53:45.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 136308 18.82 MB/sec warmup 40 sec latency 50.759 ms 2024-09-10T22:53:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:45 smithi153 ceph-mon[34951]: pgmap v383: 97 pgs: 97 active+clean; 49 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T22:53:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:45 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:53:46.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 139311 18.72 MB/sec warmup 41 sec latency 55.003 ms 2024-09-10T22:53:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:47 smithi176 ceph-mon[34982]: pgmap v384: 97 pgs: 97 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-10T22:53:47.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 142321 18.71 MB/sec warmup 42 sec latency 102.322 ms 2024-09-10T22:53:47.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:47 smithi120 ceph-mon[31247]: pgmap v384: 97 pgs: 97 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-10T22:53:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:47 smithi153 ceph-mon[34951]: pgmap v384: 97 pgs: 97 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-10T22:53:48.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 145850 18.72 MB/sec warmup 43 sec latency 11.907 ms 2024-09-10T22:53:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:49 smithi176 ceph-mon[34982]: pgmap v385: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-09-10T22:53:49.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 149329 18.72 MB/sec warmup 44 sec latency 12.577 ms 2024-09-10T22:53:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:49 smithi120 ceph-mon[31247]: pgmap v385: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-09-10T22:53:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:49 smithi153 ceph-mon[34951]: pgmap v385: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-09-10T22:53:50.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 152764 18.73 MB/sec warmup 45 sec latency 20.255 ms 2024-09-10T22:53:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:51 smithi176 ceph-mon[34982]: pgmap v386: 97 pgs: 97 active+clean; 45 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-09-10T22:53:51.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 156302 18.74 MB/sec warmup 46 sec latency 15.555 ms 2024-09-10T22:53:51.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:51 smithi120 ceph-mon[31247]: pgmap v386: 97 pgs: 97 active+clean; 45 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-09-10T22:53:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:51 smithi153 ceph-mon[34951]: pgmap v386: 97 pgs: 97 active+clean; 45 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-09-10T22:53:52.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 158988 18.60 MB/sec warmup 47 sec latency 73.065 ms 2024-09-10T22:53:53.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:53 smithi176 ceph-mon[34982]: pgmap v387: 97 pgs: 97 active+clean; 46 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-09-10T22:53:53.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 162216 18.53 MB/sec warmup 48 sec latency 13.539 ms 2024-09-10T22:53:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:53 smithi120 ceph-mon[31247]: pgmap v387: 97 pgs: 97 active+clean; 46 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-09-10T22:53:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:53 smithi153 ceph-mon[34951]: pgmap v387: 97 pgs: 97 active+clean; 46 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-09-10T22:53:54.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 165631 18.53 MB/sec warmup 49 sec latency 12.092 ms 2024-09-10T22:53:55.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:55 smithi176 ceph-mon[34982]: pgmap v388: 97 pgs: 97 active+clean; 47 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T22:53:55.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 169079 18.53 MB/sec warmup 50 sec latency 14.453 ms 2024-09-10T22:53:55.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:55 smithi120 ceph-mon[31247]: pgmap v388: 97 pgs: 97 active+clean; 47 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T22:53:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:55 smithi153 ceph-mon[34951]: pgmap v388: 97 pgs: 97 active+clean; 47 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T22:53:56.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 172099 18.51 MB/sec warmup 51 sec latency 13.102 ms 2024-09-10T22:53:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:57 smithi176 ceph-mon[34982]: pgmap v389: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 567 op/s 2024-09-10T22:53:57.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 174557 18.43 MB/sec warmup 52 sec latency 78.223 ms 2024-09-10T22:53:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:57 smithi120 ceph-mon[31247]: pgmap v389: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 567 op/s 2024-09-10T22:53:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:57 smithi153 ceph-mon[34951]: pgmap v389: 97 pgs: 97 active+clean; 53 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 567 op/s 2024-09-10T22:53:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 -- 172.21.15.120:0/852284270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 msgr2=0x7f8de4101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 --2- 172.21.15.120:0/852284270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4101a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009930 tx=0x7f8dd802a420 comp rx=0 tx=0).stop 2024-09-10T22:53:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 -- 172.21.15.120:0/852284270 shutdown_connections 2024-09-10T22:53:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 --2- 172.21.15.120:0/852284270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41031a0 0x7f8de4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 --2- 172.21.15.120:0/852284270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41027e0 0x7f8de4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 --2- 172.21.15.120:0/852284270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4101a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 -- 172.21.15.120:0/852284270 >> 172.21.15.120:0/852284270 conn(0x7f8de40fcf30 msgr2=0x7f8de40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.191+0000 7f8de9edb640 1 -- 172.21.15.120:0/852284270 shutdown_connections 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 -- 172.21.15.120:0/852284270 wait complete. 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 Processor -- start 2024-09-10T22:53:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 -- start start 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41031a0 0x7f8de4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8de410d300 con 0x7f8de4101690 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8de410d180 con 0x7f8de41027e0 2024-09-10T22:53:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.192+0000 7f8de9edb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8de410d480 con 0x7f8de41031a0 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de8ed9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de96da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41031a0 0x7f8de4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de8ed9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47584/0 (socket says 172.21.15.120:47584) 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52548/0 (socket says 172.21.15.120:52548) 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de8ed9640 1 -- 172.21.15.120:0/1362923685 learned_addr learned my addr 172.21.15.120:0/1362923685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 -- 172.21.15.120:0/1362923685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41031a0 msgr2=0x7f8de4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41031a0 0x7f8de4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 -- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 msgr2=0x7f8de4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 -- 172.21.15.120:0/1362923685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de41995f0 con 0x7f8de41027e0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de3fff640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9f0 tx=0x7f8dd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de1ffb640 1 -- 172.21.15.120:0/1362923685 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000ccc0 con 0x7f8de41027e0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de1ffb640 1 -- 172.21.15.120:0/1362923685 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd0005020 con 0x7f8de41027e0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de41998e0 con 0x7f8de41027e0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.193+0000 7f8de1ffb640 1 -- 172.21.15.120:0/1362923685 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0004a00 con 0x7f8de41027e0 2024-09-10T22:53:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.194+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de406aa10 con 0x7f8de41027e0 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8de1ffb640 1 -- 172.21.15.120:0/1362923685 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8dd0004ba0 con 0x7f8de41027e0 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 Processor -- start 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 -- start start 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5639133146f0 con 0x5639133a7dd0 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563913303b20 con 0x5639133a5390 2024-09-10T22:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.196+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563913307930 con 0x5639133a2a60 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:43292/0 (socket says 172.21.15.120:43292) 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47596/0 (socket says 172.21.15.120:47596) 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 -- 172.21.15.120:0/879315512 learned_addr learned my addr 172.21.15.120:0/879315512 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de1ffb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 0x7f8dd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 -- 172.21.15.120:0/879315512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 msgr2=0x5639133a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 -- 172.21.15.120:0/879315512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 msgr2=0x5639133aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 -- 172.21.15.120:0/879315512 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639132c9530 con 0x5639133a2a60 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de3fff640 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 0x7f8dd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de1ffb640 1 -- 172.21.15.120:0/1362923685 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8dd00e5b50 con 0x7f8de41027e0 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00e560 tx=0x7f8ddc00ea20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec55bd2cb09a38eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8dceffd640 1 -- 172.21.15.120:0/879315512 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc01e040 con 0x5639133a2a60 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8de96da640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 0x7f8dd40b77a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc024000 tx=0x7f8ddc025000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8dceffd640 1 -- 172.21.15.120:0/879315512 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ddc025dd0 con 0x5639133a2a60 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.197+0000 7f8dceffd640 1 -- 172.21.15.120:0/879315512 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc01ab50 con 0x5639133a2a60 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- 172.21.15.120:0/879315512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 msgr2=0x5639133a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00e560 tx=0x7f8ddc00ea20 comp rx=0 tx=0).stop 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- 172.21.15.120:0/879315512 shutdown_connections 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/879315512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133a4e50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- 172.21.15.120:0/879315512 >> 172.21.15.120:0/879315512 conn(0x56391339dbb0 msgr2=0x56391339e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- 172.21.15.120:0/879315512 shutdown_connections 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- 172.21.15.120:0/879315512 wait complete. 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 -- start start 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.198+0000 7f8dedfe6740 1 Processor -- start 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56391323bab0 con 0x5639133a7dd0 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5639132a9730 con 0x5639133a5390 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5639133aadb0 con 0x5639133a2a60 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8dedfe6740 10 client.0 _ll_register_callbacks cb 0x5639132fd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47606/0 (socket says 172.21.15.120:47606) 2024-09-10T22:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de8ed9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 -- 172.21.15.120:0/3149528193 learned_addr learned my addr 172.21.15.120:0/3149528193 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de3fff640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 -- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 msgr2=0x5639133b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 -- 172.21.15.120:0/3149528193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 msgr2=0x5639133b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.199+0000 7f8de96da640 1 -- 172.21.15.120:0/3149528193 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563913065790 con 0x5639133a7dd0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8de96da640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00bf10 tx=0x7f8ddc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103cc6b3cb41a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc032060 con 0x5639133a7dd0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8ddc025b20 con 0x5639133a7dd0 2024-09-10T22:53:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ddc025dd0 con 0x5639133a7dd0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ddc01e070 con 0x5639133a7dd0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 10 client.15636 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5639130de620 con 0x5639133a7dd0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ddc025dd0 con 0x5639133a7dd0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8ddc022070 con 0x5639133a7dd0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 10 client.15636 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 20 client.15636 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5639133bbb50 0x5639133bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 4 client.15636 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.200+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563913316640 con 0x5639133bbb50 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.201+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5639133bbb50 0x5639133bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.201+0000 7f8dc3fff640 10 client.15636 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.201+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5639133bbb50 0x5639133bdf40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2931734eb64d2dbe server_cookie=b1bac913f4e61f47 in_seq=0 out_seq=0 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dc3fff640 1 -- 172.21.15.120:0/3149528193 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563913316640 con 0x5639133bbb50 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dc3fff640 10 client.15636 handle_command_reply: tid=0 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 client.15636 shutdown 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 msgr2=0x5639133b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00bf10 tx=0x7f8ddc00bf40 comp rx=0 tx=0).stop 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 shutdown_connections 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5639133bbb50 0x5639133bdf40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b1860 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a5390 0x5639133b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/3149528193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a2a60 0x5639133b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 >> 172.21.15.120:0/3149528193 conn(0x56391339dbb0 msgr2=0x56391339cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dc3fff640 0 client.15636 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 shutdown_connections 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 1 -- 172.21.15.120:0/3149528193 wait complete. 2024-09-10T22:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.202+0000 7f8dedfe6740 20 client.15636 trim_cache size 0 max 16384 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 Processor -- start 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 -- start start 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x563913398bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a59d0 con 0x5639133a7dd0 2024-09-10T22:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a8250 con 0x5639133a2a60 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a6040 con 0x5639133a5390 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52580/0 (socket says 172.21.15.120:52580) 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 -- 172.21.15.120:0/1748312174 learned_addr learned my addr 172.21.15.120:0/1748312174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47618/0 (socket says 172.21.15.120:47618) 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de3fff640 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x563913398bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 -- 172.21.15.120:0/1748312174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 msgr2=0x563913398bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x563913398bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 -- 172.21.15.120:0/1748312174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 msgr2=0x5639133ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de8ed9640 1 -- 172.21.15.120:0/1748312174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639133b1000 con 0x5639133a2a60 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.218+0000 7f8de96da640 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006010 tx=0x7f8dd802ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b5cf0db782c28b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dce7fc640 1 -- 172.21.15.120:0/1748312174 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8035040 con 0x5639133a2a60 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dce7fc640 1 -- 172.21.15.120:0/1748312174 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd80035f0 con 0x5639133a2a60 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dce7fc640 1 -- 172.21.15.120:0/1748312174 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd803c6a0 con 0x5639133a2a60 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 -- 172.21.15.120:0/1748312174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 msgr2=0x5639133a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006010 tx=0x7f8dd802ab90 comp rx=0 tx=0).stop 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 -- 172.21.15.120:0/1748312174 shutdown_connections 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x563913398bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/1748312174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133a88c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.219+0000 7f8dedfe6740 1 -- 172.21.15.120:0/1748312174 >> 172.21.15.120:0/1748312174 conn(0x56391339dbb0 msgr2=0x5639133a81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- 172.21.15.120:0/1748312174 shutdown_connections 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- 172.21.15.120:0/1748312174 wait complete. 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- start start 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 Processor -- start 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x5639133b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a6710 con 0x5639133a7dd0 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a7b80 con 0x5639133a2a60 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a6de0 con 0x5639133a5390 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8dedfe6740 10 client.0 _ll_register_callbacks cb 0x7f8dd00e94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8de3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x5639133b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8de96da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47634/0 (socket says 172.21.15.120:47634) 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8de96da640 1 -- 172.21.15.120:0/4196815450 learned_addr learned my addr 172.21.15.120:0/4196815450 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.220+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 -- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 msgr2=0x5639133b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x5639133b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 -- 172.21.15.120:0/4196815450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 msgr2=0x5639133b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 -- 172.21.15.120:0/4196815450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639133a51c0 con 0x5639133a7dd0 2024-09-10T22:53:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8de96da640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5639133b1340 tx=0x7f8ddc039730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e41378b9e492058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc032070 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f8ddc004b90 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd80035f0 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ddc00e3b0 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 10 client.15642 fetch_fsmap learned FSMap version 21 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563913428a60 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd80035f0 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8ddc022040 con 0x5639133a7dd0 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 10 client.15642 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 20 client.15642 populate_metadata read hostname 'smithi120' 2024-09-10T22:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563913429400 0x56391342b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 4 client.15642 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.221+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639133120f0 con 0x563913429400 2024-09-10T22:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.222+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563913429400 0x56391342b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.222+0000 7f8dc3fff640 10 client.15642 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.222+0000 7f8de8ed9640 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563913429400 0x56391342b820 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3359662b9720830b server_cookie=6ce3b5337f8d7353 in_seq=0 out_seq=0 2024-09-10T22:53:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.379+0000 7f8dc3fff640 1 -- 172.21.15.120:0/4196815450 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639133120f0 con 0x563913429400 2024-09-10T22:53:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.379+0000 7f8dc3fff640 10 client.15642 handle_command_reply: tid=0 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.379+0000 7f8dedfe6740 1 client.15642 shutdown 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.379+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 msgr2=0x5639133b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.379+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5639133b1340 tx=0x7f8ddc039730 comp rx=0 tx=0).stop 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 shutdown_connections 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563913429400 0x56391342b820 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dc3fff640 0 client.15642 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5639133a7dd0 0x5639133b7660 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5639133a5390 0x5639133b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 --2- 172.21.15.120:0/4196815450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639133a2a60 0x5639133b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 >> 172.21.15.120:0/4196815450 conn(0x56391339dbb0 msgr2=0x5639133ab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 shutdown_connections 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 1 -- 172.21.15.120:0/4196815450 wait complete. 2024-09-10T22:53:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.380+0000 7f8dedfe6740 20 client.15642 trim_cache size 0 max 16384 2024-09-10T22:53:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:53:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:53:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:53:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:53:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 msgr2=0x7f8dd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 0x7f8dd40b77a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc024000 tx=0x7f8ddc025000 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 msgr2=0x7f8de4198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9f0 tx=0x7f8dd000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 shutdown_connections 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8dd40b52e0 0x7f8dd40b77a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8de41031a0 0x7f8de4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8de41027e0 0x7f8de4198170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 --2- 172.21.15.120:0/1362923685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de4101690 0x7f8de4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 >> 172.21.15.120:0/1362923685 conn(0x7f8de40fcf30 msgr2=0x7f8de4105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 shutdown_connections 2024-09-10T22:53:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:53:58.381+0000 7f8de9edb640 1 -- 172.21.15.120:0/1362923685 wait complete. 2024-09-10T22:53:58.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 177774 18.44 MB/sec warmup 53 sec latency 94.595 ms 2024-09-10T22:53:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:53:59 smithi176 ceph-mon[34982]: pgmap v390: 97 pgs: 97 active+clean; 52 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 351 op/s 2024-09-10T22:53:59.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 181228 18.44 MB/sec warmup 54 sec latency 19.998 ms 2024-09-10T22:53:59.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:53:59 smithi120 ceph-mon[31247]: pgmap v390: 97 pgs: 97 active+clean; 52 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 351 op/s 2024-09-10T22:53:59.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:53:59 smithi153 ceph-mon[34951]: pgmap v390: 97 pgs: 97 active+clean; 52 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 351 op/s 2024-09-10T22:54:00.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 184782 18.46 MB/sec warmup 55 sec latency 13.166 ms 2024-09-10T22:54:01.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:01 smithi176 ceph-mon[34982]: pgmap v391: 97 pgs: 97 active+clean; 45 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-09-10T22:54:01.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 188273 18.47 MB/sec warmup 56 sec latency 12.144 ms 2024-09-10T22:54:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:01 smithi120 ceph-mon[31247]: pgmap v391: 97 pgs: 97 active+clean; 45 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-09-10T22:54:01.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:01 smithi153 ceph-mon[34951]: pgmap v391: 97 pgs: 97 active+clean; 45 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-09-10T22:54:02.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 190987 18.37 MB/sec warmup 57 sec latency 66.193 ms 2024-09-10T22:54:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:03 smithi176 ceph-mon[34982]: pgmap v392: 97 pgs: 97 active+clean; 43 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T22:54:03.614 INFO:tasks.workunit.client.0.smithi120.stdout: 1 194314 18.36 MB/sec warmup 58 sec latency 13.929 ms 2024-09-10T22:54:03.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:03 smithi120 ceph-mon[31247]: pgmap v392: 97 pgs: 97 active+clean; 43 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T22:54:03.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:03 smithi153 ceph-mon[34951]: pgmap v392: 97 pgs: 97 active+clean; 43 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T22:54:04.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 197491 18.30 MB/sec warmup 59 sec latency 14.150 ms 2024-09-10T22:54:05.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 200465 18.29 MB/sec warmup 60 sec latency 19.872 ms 2024-09-10T22:54:05.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:05 smithi120 ceph-mon[31247]: pgmap v393: 97 pgs: 97 active+clean; 47 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:54:05.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:05 smithi153 ceph-mon[34951]: pgmap v393: 97 pgs: 97 active+clean; 47 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:54:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:05 smithi176 ceph-mon[34982]: pgmap v393: 97 pgs: 97 active+clean; 47 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:54:06.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 203745 18.30 MB/sec warmup 61 sec latency 17.276 ms 2024-09-10T22:54:07.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 206656 18.24 MB/sec warmup 62 sec latency 86.274 ms 2024-09-10T22:54:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:07 smithi120 ceph-mon[31247]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 572 op/s 2024-09-10T22:54:07.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:07 smithi153 ceph-mon[34951]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 572 op/s 2024-09-10T22:54:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:07 smithi176 ceph-mon[34982]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 572 op/s 2024-09-10T22:54:08.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 210000 18.24 MB/sec warmup 63 sec latency 18.240 ms 2024-09-10T22:54:09.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 213412 18.25 MB/sec warmup 64 sec latency 12.270 ms 2024-09-10T22:54:09.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:09 smithi120 ceph-mon[31247]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-10T22:54:09.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:09 smithi153 ceph-mon[34951]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-10T22:54:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:09 smithi176 ceph-mon[34982]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-09-10T22:54:10.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 216901 18.27 MB/sec warmup 65 sec latency 13.557 ms 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.468+0000 7f941c152640 1 -- 172.21.15.120:0/3069454958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 msgr2=0x7f9414107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.468+0000 7f941c152640 1 --2- 172.21.15.120:0/3069454958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414107f40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- 172.21.15.120:0/3069454958 shutdown_connections 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- 172.21.15.120:0/3069454958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 0x7f941410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- 172.21.15.120:0/3069454958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414107f40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- 172.21.15.120:0/3069454958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 0x7f94141055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- 172.21.15.120:0/3069454958 >> 172.21.15.120:0/3069454958 conn(0x7f94140fd140 msgr2=0x7f94140ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- 172.21.15.120:0/3069454958 shutdown_connections 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- 172.21.15.120:0/3069454958 wait complete. 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 Processor -- start 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- start start 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 0x7f9414102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 0x7f9414101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f941410e5b0 con 0x7f9414105b10 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f941410e430 con 0x7f941406b3d0 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.469+0000 7f941c152640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f941410e730 con 0x7f9414108680 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43930/0 (socket says 172.21.15.120:43930) 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 -- 172.21.15.120:0/4228610600 learned_addr learned my addr 172.21.15.120:0/4228610600 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f941a6ef640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 0x7f9414101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f9419eee640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 0x7f9414102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 -- 172.21.15.120:0/4228610600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 msgr2=0x7f9414101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 0x7f9414101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 -- 172.21.15.120:0/4228610600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 msgr2=0x7f9414102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 0x7f9414102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 -- 172.21.15.120:0/4228610600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9414102320 con 0x7f9414105b10 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f94196ed640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9404006010 tx=0x7f94040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f940affd640 1 -- 172.21.15.120:0/4228610600 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940403f030 con 0x7f9414105b10 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f940affd640 1 -- 172.21.15.120:0/4228610600 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9404003cb0 con 0x7f9414105b10 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94141a2e30 con 0x7f9414105b10 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f940affd640 1 -- 172.21.15.120:0/4228610600 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404035070 con 0x7f9414105b10 2024-09-10T22:54:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.470+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94141a3290 con 0x7f9414105b10 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f940affd640 1 -- 172.21.15.120:0/4228610600 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f940403bb60 con 0x7f9414105b10 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 Processor -- start 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 -- start start 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f736090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55910f6a06f0 con 0x55910f731190 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55910f68fb20 con 0x55910f72e860 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941eff8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55910f693930 con 0x55910f733bd0 2024-09-10T22:54:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f9419eee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f94196ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f736090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f9419eee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43936/0 (socket says 172.21.15.120:43936) 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41170/0 (socket says 172.21.15.120:41170) 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.473+0000 7f941a6ef640 1 -- 172.21.15.120:0/2671336223 learned_addr learned my addr 172.21.15.120:0/2671336223 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f940affd640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 0x7f94000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 -- 172.21.15.120:0/2671336223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 msgr2=0x55910f736090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f736090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 -- 172.21.15.120:0/2671336223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 msgr2=0x55910f730c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 -- 172.21.15.120:0/2671336223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910f655530 con 0x55910f731190 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f941a6ef640 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f941a6ef640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 0x7f94000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f940affd640 1 -- 172.21.15.120:0/4228610600 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f9404106460 con 0x7f9414105b10 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f9419eee640 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbe0 tx=0x7f941000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478c71150c77bf0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f93fffff640 1 -- 172.21.15.120:0/2671336223 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410015070 con 0x55910f731190 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f93fffff640 1 -- 172.21.15.120:0/2671336223 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94100035f0 con 0x55910f731190 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.474+0000 7f93fffff640 1 -- 172.21.15.120:0/2671336223 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410011640 con 0x55910f731190 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- 172.21.15.120:0/2671336223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 msgr2=0x55910f733580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941a6ef640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 0x7f94000b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f940c005f40 tx=0x7f940c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbe0 tx=0x7f941000f730 comp rx=0 tx=0).stop 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- 172.21.15.120:0/2671336223 shutdown_connections 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f736090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f733580 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 --2- 172.21.15.120:0/2671336223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f730c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- 172.21.15.120:0/2671336223 >> 172.21.15.120:0/2671336223 conn(0x55910f7299b0 msgr2=0x55910f72a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- 172.21.15.120:0/2671336223 shutdown_connections 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- 172.21.15.120:0/2671336223 wait complete. 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 -- start start 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.475+0000 7f941eff8740 1 Processor -- start 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f737ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 0x55910f738570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f739140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55910f5c7ab0 con 0x55910f733bd0 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55910f635730 con 0x55910f72e860 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55910f736bb0 con 0x55910f731190 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941eff8740 10 client.0 _ll_register_callbacks cb 0x55910f6896b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f739140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f94196ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 0x55910f738570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f737ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f739140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43942/0 (socket says 172.21.15.120:43942) 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f941a6ef640 1 -- 172.21.15.120:0/4007910604 learned_addr learned my addr 172.21.15.120:0/4007910604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 -- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 msgr2=0x55910f738570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 0x55910f738570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 -- 172.21.15.120:0/4007910604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 msgr2=0x55910f739140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f739140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 -- 172.21.15.120:0/4007910604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910f3f1790 con 0x55910f72e860 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f94196ed640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 0x55910f738570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f9419eee640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f737ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbb0 tx=0x7f941000bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83a7d8faf2bc5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.476+0000 7f93fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410028060 con 0x55910f72e860 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f94100039b0 con 0x55910f72e860 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94100035f0 con 0x55910f72e860 2024-09-10T22:54:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9410015070 con 0x55910f72e860 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 10 client.25136 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55910f46a620 con 0x55910f72e860 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94100035f0 con 0x55910f72e860 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f94100119a0 con 0x55910f72e860 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 10 client.25136 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 10 client.25136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 20 client.25136 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f743480 0x55910f745870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 4 client.25136 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55910f6a2640 con 0x55910f743480 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.477+0000 7f94196ed640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f743480 0x55910f745870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.478+0000 7f94196ed640 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f743480 0x55910f745870 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c32b0cb9720e424 server_cookie=26a5eb45935b3d88 in_seq=0 out_seq=0 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.478+0000 7f93fd7fa640 10 client.25136 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.478+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4007910604 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55910f6a2640 con 0x55910f743480 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.478+0000 7f93fd7fa640 10 client.25136 handle_command_reply: tid=0 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.478+0000 7f941eff8740 1 client.25136 shutdown 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 msgr2=0x55910f737ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f737ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbb0 tx=0x7f941000bf00 comp rx=0 tx=0).stop 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 shutdown_connections 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f743480 0x55910f745870 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f739140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f93fd7fa640 0 client.25136 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f731190 0x55910f738570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 --2- 172.21.15.120:0/4007910604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f737ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 >> 172.21.15.120:0/4007910604 conn(0x55910f7299b0 msgr2=0x55910f728bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 shutdown_connections 2024-09-10T22:54:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 1 -- 172.21.15.120:0/4007910604 wait complete. 2024-09-10T22:54:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.479+0000 7f941eff8740 20 client.25136 trim_cache size 0 max 16384 2024-09-10T22:54:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 Processor -- start 2024-09-10T22:54:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 -- start start 2024-09-10T22:54:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94141aa030 con 0x55910f733bd0 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94141ac8b0 con 0x55910f731190 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.493+0000 7f941eff8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f94141aa6a0 con 0x55910f72e860 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f94196ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f9419eee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43948/0 (socket says 172.21.15.120:43948) 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f94196ed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41196/0 (socket says 172.21.15.120:41196) 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f9419eee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59874/0 (socket says 172.21.15.120:59874) 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 -- 172.21.15.120:0/2618786843 learned_addr learned my addr 172.21.15.120:0/2618786843 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 -- 172.21.15.120:0/2618786843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 msgr2=0x55910f739120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 -- 172.21.15.120:0/2618786843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 msgr2=0x55910f7283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 -- 172.21.15.120:0/2618786843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910f72d970 con 0x55910f733bd0 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f9419eee640 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f94196ed640 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f941a6ef640 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f940c005fd0 tx=0x7f940c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbfe18d5de2fd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f93ff7fe640 1 -- 172.21.15.120:0/2618786843 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c004b20 con 0x55910f733bd0 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f93ff7fe640 1 -- 172.21.15.120:0/2618786843 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c004cc0 con 0x55910f733bd0 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.494+0000 7f93ff7fe640 1 -- 172.21.15.120:0/2618786843 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0054b0 con 0x55910f733bd0 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- 172.21.15.120:0/2618786843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 msgr2=0x55910f728910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f940c005fd0 tx=0x7f940c016000 comp rx=0 tx=0).stop 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- 172.21.15.120:0/2618786843 shutdown_connections 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f733bd0 0x55910f728910 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f731190 0x55910f7283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 --2- 172.21.15.120:0/2618786843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f72e860 0x55910f739120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- 172.21.15.120:0/2618786843 >> 172.21.15.120:0/2618786843 conn(0x55910f7299b0 msgr2=0x55910f72dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- 172.21.15.120:0/2618786843 shutdown_connections 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- 172.21.15.120:0/2618786843 wait complete. 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 -- start start 2024-09-10T22:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.495+0000 7f941eff8740 1 Processor -- start 2024-09-10T22:54:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f73ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f73f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94141aad70 con 0x55910f731190 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94141ac1e0 con 0x55910f72e860 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f94141ab440 con 0x55910f733bd0 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941eff8740 10 client.0 _ll_register_callbacks cb 0x7f940c009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43952/0 (socket says 172.21.15.120:43952) 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 -- 172.21.15.120:0/4243194993 learned_addr learned my addr 172.21.15.120:0/4243194993 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f941a6ef640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f73f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f9419eee640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f73ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 -- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 msgr2=0x55910f73f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f73f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 -- 172.21.15.120:0/4243194993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 msgr2=0x55910f73ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f73ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 -- 172.21.15.120:0/4243194993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910f73d6c0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f9419eee640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f73ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f94196ed640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94040cc950 tx=0x7f94040b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e550f5ecba10e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.496+0000 7f93fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404049040 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f94040c8ec0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c004cc0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f94040c8af0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 10 client.15660 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55910f7b0770 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c004cc0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f94041060d0 con 0x55910f731190 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 10 client.15660 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 20 client.15660 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f7b1100 0x55910f7b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 4 client.15660 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55910f69e0f0 con 0x55910f7b1100 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.497+0000 7f9419eee640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f7b1100 0x55910f7b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.498+0000 7f93fd7fa640 10 client.15660 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.498+0000 7f9419eee640 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f7b1100 0x55910f7b3520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4670ccb7955e6265 server_cookie=7951329a9e5fb67e in_seq=0 out_seq=0 2024-09-10T22:54:11.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 220193 18.28 MB/sec warmup 66 sec latency 67.953 ms 2024-09-10T22:54:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f93fd7fa640 1 -- 172.21.15.120:0/4243194993 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55910f69e0f0 con 0x55910f7b1100 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f93fd7fa640 10 client.15660 handle_command_reply: tid=0 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 client.15660 shutdown 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 msgr2=0x55910f73ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94040cc950 tx=0x7f94040b4040 comp rx=0 tx=0).stop 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 shutdown_connections 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55910f7b1100 0x55910f7b3520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55910f733bd0 0x55910f73f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f93fd7fa640 0 client.15660 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55910f731190 0x55910f73ef90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 --2- 172.21.15.120:0/4243194993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55910f72e860 0x55910f73ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 >> 172.21.15.120:0/4243194993 conn(0x55910f7299b0 msgr2=0x55910f728bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 shutdown_connections 2024-09-10T22:54:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 1 -- 172.21.15.120:0/4243194993 wait complete. 2024-09-10T22:54:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.645+0000 7f941eff8740 20 client.15660 trim_cache size 0 max 16384 2024-09-10T22:54:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:54:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:54:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:54:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 msgr2=0x7f94000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 0x7f94000b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f940c005f40 tx=0x7f940c004a40 comp rx=0 tx=0).stop 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 msgr2=0x7f9414101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9404006010 tx=0x7f94040042f0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 shutdown_connections 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f94000b52e0 0x7f94000b77a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9414108680 0x7f9414101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9414105b10 0x7f9414101570 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 --2- 172.21.15.120:0/4228610600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f941406b3d0 0x7f9414102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 >> 172.21.15.120:0/4228610600 conn(0x7f94140fd140 msgr2=0x7f9414103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 shutdown_connections 2024-09-10T22:54:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:11.646+0000 7f941c152640 1 -- 172.21.15.120:0/4228610600 wait complete. 2024-09-10T22:54:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:11 smithi120 ceph-mon[31247]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 510 op/s 2024-09-10T22:54:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:11 smithi153 ceph-mon[34951]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 510 op/s 2024-09-10T22:54:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:11 smithi176 ceph-mon[34982]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 510 op/s 2024-09-10T22:54:12.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 222548 18.13 MB/sec warmup 67 sec latency 83.158 ms 2024-09-10T22:54:13.247 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:13.539 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:13.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 225288 18.10 MB/sec warmup 68 sec latency 22.004 ms 2024-09-10T22:54:13.661 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:13 smithi120 ceph-mon[31247]: pgmap v397: 97 pgs: 97 active+clean; 54 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-09-10T22:54:13.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:13 smithi153 ceph-mon[34951]: pgmap v397: 97 pgs: 97 active+clean; 54 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-09-10T22:54:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:13 smithi176 ceph-mon[34982]: pgmap v397: 97 pgs: 97 active+clean; 54 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 440 op/s 2024-09-10T22:54:14.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.189+0000 7fde7963b640 1 -- 172.21.15.120:0/165630155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde74101c90 msgr2=0x7fde74102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.189+0000 7fde7963b640 1 --2- 172.21.15.120:0/165630155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde74101c90 0x7fde74102070 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde6402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 -- 172.21.15.120:0/165630155 shutdown_connections 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 --2- 172.21.15.120:0/165630155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74108020 0x7fde7410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 --2- 172.21.15.120:0/165630155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde741076a0 0x7fde74107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 --2- 172.21.15.120:0/165630155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde74101c90 0x7fde74102070 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 -- 172.21.15.120:0/165630155 >> 172.21.15.120:0/165630155 conn(0x7fde740fd700 msgr2=0x7fde740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 -- 172.21.15.120:0/165630155 shutdown_connections 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 -- 172.21.15.120:0/165630155 wait complete. 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 Processor -- start 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.190+0000 7fde7963b640 1 -- start start 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 0x7fde7410a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111ce0 con 0x7fde741076a0 2024-09-10T22:54:14.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111b60 con 0x7fde74101c90 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde7963b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111e60 con 0x7fde74108020 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde737fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36522/0 (socket says 172.21.15.120:36522) 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 -- 172.21.15.120:0/2362997298 learned_addr learned my addr 172.21.15.120:0/2362997298 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde737fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47052/0 (socket says 172.21.15.120:47052) 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde72ffd640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 0x7fde7410a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 -- 172.21.15.120:0/2362997298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 msgr2=0x7fde74108f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 -- 172.21.15.120:0/2362997298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 msgr2=0x7fde7410a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 0x7fde7410a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde6bfff640 1 -- 172.21.15.120:0/2362997298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde741abcb0 con 0x7fde741076a0 2024-09-10T22:54:14.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde737fe640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.191+0000 7fde72ffd640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 0x7fde7410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde6bfff640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e9f0 tx=0x7fde5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde70ff9640 1 -- 172.21.15.120:0/2362997298 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00cd80 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde741abe80 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde70ff9640 1 -- 172.21.15.120:0/2362997298 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5c0108e0 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde70ff9640 1 -- 172.21.15.120:0/2362997298 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c018980 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.192+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde741ac380 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 Processor -- start 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 -- start start 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde70ff9640 1 -- 172.21.15.120:0/2362997298 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fde5c010430 con 0x7fde741076a0 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36e3d30 0x5651f36e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5651f364de50 con 0x5651f36d7480 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651f363d280 con 0x5651f36d7da0 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde7ba10740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5651f3641090 con 0x5651f36e3d30 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.195+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36532/0 (socket says 172.21.15.120:36532) 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde6bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36e3d30 0x5651f36e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 -- 172.21.15.120:0/239199415 learned_addr learned my addr 172.21.15.120:0/239199415 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde72ffd640 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 -- 172.21.15.120:0/239199415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36e3d30 msgr2=0x5651f36e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36e3d30 0x5651f36e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 -- 172.21.15.120:0/239199415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 msgr2=0x5651f36e37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 -- 172.21.15.120:0/239199415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f339b650 con 0x5651f36d7480 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde737fe640 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011790 tx=0x7fde6c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cba6ebeb4bcffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde697fa640 1 -- 172.21.15.120:0/239199415 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c003c30 con 0x5651f36d7480 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde70ff9640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 0x7fde600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde697fa640 1 -- 172.21.15.120:0/239199415 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c00fe50 con 0x5651f36d7480 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde697fa640 1 -- 172.21.15.120:0/239199415 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0193e0 con 0x5651f36d7480 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde72ffd640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 0x7fde600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.196+0000 7fde70ff9640 1 -- 172.21.15.120:0/2362997298 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fde5c0e5e90 con 0x7fde741076a0 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 -- 172.21.15.120:0/239199415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 msgr2=0x5651f36d7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde72ffd640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 0x7fde600b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde64031440 tx=0x7fde64005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011790 tx=0x7fde6c011c50 comp rx=0 tx=0).stop 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 -- 172.21.15.120:0/239199415 shutdown_connections 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36e3d30 0x5651f36e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 --2- 172.21.15.120:0/239199415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36d7480 0x5651f36d7860 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 -- 172.21.15.120:0/239199415 >> 172.21.15.120:0/239199415 conn(0x5651f36d52b0 msgr2=0x5651f36d6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 -- 172.21.15.120:0/239199415 shutdown_connections 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.197+0000 7fde7ba10740 1 -- 172.21.15.120:0/239199415 wait complete. 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 -- start start 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 Processor -- start 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7480 0x5651f36e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5651f349e700 con 0x5651f36e3d30 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5651f35d8110 con 0x5651f36d7da0 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5651f36e69b0 con 0x5651f36d7480 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde7ba10740 10 client.0 _ll_register_callbacks cb 0x5651f35f1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36544/0 (socket says 172.21.15.120:36544) 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 -- 172.21.15.120:0/3186619297 learned_addr learned my addr 172.21.15.120:0/3186619297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:14.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde72ffd640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7480 0x5651f36e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde6bfff640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 -- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7480 msgr2=0x5651f36e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7480 0x5651f36e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 -- 172.21.15.120:0/3186619297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 msgr2=0x5651f36e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.198+0000 7fde737fe640 1 -- 172.21.15.120:0/3186619297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f3605c40 con 0x5651f36e3d30 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde737fe640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006010 tx=0x7fde6c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25914b20b57202ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02a060 con 0x5651f36e3d30 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fde6c011070 con 0x5651f36e3d30 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c00fe50 con 0x5651f36e3d30 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fde6c01f040 con 0x5651f36e3d30 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde7ba10740 10 client.15672 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:14.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5651f36f22d0 con 0x5651f36e3d30 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.199+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c00fe50 con 0x5651f36e3d30 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fde6c018070 con 0x5651f36e3d30 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 10 client.15672 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 20 client.15672 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f36f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 4 client.15672 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651f364fda0 con 0x5651f36f36d0 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde72ffd640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f36f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde5a7fc640 10 client.15672 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.200+0000 7fde72ffd640 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f36f5ac0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40ec8bcfd01731b server_cookie=7e17355dd30650c1 in_seq=0 out_seq=0 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde5a7fc640 1 -- 172.21.15.120:0/3186619297 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651f364fda0 con 0x5651f36f36d0 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde5a7fc640 10 client.15672 handle_command_reply: tid=0 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 client.15672 shutdown 2024-09-10T22:54:14.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 msgr2=0x5651f36e1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006010 tx=0x7fde6c004440 comp rx=0 tx=0).stop 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 shutdown_connections 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f36f5ac0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde5a7fc640 0 client.15672 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e1090 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7da0 0x5651f36e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 --2- 172.21.15.120:0/3186619297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7480 0x5651f36e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 >> 172.21.15.120:0/3186619297 conn(0x5651f36d52b0 msgr2=0x5651f36dae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 shutdown_connections 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.202+0000 7fde7ba10740 1 -- 172.21.15.120:0/3186619297 wait complete. 2024-09-10T22:54:14.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.203+0000 7fde7ba10740 20 client.15672 trim_cache size 0 max 16384 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 Processor -- start 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 -- start start 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b2a40 con 0x5651f36e3d30 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b52c0 con 0x5651f36d7480 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.221+0000 7fde7ba10740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b30b0 con 0x5651f36d7da0 2024-09-10T22:54:14.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36560/0 (socket says 172.21.15.120:36560) 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 -- 172.21.15.120:0/191047337 learned_addr learned my addr 172.21.15.120:0/191047337 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde6bfff640 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 -- 172.21.15.120:0/191047337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 msgr2=0x5651f36efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 -- 172.21.15.120:0/191047337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 msgr2=0x5651f36d6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36d6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 -- 172.21.15.120:0/191047337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f36def60 con 0x5651f36e3d30 2024-09-10T22:54:14.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde72ffd640 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde737fe640 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde6c002230 tx=0x7fde6c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145101595325c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde68ff9640 1 -- 172.21.15.120:0/191047337 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c035020 con 0x5651f36e3d30 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde68ff9640 1 -- 172.21.15.120:0/191047337 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c00f790 con 0x5651f36e3d30 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.222+0000 7fde68ff9640 1 -- 172.21.15.120:0/191047337 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02a040 con 0x5651f36e3d30 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.223+0000 7fde7ba10740 1 -- 172.21.15.120:0/191047337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 msgr2=0x5651f36e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.223+0000 7fde7ba10740 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde6c002230 tx=0x7fde6c005bd0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.223+0000 7fde7ba10740 1 -- 172.21.15.120:0/191047337 shutdown_connections 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.223+0000 7fde7ba10740 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e5120 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 --2- 172.21.15.120:0/191047337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 -- 172.21.15.120:0/191047337 >> 172.21.15.120:0/191047337 conn(0x5651f36d52b0 msgr2=0x5651f36d63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 -- 172.21.15.120:0/191047337 shutdown_connections 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 -- 172.21.15.120:0/191047337 wait complete. 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 -- start start 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 Processor -- start 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.224+0000 7fde7ba10740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde7ba10740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b3780 con 0x5651f36e3d30 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde7ba10740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b4bf0 con 0x5651f36d7480 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde7ba10740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b3e50 con 0x5651f36d7da0 2024-09-10T22:54:14.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53972/0 (socket says 172.21.15.120:53972) 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 -- 172.21.15.120:0/203487312 learned_addr learned my addr 172.21.15.120:0/203487312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde7ba10740 10 client.0 _ll_register_callbacks cb 0x7fde5c0e9840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde6bfff640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 -- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 msgr2=0x5651f36ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36ec190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 -- 172.21.15.120:0/203487312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 msgr2=0x5651f36e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 -- 172.21.15.120:0/203487312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f36ec920 con 0x5651f36d7480 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde737fe640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde72ffd640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde6402a930 tx=0x7fde64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f335c5d6d5acae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.225+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64035040 con 0x5651f36d7480 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fde64005040 con 0x5651f36d7480 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c00f790 con 0x5651f36d7480 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fde64005240 con 0x5651f36d7480 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 10 client.25160 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:14.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5651f37609d0 con 0x5651f36d7480 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c00f790 con 0x5651f36d7480 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fde64003710 con 0x5651f36d7480 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 10 client.25160 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 10 client.25160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 20 client.25160 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f3763130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 4 client.25160 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651f364b850 con 0x5651f36f36d0 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.226+0000 7fde6bfff640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f3763130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde5a7fc640 10 client.25160 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde6bfff640 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f3763130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f457cff310cf4203 server_cookie=f0caec4171919890 in_seq=0 out_seq=0 2024-09-10T22:54:14.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde5a7fc640 1 -- 172.21.15.120:0/203487312 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5651f364b850 con 0x5651f36f36d0 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde5a7fc640 10 client.25160 handle_command_reply: tid=0 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 client.25160 shutdown 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 msgr2=0x5651f36ebc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde6402a930 tx=0x7fde64009260 comp rx=0 tx=0).stop 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 shutdown_connections 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5651f36f36d0 0x5651f3763130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5651f36e3d30 0x5651f36e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5651f36d7da0 0x5651f36ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 --2- 172.21.15.120:0/203487312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5651f36d7480 0x5651f36ebc50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde5a7fc640 0 client.25160 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 >> 172.21.15.120:0/203487312 conn(0x5651f36d52b0 msgr2=0x5651f36e2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 shutdown_connections 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 1 -- 172.21.15.120:0/203487312 wait complete. 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.227+0000 7fde7ba10740 20 client.25160 trim_cache size 0 max 16384 2024-09-10T22:54:14.240 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:14.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:14.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:14.243 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:54:14.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 msgr2=0x7fde600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 0x7fde600b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fde64031440 tx=0x7fde64005b50 comp rx=0 tx=0).stop 2024-09-10T22:54:14.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 msgr2=0x7fde74108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:14.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e9f0 tx=0x7fde5c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 shutdown_connections 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde600b52e0 0x7fde600b77a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde74108020 0x7fde74108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde741076a0 0x7fde74108910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 --2- 172.21.15.120:0/2362997298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde74101c90 0x7fde7410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 >> 172.21.15.120:0/2362997298 conn(0x7fde740fd700 msgr2=0x7fde740fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 shutdown_connections 2024-09-10T22:54:14.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:14.228+0000 7fde7963b640 1 -- 172.21.15.120:0/2362997298 wait complete. 2024-09-10T22:54:14.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 228525 18.11 MB/sec warmup 69 sec latency 25.569 ms 2024-09-10T22:54:15.149 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:54:15.149 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:54:15.417 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:15.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:15 smithi120 ceph-mon[31247]: pgmap v398: 97 pgs: 97 active+clean; 47 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T22:54:15.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 231644 18.08 MB/sec warmup 70 sec latency 12.422 ms 2024-09-10T22:54:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:15 smithi153 ceph-mon[34951]: pgmap v398: 97 pgs: 97 active+clean; 47 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T22:54:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:15 smithi176 ceph-mon[34982]: pgmap v398: 97 pgs: 97 active+clean; 47 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T22:54:15.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 -- 172.21.15.120:0/647977447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b801076a0 msgr2=0x7f4b80107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/647977447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b801076a0 0x7f4b80107ae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009900 tx=0x7f4b7002a410 comp rx=0 tx=0).stop 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 -- 172.21.15.120:0/647977447 shutdown_connections 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/647977447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b80108020 0x7f4b8010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/647977447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b801076a0 0x7f4b80107ae0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/647977447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 0x7f4b80102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 -- 172.21.15.120:0/647977447 >> 172.21.15.120:0/647977447 conn(0x7f4b800fd700 msgr2=0x7f4b800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 -- 172.21.15.120:0/647977447 shutdown_connections 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 -- 172.21.15.120:0/647977447 wait complete. 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.970+0000 7f4b86ea3640 1 Processor -- start 2024-09-10T22:54:15.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 -- start start 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 0x7f4b8019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b801076a0 0x7f4b8019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111ca0 con 0x7f4b80108020 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111b20 con 0x7f4b80101c90 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b86ea3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80111e20 con 0x7f4b801076a0 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36592/0 (socket says 172.21.15.120:36592) 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 -- 172.21.15.120:0/2323401514 learned_addr learned my addr 172.21.15.120:0/2323401514 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b77fff640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b801076a0 0x7f4b8019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b84c3f640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 0x7f4b8019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 -- 172.21.15.120:0/2323401514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b801076a0 msgr2=0x7f4b8019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b801076a0 0x7f4b8019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:15.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 -- 172.21.15.120:0/2323401514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 msgr2=0x7f4b8019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 0x7f4b8019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 -- 172.21.15.120:0/2323401514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b801a54b0 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.971+0000 7f4b85440640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800eb10 tx=0x7f4b7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.972+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004b20 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.972+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b78004cc0 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.972+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b801a57a0 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.972+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78010670 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.972+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b801a5cb0 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.973+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b80102070 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.974+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4b78005020 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.975+0000 7f4b75ffb640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 0x7f4b680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.975+0000 7f4b84c3f640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 0x7f4b680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.975+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f4b780e6100 con 0x7f4b80108020 2024-09-10T22:54:15.992 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.975+0000 7f4b84c3f640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 0x7f4b680b77a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009bd0 tx=0x7f4b7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:15.993 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:15.979+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b780acff0 con 0x7f4b80108020 2024-09-10T22:54:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.203+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b8002e5c0 con 0x7f4b80108020 2024-09-10T22:54:16.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.204+0000 7f4b75ffb640 1 -- 172.21.15.120:0/2323401514 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11868 (secure 0 0 0) 0x7f4b780ac8f0 con 0x7f4b80108020 2024-09-10T22:54:16.210 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:54:16.210 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":21,"btime":"2024-09-10T22:53:19:261525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:53:19.261509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24998":{"gid":24998,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/4005968272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":4005968272},{"type":"v1","addr":"172.21.15.176:6837","nonce":4005968272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:54:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 21 2024-09-10T22:54:16.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 msgr2=0x7f4b680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 0x7f4b680b77a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009bd0 tx=0x7f4b7c005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 msgr2=0x7f4b8019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800eb10 tx=0x7f4b7800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 shutdown_connections 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b680b52e0 0x7f4b680b77a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b80108020 0x7f4b8019ab70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b801076a0 0x7f4b8019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 --2- 172.21.15.120:0/2323401514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b80101c90 0x7f4b8019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 >> 172.21.15.120:0/2323401514 conn(0x7f4b800fd700 msgr2=0x7f4b800ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 shutdown_connections 2024-09-10T22:54:16.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:16.206+0000 7f4b86ea3640 1 -- 172.21.15.120:0/2323401514 wait complete. 2024-09-10T22:54:16.543 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:16 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2323401514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:16.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 234833 18.07 MB/sec warmup 71 sec latency 39.263 ms 2024-09-10T22:54:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:16 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2323401514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:16.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:16 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2323401514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:16.904 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:54:17.162 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:17.590 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:17 smithi120 ceph-mon[31247]: pgmap v399: 97 pgs: 97 active+clean; 42 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 489 op/s 2024-09-10T22:54:17.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 238096 18.09 MB/sec warmup 72 sec latency 50.818 ms 2024-09-10T22:54:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 -- 172.21.15.120:0/1809645253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93541087f0 msgr2=0x7f9354108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 --2- 172.21.15.120:0/1809645253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93541087f0 0x7f9354108c30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9340009900 tx=0x7f934002a410 comp rx=0 tx=0).stop 2024-09-10T22:54:17.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 -- 172.21.15.120:0/1809645253 shutdown_connections 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 --2- 172.21.15.120:0/1809645253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354109170 0x7f935410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 --2- 172.21.15.120:0/1809645253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93541087f0 0x7f9354108c30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 --2- 172.21.15.120:0/1809645253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f93541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 -- 172.21.15.120:0/1809645253 >> 172.21.15.120:0/1809645253 conn(0x7f93540fe830 msgr2=0x7f9354100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 -- 172.21.15.120:0/1809645253 shutdown_connections 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.669+0000 7f935b05f640 1 -- 172.21.15.120:0/1809645253 wait complete. 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 Processor -- start 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 -- start start 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93541087f0 0x7f935419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 0x7f93541a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112df0 con 0x7f9354109170 2024-09-10T22:54:17.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112c70 con 0x7f9354102de0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f935b05f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112f70 con 0x7f93541087f0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53996/0 (socket says 172.21.15.120:53996) 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f93595fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 0x7f93541a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 -- 172.21.15.120:0/2544685346 learned_addr learned my addr 172.21.15.120:0/2544685346 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9353fff640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93541087f0 0x7f935419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 -- 172.21.15.120:0/2544685346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93541087f0 msgr2=0x7f935419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93541087f0 0x7f935419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 -- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 msgr2=0x7f93541a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 0x7f93541a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 -- 172.21.15.120:0/2544685346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93541a1540 con 0x7f9354102de0 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f93595fc640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 0x7f93541a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:17.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.670+0000 7f9358dfb640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9f0 tx=0x7f934800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.671+0000 7f9351ffb640 1 -- 172.21.15.120:0/2544685346 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934800cd80 con 0x7f9354102de0 2024-09-10T22:54:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.671+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93541a1830 con 0x7f9354102de0 2024-09-10T22:54:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.671+0000 7f9351ffb640 1 -- 172.21.15.120:0/2544685346 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93480108e0 con 0x7f9354102de0 2024-09-10T22:54:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.671+0000 7f9351ffb640 1 -- 172.21.15.120:0/2544685346 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348018980 con 0x7f9354102de0 2024-09-10T22:54:17.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.671+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541a80c0 con 0x7f9354102de0 2024-09-10T22:54:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.673+0000 7f9351ffb640 1 -- 172.21.15.120:0/2544685346 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9348010430 con 0x7f9354102de0 2024-09-10T22:54:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.673+0000 7f935d434740 1 Processor -- start 2024-09-10T22:54:17.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.673+0000 7f935d434740 1 -- start start 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564441ddeea0 con 0x564441e74d80 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564441dce2d0 con 0x564441e684d0 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f935d434740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564441dd20e0 con 0x564441e68df0 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54012/0 (socket says 172.21.15.120:54012) 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 -- 172.21.15.120:0/2699034824 learned_addr learned my addr 172.21.15.120:0/2699034824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f9353fff640 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f9351ffb640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 0x7f93440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 0x7f93440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 -- 172.21.15.120:0/2699034824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 msgr2=0x564441e74840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f9351ffb640 1 -- 172.21.15.120:0/2544685346 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f93480e5f50 con 0x7f9354102de0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e74840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 -- 172.21.15.120:0/2699034824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 msgr2=0x564441e77170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e77170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f93595fc640 1 -- 172.21.15.120:0/2699034824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564441b2c650 con 0x564441e684d0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.674+0000 7f9358dfb640 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e74840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f93595fc640 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f934c009fd0 tx=0x7f934c013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d60bf625fe80ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f933effd640 1 -- 172.21.15.120:0/2699034824 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c022020 con 0x564441e684d0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f933effd640 1 -- 172.21.15.120:0/2699034824 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c004620 con 0x564441e684d0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f93595fc640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 0x7f93440b77a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f934c023000 tx=0x7f934c021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f933effd640 1 -- 172.21.15.120:0/2699034824 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c01e400 con 0x564441e684d0 2024-09-10T22:54:17.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 -- 172.21.15.120:0/2699034824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 msgr2=0x564441e688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f934c009fd0 tx=0x7f934c013830 comp rx=0 tx=0).stop 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 -- 172.21.15.120:0/2699034824 shutdown_connections 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e77170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e74840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 --2- 172.21.15.120:0/2699034824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e688b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.675+0000 7f935d434740 1 -- 172.21.15.120:0/2699034824 >> 172.21.15.120:0/2699034824 conn(0x564441e66300 msgr2=0x564441e674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- 172.21.15.120:0/2699034824 shutdown_connections 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- 172.21.15.120:0/2699034824 wait complete. 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- start start 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 Processor -- start 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564441c2f700 con 0x564441e74d80 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564441d69160 con 0x564441e684d0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564441e77a00 con 0x564441e68df0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.676+0000 7f935d434740 10 client.0 _ll_register_callbacks cb 0x564441d829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f93595fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f93595fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36642/0 (socket says 172.21.15.120:36642) 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47120/0 (socket says 172.21.15.120:47120) 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54016/0 (socket says 172.21.15.120:54016) 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f93595fc640 1 -- 172.21.15.120:0/3253217718 learned_addr learned my addr 172.21.15.120:0/3253217718 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 -- 172.21.15.120:0/3253217718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 msgr2=0x564441e74640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 -- 172.21.15.120:0/3253217718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 msgr2=0x564441e72070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 -- 172.21.15.120:0/3253217718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564441d96c90 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f93595fc640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9353fff640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9340009230 tx=0x7f93400314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b15fabeb3b6304 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9333fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340035040 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f9340031810 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934c004620 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.677+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9340004040 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 10 client.25168 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564441e83330 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934c004620 con 0x564441e68df0 2024-09-10T22:54:17.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9340004750 con 0x564441e68df0 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 10 client.25168 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 10 client.25168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 20 client.25168 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e847f0 0x564441e86be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 4 client.25168 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564441de0df0 con 0x564441e847f0 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e847f0 0x564441e86be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.678+0000 7f9333fff640 10 client.25168 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:17.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.679+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e847f0 0x564441e86be0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b3c67dfc8b246a server_cookie=6bd31176acf46699 in_seq=0 out_seq=0 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.680+0000 7f9333fff640 1 -- 172.21.15.120:0/3253217718 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564441de0df0 con 0x564441e847f0 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.680+0000 7f9333fff640 10 client.25168 handle_command_reply: tid=0 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.680+0000 7f935d434740 1 client.25168 shutdown 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.680+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 msgr2=0x564441e71a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.680+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9340009230 tx=0x7f93400314a0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 shutdown_connections 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e847f0 0x564441e86be0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e74d80 0x564441e72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e68df0 0x564441e71a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f9333fff640 0 client.25168 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 --2- 172.21.15.120:0/3253217718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e684d0 0x564441e74640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 >> 172.21.15.120:0/3253217718 conn(0x564441e66300 msgr2=0x564441e6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 shutdown_connections 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 1 -- 172.21.15.120:0/3253217718 wait complete. 2024-09-10T22:54:17.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.681+0000 7f935d434740 20 client.25168 trim_cache size 0 max 16384 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 Processor -- start 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 -- start start 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e68df0 0x564441e673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e74d80 0x564441e67900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93541ae4b0 con 0x564441e684d0 2024-09-10T22:54:17.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93541b0d30 con 0x564441e68df0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f935d434740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f93541aeb20 con 0x564441e74d80 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.694+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36646/0 (socket says 172.21.15.120:36646) 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 -- 172.21.15.120:0/3995264594 learned_addr learned my addr 172.21.15.120:0/3995264594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f93595fc640 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e74d80 0x564441e67900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9353fff640 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e68df0 0x564441e673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 -- 172.21.15.120:0/3995264594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e74d80 msgr2=0x564441e67900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e74d80 0x564441e67900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 -- 172.21.15.120:0/3995264594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e68df0 msgr2=0x564441e673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e68df0 0x564441e673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 -- 172.21.15.120:0/3995264594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564441e87260 con 0x564441e684d0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f93480a8f50 tx=0x7f93480a8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20deec251347d6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f933e7fc640 1 -- 172.21.15.120:0/3995264594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480ac790 con 0x564441e684d0 2024-09-10T22:54:17.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f933e7fc640 1 -- 172.21.15.120:0/3995264594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93480b11c0 con 0x564441e684d0 2024-09-10T22:54:17.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.695+0000 7f933e7fc640 1 -- 172.21.15.120:0/3995264594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348018d80 con 0x564441e684d0 2024-09-10T22:54:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:17 smithi153 ceph-mon[34951]: pgmap v399: 97 pgs: 97 active+clean; 42 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 489 op/s 2024-09-10T22:54:17.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- 172.21.15.120:0/3995264594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 msgr2=0x564441e70f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f93480a8f50 tx=0x7f93480a8760 comp rx=0 tx=0).stop 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- 172.21.15.120:0/3995264594 shutdown_connections 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e74d80 0x564441e67900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e68df0 0x564441e673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 --2- 172.21.15.120:0/3995264594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e684d0 0x564441e70f80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- 172.21.15.120:0/3995264594 >> 172.21.15.120:0/3995264594 conn(0x564441e66300 msgr2=0x564441e70310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- 172.21.15.120:0/3995264594 shutdown_connections 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- 172.21.15.120:0/3995264594 wait complete. 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 -- start start 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.696+0000 7f935d434740 1 Processor -- start 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 0x564441e7ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e74d80 0x564441e6dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93541af1f0 con 0x564441e68df0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93541b0660 con 0x564441e74d80 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f93541af8c0 con 0x564441e684d0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 10 client.0 _ll_register_callbacks cb 0x7f9340009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 0x564441e7ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9358dfb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 0x564441e7ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47144/0 (socket says 172.21.15.120:47144) 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36658/0 (socket says 172.21.15.120:36658) 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9358dfb640 1 -- 172.21.15.120:0/3300602162 learned_addr learned my addr 172.21.15.120:0/3300602162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f93595fc640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e74d80 0x564441e6dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 -- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 msgr2=0x564441e7ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 0x564441e7ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 -- 172.21.15.120:0/3300602162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e74d80 msgr2=0x564441e6dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:17.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e74d80 0x564441e6dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 -- 172.21.15.120:0/3300602162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564441e7d840 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9353fff640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9340009850 tx=0x7f9340004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36e603d192a71ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9333fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340035070 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f9340003c10 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.697+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93480b11c0 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9340003e10 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 10 client.15702 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564441e6f0e0 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93480b11c0 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9340031510 con 0x564441e68df0 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 10 client.15702 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:17.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 20 client.15702 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e6f800 0x564441ef46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 4 client.15702 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564441ddc8a0 con 0x564441e6f800 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.698+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e6f800 0x564441ef46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.699+0000 7f9333fff640 10 client.15702 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:17.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.699+0000 7f9358dfb640 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e6f800 0x564441ef46d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fbe77a5a454f7b server_cookie=649f2206846ad8d6 in_seq=0 out_seq=0 2024-09-10T22:54:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f9333fff640 1 -- 172.21.15.120:0/3300602162 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564441ddc8a0 con 0x564441e6f800 2024-09-10T22:54:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f9333fff640 10 client.15702 handle_command_reply: tid=0 2024-09-10T22:54:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 client.15702 shutdown 2024-09-10T22:54:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 msgr2=0x564441e7d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9340009850 tx=0x7f9340004410 comp rx=0 tx=0).stop 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 shutdown_connections 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564441e6f800 0x564441ef46d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564441e74d80 0x564441e6dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564441e68df0 0x564441e7d200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 --2- 172.21.15.120:0/3300602162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564441e684d0 0x564441e7ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 >> 172.21.15.120:0/3300602162 conn(0x564441e66300 msgr2=0x564441e6ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f9333fff640 0 client.15702 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 shutdown_connections 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 1 -- 172.21.15.120:0/3300602162 wait complete. 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.700+0000 7f935d434740 20 client.15702 trim_cache size 0 max 16384 2024-09-10T22:54:17.707 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:54:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 msgr2=0x7f93440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 0x7f93440b77a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f934c023000 tx=0x7f934c021000 comp rx=0 tx=0).stop 2024-09-10T22:54:17.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 msgr2=0x7f935419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9f0 tx=0x7f934800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 shutdown_connections 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93440b52e0 0x7f93440b77a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9354109170 0x7f93541a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93541087f0 0x7f935419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 --2- 172.21.15.120:0/2544685346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9354102de0 0x7f935419b490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 >> 172.21.15.120:0/2544685346 conn(0x7f93540fe830 msgr2=0x7f9354101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 shutdown_connections 2024-09-10T22:54:17.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:17.701+0000 7f935b05f640 1 -- 172.21.15.120:0/2544685346 wait complete. 2024-09-10T22:54:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:17 smithi176 ceph-mon[34982]: pgmap v399: 97 pgs: 97 active+clean; 42 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 489 op/s 2024-09-10T22:54:18.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 241493 18.10 MB/sec warmup 73 sec latency 30.124 ms 2024-09-10T22:54:19.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 245039 18.11 MB/sec warmup 74 sec latency 14.746 ms 2024-09-10T22:54:19.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:19 smithi120 ceph-mon[31247]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T22:54:19.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:19 smithi153 ceph-mon[34951]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T22:54:19.726 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:54:19.726 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:54:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:19 smithi176 ceph-mon[34982]: pgmap v400: 97 pgs: 97 active+clean; 43 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T22:54:19.999 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:20.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 248450 18.11 MB/sec warmup 75 sec latency 40.681 ms 2024-09-10T22:54:20.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 -- 172.21.15.120:0/392909186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 msgr2=0x7fe580104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 --2- 172.21.15.120:0/392909186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580104080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009900 tx=0x7fe57c02a410 comp rx=0 tx=0).stop 2024-09-10T22:54:20.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 -- 172.21.15.120:0/392909186 shutdown_connections 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 --2- 172.21.15.120:0/392909186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 0x7fe58010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 --2- 172.21.15.120:0/392909186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe58010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 --2- 172.21.15.120:0/392909186 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580104080 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 -- 172.21.15.120:0/392909186 >> 172.21.15.120:0/392909186 conn(0x7fe5800fd700 msgr2=0x7fe5800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 -- 172.21.15.120:0/392909186 shutdown_connections 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.805+0000 7fe5881ea640 1 -- 172.21.15.120:0/392909186 wait complete. 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 Processor -- start 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 -- start start 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe5801997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 0x7fe58019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe580112d70 con 0x7fe580103ca0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe580112bf0 con 0x7fe58010d100 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe5881ea640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe580112ef0 con 0x7fe5801045c0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36684/0 (socket says 172.21.15.120:36684) 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 -- 172.21.15.120:0/1127139049 learned_addr learned my addr 172.21.15.120:0/1127139049 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585785640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe5801997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe586787640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 0x7fe58019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 -- 172.21.15.120:0/1127139049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 msgr2=0x7fe5801997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe5801997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 -- 172.21.15.120:0/1127139049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 msgr2=0x7fe58019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 0x7fe58019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585f86640 1 -- 172.21.15.120:0/1127139049 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58019f330 con 0x7fe580103ca0 2024-09-10T22:54:20.828 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.806+0000 7fe585785640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe5801997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe585f86640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005940 tx=0x7fe57c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe576ffd640 1 -- 172.21.15.120:0/1127139049 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c03f030 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58019f5c0 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe576ffd640 1 -- 172.21.15.120:0/1127139049 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c003aa0 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe576ffd640 1 -- 172.21.15.120:0/1127139049 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c035070 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.807+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5801a5ce0 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.809+0000 7fe576ffd640 1 -- 172.21.15.120:0/1127139049 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe57c003c50 con 0x7fe580103ca0 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 Processor -- start 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 -- start start 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf07f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf081b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584befe9e60 con 0x5584bf07f790 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5584befd9290 con 0x5584bf073800 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5584befdd0a0 con 0x5584bf072ee0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe576ffd640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 0x7fe56c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe586787640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe585785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf081b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe586787640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47158/0 (socket says 172.21.15.120:47158) 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe586787640 1 -- 172.21.15.120:0/1128113382 learned_addr learned my addr 172.21.15.120:0/1128113382 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe585f86640 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf07f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe586787640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 0x7fe56c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.810+0000 7fe576ffd640 1 -- 172.21.15.120:0/1127139049 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c1064a0 con 0x7fe580103ca0 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 -- 172.21.15.120:0/1128113382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 msgr2=0x5584bf07f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf07f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 -- 172.21.15.120:0/1128113382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 msgr2=0x5584bf081b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf081b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 -- 172.21.15.120:0/1128113382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bed37930 con 0x5584bf072ee0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe585785640 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf081b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe578012ab0 tx=0x7fe578010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2f2410911630f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe586787640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 0x7fe56c0b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe578010c40 tx=0x7fe57800c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe56bfff640 1 -- 172.21.15.120:0/1128113382 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578009d80 con 0x5584bf072ee0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe56bfff640 1 -- 172.21.15.120:0/1128113382 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe578022030 con 0x5584bf072ee0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.811+0000 7fe56bfff640 1 -- 172.21.15.120:0/1128113382 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780056c0 con 0x5584bf072ee0 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- 172.21.15.120:0/1128113382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 msgr2=0x5584bf0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe578012ab0 tx=0x7fe578010740 comp rx=0 tx=0).stop 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- 172.21.15.120:0/1128113382 shutdown_connections 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf081b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf07f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/1128113382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf0732c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- 172.21.15.120:0/1128113382 >> 172.21.15.120:0/1128113382 conn(0x5584bf070d10 msgr2=0x5584bf071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- 172.21.15.120:0/1128113382 shutdown_connections 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- 172.21.15.120:0/1128113382 wait complete. 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 -- start start 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.812+0000 7fe58a5bf740 1 Processor -- start 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 0x5584bf07cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584bee3a890 con 0x5584bf073800 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5584bef73d40 con 0x5584bf07f790 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5584bf0823e0 con 0x5584bf072ee0 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe58a5bf740 10 client.0 _ll_register_callbacks cb 0x5584befd7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:20.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 0x5584bf07cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47162/0 (socket says 172.21.15.120:47162) 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 -- 172.21.15.120:0/3652036437 learned_addr learned my addr 172.21.15.120:0/3652036437 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 0x5584bf07cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36710/0 (socket says 172.21.15.120:36710) 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe586787640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 -- 172.21.15.120:0/3652036437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 msgr2=0x5584bf07d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 -- 172.21.15.120:0/3652036437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 msgr2=0x5584bf07cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 0x5584bf07cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.813+0000 7fe585f86640 1 -- 172.21.15.120:0/3652036437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bf08b840 con 0x5584bf072ee0 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe585f86640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0cca40 tx=0x7fe57c04a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab5d5b324ecb891 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c049050 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c0c8fb0 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe578022030 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe57c0cc690 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe58a5bf740 10 client.25186 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5584bf08e1f0 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe578022030 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe57c040270 con 0x5584bf072ee0 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.814+0000 7fe58a5bf740 10 client.25186 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe58a5bf740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe58a5bf740 20 client.25186 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf091b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.834 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe58a5bf740 4 client.25186 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584befebdb0 con 0x5584bf08f720 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe585785640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf091b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe5697fa640 10 client.25186 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.815+0000 7fe585785640 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf091b30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d634594eb6ac69 server_cookie=7638a287a78c54d3 in_seq=0 out_seq=0 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe5697fa640 1 -- 172.21.15.120:0/3652036437 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5584befebdb0 con 0x5584bf08f720 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe5697fa640 10 client.25186 handle_command_reply: tid=0 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 client.25186 shutdown 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 msgr2=0x5584bf07c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0cca40 tx=0x7fe57c04a450 comp rx=0 tx=0).stop 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 shutdown_connections 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf091b30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe5697fa640 0 client.25186 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:20.835 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf073800 0x5584bf07cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/3652036437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf07c7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 >> 172.21.15.120:0/3652036437 conn(0x5584bf070d10 msgr2=0x5584bf06ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 shutdown_connections 2024-09-10T22:54:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 1 -- 172.21.15.120:0/3652036437 wait complete. 2024-09-10T22:54:20.836 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.817+0000 7fe58a5bf740 20 client.25186 trim_cache size 0 max 16384 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.839+0000 7fe58a5bf740 1 Processor -- start 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 -- start start 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf073800 0x5584bf07cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801aa090 con 0x5584bf072ee0 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ac910 con 0x5584bf07f790 2024-09-10T22:54:20.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801aa700 con 0x5584bf073800 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36724/0 (socket says 172.21.15.120:36724) 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 -- 172.21.15.120:0/2391769005 learned_addr learned my addr 172.21.15.120:0/2391769005 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585785640 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf073800 0x5584bf07cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe586787640 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 -- 172.21.15.120:0/2391769005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf073800 msgr2=0x5584bf07cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf073800 0x5584bf07cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 -- 172.21.15.120:0/2391769005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 msgr2=0x5584bf07d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.840+0000 7fe585f86640 1 -- 172.21.15.120:0/2391769005 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bf07ace0 con 0x5584bf072ee0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe585f86640 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009340 tx=0x7fe57c0b42e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb293ee0d52e211 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe56b7fe640 1 -- 172.21.15.120:0/2391769005 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c049050 con 0x5584bf072ee0 2024-09-10T22:54:20.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe56b7fe640 1 -- 172.21.15.120:0/2391769005 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c07c020 con 0x5584bf072ee0 2024-09-10T22:54:20.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe56b7fe640 1 -- 172.21.15.120:0/2391769005 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0cc4b0 con 0x5584bf072ee0 2024-09-10T22:54:20.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe58a5bf740 1 -- 172.21.15.120:0/2391769005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 msgr2=0x5584bf07e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.841+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009340 tx=0x7fe57c0b42e0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- 172.21.15.120:0/2391769005 shutdown_connections 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf07f790 0x5584bf07d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf073800 0x5584bf07cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/2391769005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf072ee0 0x5584bf07e520 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- 172.21.15.120:0/2391769005 >> 172.21.15.120:0/2391769005 conn(0x5584bf070d10 msgr2=0x5584bf07de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- 172.21.15.120:0/2391769005 shutdown_connections 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- 172.21.15.120:0/2391769005 wait complete. 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- start start 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 Processor -- start 2024-09-10T22:54:20.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf083920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf083e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801aadd0 con 0x5584bf07f790 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ac240 con 0x5584bf073800 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.842+0000 7fe58a5bf740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ab4a0 con 0x5584bf072ee0 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe58a5bf740 10 client.0 _ll_register_callbacks cb 0x7fe57c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36728/0 (socket says 172.21.15.120:36728) 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 -- 172.21.15.120:0/799514766 learned_addr learned my addr 172.21.15.120:0/799514766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe585f86640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf083920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe585785640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf083e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 -- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 msgr2=0x5584bf083920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf083920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 -- 172.21.15.120:0/799514766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 msgr2=0x5584bf083e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf083e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 -- 172.21.15.120:0/799514766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bf0fbb40 con 0x5584bf07f790 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe586787640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe578009ac0 tx=0x7fe578005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e9f36f2b363e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578036070 con 0x5584bf07f790 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7fe5780053d0 con 0x5584bf07f790 2024-09-10T22:54:20.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c07c020 con 0x5584bf07f790 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe5780055d0 con 0x5584bf07f790 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe58a5bf740 10 client.15720 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.843+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5584bf0fccd0 con 0x5584bf07f790 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c07c020 con 0x5584bf07f790 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe57800f040 con 0x5584bf07f790 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 10 client.15720 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 20 client.15720 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf0ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 4 client.15720 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5584befe7860 con 0x5584bf08f720 2024-09-10T22:54:20.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe585f86640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf0ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:20.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe5697fa640 10 client.15720 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:20.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.844+0000 7fe585f86640 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf0ff170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e91f51006aceb9 server_cookie=6a720d5721cd1594 in_seq=0 out_seq=0 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.846+0000 7fe5697fa640 1 -- 172.21.15.120:0/799514766 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5584befe7860 con 0x5584bf08f720 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.846+0000 7fe5697fa640 10 client.15720 handle_command_reply: tid=0 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.846+0000 7fe58a5bf740 1 client.15720 shutdown 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 msgr2=0x5584bf0fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe578009ac0 tx=0x7fe578005f10 comp rx=0 tx=0).stop 2024-09-10T22:54:20.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 shutdown_connections 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584bf08f720 0x5584bf0ff170 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584bf07f790 0x5584bf0fb600 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe5697fa640 0 client.15720 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584bf073800 0x5584bf083e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 --2- 172.21.15.120:0/799514766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584bf072ee0 0x5584bf083920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 >> 172.21.15.120:0/799514766 conn(0x5584bf070d10 msgr2=0x5584bf07e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 shutdown_connections 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 1 -- 172.21.15.120:0/799514766 wait complete. 2024-09-10T22:54:20.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.847+0000 7fe58a5bf740 20 client.15720 trim_cache size 0 max 16384 2024-09-10T22:54:20.855 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:54:20.855 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:54:20.855 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "ec85941b-87cc-4645-b788-d806c41ade31", 2024-09-10T22:54:20.855 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:54:20.855 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 msgr2=0x7fe56c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 0x7fe56c0b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe578010c40 tx=0x7fe57800c000 comp rx=0 tx=0).stop 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 msgr2=0x7fe580199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005940 tx=0x7fe57c004270 comp rx=0 tx=0).stop 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 shutdown_connections 2024-09-10T22:54:20.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe56c0b52e0 0x7fe56c0b77a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe58010d100 0x7fe58019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5801045c0 0x7fe5801997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 --2- 172.21.15.120:0/1127139049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe580103ca0 0x7fe580199280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 >> 172.21.15.120:0/1127139049 conn(0x7fe5800fd700 msgr2=0x7fe5800fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 shutdown_connections 2024-09-10T22:54:20.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:20.848+0000 7fe5881ea640 1 -- 172.21.15.120:0/1127139049 wait complete. 2024-09-10T22:54:21.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 251341 18.05 MB/sec warmup 76 sec latency 23.706 ms 2024-09-10T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:21 smithi120 ceph-mon[31247]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:21 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:21 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:21 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:54:21.618 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:21 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.619 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:21 smithi153 ceph-mon[34951]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T22:54:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:21 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:54:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:21 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:21 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:54:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:21 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:21 smithi176 ceph-mon[34982]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T22:54:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:21 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:54:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:21 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:21 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:54:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:21 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:21.889 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:22.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 -- 172.21.15.120:0/1931446011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 msgr2=0x7f6150103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 --2- 172.21.15.120:0/1931446011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150103ef0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6140009900 tx=0x7f614002a410 comp rx=0 tx=0).stop 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 -- 172.21.15.120:0/1931446011 shutdown_connections 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 --2- 172.21.15.120:0/1931446011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f615010d0c0 0x7f615010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 --2- 172.21.15.120:0/1931446011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6150104510 0x7f615010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 --2- 172.21.15.120:0/1931446011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150103ef0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 -- 172.21.15.120:0/1931446011 >> 172.21.15.120:0/1931446011 conn(0x7f61500fd700 msgr2=0x7f61500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 -- 172.21.15.120:0/1931446011 shutdown_connections 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.394+0000 7f6155f27640 1 -- 172.21.15.120:0/1931446011 wait complete. 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 Processor -- start 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 -- start start 2024-09-10T22:54:22.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6150104510 0x7f61501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f615010d0c0 0x7f615019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6150112da0 con 0x7f6150103b10 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6150112c20 con 0x7f615010d0c0 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f6155f27640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6150112f20 con 0x7f6150104510 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36736/0 (socket says 172.21.15.120:36736) 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f614f7fe640 1 -- 172.21.15.120:0/2479762589 learned_addr learned my addr 172.21.15.120:0/2479762589 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f614effd640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6150104510 0x7f61501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.395+0000 7f614ffff640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f615010d0c0 0x7f615019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 -- 172.21.15.120:0/2479762589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6150104510 msgr2=0x7f61501997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6150104510 0x7f61501997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 -- 172.21.15.120:0/2479762589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f615010d0c0 msgr2=0x7f615019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f615010d0c0 0x7f615019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 -- 172.21.15.120:0/2479762589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615019f310 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614f7fe640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6140005ab0 tx=0x7f6140003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614cff9640 1 -- 172.21.15.120:0/2479762589 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614003f070 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614cff9640 1 -- 172.21.15.120:0/2479762589 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6140031d60 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615019f5a0 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f614cff9640 1 -- 172.21.15.120:0/2479762589 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140035030 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.396+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a5cc0 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.398+0000 7f614cff9640 1 -- 172.21.15.120:0/2479762589 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f614003b580 con 0x7f6150103b10 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.398+0000 7f61582fc740 1 Processor -- start 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 -- start start 2024-09-10T22:54:22.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7ac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c712e50 con 0x55ba1c79c9c0 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c702280 con 0x55ba1c79c0a0 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f61582fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c706090 con 0x55ba1c7acd60 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54140/0 (socket says 172.21.15.120:54140) 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 -- 172.21.15.120:0/1140901158 learned_addr learned my addr 172.21.15.120:0/1140901158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614effd640 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 -- 172.21.15.120:0/1140901158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 msgr2=0x55ba1c7af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 -- 172.21.15.120:0/1140901158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 msgr2=0x55ba1c7ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614ffff640 1 -- 172.21.15.120:0/1140901158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba1c460650 con 0x55ba1c79c0a0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.399+0000 7f614cff9640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f614ffff640 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61480039f0 tx=0x7f6148003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7293cd92c0ed6344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f614f7fe640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f614cff9640 1 -- 172.21.15.120:0/2479762589 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f61401066b0 con 0x7f6150103b10 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f6135ffb640 1 -- 172.21.15.120:0/1140901158 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614801a470 con 0x55ba1c79c0a0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f6135ffb640 1 -- 172.21.15.120:0/1140901158 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6148015070 con 0x55ba1c79c0a0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.400+0000 7f6135ffb640 1 -- 172.21.15.120:0/1140901158 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480107e0 con 0x55ba1c79c0a0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- 172.21.15.120:0/1140901158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 msgr2=0x55ba1c79c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f614f7fe640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 0x7f61380b7820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f61400ca1c0 tx=0x7f614010c550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61480039f0 tx=0x7f6148003eb0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- 172.21.15.120:0/1140901158 shutdown_connections 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 --2- 172.21.15.120:0/1140901158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c79c480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- 172.21.15.120:0/1140901158 >> 172.21.15.120:0/1140901158 conn(0x55ba1c799ed0 msgr2=0x55ba1c79b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- 172.21.15.120:0/1140901158 shutdown_connections 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- 172.21.15.120:0/1140901158 wait complete. 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 -- start start 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.401+0000 7f61582fc740 1 Processor -- start 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c563700 con 0x55ba1c79c0a0 2024-09-10T22:54:22.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c69d110 con 0x55ba1c79c9c0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ba1c7afa10 con 0x55ba1c7acd60 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f61582fc740 10 client.0 _ll_register_callbacks cb 0x55ba1c6b6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36754/0 (socket says 172.21.15.120:36754) 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614ffff640 1 -- 172.21.15.120:0/3838542174 learned_addr learned my addr 172.21.15.120:0/3838542174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614ffff640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 -- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 msgr2=0x55ba1c7aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 -- 172.21.15.120:0/3838542174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 msgr2=0x55ba1c7ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614effd640 1 -- 172.21.15.120:0/3838542174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba1c6cac40 con 0x55ba1c79c9c0 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.402+0000 7f614ffff640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:22.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f614effd640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7a9ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f614400baa0 tx=0x7f614400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34882d4f5e8d80aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144003df0 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6148015070 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f6144005020 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6144022070 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 10 client.25214 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ba1c7bb400 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6148015070 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f614401b470 con 0x55ba1c79c9c0 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 10 client.25214 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 20 client.25214 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:22.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c7bc8c0 0x55ba1c7becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 4 client.25214 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba1c714da0 con 0x55ba1c7bc8c0 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.403+0000 7f614ffff640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c7bc8c0 0x55ba1c7becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.404+0000 7f614ffff640 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c7bc8c0 0x55ba1c7becb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2abda59967d60a server_cookie=870e3ba9d2a75192 in_seq=0 out_seq=0 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.404+0000 7f612effd640 10 client.25214 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f612effd640 1 -- 172.21.15.120:0/3838542174 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba1c714da0 con 0x55ba1c7bc8c0 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f612effd640 10 client.25214 handle_command_reply: tid=0 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 client.25214 shutdown 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 msgr2=0x55ba1c7a9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7a9ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f614400baa0 tx=0x7f614400bf60 comp rx=0 tx=0).stop 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 shutdown_connections 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c7bc8c0 0x55ba1c7becb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7a9ac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f612effd640 0 client.25214 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 --2- 172.21.15.120:0/3838542174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 >> 172.21.15.120:0/3838542174 conn(0x55ba1c799ed0 msgr2=0x55ba1c79fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 shutdown_connections 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.405+0000 7f61582fc740 1 -- 172.21.15.120:0/3838542174 wait complete. 2024-09-10T22:54:22.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.406+0000 7f61582fc740 20 client.25214 trim_cache size 0 max 16384 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 Processor -- start 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 -- start start 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c799710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61501aa020 con 0x55ba1c79c9c0 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ac8a0 con 0x55ba1c79c0a0 2024-09-10T22:54:22.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f61582fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61501aa690 con 0x55ba1c7acd60 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f614effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f614effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36758/0 (socket says 172.21.15.120:36758) 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.425+0000 7f614effd640 1 -- 172.21.15.120:0/3636291081 learned_addr learned my addr 172.21.15.120:0/3636291081 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614ffff640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c799710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614ffff640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 -- 172.21.15.120:0/3636291081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 msgr2=0x55ba1c799710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c799710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 -- 172.21.15.120:0/3636291081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 msgr2=0x55ba1c7b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 -- 172.21.15.120:0/3636291081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba1c7a8a40 con 0x55ba1c79c9c0 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614ffff640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f614effd640 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f614400c010 tx=0x7f6144005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a060c8876299bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f61357fa640 1 -- 172.21.15.120:0/3636291081 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400b400 con 0x55ba1c79c9c0 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f61357fa640 1 -- 172.21.15.120:0/3636291081 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614401be50 con 0x55ba1c79c9c0 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.426+0000 7f61357fa640 1 -- 172.21.15.120:0/3636291081 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144027b90 con 0x55ba1c79c9c0 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- 172.21.15.120:0/3636291081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 msgr2=0x55ba1c79b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f614400c010 tx=0x7f6144005cd0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- 172.21.15.120:0/3636291081 shutdown_connections 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c799710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c9c0 0x55ba1c79b6a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 --2- 172.21.15.120:0/3636291081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- 172.21.15.120:0/3636291081 >> 172.21.15.120:0/3636291081 conn(0x55ba1c799ed0 msgr2=0x55ba1c79af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- 172.21.15.120:0/3636291081 shutdown_connections 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- 172.21.15.120:0/3636291081 wait complete. 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.427+0000 7f61582fc740 1 -- start start 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 Processor -- start 2024-09-10T22:54:22.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61501aad60 con 0x55ba1c79c0a0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ac1d0 con 0x55ba1c79c9c0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ab430 con 0x55ba1c7acd60 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f61582fc740 10 client.0 _ll_register_callbacks cb 0x7f6144004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36772/0 (socket says 172.21.15.120:36772) 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 -- 172.21.15.120:0/553349863 learned_addr learned my addr 172.21.15.120:0/553349863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614effd640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 -- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 msgr2=0x55ba1c7b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 -- 172.21.15.120:0/553349863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 msgr2=0x55ba1c7b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614ffff640 1 -- 172.21.15.120:0/553349863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba1c7b26f0 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.428+0000 7f614effd640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f614ffff640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6148003ef0 tx=0x7f61440042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab4b0fb6ecfae7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148025040 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f6148011850 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614401be50 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f614803c070 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f61582fc740 10 client.15738 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ba1c82a520 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614401be50 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.429+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f61480104b0 con 0x55ba1c79c0a0 2024-09-10T22:54:22.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 10 client.15738 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 20 client.15738 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c82a920 0x55ba1c82cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 4 client.15738 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba1c710850 con 0x55ba1c82a920 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f614effd640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c82a920 0x55ba1c82cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f612effd640 10 client.15738 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.430+0000 7f614effd640 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c82a920 0x55ba1c82cd10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cd1f67466a64 server_cookie=b9e04b1b8147944d in_seq=0 out_seq=0 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f612effd640 1 -- 172.21.15.120:0/553349863 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55ba1c710850 con 0x55ba1c82a920 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f612effd640 10 client.15738 handle_command_reply: tid=0 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 client.15738 shutdown 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 msgr2=0x55ba1c7b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6148003ef0 tx=0x7f61440042b0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 shutdown_connections 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ba1c82a920 0x55ba1c82cd10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ba1c7acd60 0x55ba1c7b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba1c79c9c0 0x55ba1c7b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f612effd640 0 client.15738 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 --2- 172.21.15.120:0/553349863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba1c79c0a0 0x55ba1c7b5180 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 >> 172.21.15.120:0/553349863 conn(0x55ba1c799ed0 msgr2=0x55ba1c79f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 shutdown_connections 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 1 -- 172.21.15.120:0/553349863 wait complete. 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.432+0000 7f61582fc740 20 client.15738 trim_cache size 0 max 16384 2024-09-10T22:54:22.441 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "ec85941b-87cc-4645-b788-d806c41ade31", 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:54:22.443 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:22.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:22.445 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 msgr2=0x7f61380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 0x7f61380b7820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f61400ca1c0 tx=0x7f614010c550 comp rx=0 tx=0).stop 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 msgr2=0x7f6150199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6140005ab0 tx=0x7f6140003cd0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 shutdown_connections 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f615010d0c0 0x7f615019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6150104510 0x7f61501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 --2- 172.21.15.120:0/2479762589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6150103b10 0x7f6150199260 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:22.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 >> 172.21.15.120:0/2479762589 conn(0x7f61500fd700 msgr2=0x7f61500fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:22.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 shutdown_connections 2024-09-10T22:54:22.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:22.433+0000 7f6155f27640 1 -- 172.21.15.120:0/2479762589 wait complete. 2024-09-10T22:54:22.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 254328 18.00 MB/sec warmup 77 sec latency 19.909 ms 2024-09-10T22:54:23.328 INFO:tasks.cephfs.filesystem:scrub status for tag:ec85941b-87cc-4645-b788-d806c41ade31 - {'path': '/', 'tag': 'ec85941b-87cc-4645-b788-d806c41ade31', 'options': 'recursive,force'} 2024-09-10T22:54:23.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 256847 17.99 MB/sec warmup 78 sec latency 44.596 ms 2024-09-10T22:54:23.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:23 smithi120 ceph-mon[31247]: pgmap v402: 97 pgs: 97 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:54:23.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:23 smithi153 ceph-mon[34951]: pgmap v402: 97 pgs: 97 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:54:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:23 smithi176 ceph-mon[34982]: pgmap v402: 97 pgs: 97 active+clean; 49 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:54:24.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 260187 17.96 MB/sec warmup 79 sec latency 53.527 ms 2024-09-10T22:54:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- 172.21.15.120:0/674683036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 msgr2=0x7f63b010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 --2- 172.21.15.120:0/674683036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b010ab40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- 172.21.15.120:0/674683036 shutdown_connections 2024-09-10T22:54:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 --2- 172.21.15.120:0/674683036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b010ab40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 --2- 172.21.15.120:0/674683036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 0x7f63b0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 --2- 172.21.15.120:0/674683036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- 172.21.15.120:0/674683036 >> 172.21.15.120:0/674683036 conn(0x7f63b00fd140 msgr2=0x7f63b00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- 172.21.15.120:0/674683036 shutdown_connections 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- 172.21.15.120:0/674683036 wait complete. 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 Processor -- start 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.735+0000 7f63b572b640 1 -- start start 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 0x7f63b0101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e600 con 0x7f63b0108680 2024-09-10T22:54:24.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e480 con 0x7f63b006b410 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010e780 con 0x7f63b0105ad0 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 0x7f63b0101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57814/0 (socket says 172.21.15.120:57814) 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60240/0 (socket says 172.21.15.120:60240) 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63aeffd640 1 -- 172.21.15.120:0/612231618 learned_addr learned my addr 172.21.15.120:0/612231618 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 -- 172.21.15.120:0/612231618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 msgr2=0x7f63b0101530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 0x7f63b0101530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 -- 172.21.15.120:0/612231618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 msgr2=0x7f63b0102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 -- 172.21.15.120:0/612231618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01022e0 con 0x7f63b0108680 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63af7fe640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f63a8031a20 tx=0x7f63a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63acff9640 1 -- 172.21.15.120:0/612231618 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a802f380 con 0x7f63b0108680 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63acff9640 1 -- 172.21.15.120:0/612231618 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a80317c0 con 0x7f63b0108680 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a2dd0 con 0x7f63b0108680 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.736+0000 7f63acff9640 1 -- 172.21.15.120:0/612231618 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a802fba0 con 0x7f63b0108680 2024-09-10T22:54:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.737+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a3290 con 0x7f63b0108680 2024-09-10T22:54:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.739+0000 7f63b85d5740 1 Processor -- start 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.739+0000 7f63acff9640 1 -- 172.21.15.120:0/612231618 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f63a802fd40 con 0x7f63b0108680 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.739+0000 7f63b85d5740 1 -- start start 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b24f860 0x56509b251c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b254580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56509b1c16f0 con 0x56509b254bd0 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56509b1b0b20 con 0x56509b252190 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63b85d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56509b1b4930 con 0x56509b24f860 2024-09-10T22:54:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60244/0 (socket says 172.21.15.120:60244) 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b254580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 -- 172.21.15.120:0/3001081673 learned_addr learned my addr 172.21.15.120:0/3001081673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b24f860 0x56509b251c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 -- 172.21.15.120:0/3001081673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b24f860 msgr2=0x56509b251c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b24f860 0x56509b251c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 -- 172.21.15.120:0/3001081673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 msgr2=0x56509b257090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 -- 172.21.15.120:0/3001081673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56509b176530 con 0x56509b252190 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63acff9640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 0x7f639c0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63a7fff640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 0x7f639c0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b254580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005fd0 tx=0x7f63a000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307e1a898c08bd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.740+0000 7f63acff9640 1 -- 172.21.15.120:0/612231618 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f63a8106e50 con 0x7f63b0108680 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63a57fa640 1 -- 172.21.15.120:0/3001081673 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001d070 con 0x56509b252190 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63a57fa640 1 -- 172.21.15.120:0/3001081673 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a0018af0 con 0x56509b252190 2024-09-10T22:54:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63a57fa640 1 -- 172.21.15.120:0/3001081673 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0010980 con 0x56509b252190 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 -- 172.21.15.120:0/3001081673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 msgr2=0x56509b254580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63a7fff640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 0x7f639c0b7940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6398011930 tx=0x7f63980097d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b254580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005fd0 tx=0x7f63a000f790 comp rx=0 tx=0).stop 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 -- 172.21.15.120:0/3001081673 shutdown_connections 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b254bd0 0x56509b257090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b254580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3001081673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b24f860 0x56509b251c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.741+0000 7f63b85d5740 1 -- 172.21.15.120:0/3001081673 >> 172.21.15.120:0/3001081673 conn(0x56509b24a9b0 msgr2=0x56509b24b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- 172.21.15.120:0/3001081673 shutdown_connections 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- 172.21.15.120:0/3001081673 wait complete. 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- start start 2024-09-10T22:54:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 Processor -- start 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b24f860 0x56509b258ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56509b0e8ab0 con 0x56509b252190 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56509b156730 con 0x56509b24f860 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56509b257bb0 con 0x56509b254bd0 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63b85d5740 10 client.0 _ll_register_callbacks cb 0x56509b1aa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60250/0 (socket says 172.21.15.120:60250) 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 -- 172.21.15.120:0/4042208127 learned_addr learned my addr 172.21.15.120:0/4042208127 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63af7fe640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63aeffd640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b24f860 0x56509b258ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 -- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 msgr2=0x56509b25a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 -- 172.21.15.120:0/4042208127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b24f860 msgr2=0x56509b258ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.742+0000 7f63a7fff640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b24f860 0x56509b258ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63a7fff640 1 -- 172.21.15.120:0/4042208127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509af12790 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63a7fff640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f639800edf0 tx=0x7f6398004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55647d5a9b32e9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639802d060 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f639801a070 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0018af0 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6398020bb0 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 10 client.15750 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56509af8b620 con 0x56509b252190 2024-09-10T22:54:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a0018af0 con 0x56509b252190 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6398027030 con 0x56509b252190 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 10 client.15750 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 20 client.15750 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b264830 0x56509b266cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 4 client.15750 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.743+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56509b1c3640 con 0x56509b264830 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.744+0000 7f63aeffd640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b264830 0x56509b266cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.744+0000 7f63967fc640 10 client.15750 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.744+0000 7f63aeffd640 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b264830 0x56509b266cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dd331f9b1ca93d server_cookie=d19049b1696e8942 in_seq=0 out_seq=0 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63967fc640 1 -- 172.21.15.120:0/4042208127 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56509b1c3640 con 0x56509b264830 2024-09-10T22:54:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63967fc640 10 client.15750 handle_command_reply: tid=0 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 client.15750 shutdown 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 msgr2=0x56509b259590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f639800edf0 tx=0x7f6398004730 comp rx=0 tx=0).stop 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 shutdown_connections 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b264830 0x56509b266cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b252190 0x56509b259590 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 --2- 172.21.15.120:0/4042208127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b24f860 0x56509b258ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63967fc640 0 client.15750 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 >> 172.21.15.120:0/4042208127 conn(0x56509b24a9b0 msgr2=0x56509b249bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 shutdown_connections 2024-09-10T22:54:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 1 -- 172.21.15.120:0/4042208127 wait complete. 2024-09-10T22:54:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.745+0000 7f63b85d5740 20 client.15750 trim_cache size 0 max 16384 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.760+0000 7f63b85d5740 1 Processor -- start 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.760+0000 7f63b85d5740 1 -- start start 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b252190 0x56509b26bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 0x56509b26bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aa000 con 0x56509b24f860 2024-09-10T22:54:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ac880 con 0x56509b254bd0 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63b85d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aa670 con 0x56509b252190 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 0x56509b26bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60262/0 (socket says 172.21.15.120:60262) 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 -- 172.21.15.120:0/2488494396 learned_addr learned my addr 172.21.15.120:0/2488494396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63a7fff640 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b252190 0x56509b26bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 -- 172.21.15.120:0/2488494396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b252190 msgr2=0x56509b26bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b252190 0x56509b26bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 -- 172.21.15.120:0/2488494396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 msgr2=0x56509b26bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 0x56509b26bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 -- 172.21.15.120:0/2488494396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56509b25a4d0 con 0x56509b24f860 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63af7fe640 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 0x56509b26bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.761+0000 7f63aeffd640 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f63a001b040 tx=0x7f63a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23184c8b947bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63a4ff9640 1 -- 172.21.15.120:0/2488494396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001d040 con 0x56509b24f860 2024-09-10T22:54:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63a4ff9640 1 -- 172.21.15.120:0/2488494396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a00184c0 con 0x56509b24f860 2024-09-10T22:54:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63a4ff9640 1 -- 172.21.15.120:0/2488494396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00048c0 con 0x56509b24f860 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 -- 172.21.15.120:0/2488494396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 msgr2=0x56509b26b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f63a001b040 tx=0x7f63a000f790 comp rx=0 tx=0).stop 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 -- 172.21.15.120:0/2488494396 shutdown_connections 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b254bd0 0x56509b26bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b252190 0x56509b26bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 --2- 172.21.15.120:0/2488494396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b26b570 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 -- 172.21.15.120:0/2488494396 >> 172.21.15.120:0/2488494396 conn(0x56509b24a9b0 msgr2=0x56509b24da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 -- 172.21.15.120:0/2488494396 shutdown_connections 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.762+0000 7f63b85d5740 1 -- 172.21.15.120:0/2488494396 wait complete. 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 -- start start 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 Processor -- start 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b25f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aad40 con 0x56509b24f860 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ac1b0 con 0x56509b252190 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab410 con 0x56509b254bd0 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63b85d5740 10 client.0 _ll_register_callbacks cb 0x7f639800ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60266/0 (socket says 172.21.15.120:60266) 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 -- 172.21.15.120:0/3394908260 learned_addr learned my addr 172.21.15.120:0/3394908260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63a7fff640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b25f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63af7fe640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 -- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 msgr2=0x56509b25f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 -- 172.21.15.120:0/3394908260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 msgr2=0x56509b25f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b25f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.763+0000 7f63aeffd640 1 -- 172.21.15.120:0/3394908260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509b25fee0 con 0x56509b24f860 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63aeffd640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f63a000ab80 tx=0x7f63a000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02fd409e78fd72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001d070 con 0x56509b24f860 2024-09-10T22:54:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a00184c0 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f63a00050c0 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f63a00052c0 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 10 client.15756 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56509b25efd0 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a00184c0 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f63a0034070 con 0x56509b24f860 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 10 client.15756 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 20 client.15756 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b2d1e10 0x56509b2d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 4 client.15756 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56509b1bf0f0 con 0x56509b2d1e10 2024-09-10T22:54:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.764+0000 7f63a7fff640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b2d1e10 0x56509b2d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.765+0000 7f63a7fff640 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b2d1e10 0x56509b2d4230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8200b36df03d114 server_cookie=94a3883ea5ffc193 in_seq=0 out_seq=0 2024-09-10T22:54:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.765+0000 7f63967fc640 10 client.15756 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63967fc640 1 -- 172.21.15.120:0/3394908260 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56509b1bf0f0 con 0x56509b2d1e10 2024-09-10T22:54:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63967fc640 10 client.15756 handle_command_reply: tid=0 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 client.15756 shutdown 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 msgr2=0x56509b260730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f63a000ab80 tx=0x7f63a000fd40 comp rx=0 tx=0).stop 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 shutdown_connections 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56509b2d1e10 0x56509b2d4230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63967fc640 0 client.15756 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56509b254bd0 0x56509b25f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56509b252190 0x56509b25f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 --2- 172.21.15.120:0/3394908260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56509b24f860 0x56509b260730 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.923+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 >> 172.21.15.120:0/3394908260 conn(0x56509b24a9b0 msgr2=0x56509b2493d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 shutdown_connections 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b85d5740 1 -- 172.21.15.120:0/3394908260 wait complete. 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b85d5740 20 client.15756 trim_cache size 0 max 16384 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:54:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:54:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 msgr2=0x7f639c0b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 0x7f639c0b7940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6398011930 tx=0x7f63980097d0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 msgr2=0x7f63b0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f63a8031a20 tx=0x7f63a8003ba0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 shutdown_connections 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f639c0b5480 0x7f639c0b7940 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63b0108680 0x7f63b0101a70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b0105ad0 0x7f63b0101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 --2- 172.21.15.120:0/612231618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b006b410 0x7f63b0102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.924+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 >> 172.21.15.120:0/612231618 conn(0x7f63b00fd140 msgr2=0x7f63b0103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.925+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 shutdown_connections 2024-09-10T22:54:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:24.925+0000 7f63b572b640 1 -- 172.21.15.120:0/612231618 wait complete. 2024-09-10T22:54:25.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 263179 17.95 MB/sec warmup 80 sec latency 80.927 ms 2024-09-10T22:54:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:25 smithi120 ceph-mon[31247]: pgmap v403: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 461 op/s 2024-09-10T22:54:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:25 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:54:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:25 smithi153 ceph-mon[34951]: pgmap v403: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 461 op/s 2024-09-10T22:54:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:25 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:54:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:25 smithi176 ceph-mon[34982]: pgmap v403: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 461 op/s 2024-09-10T22:54:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:25 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:54:26.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 266585 17.97 MB/sec warmup 81 sec latency 18.370 ms 2024-09-10T22:54:27.615 INFO:tasks.workunit.client.0.smithi120.stdout: 1 269973 17.98 MB/sec warmup 82 sec latency 13.576 ms 2024-09-10T22:54:27.624 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:27 smithi176 ceph-mon[34982]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:54:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:27 smithi120 ceph-mon[31247]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:54:27.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:27 smithi153 ceph-mon[34951]: pgmap v404: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:54:28.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 272776 17.92 MB/sec warmup 83 sec latency 74.593 ms 2024-09-10T22:54:29.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 276130 17.93 MB/sec warmup 84 sec latency 12.955 ms 2024-09-10T22:54:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:29 smithi120 ceph-mon[31247]: pgmap v405: 97 pgs: 97 active+clean; 49 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 463 op/s 2024-09-10T22:54:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:29 smithi153 ceph-mon[34951]: pgmap v405: 97 pgs: 97 active+clean; 49 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 463 op/s 2024-09-10T22:54:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:29 smithi176 ceph-mon[34982]: pgmap v405: 97 pgs: 97 active+clean; 49 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 463 op/s 2024-09-10T22:54:30.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 279571 17.93 MB/sec warmup 85 sec latency 14.905 ms 2024-09-10T22:54:31.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 282720 17.90 MB/sec warmup 86 sec latency 12.695 ms 2024-09-10T22:54:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:31 smithi120 ceph-mon[31247]: pgmap v406: 97 pgs: 97 active+clean; 42 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-10T22:54:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:31 smithi153 ceph-mon[34951]: pgmap v406: 97 pgs: 97 active+clean; 42 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-10T22:54:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:31 smithi176 ceph-mon[34982]: pgmap v406: 97 pgs: 97 active+clean; 42 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-10T22:54:32.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 286046 17.89 MB/sec warmup 87 sec latency 11.588 ms 2024-09-10T22:54:33.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 288256 17.85 MB/sec warmup 88 sec latency 80.144 ms 2024-09-10T22:54:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:33 smithi120 ceph-mon[31247]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-09-10T22:54:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:33 smithi153 ceph-mon[34951]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-09-10T22:54:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:33 smithi176 ceph-mon[34982]: pgmap v407: 97 pgs: 97 active+clean; 46 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 496 op/s 2024-09-10T22:54:34.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 291789 17.86 MB/sec warmup 89 sec latency 11.878 ms 2024-09-10T22:54:35.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 295296 17.87 MB/sec warmup 90 sec latency 11.998 ms 2024-09-10T22:54:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:35 smithi120 ceph-mon[31247]: pgmap v408: 97 pgs: 97 active+clean; 47 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-10T22:54:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:35 smithi153 ceph-mon[34951]: pgmap v408: 97 pgs: 97 active+clean; 47 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-10T22:54:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:35 smithi176 ceph-mon[34982]: pgmap v408: 97 pgs: 97 active+clean; 47 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 423 op/s 2024-09-10T22:54:36.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 298640 17.88 MB/sec warmup 91 sec latency 14.510 ms 2024-09-10T22:54:37.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 301955 17.90 MB/sec warmup 92 sec latency 23.650 ms 2024-09-10T22:54:37.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-09-10T22:54:37.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: pgmap v409: 97 pgs: 97 active+clean; 52 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:37.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:54:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2289675791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4103370 msgr2=0x7f07a4109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2289675791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4103370 0x7f07a4109cf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f079c009cd0 tx=0x7f079c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2289675791 shutdown_connections 2024-09-10T22:54:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2289675791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4103370 0x7f07a4109cf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2289675791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a41029b0 0x7f07a4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2289675791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a4101860 0x7f07a4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2289675791 >> 172.21.15.120:0/2289675791 conn(0x7f07a40fd140 msgr2=0x7f07a40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2289675791 shutdown_connections 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2289675791 wait complete. 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 Processor -- start 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.034+0000 7f07aa0bd640 1 -- start start 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 0x7f07a41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 0x7f07a4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07a410e600 con 0x7f07a4101860 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07a410e480 con 0x7f07a4103370 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07a410e780 con 0x7f07a41029b0 2024-09-10T22:54:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48992/0 (socket says 172.21.15.120:48992) 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 -- 172.21.15.120:0/2629957385 learned_addr learned my addr 172.21.15.120:0/2629957385 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f079bfff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 0x7f07a41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 0x7f07a4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 -- 172.21.15.120:0/2629957385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 msgr2=0x7f07a41993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 0x7f07a41993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 -- 172.21.15.120:0/2629957385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 msgr2=0x7f07a4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 0x7f07a4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 -- 172.21.15.120:0/2629957385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a419a8a0 con 0x7f07a4101860 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f079bfff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 0x7f07a41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 0x7f07a4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f079400e9f0 tx=0x7f079400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a17fa640 1 -- 172.21.15.120:0/2629957385 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079400cd80 con 0x7f07a4101860 2024-09-10T22:54:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a419ab90 con 0x7f07a4101860 2024-09-10T22:54:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a17fa640 1 -- 172.21.15.120:0/2629957385 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940054d0 con 0x7f07a4101860 2024-09-10T22:54:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07a17fa640 1 -- 172.21.15.120:0/2629957385 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794010690 con 0x7f07a4101860 2024-09-10T22:54:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.035+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41a1260 con 0x7f07a4101860 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07a17fa640 1 -- 172.21.15.120:0/2629957385 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0794004e90 con 0x7f07a4101860 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 Processor -- start 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 -- start start 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f01b226f0 con 0x564f01bb3190 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564f01b11b20 con 0x564f01bb5bd0 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07ace92740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564f01b15930 con 0x564f01bb0860 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.038+0000 7f07a3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48996/0 (socket says 172.21.15.120:48996) 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55594/0 (socket says 172.21.15.120:55594) 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 -- 172.21.15.120:0/3454141370 learned_addr learned my addr 172.21.15.120:0/3454141370 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a17fa640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f079bfff640 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a17fa640 1 -- 172.21.15.120:0/2629957385 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f07940e6110 con 0x7f07a4101860 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 -- 172.21.15.120:0/3454141370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 msgr2=0x564f01bb2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 -- 172.21.15.120:0/3454141370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 msgr2=0x564f01bb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bb8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 -- 172.21.15.120:0/3454141370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f01ad7530 con 0x564f01bb3190 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a3fff640 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a37fe640 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07940a8ed0 tx=0x7f0794004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a64512af68c29a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f0799ffb640 1 -- 172.21.15.120:0/3454141370 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940ac070 con 0x564f01bb3190 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 0x7f07900b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f07a4199ae0 tx=0x7f079c02a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f0799ffb640 1 -- 172.21.15.120:0/3454141370 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940ac410 con 0x564f01bb3190 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.039+0000 7f0799ffb640 1 -- 172.21.15.120:0/3454141370 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940b5e90 con 0x564f01bb3190 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- 172.21.15.120:0/3454141370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 msgr2=0x564f01bb5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07940a8ed0 tx=0x7f0794004a90 comp rx=0 tx=0).stop 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- 172.21.15.120:0/3454141370 shutdown_connections 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb5580 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 --2- 172.21.15.120:0/3454141370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- 172.21.15.120:0/3454141370 >> 172.21.15.120:0/3454141370 conn(0x564f01bab9b0 msgr2=0x564f01bac740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- 172.21.15.120:0/3454141370 shutdown_connections 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- 172.21.15.120:0/3454141370 wait complete. 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.040+0000 7f07ace92740 1 -- start start 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 Processor -- start 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bbb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564f01a49ab0 con 0x564f01bb3190 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564f01ab7730 con 0x564f01bb5bd0 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564f01bb8b50 con 0x564f01bb0860 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07ace92740 10 client.0 _ll_register_callbacks cb 0x564f01b0b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55604/0 (socket says 172.21.15.120:55604) 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f079bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 -- 172.21.15.120:0/1066883246 learned_addr learned my addr 172.21.15.120:0/1066883246 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a3fff640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bbb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 -- 172.21.15.120:0/1066883246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 msgr2=0x564f01bbb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bbb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 -- 172.21.15.120:0/1066883246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 msgr2=0x564f01bba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.041+0000 7f07a37fe640 1 -- 172.21.15.120:0/1066883246 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f01873790 con 0x564f01bb0860 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07a37fe640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07940acac0 tx=0x7f07940057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3607f147aefa2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940ac070 con 0x564f01bb0860 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f07940e3790 con 0x564f01bb0860 2024-09-10T22:54:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940ac410 con 0x564f01bb0860 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f07940ad000 con 0x564f01bb0860 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 10 client.25240 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564f018ec620 con 0x564f01bb0860 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940ac410 con 0x564f01bb0860 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f07940a8b70 con 0x564f01bb0860 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 10 client.25240 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 10 client.25240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 20 client.25240 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01bc5330 0x564f01bc7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 4 client.25240 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.042+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f01b24640 con 0x564f01bc5330 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.043+0000 7f079bfff640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01bc5330 0x564f01bc7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.043+0000 7f0782ffd640 10 client.25240 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.043+0000 7f079bfff640 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01bc5330 0x564f01bc7720 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda81a94d835c465 server_cookie=75a8c8a561787146 in_seq=0 out_seq=0 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f0782ffd640 1 -- 172.21.15.120:0/1066883246 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f01b24640 con 0x564f01bc5330 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f0782ffd640 10 client.25240 handle_command_reply: tid=0 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 client.25240 shutdown 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 msgr2=0x564f01bb9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07940acac0 tx=0x7f07940057c0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 shutdown_connections 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01bc5330 0x564f01bc7720 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bbb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f0782ffd640 0 client.25240 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 --2- 172.21.15.120:0/1066883246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb9e90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 >> 172.21.15.120:0/1066883246 conn(0x564f01bab9b0 msgr2=0x564f01baabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 shutdown_connections 2024-09-10T22:54:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.044+0000 7f07ace92740 1 -- 172.21.15.120:0/1066883246 wait complete. 2024-09-10T22:54:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.045+0000 7f07ace92740 20 client.25240 trim_cache size 0 max 16384 2024-09-10T22:54:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 Processor -- start 2024-09-10T22:54:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 -- start start 2024-09-10T22:54:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bafd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41a7e10 con 0x564f01bb3190 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41aa690 con 0x564f01bb5bd0 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07ace92740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41a8480 con 0x564f01bb0860 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49016/0 (socket says 172.21.15.120:49016) 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 -- 172.21.15.120:0/2801781032 learned_addr learned my addr 172.21.15.120:0/2801781032 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bafd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 -- 172.21.15.120:0/2801781032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 msgr2=0x564f01bb8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 -- 172.21.15.120:0/2801781032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 msgr2=0x564f01bafd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bafd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 -- 172.21.15.120:0/2801781032 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f01bc7a90 con 0x564f01bb3190 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.059+0000 7f079bfff640 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x564f01bc7f40 tx=0x7f078c00c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f25b4f1ac8040af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07997fa640 1 -- 172.21.15.120:0/2801781032 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c004220 con 0x564f01bb3190 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07997fa640 1 -- 172.21.15.120:0/2801781032 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078c0043c0 con 0x564f01bb3190 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07997fa640 1 -- 172.21.15.120:0/2801781032 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c01c860 con 0x564f01bb3190 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 -- 172.21.15.120:0/2801781032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 msgr2=0x564f01bb9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x564f01bc7f40 tx=0x7f078c00c4f0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 -- 172.21.15.120:0/2801781032 shutdown_connections 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb5bd0 0x564f01bafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bb9460 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 --2- 172.21.15.120:0/2801781032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb0860 0x564f01bb8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 -- 172.21.15.120:0/2801781032 >> 172.21.15.120:0/2801781032 conn(0x564f01bab9b0 msgr2=0x564f01bac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 -- 172.21.15.120:0/2801781032 shutdown_connections 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.060+0000 7f07ace92740 1 -- 172.21.15.120:0/2801781032 wait complete. 2024-09-10T22:54:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 -- start start 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 Processor -- start 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 0x564f01bc1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb5bd0 0x564f01bc09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41a8b50 con 0x564f01bb3190 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41a9fc0 con 0x564f01bb0860 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41a9220 con 0x564f01bb5bd0 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07ace92740 10 client.0 _ll_register_callbacks cb 0x7f078c00c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07a37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 0x564f01bc1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:49022/0 (socket says 172.21.15.120:49022) 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 -- 172.21.15.120:0/2442712700 learned_addr learned my addr 172.21.15.120:0/2442712700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07a3fff640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb5bd0 0x564f01bc09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 -- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb5bd0 msgr2=0x564f01bc09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb5bd0 0x564f01bc09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 -- 172.21.15.120:0/2442712700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 msgr2=0x564f01bc1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 0x564f01bc1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f079bfff640 1 -- 172.21.15.120:0/2442712700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f01bc1010 con 0x564f01bb3190 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.061+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 0x564f01bc1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f079bfff640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f078c00cd70 tx=0x7f078c004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aebbd0929e24794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c032060 con 0x564f01bb3190 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8182+0+0 (secure 0 0 0) 0x7f078c003800 con 0x564f01bb3190 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c0043c0 con 0x564f01bb3190 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f078c003a00 con 0x564f01bb3190 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 10 client.15774 fetch_fsmap learned FSMap version 21 2024-09-10T22:54:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564f01c321a0 con 0x564f01bb3190 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078c0043c0 con 0x564f01bb3190 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f078c022440 con 0x564f01bb3190 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 10 client.15774 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 20 client.15774 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01c32a10 0x564f01c34e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 4 client.15774 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.062+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f01b200f0 con 0x564f01c32a10 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.063+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01c32a10 0x564f01c34e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.063+0000 7f0782ffd640 10 client.15774 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.063+0000 7f07a37fe640 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01c32a10 0x564f01c34e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf61dec9874ee7f5 server_cookie=6be263148825d855 in_seq=0 out_seq=0 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f0782ffd640 1 -- 172.21.15.120:0/2442712700 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f01b200f0 con 0x564f01c32a10 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f0782ffd640 10 client.15774 handle_command_reply: tid=0 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 client.15774 shutdown 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 msgr2=0x564f01bc0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f078c00cd70 tx=0x7f078c004230 comp rx=0 tx=0).stop 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 shutdown_connections 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564f01c32a10 0x564f01c34e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f01bb5bd0 0x564f01bc09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f0782ffd640 0 client.15774 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564f01bb3190 0x564f01bc0490 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 --2- 172.21.15.120:0/2442712700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564f01bb0860 0x564f01bc1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 >> 172.21.15.120:0/2442712700 conn(0x564f01bab9b0 msgr2=0x564f01bb8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 shutdown_connections 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 1 -- 172.21.15.120:0/2442712700 wait complete. 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.267+0000 7f07ace92740 20 client.15774 trim_cache size 0 max 16384 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:54:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:54:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 msgr2=0x7f07900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 0x7f07900b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f07a4199ae0 tx=0x7f079c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 msgr2=0x7f07a4198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f079400e9f0 tx=0x7f079400edf0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 shutdown_connections 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07a4103370 0x7f07a4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a41029b0 0x7f07a41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 --2- 172.21.15.120:0/2629957385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07a4101860 0x7f07a4198d60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 >> 172.21.15.120:0/2629957385 conn(0x7f07a40fd140 msgr2=0x7f07a4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 shutdown_connections 2024-09-10T22:54:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:38.268+0000 7f07aa0bd640 1 -- 172.21.15.120:0/2629957385 wait complete. 2024-09-10T22:54:38.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 304117 17.79 MB/sec warmup 93 sec latency 82.784 ms 2024-09-10T22:54:39.323 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:54:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setuser ceph since I am not root 2024-09-10T22:54:39.323 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:54:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setgroup ceph since I am not root 2024-09-10T22:54:39.323 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:54:39 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/1005556240 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f123e3c20 con 0x562f123c1400 2024-09-10T22:54:39.323 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:54:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T22:54:39.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 307514 17.79 MB/sec warmup 94 sec latency 13.364 ms 2024-09-10T22:54:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:39 smithi120 ceph-mon[31247]: pgmap v410: 97 pgs: 97 active+clean; 52 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-10T22:54:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:39 smithi120 ceph-mon[31247]: daemon mds.c restarted 2024-09-10T22:54:39.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:39 smithi153 ceph-mon[34951]: pgmap v410: 97 pgs: 97 active+clean; 52 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-10T22:54:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:39 smithi153 ceph-mon[34951]: daemon mds.c restarted 2024-09-10T22:54:39.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:39 smithi176 ceph-mon[34982]: pgmap v410: 97 pgs: 97 active+clean; 52 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-10T22:54:39.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:39 smithi176 ceph-mon[34982]: daemon mds.c restarted 2024-09-10T22:54:40.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 310439 17.79 MB/sec warmup 95 sec latency 12.095 ms 2024-09-10T22:54:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:40 smithi120 ceph-mon[31247]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:54:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:40 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/2789135593,v1:172.21.15.176:6837/2789135593] up:boot 2024-09-10T22:54:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:40 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:54:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:54:40.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:40 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:54:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:40 smithi153 ceph-mon[34951]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:54:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:40 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/2789135593,v1:172.21.15.176:6837/2789135593] up:boot 2024-09-10T22:54:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:40 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:54:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:54:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:40 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:54:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:40 smithi176 ceph-mon[34982]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:54:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:40 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/2789135593,v1:172.21.15.176:6837/2789135593] up:boot 2024-09-10T22:54:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:40 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:54:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:54:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:40 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:54:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:41 smithi176 ceph-mon[34982]: pgmap v412: 97 pgs: 97 active+clean; 43 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-10T22:54:41.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 313319 17.77 MB/sec warmup 96 sec latency 59.570 ms 2024-09-10T22:54:41.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:41 smithi120 ceph-mon[31247]: pgmap v412: 97 pgs: 97 active+clean; 43 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-10T22:54:41.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:41 smithi153 ceph-mon[34951]: pgmap v412: 97 pgs: 97 active+clean; 43 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-09-10T22:54:42.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 316763 17.78 MB/sec warmup 97 sec latency 21.199 ms 2024-09-10T22:54:43.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 319696 17.78 MB/sec warmup 98 sec latency 85.906 ms 2024-09-10T22:54:43.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:43 smithi120 ceph-mon[31247]: pgmap v413: 97 pgs: 97 active+clean; 41 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-09-10T22:54:43.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:43 smithi153 ceph-mon[34951]: pgmap v413: 97 pgs: 97 active+clean; 41 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-09-10T22:54:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:43 smithi176 ceph-mon[34982]: pgmap v413: 97 pgs: 97 active+clean; 41 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-09-10T22:54:44.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 323132 17.79 MB/sec warmup 99 sec latency 13.247 ms 2024-09-10T22:54:45.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 326624 17.80 MB/sec warmup 100 sec latency 15.150 ms 2024-09-10T22:54:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:45 smithi120 ceph-mon[31247]: pgmap v414: 97 pgs: 97 active+clean; 41 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-10T22:54:45.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:45 smithi153 ceph-mon[34951]: pgmap v414: 97 pgs: 97 active+clean; 41 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-10T22:54:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:45 smithi176 ceph-mon[34982]: pgmap v414: 97 pgs: 97 active+clean; 41 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-10T22:54:46.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 330154 17.81 MB/sec warmup 101 sec latency 12.602 ms 2024-09-10T22:54:47.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 333657 17.82 MB/sec warmup 102 sec latency 14.792 ms 2024-09-10T22:54:47.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:47 smithi120 ceph-mon[31247]: pgmap v415: 97 pgs: 97 active+clean; 44 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-09-10T22:54:47.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:47 smithi153 ceph-mon[34951]: pgmap v415: 97 pgs: 97 active+clean; 44 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-09-10T22:54:47.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:47 smithi176 ceph-mon[34982]: pgmap v415: 97 pgs: 97 active+clean; 44 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-09-10T22:54:48.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 336326 17.77 MB/sec warmup 103 sec latency 63.574 ms 2024-09-10T22:54:49.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 339379 17.74 MB/sec warmup 104 sec latency 12.276 ms 2024-09-10T22:54:49.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:49 smithi120 ceph-mon[31247]: pgmap v416: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-10T22:54:49.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:49 smithi153 ceph-mon[34951]: pgmap v416: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-10T22:54:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:49 smithi176 ceph-mon[34982]: pgmap v416: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-10T22:54:50.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 342896 17.75 MB/sec warmup 105 sec latency 13.608 ms 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- 172.21.15.120:0/2613844698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102b20 msgr2=0x7f70881094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 --2- 172.21.15.120:0/2613844698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102b20 0x7f70881094a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f708000ade0 tx=0x7f708002b7e0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- 172.21.15.120:0/2613844698 shutdown_connections 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 --2- 172.21.15.120:0/2613844698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102b20 0x7f70881094a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 --2- 172.21.15.120:0/2613844698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102160 0x7f70881025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 --2- 172.21.15.120:0/2613844698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 0x7f7088101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- 172.21.15.120:0/2613844698 >> 172.21.15.120:0/2613844698 conn(0x7f70880fc8f0 msgr2=0x7f70880fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- 172.21.15.120:0/2613844698 shutdown_connections 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- 172.21.15.120:0/2613844698 wait complete. 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 Processor -- start 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.364+0000 7f708f764640 1 -- start start 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 0x7f7088198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102b20 0x7f7088199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f708810c410 con 0x7f7088102160 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f708810c290 con 0x7f7088101010 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708f764640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f708810c590 con 0x7f7088102b20 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37700/0 (socket says 172.21.15.120:37700) 2024-09-10T22:54:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708d500640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 0x7f7088198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 -- 172.21.15.120:0/2053016169 learned_addr learned my addr 172.21.15.120:0/2053016169 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708dd01640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102b20 0x7f7088199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 -- 172.21.15.120:0/2053016169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102b20 msgr2=0x7f7088199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102b20 0x7f7088199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 -- 172.21.15.120:0/2053016169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 msgr2=0x7f7088198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 0x7f7088198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 -- 172.21.15.120:0/2053016169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708819a880 con 0x7f7088102160 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.365+0000 7f708ccff640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f707800e9f0 tx=0x7f707800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.366+0000 7f707e7fc640 1 -- 172.21.15.120:0/2053016169 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800cd80 con 0x7f7088102160 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.366+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708819ab70 con 0x7f7088102160 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.366+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a1240 con 0x7f7088102160 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.366+0000 7f707e7fc640 1 -- 172.21.15.120:0/2053016169 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70780054d0 con 0x7f7088102160 2024-09-10T22:54:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.366+0000 7f707e7fc640 1 -- 172.21.15.120:0/2053016169 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078010690 con 0x7f7088102160 2024-09-10T22:54:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 Processor -- start 2024-09-10T22:54:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 -- start start 2024-09-10T22:54:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f707e7fc640 1 -- 172.21.15.120:0/2053016169 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f70780046c0 con 0x7f7088102160 2024-09-10T22:54:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a475a0 0x5643c3a49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a4c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643c39b26f0 con 0x5643c3a475a0 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643c39a1b20 con 0x5643c3a49fe0 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f7092612740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5643c39a5930 con 0x5643c3a44c70 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f708d500640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a475a0 0x5643c3a49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f708dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f708dd01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:39316/0 (socket says 172.21.15.120:39316) 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f708dd01640 1 -- 172.21.15.120:0/1596773860 learned_addr learned my addr 172.21.15.120:0/1596773860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.369+0000 7f708ccff640 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a4c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 -- 172.21.15.120:0/1596773860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 msgr2=0x5643c3a4c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a4c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 -- 172.21.15.120:0/1596773860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a475a0 msgr2=0x5643c3a49990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a475a0 0x5643c3a49990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 -- 172.21.15.120:0/1596773860 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c3967530 con 0x5643c3a44c70 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708dd01640 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7080032a40 tx=0x7f7080030f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd97d178be4f567 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f707e7fc640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 0x7f70740b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f70737fe640 1 -- 172.21.15.120:0/1596773860 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080032400 con 0x5643c3a44c70 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f70737fe640 1 -- 172.21.15.120:0/1596773860 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7080003e20 con 0x5643c3a44c70 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f70737fe640 1 -- 172.21.15.120:0/1596773860 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080004640 con 0x5643c3a44c70 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f708d500640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 0x7f70740b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.370+0000 7f707e7fc640 1 -- 172.21.15.120:0/2053016169 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f70780e5f20 con 0x7f7088102160 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- 172.21.15.120:0/1596773860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 msgr2=0x5643c3a47060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7080032a40 tx=0x7f7080030f80 comp rx=0 tx=0).stop 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f708d500640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 0x7f70740b77a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7084009da0 tx=0x7f7084005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- 172.21.15.120:0/1596773860 shutdown_connections 2024-09-10T22:54:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a4c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a475a0 0x5643c3a49990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 --2- 172.21.15.120:0/1596773860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a44c70 0x5643c3a47060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- 172.21.15.120:0/1596773860 >> 172.21.15.120:0/1596773860 conn(0x5643c3a3b9b0 msgr2=0x5643c3a3c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- 172.21.15.120:0/1596773860 shutdown_connections 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- 172.21.15.120:0/1596773860 wait complete. 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 -- start start 2024-09-10T22:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.371+0000 7f7092612740 1 Processor -- start 2024-09-10T22:54:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 0x5643c3a4e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a49fe0 0x5643c3a4f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643c38d9ab0 con 0x5643c3a44c70 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643c3947730 con 0x5643c3a475a0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5643c3a4cf60 con 0x5643c3a49fe0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f7092612740 10 client.0 _ll_register_callbacks cb 0x5643c399b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37716/0 (socket says 172.21.15.120:37716) 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 -- 172.21.15.120:0/3583440741 learned_addr learned my addr 172.21.15.120:0/3583440741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708ccff640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 0x5643c3a4e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708dd01640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a49fe0 0x5643c3a4f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 -- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a49fe0 msgr2=0x5643c3a4f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a49fe0 0x5643c3a4f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 -- 172.21.15.120:0/3583440741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 msgr2=0x5643c3a4e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 0x5643c3a4e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708d500640 1 -- 172.21.15.120:0/3583440741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c3703790 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.372+0000 7f708ccff640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 0x5643c3a4e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f708d500640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70840045f0 tx=0x7f7084004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38824689ffc1bf27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708401f640 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f708401f7e0 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7080003e20 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f708401f9e0 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7092612740 10 client.15792 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5643c377c620 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.373+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7080003e20 con 0x5643c3a44c70 2024-09-10T22:54:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7084027070 con 0x5643c3a44c70 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 10 client.15792 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 20 client.15792 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3a59c80 0x5643c3a5c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 4 client.15792 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c39b4640 con 0x5643c3a59c80 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f708ccff640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3a59c80 0x5643c3a5c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f7070ff9640 10 client.15792 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.374+0000 7f708ccff640 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3a59c80 0x5643c3a5c0d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc0973b13119502 server_cookie=d46f0c22bb52d583 in_seq=0 out_seq=0 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7070ff9640 1 -- 172.21.15.120:0/3583440741 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643c39b4640 con 0x5643c3a59c80 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7070ff9640 10 client.15792 handle_command_reply: tid=0 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 client.15792 shutdown 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 msgr2=0x5643c3a4e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70840045f0 tx=0x7f7084004730 comp rx=0 tx=0).stop 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 shutdown_connections 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3a59c80 0x5643c3a5c0d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a49fe0 0x5643c3a4f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a475a0 0x5643c3a4e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 --2- 172.21.15.120:0/3583440741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a4e280 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7070ff9640 0 client.15792 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 >> 172.21.15.120:0/3583440741 conn(0x5643c3a3b9b0 msgr2=0x5643c3a3abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 shutdown_connections 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.375+0000 7f7092612740 1 -- 172.21.15.120:0/3583440741 wait complete. 2024-09-10T22:54:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.376+0000 7f7092612740 20 client.15792 trim_cache size 0 max 16384 2024-09-10T22:54:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 Processor -- start 2024-09-10T22:54:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 -- start start 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a44c70 0x5643c3a4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a4a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00c390 con 0x5643c3a49fe0 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00ec10 con 0x5643c3a44c70 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f7092612740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00ca00 con 0x5643c3a475a0 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f708dd01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f708dd01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37722/0 (socket says 172.21.15.120:37722) 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f708dd01640 1 -- 172.21.15.120:0/1294609977 learned_addr learned my addr 172.21.15.120:0/1294609977 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f708ccff640 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a4a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.392+0000 7f708d500640 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a44c70 0x5643c3a4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 -- 172.21.15.120:0/1294609977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 msgr2=0x5643c3a4a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a4a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 -- 172.21.15.120:0/1294609977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a44c70 msgr2=0x5643c3a4d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a44c70 0x5643c3a4d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 -- 172.21.15.120:0/1294609977 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c3a4f860 con 0x5643c3a49fe0 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f708dd01640 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7080005e30 tx=0x7f708000ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ea83f435509d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f7072ffd640 1 -- 172.21.15.120:0/1294609977 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080055070 con 0x5643c3a49fe0 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f7072ffd640 1 -- 172.21.15.120:0/1294609977 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708004c070 con 0x5643c3a49fe0 2024-09-10T22:54:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.393+0000 7f7072ffd640 1 -- 172.21.15.120:0/1294609977 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080004330 con 0x5643c3a49fe0 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- 172.21.15.120:0/1294609977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 msgr2=0x5643c3a4a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7080005e30 tx=0x7f708000ad60 comp rx=0 tx=0).stop 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- 172.21.15.120:0/1294609977 shutdown_connections 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a49fe0 0x5643c3a4a900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a4a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 --2- 172.21.15.120:0/1294609977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a44c70 0x5643c3a4d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- 172.21.15.120:0/1294609977 >> 172.21.15.120:0/1294609977 conn(0x5643c3a3b9b0 msgr2=0x5643c3a4de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- 172.21.15.120:0/1294609977 shutdown_connections 2024-09-10T22:54:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- 172.21.15.120:0/1294609977 wait complete. 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 -- start start 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.394+0000 7f7092612740 1 Processor -- start 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a55c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a54970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00d0d0 con 0x5643c3a44c70 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00e540 con 0x5643c3a49fe0 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f706c00d7a0 con 0x5643c3a475a0 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7092612740 10 client.0 _ll_register_callbacks cb 0x7f7084009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708d500640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a55c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708d500640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a55c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37736/0 (socket says 172.21.15.120:37736) 2024-09-10T22:54:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708d500640 1 -- 172.21.15.120:0/2741861839 learned_addr learned my addr 172.21.15.120:0/2741861839 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708dd01640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 -- 172.21.15.120:0/2741861839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 msgr2=0x5643c3a54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 -- 172.21.15.120:0/2741861839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 msgr2=0x5643c3a55c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a55c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 -- 172.21.15.120:0/2741861839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c3a473d0 con 0x5643c3a475a0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f708ccff640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a54970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f70780e3aa0 tx=0x7f70780e9980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c144e22e1daa4208 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.395+0000 7f7070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70780ac4e0 con 0x5643c3a475a0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f70780e9b30 con 0x5643c3a475a0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708004c070 con 0x5643c3a475a0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f70780abe80 con 0x5643c3a475a0 2024-09-10T22:54:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 10 client.25270 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5643c3ac6ba0 con 0x5643c3a475a0 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708004c070 con 0x5643c3a475a0 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f70780a8060 con 0x5643c3a475a0 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 10 client.25270 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 20 client.25270 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3ac73e0 0x5643c3ac9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 4 client.25270 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643c39b00f0 con 0x5643c3ac73e0 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.396+0000 7f708d500640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3ac73e0 0x5643c3ac9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.397+0000 7f7070ff9640 10 client.25270 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.397+0000 7f708d500640 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3ac73e0 0x5643c3ac9800 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435fdef388ce5898 server_cookie=ff684ca033602004 in_seq=0 out_seq=0 2024-09-10T22:54:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7070ff9640 1 -- 172.21.15.120:0/2741861839 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643c39b00f0 con 0x5643c3ac73e0 2024-09-10T22:54:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7070ff9640 10 client.25270 handle_command_reply: tid=0 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 client.25270 shutdown 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 msgr2=0x5643c3a54970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a54970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f70780e3aa0 tx=0x7f70780e9980 comp rx=0 tx=0).stop 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 shutdown_connections 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643c3ac73e0 0x5643c3ac9800 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643c3a49fe0 0x5643c3a54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643c3a475a0 0x5643c3a54970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7070ff9640 0 client.25270 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 --2- 172.21.15.120:0/2741861839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643c3a44c70 0x5643c3a55c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 >> 172.21.15.120:0/2741861839 conn(0x5643c3a3b9b0 msgr2=0x5643c3a4a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 shutdown_connections 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 1 -- 172.21.15.120:0/2741861839 wait complete. 2024-09-10T22:54:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.545+0000 7f7092612740 20 client.25270 trim_cache size 0 max 16384 2024-09-10T22:54:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:54:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:54:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:54:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 msgr2=0x7f70740b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 0x7f70740b77a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7084009da0 tx=0x7f7084005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 msgr2=0x7f7088199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f707800e9f0 tx=0x7f707800edf0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 shutdown_connections 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70740b52e0 0x7f70740b77a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7088102b20 0x7f7088199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7088102160 0x7f7088199390 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 --2- 172.21.15.120:0/2053016169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7088101010 0x7f7088198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 >> 172.21.15.120:0/2053016169 conn(0x7f70880fc8f0 msgr2=0x7f7088105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 shutdown_connections 2024-09-10T22:54:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:54:51.546+0000 7f708f764640 1 -- 172.21.15.120:0/2053016169 wait complete. 2024-09-10T22:54:51.616 INFO:tasks.workunit.client.0.smithi120.stdout: 1 345452 17.74 MB/sec warmup 106 sec latency 53.071 ms 2024-09-10T22:54:51.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:51 smithi120 ceph-mon[31247]: pgmap v417: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-09-10T22:54:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:51 smithi153 ceph-mon[34951]: pgmap v417: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-09-10T22:54:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:51 smithi176 ceph-mon[34982]: pgmap v417: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-09-10T22:54:52.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 348590 17.72 MB/sec warmup 107 sec latency 27.926 ms 2024-09-10T22:54:53.330 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:53.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 351612 17.72 MB/sec warmup 108 sec latency 59.003 ms 2024-09-10T22:54:53.642 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:53 smithi120 ceph-mon[31247]: pgmap v418: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-10T22:54:53.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:53 smithi153 ceph-mon[34951]: pgmap v418: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-10T22:54:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:53 smithi176 ceph-mon[34982]: pgmap v418: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-10T22:54:54.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 -- 172.21.15.120:0/3412876540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69681076a0 msgr2=0x7f6968107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 --2- 172.21.15.120:0/3412876540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69681076a0 0x7f6968107ae0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f695c009900 tx=0x7f695c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 -- 172.21.15.120:0/3412876540 shutdown_connections 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 --2- 172.21.15.120:0/3412876540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 0x7f696810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 --2- 172.21.15.120:0/3412876540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f69681076a0 0x7f6968107ae0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 --2- 172.21.15.120:0/3412876540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6968101c90 0x7f6968102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 -- 172.21.15.120:0/3412876540 >> 172.21.15.120:0/3412876540 conn(0x7f69680fd700 msgr2=0x7f69680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 -- 172.21.15.120:0/3412876540 shutdown_connections 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.245+0000 7f696de9b640 1 -- 172.21.15.120:0/3412876540 wait complete. 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 Processor -- start 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 -- start start 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 0x7f696819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 0x7f696819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6968111c70 con 0x7f6968101c90 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6968111af0 con 0x7f69681076a0 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f696de9b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6968111df0 con 0x7f6968108020 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f6966ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f69677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 0x7f696819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.246+0000 7f6967fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 0x7f696819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39974/0 (socket says 172.21.15.120:39974) 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 -- 172.21.15.120:0/2378036560 learned_addr learned my addr 172.21.15.120:0/2378036560 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 -- 172.21.15.120:0/2378036560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 msgr2=0x7f696819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 0x7f696819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 -- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 msgr2=0x7f696819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 0x7f696819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 -- 172.21.15.120:0/2378036560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a5480 con 0x7f69681076a0 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f69677fe640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 0x7f696819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:54.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f695c005940 tx=0x7f695c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6964ff9640 1 -- 172.21.15.120:0/2378036560 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c03f030 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a5710 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6964ff9640 1 -- 172.21.15.120:0/2378036560 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695c003aa0 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f6964ff9640 1 -- 172.21.15.120:0/2378036560 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c035070 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.247+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681a5c40 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6964ff9640 1 -- 172.21.15.120:0/2378036560 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f695c03bce0 con 0x7f69681076a0 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 Processor -- start 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 -- start start 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118da0 0x55a6a71247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7127120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6a708ee50 con 0x55a6a7118480 2024-09-10T22:54:54.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6a707e280 con 0x55a6a7124d30 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6970270740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7082090 con 0x55a6a7118da0 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6967fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57654/0 (socket says 172.21.15.120:57654) 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6967fff640 1 -- 172.21.15.120:0/4081382610 learned_addr learned my addr 172.21.15.120:0/4081382610 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f6966ffd640 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7127120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.250+0000 7f69677fe640 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118da0 0x55a6a71247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 -- 172.21.15.120:0/4081382610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118da0 msgr2=0x55a6a71247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118da0 0x55a6a71247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 -- 172.21.15.120:0/4081382610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 msgr2=0x55a6a7127120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7127120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 -- 172.21.15.120:0/4081382610 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a6ddc650 con 0x55a6a7118480 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6964ff9640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 0x7f69500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f69677fe640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 0x7f69500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6967fff640 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6960009f30 tx=0x7f69600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4de3d75e1b39ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f6964ff9640 1 -- 172.21.15.120:0/2378036560 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f695c106510 con 0x7f69681076a0 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f694dffb640 1 -- 172.21.15.120:0/4081382610 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960004d20 con 0x55a6a7118480 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f694dffb640 1 -- 172.21.15.120:0/4081382610 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6960004050 con 0x55a6a7118480 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.251+0000 7f694dffb640 1 -- 172.21.15.120:0/4081382610 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960005260 con 0x55a6a7118480 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- 172.21.15.120:0/4081382610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 msgr2=0x55a6a7118860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6960009f30 tx=0x7f69600118d0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f69677fe640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 0x7f69500b77a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55a6a7117190 tx=0x7f695800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- 172.21.15.120:0/4081382610 shutdown_connections 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7127120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118da0 0x55a6a71247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 --2- 172.21.15.120:0/4081382610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7118860 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- 172.21.15.120:0/4081382610 >> 172.21.15.120:0/4081382610 conn(0x55a6a71162b0 msgr2=0x55a6a7117450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- 172.21.15.120:0/4081382610 shutdown_connections 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- 172.21.15.120:0/4081382610 wait complete. 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 -- start start 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.252+0000 7f6970270740 1 Processor -- start 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7118da0 0x55a6a7121a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7124d30 0x55a6a71220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6a6edf700 con 0x55a6a7118480 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a6a7019110 con 0x55a6a7118da0 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a6a71279b0 con 0x55a6a7124d30 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6970270740 10 client.0 _ll_register_callbacks cb 0x55a6a7032990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57662/0 (socket says 172.21.15.120:57662) 2024-09-10T22:54:54.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 -- 172.21.15.120:0/2343611719 learned_addr learned my addr 172.21.15.120:0/2343611719 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7118da0 0x55a6a7121a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6967fff640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7124d30 0x55a6a71220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 -- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7124d30 msgr2=0x55a6a71220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7124d30 0x55a6a71220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 -- 172.21.15.120:0/2343611719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7118da0 msgr2=0x55a6a7121a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7118da0 0x55a6a7121a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 -- 172.21.15.120:0/2343611719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a7046c40 con 0x55a6a7118480 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f69677fe640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f69580109b0 tx=0x7f69580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48149ef9e35132cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.253+0000 7f6946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695802c060 con 0x55a6a7118480 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6958017030 con 0x55a6a7118480 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6960004050 con 0x55a6a7118480 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f695801fb80 con 0x55a6a7118480 2024-09-10T22:54:54.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 10 client.15810 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a6a7133370 con 0x55a6a7118480 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6960004050 con 0x55a6a7118480 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f69580323b0 con 0x55a6a7118480 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 10 client.15810 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 20 client.15810 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a7134830 0x55a6a7136c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 4 client.15810 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6a7090da0 con 0x55a6a7134830 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.254+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a7134830 0x55a6a7136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.255+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a7134830 0x55a6a7136c60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc96a6d59af8cc7 server_cookie=de8f3a2fa030d6a8 in_seq=0 out_seq=0 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.255+0000 7f6946ffd640 10 client.15810 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6946ffd640 1 -- 172.21.15.120:0/2343611719 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6a7090da0 con 0x55a6a7134830 2024-09-10T22:54:54.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6946ffd640 10 client.15810 handle_command_reply: tid=0 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 client.15810 shutdown 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 msgr2=0x55a6a7124640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f69580109b0 tx=0x7f69580046c0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 shutdown_connections 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a7134830 0x55a6a7136c60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7124d30 0x55a6a71220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7118da0 0x55a6a7121a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6946ffd640 0 client.15810 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 --2- 172.21.15.120:0/2343611719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118480 0x55a6a7124640 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 >> 172.21.15.120:0/2343611719 conn(0x55a6a71162b0 msgr2=0x55a6a711be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 shutdown_connections 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 1 -- 172.21.15.120:0/2343611719 wait complete. 2024-09-10T22:54:54.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.256+0000 7f6970270740 20 client.15810 trim_cache size 0 max 16384 2024-09-10T22:54:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 Processor -- start 2024-09-10T22:54:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 -- start start 2024-09-10T22:54:54.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7130be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7126660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ac2d0 con 0x55a6a7118da0 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69681aeb50 con 0x55a6a7124d30 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.274+0000 7f6970270740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ac940 con 0x55a6a7118480 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6966ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7126660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6966ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57674/0 (socket says 172.21.15.120:57674) 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6966ffd640 1 -- 172.21.15.120:0/4084109301 learned_addr learned my addr 172.21.15.120:0/4084109301 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f69677fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7130be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 -- 172.21.15.120:0/4084109301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 msgr2=0x55a6a7130be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7130be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 -- 172.21.15.120:0/4084109301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 msgr2=0x55a6a7126120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 -- 172.21.15.120:0/4084109301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a7118bd0 con 0x55a6a7124d30 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6966ffd640 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f6967fff640 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7126660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6960000f80 tx=0x7f6960009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ff95b62f7e65f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f694d7fa640 1 -- 172.21.15.120:0/4084109301 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960003d70 con 0x55a6a7124d30 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f694d7fa640 1 -- 172.21.15.120:0/4084109301 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69600054f0 con 0x55a6a7124d30 2024-09-10T22:54:54.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.275+0000 7f694d7fa640 1 -- 172.21.15.120:0/4084109301 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696002b8e0 con 0x55a6a7124d30 2024-09-10T22:54:54.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- 172.21.15.120:0/4084109301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 msgr2=0x55a6a7126660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7126660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6960000f80 tx=0x7f6960009c40 comp rx=0 tx=0).stop 2024-09-10T22:54:54.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- 172.21.15.120:0/4084109301 shutdown_connections 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7126660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a7126120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 --2- 172.21.15.120:0/4084109301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7130be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- 172.21.15.120:0/4084109301 >> 172.21.15.120:0/4084109301 conn(0x55a6a71162b0 msgr2=0x55a6a7117370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- 172.21.15.120:0/4084109301 shutdown_connections 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- 172.21.15.120:0/4084109301 wait complete. 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.276+0000 7f6970270740 1 -- start start 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 Processor -- start 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7127cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a71281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ad010 con 0x55a6a7118da0 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ae480 con 0x55a6a7124d30 2024-09-10T22:54:54.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ad6e0 con 0x55a6a7118480 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6970270740 10 client.0 _ll_register_callbacks cb 0x7f6958010650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f69677fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7127cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6966ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a71281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40008/0 (socket says 172.21.15.120:40008) 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 -- 172.21.15.120:0/2136791270 learned_addr learned my addr 172.21.15.120:0/2136791270 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6966ffd640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a71281f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57682/0 (socket says 172.21.15.120:57682) 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 -- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 msgr2=0x55a6a7127cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7127cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 -- 172.21.15.120:0/2136791270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 msgr2=0x55a6a71281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a71281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.277+0000 7f6967fff640 1 -- 172.21.15.120:0/2136791270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a71a18e0 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6967fff640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69600052e0 tx=0x7f6960005310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd831512dbafa013 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960009a50 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f696002b660 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69600054f0 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f696002b860 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6970270740 10 client.25292 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a6a71a2bf0 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69600054f0 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.278+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f696000f8e0 con 0x55a6a7124d30 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 10 client.25292 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:54.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 20 client.25292 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a71a30a0 0x55a6a71a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 4 client.25292 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6a708c850 con 0x55a6a71a30a0 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f69677fe640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a71a30a0 0x55a6a71a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f6946ffd640 10 client.25292 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.279+0000 7f69677fe640 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a71a30a0 0x55a6a71a54c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0a3f59776535d server_cookie=4691bbe2794685f5 in_seq=0 out_seq=0 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6946ffd640 1 -- 172.21.15.120:0/2136791270 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55a6a708c850 con 0x55a6a71a30a0 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6946ffd640 10 client.25292 handle_command_reply: tid=0 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 client.25292 shutdown 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 msgr2=0x55a6a7128730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69600052e0 tx=0x7f6960005310 comp rx=0 tx=0).stop 2024-09-10T22:54:54.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 shutdown_connections 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a6a71a30a0 0x55a6a71a54c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6a7124d30 0x55a6a7128730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6946ffd640 0 client.25292 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6a7118da0 0x55a6a71281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 --2- 172.21.15.120:0/2136791270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6a7118480 0x55a6a7127cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 >> 172.21.15.120:0/2136791270 conn(0x55a6a71162b0 msgr2=0x55a6a71230c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 shutdown_connections 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 1 -- 172.21.15.120:0/2136791270 wait complete. 2024-09-10T22:54:54.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.280+0000 7f6970270740 20 client.25292 trim_cache size 0 max 16384 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:54:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:54:54.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 msgr2=0x7f69500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 0x7f69500b77a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55a6a7117190 tx=0x7f695800b410 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 msgr2=0x7f696819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f695c005940 tx=0x7f695c004270 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 shutdown_connections 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f69500b52e0 0x7f69500b77a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6968108020 0x7f696819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69681076a0 0x7f696819a5e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 --2- 172.21.15.120:0/2378036560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6968101c90 0x7f696819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 >> 172.21.15.120:0/2378036560 conn(0x7f69680fd700 msgr2=0x7f69680ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 shutdown_connections 2024-09-10T22:54:54.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:54.281+0000 7f696de9b640 1 -- 172.21.15.120:0/2378036560 wait complete. 2024-09-10T22:54:54.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 355029 17.73 MB/sec warmup 109 sec latency 13.571 ms 2024-09-10T22:54:55.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:54:55.264 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:54:55.530 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:55.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 358567 17.75 MB/sec warmup 110 sec latency 13.467 ms 2024-09-10T22:54:55.651 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:55 smithi120 ceph-mon[31247]: pgmap v419: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 292 op/s 2024-09-10T22:54:55.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:55 smithi153 ceph-mon[34951]: pgmap v419: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 292 op/s 2024-09-10T22:54:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:55 smithi176 ceph-mon[34982]: pgmap v419: 97 pgs: 97 active+clean; 51 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 292 op/s 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 -- 172.21.15.120:0/2606532659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4400108020 msgr2=0x7f440010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 --2- 172.21.15.120:0/2606532659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4400108020 0x7f440010e820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009ca0 tx=0x7f43f802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 -- 172.21.15.120:0/2606532659 shutdown_connections 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 --2- 172.21.15.120:0/2606532659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4400108020 0x7f440010e820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 --2- 172.21.15.120:0/2606532659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44001076a0 0x7f4400107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 --2- 172.21.15.120:0/2606532659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 0x7f4400102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 -- 172.21.15.120:0/2606532659 >> 172.21.15.120:0/2606532659 conn(0x7f44000fd700 msgr2=0x7f44000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 -- 172.21.15.120:0/2606532659 shutdown_connections 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.008+0000 7f44048ac640 1 -- 172.21.15.120:0/2606532659 wait complete. 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.009+0000 7f44048ac640 1 Processor -- start 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.009+0000 7f44048ac640 1 -- start start 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 0x7f440019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4400108020 0x7f44001a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111ca0 con 0x7f44001076a0 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111b20 con 0x7f4400101c90 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111e20 con 0x7f4400108020 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57704/0 (socket says 172.21.15.120:57704) 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 -- 172.21.15.120:0/2371597258 learned_addr learned my addr 172.21.15.120:0/2371597258 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:56.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43fed9d640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4400108020 0x7f44001a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43fe59c640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 0x7f440019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 -- 172.21.15.120:0/2371597258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4400108020 msgr2=0x7f44001a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4400108020 0x7f44001a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 -- 172.21.15.120:0/2371597258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 msgr2=0x7f440019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 0x7f440019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 -- 172.21.15.120:0/2371597258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44001a1500 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f5d9b640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e800cd80 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a17f0 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e80054d0 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8010690 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.010+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a7eb0 con 0x7f44001076a0 2024-09-10T22:54:56.040 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.011+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400102070 con 0x7f44001076a0 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.013+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f43e8004e30 con 0x7f44001076a0 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.014+0000 7f43f7fff640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 0x7f43ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.014+0000 7f43fe59c640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 0x7f43ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.014+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f43e80e6180 con 0x7f44001076a0 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.014+0000 7f43fe59c640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 0x7f43ec0b77a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009bd0 tx=0x7f43f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.041 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.017+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f43e80ad070 con 0x7f44001076a0 2024-09-10T22:54:56.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.241+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44001a8200 con 0x7f44001076a0 2024-09-10T22:54:56.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.241+0000 7f43f7fff640 1 -- 172.21.15.120:0/2371597258 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f43e80ac970 con 0x7f44001076a0 2024-09-10T22:54:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:54:56.244 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":23,"btime":"2024-09-10T22:54:39:370770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:54:39.370760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25252":{"gid":25252,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2789135593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2789135593},{"type":"v1","addr":"172.21.15.176:6837","nonce":2789135593}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:54:56.244 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 23 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 msgr2=0x7f43ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 0x7f43ec0b77a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009bd0 tx=0x7f43f0005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 msgr2=0x7f440019b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800edf0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 shutdown_connections 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f43ec0b52e0 0x7f43ec0b77a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4400108020 0x7f44001a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44001076a0 0x7f440019b990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 --2- 172.21.15.120:0/2371597258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4400101c90 0x7f440019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 >> 172.21.15.120:0/2371597258 conn(0x7f44000fd700 msgr2=0x7f440006abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 shutdown_connections 2024-09-10T22:54:56.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:56.244+0000 7f44048ac640 1 -- 172.21.15.120:0/2371597258 wait complete. 2024-09-10T22:54:56.601 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:56 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2371597258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:56.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 362083 17.76 MB/sec warmup 111 sec latency 15.059 ms 2024-09-10T22:54:56.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:56 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2371597258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:56.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:56 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2371597258' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:57.080 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:54:57.335 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:57.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 365551 17.77 MB/sec warmup 112 sec latency 20.464 ms 2024-09-10T22:54:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:57 smithi120 ceph-mon[31247]: pgmap v420: 97 pgs: 97 active+clean; 51 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-09-10T22:54:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:57 smithi153 ceph-mon[34951]: pgmap v420: 97 pgs: 97 active+clean; 51 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-09-10T22:54:57.724 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:57 smithi176 ceph-mon[34982]: pgmap v420: 97 pgs: 97 active+clean; 51 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.821+0000 7f3b8ea55640 1 -- 172.21.15.120:0/4123198633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881055c0 msgr2=0x7f3b88105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.821+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/4123198633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881055c0 0x7f3b88105a20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009900 tx=0x7f3b7c02a410 comp rx=0 tx=0).stop 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 -- 172.21.15.120:0/4123198633 shutdown_connections 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/4123198633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b88105f60 0x7f3b8810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/4123198633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881055c0 0x7f3b88105a20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/4123198633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b881012a0 0x7f3b88101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 -- 172.21.15.120:0/4123198633 >> 172.21.15.120:0/4123198633 conn(0x7f3b88078060 msgr2=0x7f3b880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 -- 172.21.15.120:0/4123198633 shutdown_connections 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 -- 172.21.15.120:0/4123198633 wait complete. 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 Processor -- start 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.822+0000 7f3b8ea55640 1 -- start start 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 0x7f3b88199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b88105f60 0x7f3b8819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112db0 con 0x7f3b881012a0 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112c30 con 0x7f3b881055c0 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88112f30 con 0x7f3b88105f60 2024-09-10T22:54:57.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57736/0 (socket says 172.21.15.120:57736) 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 0x7f3b88199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 -- 172.21.15.120:0/2733420926 learned_addr learned my addr 172.21.15.120:0/2733420926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b88105f60 0x7f3b8819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 -- 172.21.15.120:0/2733420926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b88105f60 msgr2=0x7f3b8819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b88105f60 0x7f3b8819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 -- 172.21.15.120:0/2733420926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 msgr2=0x7f3b88199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 0x7f3b88199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 -- 172.21.15.120:0/2733420926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8819f2f0 con 0x7f3b881012a0 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b877fe640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 0x7f3b88199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800d950 tx=0x7f3b7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b857fa640 1 -- 172.21.15.120:0/2733420926 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78017030 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8819f5e0 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b857fa640 1 -- 172.21.15.120:0/2733420926 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b78004020 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.823+0000 7f3b857fa640 1 -- 172.21.15.120:0/2733420926 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780047a0 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.824+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a5ca0 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b857fa640 1 -- 172.21.15.120:0/2733420926 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3b78009a30 con 0x7f3b881012a0 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 Processor -- start 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 -- start start 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b91a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b928d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55742b890ea0 con 0x55742b91aa10 2024-09-10T22:54:57.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55742b8802d0 con 0x55742b91a0f0 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55742b8840e0 con 0x55742b9269a0 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57744/0 (socket says 172.21.15.120:57744) 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b87fff640 1 -- 172.21.15.120:0/3753264993 learned_addr learned my addr 172.21.15.120:0/3753264993 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b91a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.826+0000 7f3b877fe640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b928d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 -- 172.21.15.120:0/3753264993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 msgr2=0x55742b928d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b928d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 -- 172.21.15.120:0/3753264993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 msgr2=0x55742b91a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b91a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 -- 172.21.15.120:0/3753264993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742b5de650 con 0x55742b91aa10 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b857fa640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 0x7f3b700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b877fe640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b928d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b91a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 0x7f3b700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b857fa640 1 -- 172.21.15.120:0/2733420926 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3b780e55d0 con 0x7f3b881012a0 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b87fff640 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b780eb5b0 tx=0x7f3b780a8410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ce7bf1ed1f709b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b6e7fc640 1 -- 172.21.15.120:0/3753264993 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780abb90 con 0x55742b91aa10 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b6e7fc640 1 -- 172.21.15.120:0/3753264993 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b780ebe40 con 0x55742b91aa10 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.827+0000 7f3b6e7fc640 1 -- 172.21.15.120:0/3753264993 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780ab530 con 0x55742b91aa10 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 -- 172.21.15.120:0/3753264993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 msgr2=0x55742b926460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 0x7f3b700b77a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b80010540 tx=0x7f3b80007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b780eb5b0 tx=0x7f3b780a8410 comp rx=0 tx=0).stop 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 -- 172.21.15.120:0/3753264993 shutdown_connections 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b928d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b926460 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/3753264993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b91a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 -- 172.21.15.120:0/3753264993 >> 172.21.15.120:0/3753264993 conn(0x55742b917f20 msgr2=0x55742b9190c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 -- 172.21.15.120:0/3753264993 shutdown_connections 2024-09-10T22:54:57.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.828+0000 7f3b90e2a740 1 -- 172.21.15.120:0/3753264993 wait complete. 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 -- start start 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 Processor -- start 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91aa10 0x55742b9236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 0x55742b923d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55742b6e1700 con 0x55742b91a0f0 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55742b81b160 con 0x55742b9269a0 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55742b9295b0 con 0x55742b91aa10 2024-09-10T22:54:57.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b90e2a740 10 client.0 _ll_register_callbacks cb 0x55742b8349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57754/0 (socket says 172.21.15.120:57754) 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91aa10 0x55742b9236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 -- 172.21.15.120:0/2211622283 learned_addr learned my addr 172.21.15.120:0/2211622283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 0x55742b923d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 -- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91aa10 msgr2=0x55742b9236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91aa10 0x55742b9236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 -- 172.21.15.120:0/2211622283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 msgr2=0x55742b923d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 0x55742b923d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b87fff640 1 -- 172.21.15.120:0/2211622283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742b848c90 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.829+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 0x55742b923d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b87fff640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3b780e3470 tx=0x7f3b780a7d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3b2e0165a86e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780ab710 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3b780930d0 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55742b892df0 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3b780ab900 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 10 client.15834 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55742b934f70 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55742b892df0 con 0x55742b91a0f0 2024-09-10T22:54:57.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f3b78091be0 con 0x55742b91a0f0 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 10 client.15834 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 20 client.15834 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9362c0 0x55742b9386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 4 client.15834 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.830+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742b88e8a0 con 0x55742b9362c0 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.831+0000 7f3b877fe640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9362c0 0x55742b9386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.831+0000 7f3b677fe640 10 client.15834 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.831+0000 7f3b877fe640 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9362c0 0x55742b9386b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426e7af83cddbe6d server_cookie=716fdb3f078859d0 in_seq=0 out_seq=0 2024-09-10T22:54:57.850 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b677fe640 1 -- 172.21.15.120:0/2211622283 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55742b88e8a0 con 0x55742b9362c0 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b677fe640 10 client.15834 handle_command_reply: tid=0 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 client.15834 shutdown 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 msgr2=0x55742b926290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3b780e3470 tx=0x7f3b780a7d60 comp rx=0 tx=0).stop 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 shutdown_connections 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9362c0 0x55742b9386b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b9269a0 0x55742b923d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.851 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b677fe640 0 client.15834 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91aa10 0x55742b9236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/2211622283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91a0f0 0x55742b926290 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 >> 172.21.15.120:0/2211622283 conn(0x55742b917f20 msgr2=0x55742b91da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 shutdown_connections 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 1 -- 172.21.15.120:0/2211622283 wait complete. 2024-09-10T22:54:57.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.832+0000 7f3b90e2a740 20 client.15834 trim_cache size 0 max 16384 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.845+0000 7f3b90e2a740 1 Processor -- start 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.845+0000 7f3b90e2a740 1 -- start start 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b917760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b9282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881ac220 con 0x55742b91aa10 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881aeaa0 con 0x55742b91a0f0 2024-09-10T22:54:57.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881ac890 con 0x55742b9269a0 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57764/0 (socket says 172.21.15.120:57764) 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 -- 172.21.15.120:0/876756189 learned_addr learned my addr 172.21.15.120:0/876756189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b9282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b87fff640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b917760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 -- 172.21.15.120:0/876756189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 msgr2=0x55742b9282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b9282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 -- 172.21.15.120:0/876756189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 msgr2=0x55742b917760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b917760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 -- 172.21.15.120:0/876756189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742b9137e0 con 0x55742b91aa10 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b87fff640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b917760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b877fe640 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c02a930 tx=0x7f3b7c0315e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370b0f0291e3a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b6dffb640 1 -- 172.21.15.120:0/876756189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c035040 con 0x55742b91aa10 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b6dffb640 1 -- 172.21.15.120:0/876756189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b7c0054f0 con 0x55742b91aa10 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.846+0000 7f3b6dffb640 1 -- 172.21.15.120:0/876756189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c004d00 con 0x55742b91aa10 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- 172.21.15.120:0/876756189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 msgr2=0x55742b927d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c02a930 tx=0x7f3b7c0315e0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- 172.21.15.120:0/876756189 shutdown_connections 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b9269a0 0x55742b9282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b91aa10 0x55742b927d90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/876756189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91a0f0 0x55742b917760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- 172.21.15.120:0/876756189 >> 172.21.15.120:0/876756189 conn(0x55742b917f20 msgr2=0x55742b91e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- 172.21.15.120:0/876756189 shutdown_connections 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- 172.21.15.120:0/876756189 wait complete. 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.847+0000 7f3b90e2a740 1 -- start start 2024-09-10T22:54:57.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 Processor -- start 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 0x55742b93bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 0x55742b9370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881acf60 con 0x55742b9269a0 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881ae3d0 con 0x55742b91aa10 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3b881ad630 con 0x55742b91a0f0 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b90e2a740 10 client.0 _ll_register_callbacks cb 0x7f3b80007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:57.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b8cff2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b8cff2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57766/0 (socket says 172.21.15.120:57766) 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 0x55742b93bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b8cff2640 1 -- 172.21.15.120:0/305377579 learned_addr learned my addr 172.21.15.120:0/305377579 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.848+0000 7f3b877fe640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 0x55742b9370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 -- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 msgr2=0x55742b93bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 0x55742b93bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 -- 172.21.15.120:0/305377579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 msgr2=0x55742b9370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:57.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 0x55742b9370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 -- 172.21.15.120:0/305377579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742b9a1e20 con 0x55742b9269a0 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b87fff640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 0x55742b93bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b877fe640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 0x55742b9370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b8cff2640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000fca0 tx=0x7f3b80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd5f520324990a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8002c060 con 0x55742b9269a0 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3b8000fcd0 con 0x55742b9269a0 2024-09-10T22:54:57.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b7c0054f0 con 0x55742b9269a0 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3b80027400 con 0x55742b9269a0 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b90e2a740 10 client.15840 fetch_fsmap learned FSMap version 23 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55742b9a3040 con 0x55742b9269a0 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b7c0054f0 con 0x55742b9269a0 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.849+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f3b8001b500 con 0x55742b9269a0 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 10 client.15840 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:54:57.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 20 client.15840 populate_metadata read hostname 'smithi120' 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9a3820 0x55742b9a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 4 client.15840 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55742b88bcd0 con 0x55742b9a3820 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b87fff640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9a3820 0x55742b9a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b677fe640 10 client.15840 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.850+0000 7f3b87fff640 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9a3820 0x55742b9a5c40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5d19d777257cc9 server_cookie=70aa080e134739db in_seq=0 out_seq=0 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b677fe640 1 -- 172.21.15.120:0/305377579 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55742b88bcd0 con 0x55742b9a3820 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b677fe640 10 client.15840 handle_command_reply: tid=0 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 client.15840 shutdown 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 msgr2=0x55742b9377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000fca0 tx=0x7f3b80004a40 comp rx=0 tx=0).stop 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 shutdown_connections 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55742b9a3820 0x55742b9a5c40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55742b9269a0 0x55742b9377b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55742b91aa10 0x55742b9370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b677fe640 0 client.15840 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 --2- 172.21.15.120:0/305377579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55742b91a0f0 0x55742b93bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 >> 172.21.15.120:0/305377579 conn(0x55742b917f20 msgr2=0x55742b924d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 shutdown_connections 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 1 -- 172.21.15.120:0/305377579 wait complete. 2024-09-10T22:54:57.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.852+0000 7f3b90e2a740 20 client.15840 trim_cache size 0 max 16384 2024-09-10T22:54:57.869 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:54:57.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 msgr2=0x7f3b700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 0x7f3b700b77a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b80010540 tx=0x7f3b80007520 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 msgr2=0x7f3b88199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800d950 tx=0x7f3b7800de10 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 shutdown_connections 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b700b52e0 0x7f3b700b77a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b88105f60 0x7f3b8819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b881055c0 0x7f3b88199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 --2- 172.21.15.120:0/2733420926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b881012a0 0x7f3b88199240 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 >> 172.21.15.120:0/2733420926 conn(0x7f3b88078060 msgr2=0x7f3b88108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 shutdown_connections 2024-09-10T22:54:57.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:54:57.853+0000 7f3b8ea55640 1 -- 172.21.15.120:0/2733420926 wait complete. 2024-09-10T22:54:58.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 368349 17.73 MB/sec warmup 113 sec latency 71.643 ms 2024-09-10T22:54:59.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 371340 17.69 MB/sec warmup 114 sec latency 38.876 ms 2024-09-10T22:54:59.660 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:54:59.660 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:54:59.683 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:54:59 smithi120 ceph-mon[31247]: pgmap v421: 97 pgs: 97 active+clean; 43 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-09-10T22:54:59.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:54:59 smithi153 ceph-mon[34951]: pgmap v421: 97 pgs: 97 active+clean; 43 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-09-10T22:54:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:54:59 smithi176 ceph-mon[34982]: pgmap v421: 97 pgs: 97 active+clean; 43 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-09-10T22:54:59.928 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:00.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 -- 172.21.15.120:0/1192265028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a010d100 msgr2=0x7f63a010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 --2- 172.21.15.120:0/1192265028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a010d100 0x7f63a010f560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6398009ca0 tx=0x7f639802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 -- 172.21.15.120:0/1192265028 shutdown_connections 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 --2- 172.21.15.120:0/1192265028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a010d100 0x7f63a010f560 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 --2- 172.21.15.120:0/1192265028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a01045c0 0x7f63a010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 --2- 172.21.15.120:0/1192265028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a0103ca0 0x7f63a0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 -- 172.21.15.120:0/1192265028 >> 172.21.15.120:0/1192265028 conn(0x7f63a00fd700 msgr2=0x7f63a00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.432+0000 7f63a5e01640 1 -- 172.21.15.120:0/1192265028 shutdown_connections 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.433+0000 7f63a5e01640 1 -- 172.21.15.120:0/1192265028 wait complete. 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.433+0000 7f63a5e01640 1 Processor -- start 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.433+0000 7f63a5e01640 1 -- start start 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a0103ca0 0x7f63a019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 0x7f63a01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0112d70 con 0x7f63a01045c0 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0112bf0 con 0x7f63a010d100 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f63a5e01640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0112ef0 con 0x7f63a0103ca0 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57788/0 (socket says 172.21.15.120:57788) 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 -- 172.21.15.120:0/2865973826 learned_addr learned my addr 172.21.15.120:0/2865973826 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639f7fe640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a0103ca0 0x7f63a019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639ffff640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 0x7f63a01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 -- 172.21.15.120:0/2865973826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a0103ca0 msgr2=0x7f63a019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a0103ca0 0x7f63a019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 -- 172.21.15.120:0/2865973826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 msgr2=0x7f63a01a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 0x7f63a01a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 -- 172.21.15.120:0/2865973826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a01a1500 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639ffff640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 0x7f63a01a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.434+0000 7f639effd640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f639400e9c0 tx=0x7f639400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.435+0000 7f639cff9640 1 -- 172.21.15.120:0/2865973826 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400ccc0 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.435+0000 7f639cff9640 1 -- 172.21.15.120:0/2865973826 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63940035f0 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.435+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a01a17f0 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.435+0000 7f639cff9640 1 -- 172.21.15.120:0/2865973826 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400ce60 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.435+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a01a7eb0 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.437+0000 7f639cff9640 1 -- 172.21.15.120:0/2865973826 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6394004000 con 0x7f63a01045c0 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.437+0000 7f63a81d6740 1 Processor -- start 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.437+0000 7f63a81d6740 1 -- start start 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260c3790 0x5624260c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56242602de60 con 0x5624260b6ee0 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56242601d290 con 0x5624260c3790 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f63a81d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5624260210a0 con 0x5624260b7800 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57794/0 (socket says 172.21.15.120:57794) 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639ffff640 1 -- 172.21.15.120:0/397631731 learned_addr learned my addr 172.21.15.120:0/397631731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639cff9640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 0x7f63880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639effd640 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260c3790 0x5624260c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 -- 172.21.15.120:0/397631731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260c3790 msgr2=0x5624260c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260c3790 0x5624260c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 -- 172.21.15.120:0/397631731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 msgr2=0x5624260b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 -- 172.21.15.120:0/397631731 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562425d7b930 con 0x5624260b7800 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639ffff640 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639ffff640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 0x7f63880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639cff9640 1 -- 172.21.15.120:0/2865973826 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f63940e5e70 con 0x7f63a01045c0 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.438+0000 7f639f7fe640 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c3250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6390009c50 tx=0x7f6390011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e0c0390954cfddc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.439+0000 7f6385ffb640 1 -- 172.21.15.120:0/397631731 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390009e00 con 0x5624260b7800 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.439+0000 7f6385ffb640 1 -- 172.21.15.120:0/397631731 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6390005020 con 0x5624260b7800 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.439+0000 7f6385ffb640 1 -- 172.21.15.120:0/397631731 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390004b60 con 0x5624260b7800 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- 172.21.15.120:0/397631731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 msgr2=0x5624260c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f639ffff640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 0x7f63880b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6398035000 tx=0x7f63980095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c3250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6390009c50 tx=0x7f6390011810 comp rx=0 tx=0).stop 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- 172.21.15.120:0/397631731 shutdown_connections 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260c3790 0x5624260c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c3250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 --2- 172.21.15.120:0/397631731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b6ee0 0x5624260b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- 172.21.15.120:0/397631731 >> 172.21.15.120:0/397631731 conn(0x5624260b4d10 msgr2=0x5624260b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- 172.21.15.120:0/397631731 shutdown_connections 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- 172.21.15.120:0/397631731 wait complete. 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 -- start start 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.440+0000 7f63a81d6740 1 Processor -- start 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562425e7e890 con 0x5624260c3790 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562425fb7d40 con 0x5624260b6ee0 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5624260c6440 con 0x5624260b7800 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f63a81d6740 10 client.0 _ll_register_callbacks cb 0x56242601bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:00.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35484/0 (socket says 172.21.15.120:35484) 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 -- 172.21.15.120:0/3615485974 learned_addr learned my addr 172.21.15.120:0/3615485974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57808/0 (socket says 172.21.15.120:57808) 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 -- 172.21.15.120:0/3615485974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 msgr2=0x5624260c0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260c0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 -- 172.21.15.120:0/3615485974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 msgr2=0x5624260c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 -- 172.21.15.120:0/3615485974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624260cf820 con 0x5624260b7800 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639ffff640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:00.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f639effd640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63940a8c30 tx=0x7f63940abe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686d5ceea62c9bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f637effd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.441+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63940e3e60 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6390005020 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f63940037a0 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f63940ac170 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 10 client.25324 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5624260d2200 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6390005020 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f63940a7fb0 con 0x5624260b7800 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 10 client.25324 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 20 client.25324 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5624260d3550 0x5624260d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 4 client.25324 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:00.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242602fdb0 con 0x5624260d3550 2024-09-10T22:55:00.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.442+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5624260d3550 0x5624260d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.443+0000 7f637effd640 10 client.25324 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.443+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5624260d3550 0x5624260d5970 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d52a654e509b53 server_cookie=c6789d3e175789fe in_seq=0 out_seq=0 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f637effd640 1 -- 172.21.15.120:0/3615485974 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56242602fdb0 con 0x5624260d3550 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f637effd640 10 client.25324 handle_command_reply: tid=0 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 client.25324 shutdown 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 msgr2=0x5624260c0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63940a8c30 tx=0x7f63940abe90 comp rx=0 tx=0).stop 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 shutdown_connections 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5624260d3550 0x5624260d5970 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b7800 0x5624260c0c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f637effd640 0 client.25324 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3615485974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 >> 172.21.15.120:0/3615485974 conn(0x5624260b4d10 msgr2=0x5624260b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 shutdown_connections 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 1 -- 172.21.15.120:0/3615485974 wait complete. 2024-09-10T22:55:00.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.444+0000 7f63a81d6740 20 client.25324 trim_cache size 0 max 16384 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 Processor -- start 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 -- start start 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 0x5624260db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01ae4d0 con 0x5624260b7800 2024-09-10T22:55:00.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01b0d50 con 0x5624260b6ee0 2024-09-10T22:55:00.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f63a81d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01aeb40 con 0x5624260c3790 2024-09-10T22:55:00.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f639effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f639effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57816/0 (socket says 172.21.15.120:57816) 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f639ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 0x5624260db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f639effd640 1 -- 172.21.15.120:0/2070678860 learned_addr learned my addr 172.21.15.120:0/2070678860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.463+0000 7f639f7fe640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 -- 172.21.15.120:0/2070678860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 msgr2=0x5624260db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 0x5624260db260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 -- 172.21.15.120:0/2070678860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 msgr2=0x5624260bde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 -- 172.21.15.120:0/2070678860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624260b5560 con 0x5624260b6ee0 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639ffff640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 0x5624260db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639effd640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f639f7fe640 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260bd950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6390009b70 tx=0x7f63900096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b88b14fc8414a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f63857fa640 1 -- 172.21.15.120:0/2070678860 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390005250 con 0x5624260b6ee0 2024-09-10T22:55:00.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f63857fa640 1 -- 172.21.15.120:0/2070678860 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63900053f0 con 0x5624260b6ee0 2024-09-10T22:55:00.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.464+0000 7f63857fa640 1 -- 172.21.15.120:0/2070678860 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390025670 con 0x5624260b6ee0 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- 172.21.15.120:0/2070678860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 msgr2=0x5624260bd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260bd950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6390009b70 tx=0x7f63900096a0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- 172.21.15.120:0/2070678860 shutdown_connections 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260c3790 0x5624260db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260b7800 0x5624260bde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 --2- 172.21.15.120:0/2070678860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b6ee0 0x5624260bd950 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- 172.21.15.120:0/2070678860 >> 172.21.15.120:0/2070678860 conn(0x5624260b4d10 msgr2=0x5624260befc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- 172.21.15.120:0/2070678860 shutdown_connections 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- 172.21.15.120:0/2070678860 wait complete. 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.465+0000 7f63a81d6740 1 -- start start 2024-09-10T22:55:00.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 Processor -- start 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 0x5624260cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b7800 0x5624260c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01af210 con 0x5624260c3790 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01b0680 con 0x5624260b7800 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01af8e0 con 0x5624260b6ee0 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f63a81d6740 10 client.0 _ll_register_callbacks cb 0x7f63940a8da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57820/0 (socket says 172.21.15.120:57820) 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 -- 172.21.15.120:0/3519966586 learned_addr learned my addr 172.21.15.120:0/3519966586 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 0x5624260cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639effd640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b7800 0x5624260c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 -- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 msgr2=0x5624260cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 0x5624260cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 -- 172.21.15.120:0/3519966586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b7800 msgr2=0x5624260c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b7800 0x5624260c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639ffff640 1 -- 172.21.15.120:0/3519966586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624260c8cd0 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.466+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 0x5624260cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f639ffff640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6398003e70 tx=0x7f639802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b2d5689ee8dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639805b060 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63900053f0 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f639802f660 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f639802f860 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 10 client.15858 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562426141350 con 0x5624260c3790 2024-09-10T22:55:00.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63900053f0 con 0x5624260c3790 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6398030b30 con 0x5624260c3790 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 10 client.15858 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 20 client.15858 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562426141a90 0x562426143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 4 client.15858 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.467+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56242602b860 con 0x562426141a90 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.468+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562426141a90 0x562426143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.469+0000 7f639f7fe640 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562426141a90 0x562426143eb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444a96876897e4ab server_cookie=e6c2942ac614529f in_seq=0 out_seq=0 2024-09-10T22:55:00.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.469+0000 7f637effd640 10 client.15858 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:00.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f637effd640 1 -- 172.21.15.120:0/3519966586 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56242602b860 con 0x562426141a90 2024-09-10T22:55:00.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f637effd640 10 client.15858 handle_command_reply: tid=0 2024-09-10T22:55:00.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 client.15858 shutdown 2024-09-10T22:55:00.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 msgr2=0x5624260c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6398003e70 tx=0x7f639802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 shutdown_connections 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562426141a90 0x562426143eb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624260c3790 0x5624260c8690 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5624260b7800 0x5624260c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 --2- 172.21.15.120:0/3519966586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624260b6ee0 0x5624260cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 >> 172.21.15.120:0/3519966586 conn(0x5624260b4d10 msgr2=0x5624260cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f637effd640 0 client.15858 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 shutdown_connections 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 1 -- 172.21.15.120:0/3519966586 wait complete. 2024-09-10T22:55:00.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.470+0000 7f63a81d6740 20 client.15858 trim_cache size 0 max 16384 2024-09-10T22:55:00.477 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:55:00.477 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:55:00.477 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "5b13ee9d-06c7-493a-b472-734d0fb7db89", 2024-09-10T22:55:00.477 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:55:00.477 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 msgr2=0x7f63880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 0x7f63880b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6398035000 tx=0x7f63980095c0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 msgr2=0x7f63a019b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f639400e9c0 tx=0x7f639400ee80 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 shutdown_connections 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f63880b52e0 0x7f63880b77a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63a010d100 0x7f63a01a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63a01045c0 0x7f63a019b990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 --2- 172.21.15.120:0/2865973826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63a0103ca0 0x7f63a019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 >> 172.21.15.120:0/2865973826 conn(0x7f63a00fd700 msgr2=0x7f63a010f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 shutdown_connections 2024-09-10T22:55:00.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:00.471+0000 7f63a5e01640 1 -- 172.21.15.120:0/2865973826 wait complete. 2024-09-10T22:55:00.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 373857 17.69 MB/sec warmup 115 sec latency 51.559 ms 2024-09-10T22:55:01.302 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:55:01.616 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:01.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 377225 17.67 MB/sec warmup 116 sec latency 13.813 ms 2024-09-10T22:55:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:01 smithi120 ceph-mon[31247]: pgmap v422: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-09-10T22:55:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:01 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:55:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:01 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:01 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:55:01.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:01 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:01 smithi153 ceph-mon[34951]: pgmap v422: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-09-10T22:55:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:01 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:55:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:01 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:01 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:55:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:01 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:01 smithi176 ceph-mon[34982]: pgmap v422: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-09-10T22:55:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:01 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:55:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:01 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:01 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:55:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:01 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 -- 172.21.15.120:0/3818990420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 msgr2=0x7f800410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 --2- 172.21.15.120:0/3818990420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f800410cab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009900 tx=0x7f7ff402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 -- 172.21.15.120:0/3818990420 shutdown_connections 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 --2- 172.21.15.120:0/3818990420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f800410d100 0x7f800410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 --2- 172.21.15.120:0/3818990420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f800410cab0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 --2- 172.21.15.120:0/3818990420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8004103ca0 0x7f8004104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 -- 172.21.15.120:0/3818990420 >> 172.21.15.120:0/3818990420 conn(0x7f80040fd700 msgr2=0x7f80040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 -- 172.21.15.120:0/3818990420 shutdown_connections 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.104+0000 7f800b6a3640 1 -- 172.21.15.120:0/3818990420 wait complete. 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.105+0000 7f800b6a3640 1 Processor -- start 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.105+0000 7f800b6a3640 1 -- start start 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.105+0000 7f800b6a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8004103ca0 0x7f8004101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.105+0000 7f800b6a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.105+0000 7f800b6a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57824/0 (socket says 172.21.15.120:57824) 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35524/0 (socket says 172.21.15.120:35524) 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f800b6a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8004112db0 con 0x7f80041045c0 2024-09-10T22:55:02.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8008c3e640 1 -- 172.21.15.120:0/1544046519 learned_addr learned my addr 172.21.15.120:0/1544046519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8004112c30 con 0x7f8004103ca0 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8004112f30 con 0x7f800410d100 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f800943f640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8004103ca0 0x7f8004101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 -- 172.21.15.120:0/1544046519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8004103ca0 msgr2=0x7f8004101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8004103ca0 0x7f8004101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 -- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 msgr2=0x7f8004101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 -- 172.21.15.120:0/1544046519 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80041a76e0 con 0x7f800410d100 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f8009c40640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00be20 tx=0x7f7ffc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.106+0000 7f7ffa7fc640 1 -- 172.21.15.120:0/1544046519 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc019070 con 0x7f800410d100 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.107+0000 7f7ffa7fc640 1 -- 172.21.15.120:0/1544046519 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc0148f0 con 0x7f800410d100 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.107+0000 7f7ffa7fc640 1 -- 172.21.15.120:0/1544046519 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc012840 con 0x7f800410d100 2024-09-10T22:55:02.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.107+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041a79d0 con 0x7f800410d100 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.107+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041a7ec0 con 0x7f800410d100 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.109+0000 7f800da78740 1 Processor -- start 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f7ffa7fc640 1 -- 172.21.15.120:0/1544046519 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7ffc014440 con 0x7f800410d100 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 -- start start 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487480 0x55c040487860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 0x55c0404937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c0403fde50 con 0x55c040493d30 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c0403ed280 con 0x55c040487da0 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800da78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c0403f1090 con 0x55c040487480 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57832/0 (socket says 172.21.15.120:57832) 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f8008c3e640 1 -- 172.21.15.120:0/2134030563 learned_addr learned my addr 172.21.15.120:0/2134030563 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:02.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f8008c3e640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487480 0x55c040487860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.110+0000 7f800943f640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 0x55c0404937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 -- 172.21.15.120:0/2134030563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487480 msgr2=0x55c040487860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487480 0x55c040487860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 -- 172.21.15.120:0/2134030563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 msgr2=0x55c040496120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 -- 172.21.15.120:0/2134030563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c04014b650 con 0x55c040487da0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f8008c3e640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f7ffa7fc640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 0x7f7ff00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f800943f640 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 0x55c0404937f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8000005fd0 tx=0x7f80000107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c8eb2a286b2025 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f7ffa7fc640 1 -- 172.21.15.120:0/1544046519 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f7ffc0e5590 con 0x7f800410d100 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 0x7f7ff00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f7fef7fe640 1 -- 172.21.15.120:0/2134030563 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800001c680 con 0x55c040487da0 2024-09-10T22:55:02.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f7fef7fe640 1 -- 172.21.15.120:0/2134030563 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800000be20 con 0x55c040487da0 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.111+0000 7f7fef7fe640 1 -- 172.21.15.120:0/2134030563 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000011a80 con 0x55c040487da0 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 -- 172.21.15.120:0/2134030563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 msgr2=0x55c0404937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 0x7f7ff00b77a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4031a90 tx=0x7f7ff4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 0x55c0404937f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8000005fd0 tx=0x7f80000107a0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 -- 172.21.15.120:0/2134030563 shutdown_connections 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040496120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487da0 0x55c0404937f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 --2- 172.21.15.120:0/2134030563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487480 0x55c040487860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.112+0000 7f800da78740 1 -- 172.21.15.120:0/2134030563 >> 172.21.15.120:0/2134030563 conn(0x55c0404852b0 msgr2=0x55c040486450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.113+0000 7f800da78740 1 -- 172.21.15.120:0/2134030563 shutdown_connections 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.113+0000 7f800da78740 1 -- 172.21.15.120:0/2134030563 wait complete. 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.113+0000 7f800da78740 1 -- start start 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.113+0000 7f800da78740 1 Processor -- start 2024-09-10T22:55:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040490a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c04024e700 con 0x55c040493d30 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c040388110 con 0x55c040487480 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c0404969b0 con 0x55c040487da0 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800da78740 10 client.0 _ll_register_callbacks cb 0x55c0403a1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8009c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040490a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f800943f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8009c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57838/0 (socket says 172.21.15.120:57838) 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8009c40640 1 -- 172.21.15.120:0/3459891625 learned_addr learned my addr 172.21.15.120:0/3459891625 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8008c3e640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040490a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35532/0 (socket says 172.21.15.120:35532) 2024-09-10T22:55:02.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8009c40640 1 -- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 msgr2=0x55c040490a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.114+0000 7f8009c40640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040490a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f8009c40640 1 -- 172.21.15.120:0/3459891625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 msgr2=0x55c0404935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f8009c40640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f8009c40640 1 -- 172.21.15.120:0/3459891625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0403b5c40 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f8009c40640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bdf0 tx=0x7f7ffc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dee94b6fd126b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0932b0 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f7ffc0ac090 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800000be20 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7ffc0abe30 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f800da78740 10 client.15870 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c0404aab10 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800000be20 con 0x55c040493d30 2024-09-10T22:55:02.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.115+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7ffc0b4e80 con 0x55c040493d30 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 10 client.15870 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 20 client.15870 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c0404abf70 0x55c0404ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 4 client.15870 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0403ffda0 con 0x55c0404abf70 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800943f640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c0404abf70 0x55c0404ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f7fecff9640 10 client.15870 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.116+0000 7f800943f640 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c0404abf70 0x55c0404ae3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c12359c4eb3a7b8 server_cookie=732ced9e489689b4 in_seq=0 out_seq=0 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.117+0000 7f7fecff9640 1 -- 172.21.15.120:0/3459891625 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0403ffda0 con 0x55c0404abf70 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.117+0000 7f7fecff9640 10 client.15870 handle_command_reply: tid=0 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.117+0000 7f800da78740 1 client.15870 shutdown 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 msgr2=0x55c040490ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bdf0 tx=0x7f7ffc004b50 comp rx=0 tx=0).stop 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 shutdown_connections 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c0404abf70 0x55c0404ae3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c040490ff0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040490a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 --2- 172.21.15.120:0/3459891625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 >> 172.21.15.120:0/3459891625 conn(0x55c0404852b0 msgr2=0x55c04048ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f7fecff9640 0 client.15870 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 shutdown_connections 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 1 -- 172.21.15.120:0/3459891625 wait complete. 2024-09-10T22:55:02.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.118+0000 7f800da78740 20 client.15870 trim_cache size 0 max 16384 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 Processor -- start 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 -- start start 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80041aa030 con 0x55c040493d30 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f80041ac850 con 0x55c040487480 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800da78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f80041aa680 con 0x55c040487da0 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57848/0 (socket says 172.21.15.120:57848) 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35546/0 (socket says 172.21.15.120:35546) 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 -- 172.21.15.120:0/3771761981 learned_addr learned my addr 172.21.15.120:0/3771761981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f800943f640 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 -- 172.21.15.120:0/3771761981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 msgr2=0x55c040492770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 -- 172.21.15.120:0/3771761981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 msgr2=0x55c0404a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8009c40640 1 -- 172.21.15.120:0/3771761981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c040486990 con 0x55c040493d30 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.137+0000 7f8008c3e640 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.138+0000 7f8009c40640 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc095040 tx=0x7f7ffc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef616e180d8b527 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.138+0000 7f7feeffd640 1 -- 172.21.15.120:0/3771761981 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0cb070 con 0x55c040493d30 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.138+0000 7f7feeffd640 1 -- 172.21.15.120:0/3771761981 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc0ab380 con 0x55c040493d30 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.138+0000 7f7feeffd640 1 -- 172.21.15.120:0/3771761981 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0a78c0 con 0x55c040493d30 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- 172.21.15.120:0/3771761981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 msgr2=0x55c04048dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc095040 tx=0x7f7ffc00cf00 comp rx=0 tx=0).stop 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- 172.21.15.120:0/3771761981 shutdown_connections 2024-09-10T22:55:02.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040493d30 0x55c04048dc60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040487da0 0x55c040492770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 --2- 172.21.15.120:0/3771761981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- 172.21.15.120:0/3771761981 >> 172.21.15.120:0/3771761981 conn(0x55c0404852b0 msgr2=0x55c040492060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- 172.21.15.120:0/3771761981 shutdown_connections 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- 172.21.15.120:0/3771761981 wait complete. 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 -- start start 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.139+0000 7f800da78740 1 Processor -- start 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 0x55c04048cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80041aad10 con 0x55c040487da0 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f80041ac180 con 0x55c040487480 2024-09-10T22:55:02.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f80041ab3e0 con 0x55c040493d30 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800da78740 10 client.0 _ll_register_callbacks cb 0x7f800000b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8008c3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57854/0 (socket says 172.21.15.120:57854) 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8008c3e640 1 -- 172.21.15.120:0/1255898378 learned_addr learned my addr 172.21.15.120:0/1255898378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8009c40640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 0x55c04048cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 -- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 msgr2=0x55c04048cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 0x55c04048cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 -- 172.21.15.120:0/1255898378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 msgr2=0x55c0404a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 -- 172.21.15.120:0/1255898378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c04048d210 con 0x55c040487480 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8009c40640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 0x55c04048cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:02.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f800943f640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a4530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8000005890 tx=0x7f8000005ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8756d30b8d1a3a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.140+0000 7f7fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000004020 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f80000041c0 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ffc0ab380 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8000036070 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 10 client.25343 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c04048e4e0 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc0ab380 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f80000117f0 con 0x55c040487480 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 10 client.25343 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 20 client.25343 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:02.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c040519eb0 0x55c04051c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 4 client.25343 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.141+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0403fb850 con 0x55c040519eb0 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.142+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c040519eb0 0x55c04051c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.142+0000 7f7fecff9640 10 client.25343 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.142+0000 7f8008c3e640 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c040519eb0 0x55c04051c2a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd1ae71a0e32f5b server_cookie=a5db9170cce75356 in_seq=0 out_seq=0 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.143+0000 7f7fecff9640 1 -- 172.21.15.120:0/1255898378 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x55c0403fb850 con 0x55c040519eb0 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.143+0000 7f7fecff9640 10 client.25343 handle_command_reply: tid=0 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.143+0000 7f800da78740 1 client.25343 shutdown 2024-09-10T22:55:02.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 msgr2=0x55c0404a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a4530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8000005890 tx=0x7f8000005ae0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 shutdown_connections 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c040519eb0 0x55c04051c2a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c040493d30 0x55c04048cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c040487da0 0x55c0404a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f7fecff9640 0 client.25343 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 --2- 172.21.15.120:0/1255898378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c040487480 0x55c0404a4530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 >> 172.21.15.120:0/1255898378 conn(0x55c0404852b0 msgr2=0x55c04048eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 shutdown_connections 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 1 -- 172.21.15.120:0/1255898378 wait complete. 2024-09-10T22:55:02.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800da78740 20 client.25343 trim_cache size 0 max 16384 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.159 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "5b13ee9d-06c7-493a-b472-734d0fb7db89", 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.160 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:55:02.162 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:02.163 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:02.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:02.164 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:02.164 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:55:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 msgr2=0x7f7ff00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 0x7f7ff00b77a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4031a90 tx=0x7f7ff4005d90 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 msgr2=0x7f80041023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.144+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00be20 tx=0x7f7ffc00bf10 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 shutdown_connections 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7ff00b52e0 0x7f7ff00b77a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800410d100 0x7f80041023d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80041045c0 0x7f8004101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 --2- 172.21.15.120:0/1544046519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8004103ca0 0x7f8004101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 >> 172.21.15.120:0/1544046519 conn(0x7f80040fd700 msgr2=0x7f800410f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 shutdown_connections 2024-09-10T22:55:02.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:02.145+0000 7f800b6a3640 1 -- 172.21.15.120:0/1544046519 wait complete. 2024-09-10T22:55:02.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 380475 17.67 MB/sec warmup 117 sec latency 37.060 ms 2024-09-10T22:55:02.912 INFO:tasks.cephfs.filesystem:scrub status for tag:5b13ee9d-06c7-493a-b472-734d0fb7db89 - {'path': '/', 'tag': '5b13ee9d-06c7-493a-b472-734d0fb7db89', 'options': 'recursive,force'} 2024-09-10T22:55:03.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 383648 17.68 MB/sec warmup 118 sec latency 20.268 ms 2024-09-10T22:55:03.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:03 smithi120 ceph-mon[31247]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-09-10T22:55:03.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:03 smithi153 ceph-mon[34951]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-09-10T22:55:03.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:03 smithi176 ceph-mon[34982]: pgmap v423: 97 pgs: 97 active+clean; 49 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-09-10T22:55:04.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 386914 17.69 MB/sec warmup 119 sec latency 38.780 ms 2024-09-10T22:55:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- 172.21.15.120:0/2957996454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 msgr2=0x7f2de8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 --2- 172.21.15.120:0/2957996454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8101c60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009930 tx=0x7f2dd802a420 comp rx=0 tx=0).stop 2024-09-10T22:55:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- 172.21.15.120:0/2957996454 shutdown_connections 2024-09-10T22:55:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 --2- 172.21.15.120:0/2957996454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de8103370 0x7f2de8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 --2- 172.21.15.120:0/2957996454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de81029b0 0x7f2de8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 --2- 172.21.15.120:0/2957996454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8101c60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- 172.21.15.120:0/2957996454 >> 172.21.15.120:0/2957996454 conn(0x7f2de80fd140 msgr2=0x7f2de80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- 172.21.15.120:0/2957996454 shutdown_connections 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- 172.21.15.120:0/2957996454 wait complete. 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 Processor -- start 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.657+0000 7f2dee73a640 1 -- start start 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de81029b0 0x7f2de8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de8103370 0x7f2de819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810d4b0 con 0x7f2de8101860 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810d330 con 0x7f2de81029b0 2024-09-10T22:55:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2dee73a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810d630 con 0x7f2de8103370 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45110/0 (socket says 172.21.15.120:45110) 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 -- 172.21.15.120:0/879651870 learned_addr learned my addr 172.21.15.120:0/879651870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de77fe640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de81029b0 0x7f2de8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2deccd7640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de8103370 0x7f2de819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 -- 172.21.15.120:0/879651870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de8103370 msgr2=0x7f2de819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de8103370 0x7f2de819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 -- 172.21.15.120:0/879651870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de81029b0 msgr2=0x7f2de8199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de81029b0 0x7f2de8199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 -- 172.21.15.120:0/879651870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de819a8f0 con 0x7f2de8101860 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.658+0000 7f2de7fff640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009a50 tx=0x7f2dd8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.659+0000 7f2de57fa640 1 -- 172.21.15.120:0/879651870 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd803f030 con 0x7f2de8101860 2024-09-10T22:55:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.659+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de819ab80 con 0x7f2de8101860 2024-09-10T22:55:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.659+0000 7f2de57fa640 1 -- 172.21.15.120:0/879651870 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80044b0 con 0x7f2de8101860 2024-09-10T22:55:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.659+0000 7f2de57fa640 1 -- 172.21.15.120:0/879651870 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8035070 con 0x7f2de8101860 2024-09-10T22:55:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.659+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de81a12b0 con 0x7f2de8101860 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2de57fa640 1 -- 172.21.15.120:0/879651870 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2dd803bbb0 con 0x7f2de8101860 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 Processor -- start 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 -- start start 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 0x55906a321580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a324090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55906a28e6f0 con 0x55906a31f190 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55906a27db20 con 0x55906a31c860 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2df15e2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55906a281930 con 0x55906a321bd0 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 0x55906a321580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.661+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55424/0 (socket says 172.21.15.120:55424) 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 -- 172.21.15.120:0/3191577098 learned_addr learned my addr 172.21.15.120:0/3191577098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2de77fe640 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a324090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 -- 172.21.15.120:0/3191577098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 msgr2=0x55906a324090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a324090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 -- 172.21.15.120:0/3191577098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 msgr2=0x55906a321580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 0x55906a321580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 -- 172.21.15.120:0/3191577098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906a243530 con 0x55906a31c860 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2de57fa640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 0x7f2dd00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2de7fff640 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 0x55906a321580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2de7fff640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 0x7f2dd00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009f40 tx=0x7f2de0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b3d1694a25ea19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2de57fa640 1 -- 172.21.15.120:0/879651870 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f2dd8106530 con 0x7f2de8101860 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2dce7fc640 1 -- 172.21.15.120:0/3191577098 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0016070 con 0x55906a31c860 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2dce7fc640 1 -- 172.21.15.120:0/3191577098 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de0004d20 con 0x55906a31c860 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.662+0000 7f2dce7fc640 1 -- 172.21.15.120:0/3191577098 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0012850 con 0x55906a31c860 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- 172.21.15.120:0/3191577098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 msgr2=0x55906a31ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2de7fff640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 0x7f2dd00b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80ccad0 tx=0x7f2dd8109f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009f40 tx=0x7f2de0010800 comp rx=0 tx=0).stop 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- 172.21.15.120:0/3191577098 shutdown_connections 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a324090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31f190 0x55906a321580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3191577098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31c860 0x55906a31ec50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- 172.21.15.120:0/3191577098 >> 172.21.15.120:0/3191577098 conn(0x55906a3179b0 msgr2=0x55906a318740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- 172.21.15.120:0/3191577098 shutdown_connections 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- 172.21.15.120:0/3191577098 wait complete. 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.663+0000 7f2df15e2740 1 -- start start 2024-09-10T22:55:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 Processor -- start 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 0x55906a326570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a321bd0 0x55906a327140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55906a1b5ab0 con 0x55906a31c860 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55906a223730 con 0x55906a321bd0 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55906a324bb0 con 0x55906a31f190 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2df15e2740 10 client.0 _ll_register_callbacks cb 0x55906a2776b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45118/0 (socket says 172.21.15.120:45118) 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2deccd7640 1 -- 172.21.15.120:0/381743507 learned_addr learned my addr 172.21.15.120:0/381743507 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 0x55906a326570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2deccd7640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a321bd0 0x55906a327140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 -- 172.21.15.120:0/381743507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a321bd0 msgr2=0x55906a327140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a321bd0 0x55906a327140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 -- 172.21.15.120:0/381743507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 msgr2=0x55906a325ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2de77fe640 1 -- 172.21.15.120:0/381743507 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559069fdf790 con 0x55906a31f190 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.664+0000 7f2deccd7640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2de77fe640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 0x55906a326570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc005fd0 tx=0x7f2ddc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a10354ff3f80792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004020 con 0x55906a31f190 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f2ddc0041c0 con 0x55906a31f190 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0004d20 con 0x55906a31f190 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc020020 con 0x55906a31f190 2024-09-10T22:55:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 10 client.25360 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55906a058620 con 0x55906a31f190 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de0004d20 con 0x55906a31f190 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f2ddc003820 con 0x55906a31f190 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 10 client.25360 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 20 client.25360 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a331820 0x55906a333c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 4 client.25360 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.665+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55906a290640 con 0x55906a331820 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.666+0000 7f2deccd7640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a331820 0x55906a333c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.666+0000 7f2dc77fe640 10 client.25360 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.666+0000 7f2deccd7640 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a331820 0x55906a333c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60ef55ac7e44bd6 server_cookie=4472beb880c4d7a4 in_seq=0 out_seq=0 2024-09-10T22:55:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.667+0000 7f2dc77fe640 1 -- 172.21.15.120:0/381743507 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55906a290640 con 0x55906a331820 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.667+0000 7f2dc77fe640 10 client.25360 handle_command_reply: tid=0 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.667+0000 7f2df15e2740 1 client.25360 shutdown 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.667+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 msgr2=0x55906a326570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.667+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 0x55906a326570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc005fd0 tx=0x7f2ddc0118d0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 shutdown_connections 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a331820 0x55906a333c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a321bd0 0x55906a327140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a31f190 0x55906a326570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2dc77fe640 0 client.25360 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 --2- 172.21.15.120:0/381743507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a325ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 >> 172.21.15.120:0/381743507 conn(0x55906a3179b0 msgr2=0x55906a316bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 shutdown_connections 2024-09-10T22:55:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 1 -- 172.21.15.120:0/381743507 wait complete. 2024-09-10T22:55:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.668+0000 7f2df15e2740 20 client.25360 trim_cache size 0 max 16384 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 Processor -- start 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 -- start start 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a321fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a3224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81a7e40 con 0x55906a31c860 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81aa6c0 con 0x55906a31f190 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2df15e2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81a84b0 con 0x55906a321bd0 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45134/0 (socket says 172.21.15.120:45134) 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.683+0000 7f2deccd7640 1 -- 172.21.15.120:0/3918234921 learned_addr learned my addr 172.21.15.120:0/3918234921 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2de77fe640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a321fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a3224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 -- 172.21.15.120:0/3918234921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 msgr2=0x55906a3224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a3224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 -- 172.21.15.120:0/3918234921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 msgr2=0x55906a321fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a321fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 -- 172.21.15.120:0/3918234921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906a3342d0 con 0x55906a31c860 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2de77fe640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a321fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2deccd7640 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc75355114398dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2dcdffb640 1 -- 172.21.15.120:0/3918234921 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0016070 con 0x55906a31c860 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2dcdffb640 1 -- 172.21.15.120:0/3918234921 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de0003e20 con 0x55906a31c860 2024-09-10T22:55:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.684+0000 7f2dcdffb640 1 -- 172.21.15.120:0/3918234921 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004050 con 0x55906a31c860 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- 172.21.15.120:0/3918234921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 msgr2=0x55906a327a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de0009870 comp rx=0 tx=0).stop 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- 172.21.15.120:0/3918234921 shutdown_connections 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a3224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a321fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- 172.21.15.120:0/3918234921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a327a10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- 172.21.15.120:0/3918234921 >> 172.21.15.120:0/3918234921 conn(0x55906a3179b0 msgr2=0x55906a31bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- 172.21.15.120:0/3918234921 shutdown_connections 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- 172.21.15.120:0/3918234921 wait complete. 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 -- start start 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 Processor -- start 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a32d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.685+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a32dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a32c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81a8b80 con 0x55906a31c860 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81a9ff0 con 0x55906a31f190 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81a9250 con 0x55906a321bd0 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 10 client.0 _ll_register_callbacks cb 0x7f2ddc00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a32d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a32dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2deccd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a32d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45136/0 (socket says 172.21.15.120:45136) 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2deccd7640 1 -- 172.21.15.120:0/1254035396 learned_addr learned my addr 172.21.15.120:0/1254035396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2deccd7640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a32c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 -- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 msgr2=0x55906a32c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a32c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 -- 172.21.15.120:0/1254035396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 msgr2=0x55906a32d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a32d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 -- 172.21.15.120:0/1254035396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906a322a30 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2de77fe640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a32dbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001eb0 tx=0x7f2ddc0040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c6d991eed4a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2dc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc000e60 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0003e20 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f2ddc017070 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc00b480 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.686+0000 7f2df15e2740 10 client.25361 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55906a32b7a0 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de0003e20 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f2ddc0107c0 con 0x55906a31f190 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 10 client.25361 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 20 client.25361 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a39ebf0 0x55906a3a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 4 client.25361 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55906a28c0f0 con 0x55906a39ebf0 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2deccd7640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a39ebf0 0x55906a3a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2dc77fe640 10 client.25361 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.687+0000 7f2deccd7640 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a39ebf0 0x55906a3a1010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cacb6aecb6874e5e server_cookie=5872c19aca3fa921 in_seq=0 out_seq=0 2024-09-10T22:55:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.831+0000 7f2dc77fe640 1 -- 172.21.15.120:0/1254035396 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55906a28c0f0 con 0x55906a39ebf0 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.831+0000 7f2dc77fe640 10 client.25361 handle_command_reply: tid=0 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.831+0000 7f2df15e2740 1 client.25361 shutdown 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.831+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 msgr2=0x55906a32dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a32dbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc001eb0 tx=0x7f2ddc0040e0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 shutdown_connections 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55906a39ebf0 0x55906a3a1010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55906a321bd0 0x55906a32c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2dc77fe640 0 client.25361 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55906a31f190 0x55906a32dbc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 --2- 172.21.15.120:0/1254035396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55906a31c860 0x55906a32d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 >> 172.21.15.120:0/1254035396 conn(0x55906a3179b0 msgr2=0x55906a31bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 shutdown_connections 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 1 -- 172.21.15.120:0/1254035396 wait complete. 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.832+0000 7f2df15e2740 20 client.25361 trim_cache size 0 max 16384 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:55:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:55:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 msgr2=0x7f2dd00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 0x7f2dd00b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80ccad0 tx=0x7f2dd8109f90 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 msgr2=0x7f2de8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009a50 tx=0x7f2dd8031a50 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 shutdown_connections 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2dd00b52e0 0x7f2dd00b77a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2de8103370 0x7f2de819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2de81029b0 0x7f2de8199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 --2- 172.21.15.120:0/879651870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2de8101860 0x7f2de8198db0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 >> 172.21.15.120:0/879651870 conn(0x7f2de80fd140 msgr2=0x7f2de80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 shutdown_connections 2024-09-10T22:55:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:04.833+0000 7f2dee73a640 1 -- 172.21.15.120:0/879651870 wait complete. 2024-09-10T22:55:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:05 smithi120 ceph-mon[31247]: pgmap v424: 97 pgs: 97 active+clean; 50 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-10T22:55:05.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:05 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:55:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:05 smithi153 ceph-mon[34951]: pgmap v424: 97 pgs: 97 active+clean; 50 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-10T22:55:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:05 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:55:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:05 smithi176 ceph-mon[34982]: pgmap v424: 97 pgs: 97 active+clean; 50 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-10T22:55:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:05 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:55:06.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 392660 15.26 MB/sec execute 1 sec latency 97.523 ms 2024-09-10T22:55:07.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 395459 16.25 MB/sec execute 2 sec latency 39.128 ms 2024-09-10T22:55:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:07 smithi120 ceph-mon[31247]: pgmap v425: 97 pgs: 97 active+clean; 40 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2024-09-10T22:55:07.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:07 smithi153 ceph-mon[34951]: pgmap v425: 97 pgs: 97 active+clean; 40 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2024-09-10T22:55:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:07 smithi176 ceph-mon[34982]: pgmap v425: 97 pgs: 97 active+clean; 40 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 573 op/s 2024-09-10T22:55:08.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 398496 16.14 MB/sec execute 3 sec latency 20.119 ms 2024-09-10T22:55:09.617 INFO:tasks.workunit.client.0.smithi120.stdout: 1 401312 16.36 MB/sec execute 4 sec latency 56.155 ms 2024-09-10T22:55:09.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:09 smithi120 ceph-mon[31247]: pgmap v426: 97 pgs: 97 active+clean; 44 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-09-10T22:55:09.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:09 smithi153 ceph-mon[34951]: pgmap v426: 97 pgs: 97 active+clean; 44 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-09-10T22:55:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:09 smithi176 ceph-mon[34982]: pgmap v426: 97 pgs: 97 active+clean; 44 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 450 op/s 2024-09-10T22:55:10.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 404704 16.85 MB/sec execute 5 sec latency 22.219 ms 2024-09-10T22:55:11.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 408205 17.23 MB/sec execute 6 sec latency 14.881 ms 2024-09-10T22:55:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:11 smithi120 ceph-mon[31247]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-10T22:55:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:11 smithi153 ceph-mon[34951]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-10T22:55:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:11 smithi176 ceph-mon[34982]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-10T22:55:12.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 411631 17.36 MB/sec execute 7 sec latency 20.953 ms 2024-09-10T22:55:13.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 415123 17.56 MB/sec execute 8 sec latency 11.660 ms 2024-09-10T22:55:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:13 smithi120 ceph-mon[31247]: pgmap v428: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T22:55:13.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:13 smithi153 ceph-mon[34951]: pgmap v428: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T22:55:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:13 smithi176 ceph-mon[34982]: pgmap v428: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T22:55:14.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 417599 16.66 MB/sec execute 9 sec latency 57.811 ms 2024-09-10T22:55:15.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 420562 16.66 MB/sec execute 10 sec latency 21.192 ms 2024-09-10T22:55:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:15 smithi120 ceph-mon[31247]: pgmap v429: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T22:55:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:15 smithi153 ceph-mon[34951]: pgmap v429: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T22:55:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:15 smithi176 ceph-mon[34982]: pgmap v429: 97 pgs: 97 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T22:55:16.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 423824 16.86 MB/sec execute 11 sec latency 16.925 ms 2024-09-10T22:55:17.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 427203 17.04 MB/sec execute 12 sec latency 12.469 ms 2024-09-10T22:55:17.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:17 smithi120 ceph-mon[31247]: pgmap v430: 97 pgs: 97 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-09-10T22:55:17.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:17 smithi153 ceph-mon[34951]: pgmap v430: 97 pgs: 97 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-09-10T22:55:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:17 smithi176 ceph-mon[34982]: pgmap v430: 97 pgs: 97 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- 172.21.15.120:0/652819099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 msgr2=0x7f12f8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 --2- 172.21.15.120:0/652819099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f8102e30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009930 tx=0x7f12e802a420 comp rx=0 tx=0).stop 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- 172.21.15.120:0/652819099 shutdown_connections 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 --2- 172.21.15.120:0/652819099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8103370 0x7f12f8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 --2- 172.21.15.120:0/652819099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f8102e30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 --2- 172.21.15.120:0/652819099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8101860 0x7f12f8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- 172.21.15.120:0/652819099 >> 172.21.15.120:0/652819099 conn(0x7f12f80fd140 msgr2=0x7f12f80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- 172.21.15.120:0/652819099 shutdown_connections 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- 172.21.15.120:0/652819099 wait complete. 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 Processor -- start 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.925+0000 7f130062a640 1 -- start start 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8101860 0x7f12f8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8103370 0x7f12f8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f810d4b0 con 0x7f12f81029b0 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f810d330 con 0x7f12f8101860 2024-09-10T22:55:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12f810d630 con 0x7f12f8103370 2024-09-10T22:55:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fe3c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8101860 0x7f12f8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33532/0 (socket says 172.21.15.120:33532) 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 -- 172.21.15.120:0/3939598267 learned_addr learned my addr 172.21.15.120:0/3939598267 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12febc7640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8103370 0x7f12f8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 -- 172.21.15.120:0/3939598267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8103370 msgr2=0x7f12f8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8103370 0x7f12f8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 -- 172.21.15.120:0/3939598267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8101860 msgr2=0x7f12f8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8101860 0x7f12f8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 -- 172.21.15.120:0/3939598267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f81997b0 con 0x7f12f81029b0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009a50 tx=0x7f12e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12ef7fe640 1 -- 172.21.15.120:0/3939598267 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e803f030 con 0x7f12f81029b0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12ef7fe640 1 -- 172.21.15.120:0/3939598267 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e8031910 con 0x7f12f81029b0 2024-09-10T22:55:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f8199a40 con 0x7f12f81029b0 2024-09-10T22:55:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f12ef7fe640 1 -- 172.21.15.120:0/3939598267 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8035070 con 0x7f12f81029b0 2024-09-10T22:55:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.926+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f8199d50 con 0x7f12f81029b0 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.929+0000 7f12ef7fe640 1 -- 172.21.15.120:0/3939598267 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f12e803ba60 con 0x7f12f81029b0 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.929+0000 7f13033fa740 1 Processor -- start 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.929+0000 7f13033fa740 1 -- start start 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce964010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ce8d46f0 con 0x5589ce963b60 2024-09-10T22:55:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5589ce8c3b20 con 0x5589ce962860 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f13033fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5589ce8c7930 con 0x5589ce9631a0 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce964010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fe3c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12febc7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fe3c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48152/0 (socket says 172.21.15.120:48152) 2024-09-10T22:55:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12febc7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33782/0 (socket says 172.21.15.120:33782) 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fe3c6640 1 -- 172.21.15.120:0/1753840122 learned_addr learned my addr 172.21.15.120:0/1753840122 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12ef7fe640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 0x7f12e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 -- 172.21.15.120:0/1753840122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 msgr2=0x5589ce963620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 -- 172.21.15.120:0/1753840122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 msgr2=0x5589ce962c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 -- 172.21.15.120:0/1753840122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ce889530 con 0x5589ce963b60 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12febc7640 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12febc7640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 0x7f12e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12ef7fe640 1 -- 172.21.15.120:0/3939598267 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f12e8106400 con 0x7f12f81029b0 2024-09-10T22:55:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.930+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce964010 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12e8103ea0 tx=0x7f12e80913b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edb99d0ab3285d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.931+0000 7f12ecbfe640 1 -- 172.21.15.120:0/1753840122 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e80cca10 con 0x5589ce963b60 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.931+0000 7f12febc7640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 0x7f12e40b77a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12f81989f0 tx=0x7f12f0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.931+0000 7f12ecbfe640 1 -- 172.21.15.120:0/1753840122 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80cc750 con 0x5589ce963b60 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.931+0000 7f12ecbfe640 1 -- 172.21.15.120:0/1753840122 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8096d80 con 0x5589ce963b60 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- 172.21.15.120:0/1753840122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 msgr2=0x5589ce964010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce964010 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12e8103ea0 tx=0x7f12e80913b0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- 172.21.15.120:0/1753840122 shutdown_connections 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce964010 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce963620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- 172.21.15.120:0/1753840122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce962c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- 172.21.15.120:0/1753840122 >> 172.21.15.120:0/1753840122 conn(0x5589ce95d9b0 msgr2=0x5589ce95e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- 172.21.15.120:0/1753840122 shutdown_connections 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- 172.21.15.120:0/1753840122 wait complete. 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- start start 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 Processor -- start 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce96ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce963b60 0x5589ce971220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5589ce7fbab0 con 0x5589ce9631a0 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5589ce869730 con 0x5589ce962860 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5589ce8d46f0 con 0x5589ce963b60 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.932+0000 7f13033fa740 10 client.0 _ll_register_callbacks cb 0x5589ce8bd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33544/0 (socket says 172.21.15.120:33544) 2024-09-10T22:55:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2580889895 learned_addr learned my addr 172.21.15.120:0/2580889895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce96ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12febc7640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce963b60 0x5589ce971220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce963b60 msgr2=0x5589ce971220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce963b60 0x5589ce971220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2580889895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 msgr2=0x5589ce96ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce96ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2580889895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ce625790 con 0x5589ce9631a0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12e809a040 tx=0x7f12e8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ce65bc95ba2d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8049040 con 0x5589ce9631a0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f12e8091130 con 0x5589ce9631a0 2024-09-10T22:55:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e80cc750 con 0x5589ce9631a0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f12e8090f90 con 0x5589ce9631a0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f13033fa740 10 client.15906 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5589ce69e620 con 0x5589ce9631a0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.933+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80cc750 con 0x5589ce9631a0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f12e8103f50 con 0x5589ce9631a0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 10 client.15906 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 20 client.15906 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce97b720 0x5589ce97db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 4 client.15906 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ce8d6640 con 0x5589ce97b720 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce97b720 0x5589ce97db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce97b720 0x5589ce97db10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f36d75d960fbcd8 server_cookie=24d9de50a3be1dde in_seq=0 out_seq=0 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.934+0000 7f12e1ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f12e1ffb640 1 -- 172.21.15.120:0/2580889895 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589ce8d6640 con 0x5589ce97b720 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f12e1ffb640 10 client.15906 handle_command_reply: tid=0 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 client.15906 shutdown 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 msgr2=0x5589ce970650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12e809a040 tx=0x7f12e8005e60 comp rx=0 tx=0).stop 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 shutdown_connections 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce97b720 0x5589ce97db10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce963b60 0x5589ce971220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce970650 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f12e1ffb640 0 client.15906 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 --2- 172.21.15.120:0/2580889895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce96ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 >> 172.21.15.120:0/2580889895 conn(0x5589ce95d9b0 msgr2=0x5589ce95cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 shutdown_connections 2024-09-10T22:55:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 1 -- 172.21.15.120:0/2580889895 wait complete. 2024-09-10T22:55:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.935+0000 7f13033fa740 20 client.15906 trim_cache size 0 max 16384 2024-09-10T22:55:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 Processor -- start 2024-09-10T22:55:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 -- start start 2024-09-10T22:55:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce962860 0x5589ce96f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce963b60 0x5589ce96e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a5c20 con 0x5589ce9631a0 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a84a0 con 0x5589ce963b60 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f13033fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a6290 con 0x5589ce962860 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f12fdbc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33558/0 (socket says 172.21.15.120:33558) 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2898505789 learned_addr learned my addr 172.21.15.120:0/2898505789 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.950+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce962860 0x5589ce96f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12febc7640 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce963b60 0x5589ce96e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2898505789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce962860 msgr2=0x5589ce96f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce962860 0x5589ce96f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2898505789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce963b60 msgr2=0x5589ce96e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce963b60 0x5589ce96e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 -- 172.21.15.120:0/2898505789 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ce96df60 con 0x5589ce9631a0 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e80c9150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d684a10be486d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12e3fff640 1 -- 172.21.15.120:0/2898505789 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8049070 con 0x5589ce9631a0 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12e3fff640 1 -- 172.21.15.120:0/2898505789 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80cc750 con 0x5589ce9631a0 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.951+0000 7f12e3fff640 1 -- 172.21.15.120:0/2898505789 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e80cc3b0 con 0x5589ce9631a0 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- 172.21.15.120:0/2898505789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 msgr2=0x5589ce963fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e80c9150 comp rx=0 tx=0).stop 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- 172.21.15.120:0/2898505789 shutdown_connections 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce963b60 0x5589ce96e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce9631a0 0x5589ce963fd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 --2- 172.21.15.120:0/2898505789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce962860 0x5589ce96f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- 172.21.15.120:0/2898505789 >> 172.21.15.120:0/2898505789 conn(0x5589ce95d9b0 msgr2=0x5589ce96f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- 172.21.15.120:0/2898505789 shutdown_connections 2024-09-10T22:55:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- 172.21.15.120:0/2898505789 wait complete. 2024-09-10T22:55:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.952+0000 7f13033fa740 1 -- start start 2024-09-10T22:55:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 Processor -- start 2024-09-10T22:55:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce977c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a6960 con 0x5589ce963b60 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a7dd0 con 0x5589ce962860 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12f81a7030 con 0x5589ce9631a0 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f13033fa740 10 client.0 _ll_register_callbacks cb 0x7f12f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12febc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33798/0 (socket says 172.21.15.120:33798) 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 -- 172.21.15.120:0/1675781729 learned_addr learned my addr 172.21.15.120:0/1675781729 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12febc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33572/0 (socket says 172.21.15.120:33572) 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce977c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 -- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 msgr2=0x5589ce977c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce977c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 -- 172.21.15.120:0/1675781729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 msgr2=0x5589ce9733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 -- 172.21.15.120:0/1675781729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ce973bd0 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12febc7640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12fe3c6640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000c00 tx=0x7f12f401fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfcc5d8c84f503b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.953+0000 7f12e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4028020 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f12f4005220 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e80cc750 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f12f4005420 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 10 client.25379 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5589ce9e85e0 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80cc750 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f12f4023070 con 0x5589ce962860 2024-09-10T22:55:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 10 client.25379 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 10 client.25379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 20 client.25379 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce9e8ee0 0x5589ce9eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 4 client.25379 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.954+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589ce8d20f0 con 0x5589ce9e8ee0 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.955+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce9e8ee0 0x5589ce9eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.955+0000 7f12fdbc5640 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce9e8ee0 0x5589ce9eb300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c10749f48711c1c server_cookie=3626fd109ac5531d in_seq=0 out_seq=0 2024-09-10T22:55:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:17.955+0000 7f12e1ffb640 10 client.25379 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f12e1ffb640 1 -- 172.21.15.120:0/1675781729 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589ce8d20f0 con 0x5589ce9e8ee0 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f12e1ffb640 10 client.25379 handle_command_reply: tid=0 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 client.25379 shutdown 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 msgr2=0x5589ce9775f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000c00 tx=0x7f12f401fa30 comp rx=0 tx=0).stop 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 shutdown_connections 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5589ce9e8ee0 0x5589ce9eb300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5589ce963b60 0x5589ce9733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f12e1ffb640 0 client.25379 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5589ce9631a0 0x5589ce977c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 --2- 172.21.15.120:0/1675781729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5589ce962860 0x5589ce9775f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 >> 172.21.15.120:0/1675781729 conn(0x5589ce95d9b0 msgr2=0x5589ce96e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.108+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 shutdown_connections 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f13033fa740 1 -- 172.21.15.120:0/1675781729 wait complete. 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f13033fa740 20 client.25379 trim_cache size 0 max 16384 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:55:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:55:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 msgr2=0x7f12e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 0x7f12e40b77a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12f81989f0 tx=0x7f12f0004a40 comp rx=0 tx=0).stop 2024-09-10T22:55:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 msgr2=0x7f12f81982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009a50 tx=0x7f12e80042f0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 shutdown_connections 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f12e40b52e0 0x7f12e40b77a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12f8103370 0x7f12f8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12f81029b0 0x7f12f81982c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 --2- 172.21.15.120:0/3939598267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12f8101860 0x7f12f8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 >> 172.21.15.120:0/3939598267 conn(0x7f12f80fd140 msgr2=0x7f12f8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 shutdown_connections 2024-09-10T22:55:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:18.109+0000 7f130062a640 1 -- 172.21.15.120:0/3939598267 wait complete. 2024-09-10T22:55:18.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 430045 16.84 MB/sec execute 13 sec latency 73.561 ms 2024-09-10T22:55:19.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 433175 16.97 MB/sec execute 14 sec latency 77.437 ms 2024-09-10T22:55:19.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:19 smithi120 ceph-mon[31247]: pgmap v431: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-09-10T22:55:19.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:19 smithi153 ceph-mon[34951]: pgmap v431: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-09-10T22:55:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:19 smithi176 ceph-mon[34982]: pgmap v431: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 342 op/s 2024-09-10T22:55:20.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 436557 17.09 MB/sec execute 15 sec latency 19.708 ms 2024-09-10T22:55:21.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 440002 17.16 MB/sec execute 16 sec latency 13.239 ms 2024-09-10T22:55:21.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:21 smithi120 ceph-mon[31247]: pgmap v432: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T22:55:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:21 smithi153 ceph-mon[34951]: pgmap v432: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T22:55:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:21 smithi176 ceph-mon[34982]: pgmap v432: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T22:55:22.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 443511 17.27 MB/sec execute 17 sec latency 12.474 ms 2024-09-10T22:55:23.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 446955 17.35 MB/sec execute 18 sec latency 12.292 ms 2024-09-10T22:55:23.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:23 smithi120 ceph-mon[31247]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:55:23.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:23 smithi153 ceph-mon[34951]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:55:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:23 smithi176 ceph-mon[34982]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 498 op/s 2024-09-10T22:55:24.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 449554 16.95 MB/sec execute 19 sec latency 85.716 ms 2024-09-10T22:55:25.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 452841 17.00 MB/sec execute 20 sec latency 34.381 ms 2024-09-10T22:55:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:25 smithi120 ceph-mon[31247]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-10T22:55:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:25 smithi153 ceph-mon[34951]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-10T22:55:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:25 smithi176 ceph-mon[34982]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 415 op/s 2024-09-10T22:55:26.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 456239 17.05 MB/sec execute 21 sec latency 12.864 ms 2024-09-10T22:55:27.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 458709 17.48 MB/sec execute 22 sec latency 37.965 ms 2024-09-10T22:55:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:27 smithi120 ceph-mon[31247]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-10T22:55:27.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:27 smithi153 ceph-mon[34951]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-10T22:55:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:27 smithi176 ceph-mon[34982]: pgmap v435: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-10T22:55:28.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 462296 17.90 MB/sec execute 23 sec latency 21.834 ms 2024-09-10T22:55:29.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 465648 17.94 MB/sec execute 24 sec latency 66.064 ms 2024-09-10T22:55:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:29 smithi120 ceph-mon[31247]: pgmap v436: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-09-10T22:55:29.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:29 smithi153 ceph-mon[34951]: pgmap v436: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-09-10T22:55:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:29 smithi176 ceph-mon[34982]: pgmap v436: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-09-10T22:55:30.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 468794 17.81 MB/sec execute 25 sec latency 21.538 ms 2024-09-10T22:55:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 -- 172.21.15.120:0/3020381335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78103ad0 msgr2=0x7f8a78103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 --2- 172.21.15.120:0/3020381335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78103ad0 0x7f8a78103f50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009930 tx=0x7f8a6802a420 comp rx=0 tx=0).stop 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 -- 172.21.15.120:0/3020381335 shutdown_connections 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 --2- 172.21.15.120:0/3020381335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78104490 0x7f8a7810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 --2- 172.21.15.120:0/3020381335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78103ad0 0x7f8a78103f50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 --2- 172.21.15.120:0/3020381335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 -- 172.21.15.120:0/3020381335 >> 172.21.15.120:0/3020381335 conn(0x7f8a780fe240 msgr2=0x7f8a78100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 -- 172.21.15.120:0/3020381335 shutdown_connections 2024-09-10T22:55:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.201+0000 7f8a80423640 1 -- 172.21.15.120:0/3020381335 wait complete. 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 Processor -- start 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 -- start start 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78103ad0 0x7f8a78199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 0x7f8a7819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7810e5f0 con 0x7f8a78104490 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7810e470 con 0x7f8a78103ad0 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a80423640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7810e770 con 0x7f8a78102980 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35120/0 (socket says 172.21.15.120:35120) 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 0x7f8a7819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 -- 172.21.15.120:0/2976461241 learned_addr learned my addr 172.21.15.120:0/2976461241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78103ad0 0x7f8a78199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 -- 172.21.15.120:0/2976461241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78103ad0 msgr2=0x7f8a78199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78103ad0 0x7f8a78199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 -- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 msgr2=0x7f8a7819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 0x7f8a7819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e1bf640 1 -- 172.21.15.120:0/2976461241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7819a8f0 con 0x7f8a78102980 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.202+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 0x7f8a7819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9f0 tx=0x7f8a7400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2976461241 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400cd80 con 0x7f8a78102980 2024-09-10T22:55:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7819abe0 con 0x7f8a78102980 2024-09-10T22:55:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2976461241 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a740054d0 con 0x7f8a78102980 2024-09-10T22:55:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a781a12b0 con 0x7f8a78102980 2024-09-10T22:55:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.203+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2976461241 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74018630 con 0x7f8a78102980 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.205+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2976461241 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8a74004e70 con 0x7f8a78102980 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.205+0000 7f8a831f2740 1 Processor -- start 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 -- start start 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c082f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606c07996f0 con 0x5606c0827860 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606c0788b20 con 0x5606c082a190 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a831f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606c078c930 con 0x5606c082cbd0 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44166/0 (socket says 172.21.15.120:44166) 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1307481677 learned_addr learned my addr 172.21.15.120:0/1307481677 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c082f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1307481677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 msgr2=0x5606c082f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c082f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1307481677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 msgr2=0x5606c082c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1307481677 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c074e530 con 0x5606c0827860 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.206+0000 7f8a6f7fe640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 0x7f8a640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8a70010740 tx=0x7f8a70010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4f2c3b6686f518 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 0x7f8a640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a6cbfe640 1 -- 172.21.15.120:0/1307481677 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000ead0 con 0x5606c0827860 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a6cbfe640 1 -- 172.21.15.120:0/1307481677 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70010e50 con 0x5606c0827860 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a6cbfe640 1 -- 172.21.15.120:0/1307481677 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7001c410 con 0x5606c0827860 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a6f7fe640 1 -- 172.21.15.120:0/2976461241 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8a740e60c0 con 0x7f8a78102980 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 -- 172.21.15.120:0/1307481677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 msgr2=0x5606c0829c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a7e1bf640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 0x7f8a640b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a740a8e60 tx=0x7f8a74004a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8a70010740 tx=0x7f8a70010c00 comp rx=0 tx=0).stop 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 -- 172.21.15.120:0/1307481677 shutdown_connections 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c082f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.207+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1307481677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c0829c50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- 172.21.15.120:0/1307481677 >> 172.21.15.120:0/1307481677 conn(0x5606c08229b0 msgr2=0x5606c0823740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- 172.21.15.120:0/1307481677 shutdown_connections 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- 172.21.15.120:0/1307481677 wait complete. 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- start start 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 Processor -- start 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0830ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c0832130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606c06c0ab0 con 0x5606c082a190 2024-09-10T22:55:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606c072e730 con 0x5606c082cbd0 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606c082fbb0 con 0x5606c0827860 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a831f2740 10 client.0 _ll_register_callbacks cb 0x5606c07826b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a7d9be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a7d9be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44176/0 (socket says 172.21.15.120:44176) 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a7d9be640 1 -- 172.21.15.120:0/629104243 learned_addr learned my addr 172.21.15.120:0/629104243 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0830ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.208+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c0832130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 -- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 msgr2=0x5606c0830ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0830ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 -- 172.21.15.120:0/629104243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 msgr2=0x5606c0832130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c0832130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 -- 172.21.15.120:0/629104243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c04ea790 con 0x5606c082a190 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c0832130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802a940 tx=0x7f8a68031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877366bab4ef77e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68035040 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8a68004240 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a70010e50 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a6804d070 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 10 client.15924 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5606c0563620 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70010e50 con 0x5606c082a190 2024-09-10T22:55:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8a68044020 con 0x5606c082a190 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 10 client.15924 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 20 client.15924 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c083c7c0 0x5606c083ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 4 client.15924 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.209+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c079b640 con 0x5606c083c7c0 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.210+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c083c7c0 0x5606c083ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.210+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c083c7c0 0x5606c083ebb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322965e9af217af server_cookie=ae28e990475af375 in_seq=0 out_seq=0 2024-09-10T22:55:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.210+0000 7f8a61ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a61ffb640 1 -- 172.21.15.120:0/629104243 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606c079b640 con 0x5606c083c7c0 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a61ffb640 10 client.15924 handle_command_reply: tid=0 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 client.15924 shutdown 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 msgr2=0x5606c0831560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802a940 tx=0x7f8a68031e80 comp rx=0 tx=0).stop 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 shutdown_connections 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c083c7c0 0x5606c083ebb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c0832130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a61ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0831560 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 --2- 172.21.15.120:0/629104243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0830ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 >> 172.21.15.120:0/629104243 conn(0x5606c08229b0 msgr2=0x5606c0821bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 shutdown_connections 2024-09-10T22:55:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 1 -- 172.21.15.120:0/629104243 wait complete. 2024-09-10T22:55:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.211+0000 7f8a831f2740 20 client.15924 trim_cache size 0 max 16384 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 Processor -- start 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 -- start start 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c0830490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781a7e90 con 0x5606c0827860 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781aa710 con 0x5606c082a190 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a831f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781a8500 con 0x5606c082cbd0 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44182/0 (socket says 172.21.15.120:44182) 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1120663325 learned_addr learned my addr 172.21.15.120:0/1120663325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c0830490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1120663325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 msgr2=0x5606c0830490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c0830490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1120663325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 msgr2=0x5606c082ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 -- 172.21.15.120:0/1120663325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c083ef20 con 0x5606c0827860 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.226+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000f80 tx=0x7f8a70009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fc1c15354037c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a63fff640 1 -- 172.21.15.120:0/1120663325 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70010e50 con 0x5606c0827860 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a63fff640 1 -- 172.21.15.120:0/1120663325 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70021070 con 0x5606c0827860 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a63fff640 1 -- 172.21.15.120:0/1120663325 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000eba0 con 0x5606c0827860 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 -- 172.21.15.120:0/1120663325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 msgr2=0x5606c08315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000f80 tx=0x7f8a70009f60 comp rx=0 tx=0).stop 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 -- 172.21.15.120:0/1120663325 shutdown_connections 2024-09-10T22:55:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c082cbd0 0x5606c0830490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082a190 0x5606c082ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 --2- 172.21.15.120:0/1120663325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c0827860 0x5606c08315d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 -- 172.21.15.120:0/1120663325 >> 172.21.15.120:0/1120663325 conn(0x5606c08229b0 msgr2=0x5606c0830ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 -- 172.21.15.120:0/1120663325 shutdown_connections 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.227+0000 7f8a831f2740 1 -- 172.21.15.120:0/1120663325 wait complete. 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 -- start start 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 Processor -- start 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0838670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c08378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781a8bd0 con 0x5606c082a190 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781aa040 con 0x5606c082cbd0 2024-09-10T22:55:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781a92a0 con 0x5606c0827860 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a831f2740 10 client.0 _ll_register_callbacks cb 0x7f8a68009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44198/0 (socket says 172.21.15.120:44198) 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 -- 172.21.15.120:0/960190824 learned_addr learned my addr 172.21.15.120:0/960190824 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c08378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0838670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 -- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 msgr2=0x5606c0838670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0838670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 -- 172.21.15.120:0/960190824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 msgr2=0x5606c08378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c08378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 -- 172.21.15.120:0/960190824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c08324a0 con 0x5606c082a190 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c08378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a7d9be640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802a940 tx=0x7f8a680098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cbe76db8619da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.228+0000 7f8a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68056020 con 0x5606c082a190 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8a68003fe0 con 0x5606c082a190 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a70021070 con 0x5606c082a190 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a68035070 con 0x5606c082a190 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 10 client.15930 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5606c0836b30 con 0x5606c082a190 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70021070 con 0x5606c082a190 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8a68044040 con 0x5606c082a190 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 10 client.15930 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 20 client.15930 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c08a9ca0 0x5606c08ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 4 client.15930 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606c07970f0 con 0x5606c08a9ca0 2024-09-10T22:55:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.229+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c08a9ca0 0x5606c08ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.230+0000 7f8a61ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.230+0000 7f8a7e9c0640 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c08a9ca0 0x5606c08ac0c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f6fdb62939cebd server_cookie=5a02074d8e5427d3 in_seq=0 out_seq=0 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a61ffb640 1 -- 172.21.15.120:0/960190824 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606c07970f0 con 0x5606c08a9ca0 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a61ffb640 10 client.15930 handle_command_reply: tid=0 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 client.15930 shutdown 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 msgr2=0x5606c0837370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a6802a940 tx=0x7f8a680098a0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 shutdown_connections 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606c08a9ca0 0x5606c08ac0c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a61ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606c082cbd0 0x5606c08378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606c082a190 0x5606c0837370 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 --2- 172.21.15.120:0/960190824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606c0827860 0x5606c0838670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 >> 172.21.15.120:0/960190824 conn(0x5606c08229b0 msgr2=0x5606c082ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 shutdown_connections 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 1 -- 172.21.15.120:0/960190824 wait complete. 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.429+0000 7f8a831f2740 20 client.15930 trim_cache size 0 max 16384 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:55:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:55:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:55:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 msgr2=0x7f8a640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 0x7f8a640b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a740a8e60 tx=0x7f8a74004a90 comp rx=0 tx=0).stop 2024-09-10T22:55:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 msgr2=0x7f8a78198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9f0 tx=0x7f8a7400edf0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 shutdown_connections 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a640b52e0 0x7f8a640b77a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a78104490 0x7f8a7819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a78103ad0 0x7f8a78199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 --2- 172.21.15.120:0/2976461241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a78102980 0x7f8a78198db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 >> 172.21.15.120:0/2976461241 conn(0x7f8a780fe240 msgr2=0x7f8a78104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 shutdown_connections 2024-09-10T22:55:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:31.430+0000 7f8a80423640 1 -- 172.21.15.120:0/2976461241 wait complete. 2024-09-10T22:55:31.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 471332 17.57 MB/sec execute 26 sec latency 99.029 ms 2024-09-10T22:55:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:31 smithi120 ceph-mon[31247]: pgmap v437: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-09-10T22:55:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:31 smithi153 ceph-mon[34951]: pgmap v437: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-09-10T22:55:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:31 smithi176 ceph-mon[34982]: pgmap v437: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-09-10T22:55:32.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 474161 17.57 MB/sec execute 27 sec latency 14.675 ms 2024-09-10T22:55:32.913 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:55:33.168 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:33.585 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:33 smithi120 ceph-mon[31247]: pgmap v438: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-10T22:55:33.618 INFO:tasks.workunit.client.0.smithi120.stdout: 1 477267 17.51 MB/sec execute 28 sec latency 17.465 ms 2024-09-10T22:55:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:33 smithi153 ceph-mon[34951]: pgmap v438: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-10T22:55:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:33 smithi176 ceph-mon[34982]: pgmap v438: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 377 op/s 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 -- 172.21.15.120:0/2062383940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581076a0 msgr2=0x7f6858107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 --2- 172.21.15.120:0/2062383940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581076a0 0x7f6858107ae0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6844009900 tx=0x7f684402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 -- 172.21.15.120:0/2062383940 shutdown_connections 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 --2- 172.21.15.120:0/2062383940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6858108020 0x7f685810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 --2- 172.21.15.120:0/2062383940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581076a0 0x7f6858107ae0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 --2- 172.21.15.120:0/2062383940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858101c90 0x7f6858102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.835+0000 7f685ea79640 1 -- 172.21.15.120:0/2062383940 >> 172.21.15.120:0/2062383940 conn(0x7f68580fd700 msgr2=0x7f68580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- 172.21.15.120:0/2062383940 shutdown_connections 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- 172.21.15.120:0/2062383940 wait complete. 2024-09-10T22:55:34.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 Processor -- start 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- start start 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 0x7f68581986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858108020 0x7f685819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6858111ca0 con 0x7f6858101c90 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6858111b20 con 0x7f6858108020 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.836+0000 7f685ea79640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6858111e20 con 0x7f68581076a0 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36334/0 (socket says 172.21.15.120:36334) 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 -- 172.21.15.120:0/1970335274 learned_addr learned my addr 172.21.15.120:0/1970335274 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f6857fff640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 0x7f68581986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685d016640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858108020 0x7f685819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 -- 172.21.15.120:0/1970335274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 msgr2=0x7f68581986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 0x7f68581986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 -- 172.21.15.120:0/1970335274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858108020 msgr2=0x7f685819da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858108020 0x7f685819da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 -- 172.21.15.120:0/1970335274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685819e260 con 0x7f6858101c90 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f6857fff640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 0x7f68581986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685c815640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f6855ffb640 1 -- 172.21.15.120:0/1970335274 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c00cd80 con 0x7f6858101c90 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685819e550 con 0x7f6858101c90 2024-09-10T22:55:34.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a4de0 con 0x7f6858101c90 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f6855ffb640 1 -- 172.21.15.120:0/1970335274 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684c0108e0 con 0x7f6858101c90 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.837+0000 7f6855ffb640 1 -- 172.21.15.120:0/1970335274 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c0188e0 con 0x7f6858101c90 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6855ffb640 1 -- 172.21.15.120:0/1970335274 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f684c010430 con 0x7f6858101c90 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 Processor -- start 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 -- start start 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f32811aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32811af40 0x55f32811d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f328082e50 con 0x55f32810c480 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f328072280 con 0x55f32811af40 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6860e4e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f328076090 con 0x55f32810cda0 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f685d016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f685d016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36338/0 (socket says 172.21.15.120:36338) 2024-09-10T22:55:34.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f685d016640 1 -- 172.21.15.120:0/3549652946 learned_addr learned my addr 172.21.15.120:0/3549652946 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f685c815640 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f32811aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.840+0000 7f6857fff640 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32811af40 0x55f32811d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 -- 172.21.15.120:0/3549652946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 msgr2=0x55f32811aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f32811aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 -- 172.21.15.120:0/3549652946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32811af40 msgr2=0x55f32811d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32811af40 0x55f32811d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 -- 172.21.15.120:0/3549652946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f327dd0650 con 0x55f32810c480 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f6855ffb640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 0x7f68480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685d016640 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f68500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3849c66485d98396 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f685c815640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 0x7f68480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f6842ffd640 1 -- 172.21.15.120:0/3549652946 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850004d20 con 0x55f32810c480 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f6855ffb640 1 -- 172.21.15.120:0/1970335274 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f684c0e5ea0 con 0x7f6858101c90 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f6842ffd640 1 -- 172.21.15.120:0/3549652946 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6850004050 con 0x55f32810c480 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.841+0000 7f6842ffd640 1 -- 172.21.15.120:0/3549652946 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850005280 con 0x55f32810c480 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- 172.21.15.120:0/3549652946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 msgr2=0x55f32810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f68500118d0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f685c815640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 0x7f68480b77a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f684c0e3d40 tx=0x7f684c0e9980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- 172.21.15.120:0/3549652946 shutdown_connections 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32811af40 0x55f32811d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f32811aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 --2- 172.21.15.120:0/3549652946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810c860 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- 172.21.15.120:0/3549652946 >> 172.21.15.120:0/3549652946 conn(0x55f32810a2b0 msgr2=0x55f32810b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- 172.21.15.120:0/3549652946 shutdown_connections 2024-09-10T22:55:34.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- 172.21.15.120:0/3549652946 wait complete. 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 -- start start 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.842+0000 7f6860e4e740 1 Processor -- start 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f328114000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32811af40 0x55f328122600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f327ed3700 con 0x55f32811af40 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f32800d110 con 0x55f32810c480 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f32811dbc0 con 0x55f32810cda0 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6860e4e740 10 client.0 _ll_register_callbacks cb 0x55f328026990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f685d016640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f328114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f685d016640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35022/0 (socket says 172.21.15.120:35022) 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f685d016640 1 -- 172.21.15.120:0/1451834060 learned_addr learned my addr 172.21.15.120:0/1451834060 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:34.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f685d016640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32811af40 0x55f328122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 -- 172.21.15.120:0/1451834060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 msgr2=0x55f328113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 -- 172.21.15.120:0/1451834060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32811af40 msgr2=0x55f328122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32811af40 0x55f328122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f6857fff640 1 -- 172.21.15.120:0/1451834060 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32803ac40 con 0x55f32810cda0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.843+0000 7f685d016640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6857fff640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f328114000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6844009230 tx=0x7f6844031750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf2aa7ce167034b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844035040 con 0x55f32810cda0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6850004050 con 0x55f32810cda0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6844004240 con 0x55f32810cda0 2024-09-10T22:55:34.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f684404c070 con 0x55f32810cda0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 10 client.25414 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f32812b700 con 0x55f32810cda0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6850004050 con 0x55f32810cda0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6844004790 con 0x55f32810cda0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 10 client.25414 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 10 client.25414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 20 client.25414 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32812efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 4 client.25414 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.844+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f328084da0 con 0x55f32812cbb0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.845+0000 7f685d016640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32812efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.845+0000 7f685d016640 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32812efd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d850912d4197ae07 server_cookie=7678ca4e0e722650 in_seq=0 out_seq=0 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.845+0000 7f6837fff640 10 client.25414 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:34.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6837fff640 1 -- 172.21.15.120:0/1451834060 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f328084da0 con 0x55f32812cbb0 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6837fff640 10 client.25414 handle_command_reply: tid=0 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 client.25414 shutdown 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 msgr2=0x55f328114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f328114000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6844009230 tx=0x7f6844031750 comp rx=0 tx=0).stop 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 shutdown_connections 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32812efd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32811af40 0x55f328122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32810cda0 0x55f328114000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6837fff640 0 client.25414 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 --2- 172.21.15.120:0/1451834060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f328113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 >> 172.21.15.120:0/1451834060 conn(0x55f32810a2b0 msgr2=0x55f32810fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 shutdown_connections 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 1 -- 172.21.15.120:0/1451834060 wait complete. 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.846+0000 7f6860e4e740 20 client.25414 trim_cache size 0 max 16384 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 Processor -- start 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 -- start start 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810cda0 0x55f32810b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f3281348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00c360 con 0x55f32810c480 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00ebe0 con 0x55f32810cda0 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6860e4e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00c9d0 con 0x55f32811af40 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f685d016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f685d016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36350/0 (socket says 172.21.15.120:36350) 2024-09-10T22:55:34.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f685d016640 1 -- 172.21.15.120:0/2461856505 learned_addr learned my addr 172.21.15.120:0/2461856505 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:55:34.152 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:34.153 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:34.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:34.157 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:55:34.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f6857fff640 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810cda0 0x55f32810b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.865+0000 7f685d016640 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f3281348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 -- 172.21.15.120:0/2461856505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 msgr2=0x55f3281348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f3281348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 -- 172.21.15.120:0/2461856505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810cda0 msgr2=0x55f32810b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810cda0 0x55f32810b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 -- 172.21.15.120:0/2461856505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f328111dc0 con 0x55f32810c480 2024-09-10T22:55:34.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f685d016640 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f685000a010 tx=0x7f6850009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7722a5b5aad1cb0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f68427fc640 1 -- 172.21.15.120:0/2461856505 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500049e0 con 0x55f32810c480 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f68427fc640 1 -- 172.21.15.120:0/2461856505 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685002c030 con 0x55f32810c480 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.866+0000 7f68427fc640 1 -- 172.21.15.120:0/2461856505 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685000f740 con 0x55f32810c480 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- 172.21.15.120:0/2461856505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 msgr2=0x55f32810b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f685000a010 tx=0x7f6850009a50 comp rx=0 tx=0).stop 2024-09-10T22:55:34.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- 172.21.15.120:0/2461856505 shutdown_connections 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f3281348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810cda0 0x55f32810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 --2- 172.21.15.120:0/2461856505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810c480 0x55f32810b2e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- 172.21.15.120:0/2461856505 >> 172.21.15.120:0/2461856505 conn(0x55f32810a2b0 msgr2=0x55f32811cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- 172.21.15.120:0/2461856505 shutdown_connections 2024-09-10T22:55:34.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- 172.21.15.120:0/2461856505 wait complete. 2024-09-10T22:55:34.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.867+0000 7f6860e4e740 1 -- start start 2024-09-10T22:55:34.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 Processor -- start 2024-09-10T22:55:34.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f3281253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f328125e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00d0a0 con 0x55f32810cda0 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00e510 con 0x55f32810c480 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f683c00d770 con 0x55f32811af40 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6860e4e740 10 client.0 _ll_register_callbacks cb 0x7f68500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6857fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6857fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36360/0 (socket says 172.21.15.120:36360) 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f6857fff640 1 -- 172.21.15.120:0/316222953 learned_addr learned my addr 172.21.15.120:0/316222953 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f685d016640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f3281253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.868+0000 7f685d016640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f328125e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 -- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 msgr2=0x55f328125e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f328125e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 -- 172.21.15.120:0/316222953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 msgr2=0x55f3281253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f3281253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 -- 172.21.15.120:0/316222953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f328198ba0 con 0x55f32810cda0 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6857fff640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f684402a450 tx=0x7f68440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef766018cb8f12f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6837fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844035040 con 0x55f32810cda0 2024-09-10T22:55:34.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6844031bd0 con 0x55f32810cda0 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f685002c030 con 0x55f32810cda0 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6844009520 con 0x55f32810cda0 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6860e4e740 10 client.15948 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f328199eb0 con 0x55f32810cda0 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.869+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685002c030 con 0x55f32810cda0 2024-09-10T22:55:34.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f68440038c0 con 0x55f32810cda0 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 10 client.15948 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 20 client.15948 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 4 client.15948 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:55:34.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f328080850 con 0x55f32812cbb0 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f685d016640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6837fff640 10 client.15948 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f685d016640 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32819c6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129f6b927621649a server_cookie=bd61aca03994d758 in_seq=0 out_seq=0 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.870+0000 7f6837fff640 1 -- 172.21.15.120:0/316222953 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55f328080850 con 0x55f32812cbb0 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6837fff640 10 client.15948 handle_command_reply: tid=0 2024-09-10T22:55:34.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 client.15948 shutdown 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 msgr2=0x55f328125900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f684402a450 tx=0x7f68440048f0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 shutdown_connections 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f32812cbb0 0x55f32819c6a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f32811af40 0x55f328125e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f32810cda0 0x55f328125900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6837fff640 0 client.15948 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:34.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 --2- 172.21.15.120:0/316222953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f32810c480 0x55f3281253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 >> 172.21.15.120:0/316222953 conn(0x55f32810a2b0 msgr2=0x55f32810b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 shutdown_connections 2024-09-10T22:55:34.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 1 -- 172.21.15.120:0/316222953 wait complete. 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.871+0000 7f6860e4e740 20 client.15948 trim_cache size 0 max 16384 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 msgr2=0x7f68480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 0x7f68480b77a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f684c0e3d40 tx=0x7f684c0e9980 comp rx=0 tx=0).stop 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 msgr2=0x7f68581981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 shutdown_connections 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68480b52e0 0x7f68480b77a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6858108020 0x7f685819da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68581076a0 0x7f68581986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 --2- 172.21.15.120:0/1970335274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6858101c90 0x7f68581981b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 >> 172.21.15.120:0/1970335274 conn(0x7f68580fd700 msgr2=0x7f68580ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:34.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 shutdown_connections 2024-09-10T22:55:34.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:33.872+0000 7f685ea79640 1 -- 172.21.15.120:0/1970335274 wait complete. 2024-09-10T22:55:34.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 480288 17.50 MB/sec execute 29 sec latency 65.841 ms 2024-09-10T22:55:35.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:55:35.106 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:35.388 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:35.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 483579 17.55 MB/sec execute 30 sec latency 29.399 ms 2024-09-10T22:55:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:35 smithi120 ceph-mon[31247]: pgmap v439: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-09-10T22:55:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:35 smithi153 ceph-mon[34951]: pgmap v439: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-09-10T22:55:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:35 smithi176 ceph-mon[34982]: pgmap v439: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 -- 172.21.15.120:0/511093334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4698102de0 msgr2=0x7f46981031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 --2- 172.21.15.120:0/511093334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4698102de0 0x7f46981031c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02a410 comp rx=0 tx=0).stop 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 -- 172.21.15.120:0/511093334 shutdown_connections 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 --2- 172.21.15.120:0/511093334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698109170 0x7f469810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 --2- 172.21.15.120:0/511093334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46981087f0 0x7f4698108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 --2- 172.21.15.120:0/511093334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4698102de0 0x7f46981031c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 -- 172.21.15.120:0/511093334 >> 172.21.15.120:0/511093334 conn(0x7f46980fe830 msgr2=0x7f4698100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 -- 172.21.15.120:0/511093334 shutdown_connections 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.007+0000 7f469eadf640 1 -- 172.21.15.120:0/511093334 wait complete. 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 Processor -- start 2024-09-10T22:55:36.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 -- start start 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 0x7f46980754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4698109170 0x7f4698075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4698112e30 con 0x7f46981087f0 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4698112cb0 con 0x7f4698102de0 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f469eadf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4698112fb0 con 0x7f4698109170 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f4697fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f4697fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36372/0 (socket says 172.21.15.120:36372) 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.008+0000 7f4697fff640 1 -- 172.21.15.120:0/3908527534 learned_addr learned my addr 172.21.15.120:0/3908527534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f469d07c640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4698109170 0x7f4698075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f469c87b640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 0x7f46980754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 -- 172.21.15.120:0/3908527534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4698109170 msgr2=0x7f4698075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:36.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4698109170 0x7f4698075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 -- 172.21.15.120:0/3908527534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 msgr2=0x7f46980754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 0x7f46980754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 -- 172.21.15.120:0/3908527534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46981a76b0 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f469c87b640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 0x7f46980754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4697fff640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468400cd80 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46840108e0 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46981a79a0 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684018980 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.009+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46981a7f30 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.010+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46981031c0 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.012+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4684010430 con 0x7f46981087f0 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.013+0000 7f4695ffb640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 0x7f46880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:36.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.013+0000 7f469c87b640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 0x7f46880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:36.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.013+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f46840e63a0 con 0x7f46981087f0 2024-09-10T22:55:36.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.013+0000 7f469c87b640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 0x7f46880b7820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f468c031440 tx=0x7f468c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:36.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.016+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f46840ad290 con 0x7f46981087f0 2024-09-10T22:55:36.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.260+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46980767a0 con 0x7f46981087f0 2024-09-10T22:55:36.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.261+0000 7f4695ffb640 1 -- 172.21.15.120:0/3908527534 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f46840acb90 con 0x7f46981087f0 2024-09-10T22:55:36.263 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:55:36.263 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":23,"btime":"2024-09-10T22:54:39:370770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:54:39.370760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25252":{"gid":25252,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2789135593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2789135593},{"type":"v1","addr":"172.21.15.176:6837","nonce":2789135593}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:55:36.263 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 23 2024-09-10T22:55:36.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 msgr2=0x7f46880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 0x7f46880b7820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f468c031440 tx=0x7f468c005a20 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 msgr2=0x7f46980759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 shutdown_connections 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46880b5360 0x7f46880b7820 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4698109170 0x7f4698075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46981087f0 0x7f46980759e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 --2- 172.21.15.120:0/3908527534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4698102de0 0x7f46980754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.263+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 >> 172.21.15.120:0/3908527534 conn(0x7f46980fe830 msgr2=0x7f4698100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.264+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 shutdown_connections 2024-09-10T22:55:36.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:36.264+0000 7f469eadf640 1 -- 172.21.15.120:0/3908527534 wait complete. 2024-09-10T22:55:36.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 486983 17.59 MB/sec execute 31 sec latency 21.219 ms 2024-09-10T22:55:36.636 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:36 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3908527534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:36.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:36 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3908527534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:36.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:36 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3908527534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:37.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:55:37.383 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:37.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 490314 17.60 MB/sec execute 32 sec latency 40.243 ms 2024-09-10T22:55:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:37 smithi120 ceph-mon[31247]: pgmap v440: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-10T22:55:37.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:37 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:37 smithi153 ceph-mon[34951]: pgmap v440: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-10T22:55:37.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:37 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:37 smithi176 ceph-mon[34982]: pgmap v440: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-10T22:55:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:37 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:37.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.937+0000 7f1177864640 1 -- 172.21.15.120:0/427996622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 msgr2=0x7f1170108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.937+0000 7f1177864640 1 --2- 172.21.15.120:0/427996622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170108c10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f116402a410 comp rx=0 tx=0).stop 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 -- 172.21.15.120:0/427996622 shutdown_connections 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 --2- 172.21.15.120:0/427996622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 0x7f117010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 --2- 172.21.15.120:0/427996622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170108c10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 --2- 172.21.15.120:0/427996622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 0x7f11701031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 -- 172.21.15.120:0/427996622 >> 172.21.15.120:0/427996622 conn(0x7f11700fe830 msgr2=0x7f1170100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 -- 172.21.15.120:0/427996622 shutdown_connections 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 -- 172.21.15.120:0/427996622 wait complete. 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 Processor -- start 2024-09-10T22:55:37.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.938+0000 7f1177864640 1 -- start start 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 0x7f11701992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 0x7f117019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112e10 con 0x7f11701087d0 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112c90 con 0x7f1170109150 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1177864640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112f90 con 0x7f1170102dc0 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36378/0 (socket says 172.21.15.120:36378) 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 -- 172.21.15.120:0/1378312254 learned_addr learned my addr 172.21.15.120:0/1378312254 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1175600640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 0x7f11701992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1175e01640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 0x7f117019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 -- 172.21.15.120:0/1378312254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 msgr2=0x7f11701992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 0x7f11701992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 -- 172.21.15.120:0/1378312254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 msgr2=0x7f117019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 0x7f117019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 -- 172.21.15.120:0/1378312254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117019f370 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f1174dff640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1164005940 tx=0x7f1164004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.939+0000 7f116e7fc640 1 -- 172.21.15.120:0/1378312254 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403f030 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.940+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117019f600 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.940+0000 7f116e7fc640 1 -- 172.21.15.120:0/1378312254 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1164003aa0 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.940+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a5d20 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.940+0000 7f116e7fc640 1 -- 172.21.15.120:0/1378312254 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164049060 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.942+0000 7f116e7fc640 1 -- 172.21.15.120:0/1378312254 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1164003c50 con 0x7f11701087d0 2024-09-10T22:55:37.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.943+0000 7f116e7fc640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 0x7f11580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.943+0000 7f1179c39740 1 Processor -- start 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.943+0000 7f1175600640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 0x7f11580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.943+0000 7f1179c39740 1 -- start start 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.943+0000 7f116e7fc640 1 -- 172.21.15.120:0/1378312254 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f11641067f0 con 0x7f11701087d0 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fedf0 0x55799c20a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 0x7f11580b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1168005fd0 tx=0x7f1168009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c20d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55799c174ea0 con 0x55799c20ad80 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55799c1642d0 con 0x55799c1fedf0 2024-09-10T22:55:37.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1179c39740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55799c1680e0 con 0x55799c1fe4d0 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175e01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c20d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40674/0 (socket says 172.21.15.120:40674) 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175e01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c20d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36382/0 (socket says 172.21.15.120:36382) 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 -- 172.21.15.120:0/4277619131 learned_addr learned my addr 172.21.15.120:0/4277619131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1174dff640 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fedf0 0x55799c20a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 -- 172.21.15.120:0/4277619131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fedf0 msgr2=0x55799c20a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fedf0 0x55799c20a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 -- 172.21.15.120:0/4277619131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 msgr2=0x55799c20d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c20d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.944+0000 7f1175600640 1 -- 172.21.15.120:0/4277619131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799bec2650 con 0x55799c1fe4d0 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.945+0000 7f1175600640 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1168003d10 tx=0x7f1168014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5c65c398cb8823 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.945+0000 7f11577fe640 1 -- 172.21.15.120:0/4277619131 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168027670 con 0x55799c1fe4d0 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.945+0000 7f11577fe640 1 -- 172.21.15.120:0/4277619131 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168022070 con 0x55799c1fe4d0 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.945+0000 7f11577fe640 1 -- 172.21.15.120:0/4277619131 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801e670 con 0x55799c1fe4d0 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- 172.21.15.120:0/4277619131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 msgr2=0x55799c1fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1168003d10 tx=0x7f1168014000 comp rx=0 tx=0).stop 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- 172.21.15.120:0/4277619131 shutdown_connections 2024-09-10T22:55:37.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c20d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fedf0 0x55799c20a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 --2- 172.21.15.120:0/4277619131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fe8b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- 172.21.15.120:0/4277619131 >> 172.21.15.120:0/4277619131 conn(0x55799c1fc300 msgr2=0x55799c1fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- 172.21.15.120:0/4277619131 shutdown_connections 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- 172.21.15.120:0/4277619131 wait complete. 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 -- start start 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.946+0000 7f1179c39740 1 Processor -- start 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 0x55799c20a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c207a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c208000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55799bfc5700 con 0x55799c20ad80 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55799c0ff160 con 0x55799c1fe4d0 2024-09-10T22:55:37.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55799c20da00 con 0x55799c1fedf0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1179c39740 10 client.0 _ll_register_callbacks cb 0x55799c1189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1174dff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c207a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175e01640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c208000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1174dff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c207a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40680/0 (socket says 172.21.15.120:40680) 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1174dff640 1 -- 172.21.15.120:0/3617375568 learned_addr learned my addr 172.21.15.120:0/3617375568 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 0x55799c20a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 -- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 msgr2=0x55799c207a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c207a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 -- 172.21.15.120:0/3617375568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 msgr2=0x55799c208000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c208000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 -- 172.21.15.120:0/3617375568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799c12cc90 con 0x55799c1fe4d0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1175600640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 0x55799c20a610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1168003b20 tx=0x7f116801d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db11830ac921f361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.947+0000 7f1154ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801dc50 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f1168004a40 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1168022070 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1168004c40 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 10 client.25433 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55799c2192d0 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168004de0 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f1168027590 con 0x55799c1fe4d0 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 10 client.25433 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 20 client.25433 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c21a6d0 0x55799c21cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.948+0000 7f1179c39740 4 client.25433 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:37.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.949+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1168022070 con 0x55799c21a6d0 2024-09-10T22:55:37.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.949+0000 7f1174dff640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c21a6d0 0x55799c21cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.949+0000 7f1154ff9640 10 client.25433 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:37.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.949+0000 7f1174dff640 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c21a6d0 0x55799c21cac0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723ff65d4190ccf7 server_cookie=907a604962a7d17d in_seq=0 out_seq=0 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1154ff9640 1 -- 172.21.15.120:0/3617375568 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1168022070 con 0x55799c21a6d0 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1154ff9640 10 client.25433 handle_command_reply: tid=0 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 client.25433 shutdown 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 msgr2=0x55799c20a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 0x55799c20a610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1168003b20 tx=0x7f116801d950 comp rx=0 tx=0).stop 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 shutdown_connections 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c21a6d0 0x55799c21cac0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c20ad80 0x55799c208000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c207a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1154ff9640 0 client.25433 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:37.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 --2- 172.21.15.120:0/3617375568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c1fe4d0 0x55799c20a610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 >> 172.21.15.120:0/3617375568 conn(0x55799c1fc300 msgr2=0x55799c201ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 shutdown_connections 2024-09-10T22:55:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 1 -- 172.21.15.120:0/3617375568 wait complete. 2024-09-10T22:55:37.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.951+0000 7f1179c39740 20 client.25433 trim_cache size 0 max 16384 2024-09-10T22:55:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 Processor -- start 2024-09-10T22:55:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 -- start start 2024-09-10T22:55:37.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c20c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11701aa100 con 0x55799c1fedf0 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ac980 con 0x55799c20ad80 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1179c39740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11701aa770 con 0x55799c1fe4d0 2024-09-10T22:55:37.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1174dff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1175600640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1174dff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36404/0 (socket says 172.21.15.120:36404) 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.968+0000 7f1174dff640 1 -- 172.21.15.120:0/1008020617 learned_addr learned my addr 172.21.15.120:0/1008020617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1175e01640 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c20c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 -- 172.21.15.120:0/1008020617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 msgr2=0x55799c1fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 -- 172.21.15.120:0/1008020617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 msgr2=0x55799c20c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c20c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 -- 172.21.15.120:0/1008020617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799c1fcd80 con 0x55799c1fedf0 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1174dff640 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1164104370 tx=0x7f11640b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990a57278798d071 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1156ffd640 1 -- 172.21.15.120:0/1008020617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035040 con 0x55799c1fedf0 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1156ffd640 1 -- 172.21.15.120:0/1008020617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11640ccba0 con 0x55799c1fedf0 2024-09-10T22:55:37.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.969+0000 7f1156ffd640 1 -- 172.21.15.120:0/1008020617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640cd2f0 con 0x55799c1fedf0 2024-09-10T22:55:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- 172.21.15.120:0/1008020617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 msgr2=0x55799c20c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1164104370 tx=0x7f11640b4040 comp rx=0 tx=0).stop 2024-09-10T22:55:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- 172.21.15.120:0/1008020617 shutdown_connections 2024-09-10T22:55:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c20c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fedf0 0x55799c20c170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 --2- 172.21.15.120:0/1008020617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fe4d0 0x55799c1fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- 172.21.15.120:0/1008020617 >> 172.21.15.120:0/1008020617 conn(0x55799c1fc300 msgr2=0x55799c2098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- 172.21.15.120:0/1008020617 shutdown_connections 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- 172.21.15.120:0/1008020617 wait complete. 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 -- start start 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.970+0000 7f1179c39740 1 Processor -- start 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c210ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c211430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11701aae40 con 0x55799c1fe4d0 2024-09-10T22:55:37.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ac2b0 con 0x55799c20ad80 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ab510 con 0x55799c1fedf0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1179c39740 10 client.0 _ll_register_callbacks cb 0x7f1164005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36412/0 (socket says 172.21.15.120:36412) 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 -- 172.21.15.120:0/1093169425 learned_addr learned my addr 172.21.15.120:0/1093169425 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1174dff640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c210ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175e01640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c211430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 -- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 msgr2=0x55799c210ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c210ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 -- 172.21.15.120:0/1093169425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 msgr2=0x55799c211430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c211430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 -- 172.21.15.120:0/1093169425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799c211c40 con 0x55799c1fe4d0 2024-09-10T22:55:37.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175e01640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c211430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1175600640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1168016040 tx=0x7f116801de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a5355a94127d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1154ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168040070 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.971+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f1168027470 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11640ccba0 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f116801dc50 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 10 client.15972 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55799c288350 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11640ccba0 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f1168045c60 con 0x55799c1fe4d0 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 10 client.15972 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 20 client.15972 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:37.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c288a90 0x55799c28aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 4 client.15972 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55799c176df0 con 0x55799c288a90 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.972+0000 7f1174dff640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c288a90 0x55799c28aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.973+0000 7f1154ff9640 10 client.15972 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.973+0000 7f1174dff640 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c288a90 0x55799c28aeb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed32cb8b8ee5c38 server_cookie=94a5778a24028fe8 in_seq=0 out_seq=0 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.974+0000 7f1154ff9640 1 -- 172.21.15.120:0/1093169425 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55799c176df0 con 0x55799c288a90 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1154ff9640 10 client.15972 handle_command_reply: tid=0 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 client.15972 shutdown 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 msgr2=0x55799c213440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.983 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1168016040 tx=0x7f116801de90 comp rx=0 tx=0).stop 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 shutdown_connections 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55799c288a90 0x55799c28aeb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55799c20ad80 0x55799c211430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55799c1fedf0 0x55799c210ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 --2- 172.21.15.120:0/1093169425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55799c1fe4d0 0x55799c213440 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1154ff9640 0 client.15972 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:37.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 >> 172.21.15.120:0/1093169425 conn(0x55799c1fc300 msgr2=0x55799c2062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 shutdown_connections 2024-09-10T22:55:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 1 -- 172.21.15.120:0/1093169425 wait complete. 2024-09-10T22:55:37.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.975+0000 7f1179c39740 20 client.15972 trim_cache size 0 max 16384 2024-09-10T22:55:37.985 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 msgr2=0x7f11580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 0x7f11580b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1168005fd0 tx=0x7f1168009500 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 msgr2=0x7f1170199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1164005940 tx=0x7f1164004270 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 shutdown_connections 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11580b52e0 0x7f11580b77a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1170109150 0x7f117019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11701087d0 0x7f1170199800 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 --2- 172.21.15.120:0/1378312254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1170102dc0 0x7f11701992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 >> 172.21.15.120:0/1378312254 conn(0x7f11700fe830 msgr2=0x7f1170100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 shutdown_connections 2024-09-10T22:55:37.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:37.976+0000 7f1177864640 1 -- 172.21.15.120:0/1378312254 wait complete. 2024-09-10T22:55:38.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 493417 17.50 MB/sec execute 33 sec latency 24.299 ms 2024-09-10T22:55:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:38.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:55:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:38.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:55:38.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:38.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:38.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:55:39.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 496447 17.41 MB/sec execute 34 sec latency 69.542 ms 2024-09-10T22:55:39.777 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:55:39.778 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:55:40.056 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:40.177 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:39 smithi120 ceph-mon[31247]: pgmap v441: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-10T22:55:40.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:39 smithi153 ceph-mon[34951]: pgmap v441: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-10T22:55:40.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:39 smithi176 ceph-mon[34982]: pgmap v441: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 467 op/s 2024-09-10T22:55:40.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 499247 17.38 MB/sec execute 35 sec latency 39.091 ms 2024-09-10T22:55:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 -- 172.21.15.120:0/258139112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa001087d0 msgr2=0x7faa00108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 --2- 172.21.15.120:0/258139112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa001087d0 0x7faa00108c10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009900 tx=0x7fa9f402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 -- 172.21.15.120:0/258139112 shutdown_connections 2024-09-10T22:55:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 --2- 172.21.15.120:0/258139112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa00109150 0x7faa0010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 --2- 172.21.15.120:0/258139112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa001087d0 0x7faa00108c10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 --2- 172.21.15.120:0/258139112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 0x7faa001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 -- 172.21.15.120:0/258139112 >> 172.21.15.120:0/258139112 conn(0x7faa000fe870 msgr2=0x7faa00100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.637+0000 7faa04893640 1 -- 172.21.15.120:0/258139112 shutdown_connections 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.638+0000 7faa04893640 1 -- 172.21.15.120:0/258139112 wait complete. 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.638+0000 7faa04893640 1 Processor -- start 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.638+0000 7faa04893640 1 -- start start 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 0x7faa001992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa001087d0 0x7faa001997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa00112dd0 con 0x7faa00109150 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa00112c50 con 0x7faa00102dc0 2024-09-10T22:55:40.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7faa04893640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faa00112f50 con 0x7faa001087d0 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36424/0 (socket says 172.21.15.120:36424) 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 -- 172.21.15.120:0/677370584 learned_addr learned my addr 172.21.15.120:0/677370584 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa001087d0 0x7faa001997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 0x7faa001992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 -- 172.21.15.120:0/677370584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa001087d0 msgr2=0x7faa001997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa001087d0 0x7faa001997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 -- 172.21.15.120:0/677370584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 msgr2=0x7faa001992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 0x7faa001992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 -- 172.21.15.120:0/677370584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0019f350 con 0x7faa00109150 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800eb10 tx=0x7fa9f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.639+0000 7fa9ef7fe640 1 -- 172.21.15.120:0/677370584 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004b20 con 0x7faa00109150 2024-09-10T22:55:40.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.640+0000 7fa9ef7fe640 1 -- 172.21.15.120:0/677370584 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8004cc0 con 0x7faa00109150 2024-09-10T22:55:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.640+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0019f640 con 0x7faa00109150 2024-09-10T22:55:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.640+0000 7fa9ef7fe640 1 -- 172.21.15.120:0/677370584 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8010670 con 0x7faa00109150 2024-09-10T22:55:40.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.640+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a5ed0 con 0x7faa00109150 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.642+0000 7fa9ef7fe640 1 -- 172.21.15.120:0/677370584 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa9f8005020 con 0x7faa00109150 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.642+0000 7faa06c68740 1 Processor -- start 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 -- start start 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902daee0 0x55e2902db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 0x55e2902e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e290251e60 con 0x55e2902db800 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e290241290 con 0x55e2902daee0 2024-09-10T22:55:40.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7faa06c68740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e2902450a0 con 0x55e2902e7790 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902daee0 0x55e2902db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36430/0 (socket says 172.21.15.120:36430) 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1918352216 learned_addr learned my addr 172.21.15.120:0/1918352216 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 0x55e2902e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1918352216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 msgr2=0x55e2902e9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 0x55e2902e9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1918352216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902daee0 msgr2=0x55e2902db2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902daee0 0x55e2902db2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.643+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1918352216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e28ff9f930 con 0x55e2902db800 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 0x55e2902e9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:40.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9ef7fe640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 0x7fa9e80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009d20 tx=0x7fa9f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8330d249e7ee84e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 0x7fa9e80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9ef7fe640 1 -- 172.21.15.120:0/677370584 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f80e60e0 con 0x7faa00109150 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9ecff9640 1 -- 172.21.15.120:0/1918352216 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0004230 con 0x55e2902db800 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9ecff9640 1 -- 172.21.15.120:0/1918352216 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f00043d0 con 0x55e2902db800 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.644+0000 7fa9ecff9640 1 -- 172.21.15.120:0/1918352216 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00198c0 con 0x55e2902db800 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- 172.21.15.120:0/1918352216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 msgr2=0x55e2902e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 0x7fa9e80b7820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4031440 tx=0x7fa9f4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009d20 tx=0x7fa9f0011810 comp rx=0 tx=0).stop 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- 172.21.15.120:0/1918352216 shutdown_connections 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902e7790 0x55e2902e9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e7250 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 --2- 172.21.15.120:0/1918352216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902daee0 0x55e2902db2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- 172.21.15.120:0/1918352216 >> 172.21.15.120:0/1918352216 conn(0x55e2902d8d10 msgr2=0x55e2902d9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- 172.21.15.120:0/1918352216 shutdown_connections 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- 172.21.15.120:0/1918352216 wait complete. 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.645+0000 7faa06c68740 1 -- start start 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 Processor -- start 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 0x55e2902e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e2900a2890 con 0x55e2902db800 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e2901dbd40 con 0x55e2902e7790 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e2902ea440 con 0x55e2902daee0 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7faa06c68740 10 client.0 _ll_register_callbacks cb 0x55e29023fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 0x55e2902e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40720/0 (socket says 172.21.15.120:40720) 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1976020284 learned_addr learned my addr 172.21.15.120:0/1976020284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1976020284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 msgr2=0x55e2902e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 0x55e2902e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1976020284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 msgr2=0x55e2902e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 -- 172.21.15.120:0/1976020284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2902f3820 con 0x55e2902daee0 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 0x55e2902e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:40.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0019520 tx=0x7fa9f0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29675e395d1956e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.646+0000 7fa9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027070 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f00043d0 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f0005240 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0033050 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 10 client.25456 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e2902f61d0 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f00043d0 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fa9f0004b70 con 0x55e2902daee0 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 10 client.25456 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 10 client.25456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 20 client.25456 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e2902f75e0 0x55e2902f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 4 client.25456 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.647+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e290253db0 con 0x55e2902f75e0 2024-09-10T22:55:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.648+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e2902f75e0 0x55e2902f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.648+0000 7fa9e5ffb640 10 client.25456 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:40.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.648+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e2902f75e0 0x55e2902f99d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1715e52ee392f7 server_cookie=230d4fe6f1bcb361 in_seq=0 out_seq=0 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/1976020284 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e290253db0 con 0x55e2902f75e0 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7fa9e5ffb640 10 client.25456 handle_command_reply: tid=0 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 client.25456 shutdown 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 msgr2=0x55e2902e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0019520 tx=0x7fa9f0003820 comp rx=0 tx=0).stop 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 shutdown_connections 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e2902f75e0 0x55e2902f99d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902e7790 0x55e2902e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902db800 0x55e2902e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 --2- 172.21.15.120:0/1976020284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902e4760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7fa9e5ffb640 0 client.25456 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:40.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 >> 172.21.15.120:0/1976020284 conn(0x55e2902d8d10 msgr2=0x55e2902d7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 shutdown_connections 2024-09-10T22:55:40.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 1 -- 172.21.15.120:0/1976020284 wait complete. 2024-09-10T22:55:40.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.650+0000 7faa06c68740 20 client.25456 trim_cache size 0 max 16384 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 Processor -- start 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 -- start start 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ae520 con 0x55e2902e7790 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0da0 con 0x55e2902db800 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7faa06c68740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faa001aeb90 con 0x55e2902daee0 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.668+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35154/0 (socket says 172.21.15.120:35154) 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36438/0 (socket says 172.21.15.120:36438) 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 -- 172.21.15.120:0/46517139 learned_addr learned my addr 172.21.15.120:0/46517139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 -- 172.21.15.120:0/46517139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 msgr2=0x55e2902da310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902da310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 -- 172.21.15.120:0/46517139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 msgr2=0x55e2902ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 -- 172.21.15.120:0/46517139 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2902d9790 con 0x55e2902db800 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:40.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40316d0 tx=0x7fa9f4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915a14c51cad3095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9e7fff640 1 -- 172.21.15.120:0/46517139 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4046070 con 0x55e2902db800 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9e7fff640 1 -- 172.21.15.120:0/46517139 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4003ca0 con 0x55e2902db800 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.669+0000 7fa9e7fff640 1 -- 172.21.15.120:0/46517139 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4035040 con 0x55e2902db800 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- 172.21.15.120:0/46517139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 msgr2=0x55e2902ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40316d0 tx=0x7fa9f4009260 comp rx=0 tx=0).stop 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- 172.21.15.120:0/46517139 shutdown_connections 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ff1e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 --2- 172.21.15.120:0/46517139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902da310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- 172.21.15.120:0/46517139 >> 172.21.15.120:0/46517139 conn(0x55e2902d8d10 msgr2=0x55e2902d9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- 172.21.15.120:0/46517139 shutdown_connections 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- 172.21.15.120:0/46517139 wait complete. 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 -- start start 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.670+0000 7faa06c68740 1 Processor -- start 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af260 con 0x55e2902e7790 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b06d0 con 0x55e2902db800 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af930 con 0x55e2902daee0 2024-09-10T22:55:40.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7faa06c68740 10 client.0 _ll_register_callbacks cb 0x7fa9f80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35162/0 (socket says 172.21.15.120:35162) 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fdd9b640 1 -- 172.21.15.120:0/946491509 learned_addr learned my addr 172.21.15.120:0/946491509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40734/0 (socket says 172.21.15.120:40734) 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 -- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 msgr2=0x55e2902eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 -- 172.21.15.120:0/946491509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 msgr2=0x55e2902ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 -- 172.21.15.120:0/946491509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2902e9f70 con 0x55e2902e7790 2024-09-10T22:55:40.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fdd9b640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9fed9d640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ec190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80e3bb0 tx=0x7fa9f80ace10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5a394f9368ded6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.671+0000 7fa9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80abfe0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f80adba0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f4003ca0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa9f80b1130 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 10 client.15990 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e2903646e0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4003ca0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fa9f80a81c0 con 0x55e2902e7790 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 10 client.15990 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:40.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 20 client.15990 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e290364e60 0x55e290367280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 4 client.15990 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e29024f860 con 0x55e290364e60 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.672+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e290364e60 0x55e290367280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.673+0000 7fa9e5ffb640 10 client.15990 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:40.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.673+0000 7fa9fe59c640 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e290364e60 0x55e290367280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50203a4fa72c37c server_cookie=30f10d857f99ee97 in_seq=0 out_seq=0 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7fa9e5ffb640 1 -- 172.21.15.120:0/946491509 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e29024f860 con 0x55e290364e60 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7fa9e5ffb640 10 client.15990 handle_command_reply: tid=0 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 client.15990 shutdown 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 msgr2=0x55e2902ec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ec190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80e3bb0 tx=0x7fa9f80ace10 comp rx=0 tx=0).stop 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 shutdown_connections 2024-09-10T22:55:40.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e290364e60 0x55e290367280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e2902e7790 0x55e2902ec190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e2902db800 0x55e2902ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 --2- 172.21.15.120:0/946491509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e2902daee0 0x55e2902eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7fa9e5ffb640 0 client.15990 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 >> 172.21.15.120:0/946491509 conn(0x55e2902d8d10 msgr2=0x55e2902e5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 shutdown_connections 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 1 -- 172.21.15.120:0/946491509 wait complete. 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.675+0000 7faa06c68740 20 client.15990 trim_cache size 0 max 16384 2024-09-10T22:55:40.682 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:55:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:55:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "13b316f4-2fb8-4178-b82e-6e02b551a316", 2024-09-10T22:55:40.683 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:55:40.683 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:55:40.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 msgr2=0x7fa9e80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 0x7fa9e80b7820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4031440 tx=0x7fa9f4005b50 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 msgr2=0x7faa0019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800eb10 tx=0x7fa9f800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 shutdown_connections 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa9e80b5360 0x7fa9e80b7820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa00109150 0x7faa0019eb80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa001087d0 0x7faa001997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 --2- 172.21.15.120:0/677370584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa00102dc0 0x7faa001992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 >> 172.21.15.120:0/677370584 conn(0x7faa000fe870 msgr2=0x7faa00100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 shutdown_connections 2024-09-10T22:55:40.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:40.676+0000 7faa04893640 1 -- 172.21.15.120:0/677370584 wait complete. 2024-09-10T22:55:41.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 502424 17.42 MB/sec execute 36 sec latency 31.411 ms 2024-09-10T22:55:41.837 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:55:42.104 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:41 smithi120 ceph-mon[31247]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-09-10T22:55:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:41 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:55:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:41 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:55:42.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:41 smithi153 ceph-mon[34951]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-09-10T22:55:42.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:41 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:55:42.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:41 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:55:42.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:41 smithi176 ceph-mon[34982]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-09-10T22:55:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:41 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:55:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:41 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:55:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:42.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 505775 17.39 MB/sec execute 37 sec latency 19.710 ms 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 -- 172.21.15.120:0/2078602025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec08108040 msgr2=0x7fec0810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 --2- 172.21.15.120:0/2078602025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec08108040 0x7fec0810e840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fec00009ca0 tx=0x7fec0002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 -- 172.21.15.120:0/2078602025 shutdown_connections 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 --2- 172.21.15.120:0/2078602025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec08108040 0x7fec0810e840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 --2- 172.21.15.120:0/2078602025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec081076c0 0x7fec08107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 --2- 172.21.15.120:0/2078602025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec08102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 -- 172.21.15.120:0/2078602025 >> 172.21.15.120:0/2078602025 conn(0x7fec080fd700 msgr2=0x7fec080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 -- 172.21.15.120:0/2078602025 shutdown_connections 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.644+0000 7fec0d815640 1 -- 172.21.15.120:0/2078602025 wait complete. 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 Processor -- start 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 -- start start 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec081991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec08108040 0x7fec0819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec08111d00 con 0x7fec081076c0 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec08111b80 con 0x7fec08108040 2024-09-10T22:55:42.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec0d815640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec08111e80 con 0x7fec08101cb0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7febfffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7febfffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36458/0 (socket says 172.21.15.120:36458) 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec081991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7febfffff640 1 -- 172.21.15.120:0/529378343 learned_addr learned my addr 172.21.15.120:0/529378343 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec077fe640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec08108040 0x7fec0819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 -- 172.21.15.120:0/529378343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec08108040 msgr2=0x7fec0819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec08108040 0x7fec0819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 -- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 msgr2=0x7fec081996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7fec06ffd640 1 -- 172.21.15.120:0/529378343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0819f250 con 0x7fec08101cb0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.645+0000 7febfffff640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec06ffd640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec081991a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9f0 tx=0x7febf800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec0c813640 1 -- 172.21.15.120:0/529378343 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800ccc0 con 0x7fec08101cb0 2024-09-10T22:55:42.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec0c813640 1 -- 172.21.15.120:0/529378343 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf8005020 con 0x7fec08101cb0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0819f540 con 0x7fec08101cb0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec0c813640 1 -- 172.21.15.120:0/529378343 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004a00 con 0x7fec08101cb0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.646+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec081a5c00 con 0x7fec08101cb0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.648+0000 7fec0fbea740 1 Processor -- start 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.648+0000 7fec0c813640 1 -- 172.21.15.120:0/529378343 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7febf8004ba0 con 0x7fec08101cb0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.648+0000 7fec0fbea740 1 -- start start 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 0x561d07ca77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07ca7d30 0x561d07caa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d07c11e50 con 0x561d07c9b480 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d07c01280 con 0x561d07ca7d30 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0fbea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d07c05090 con 0x561d07c9bda0 2024-09-10T22:55:42.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36468/0 (socket says 172.21.15.120:36468) 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 -- 172.21.15.120:0/3684982041 learned_addr learned my addr 172.21.15.120:0/3684982041 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec06ffd640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 0x561d07ca77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7febfffff640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07ca7d30 0x561d07caa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 -- 172.21.15.120:0/3684982041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 msgr2=0x561d07ca77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 0x561d07ca77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 -- 172.21.15.120:0/3684982041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07ca7d30 msgr2=0x561d07caa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07ca7d30 0x561d07caa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 -- 172.21.15.120:0/3684982041 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0795f650 con 0x561d07c9b480 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec06ffd640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 0x561d07ca77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec077fe640 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec000098b0 tx=0x7fec000098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34862a165652672e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec0c813640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 0x7febf40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7fec06ffd640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 0x7febf40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.649+0000 7febfdffb640 1 -- 172.21.15.120:0/3684982041 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00031cf0 con 0x561d07c9b480 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7febfdffb640 1 -- 172.21.15.120:0/3684982041 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec00036030 con 0x561d07c9b480 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7febfdffb640 1 -- 172.21.15.120:0/3684982041 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000042a0 con 0x561d07c9b480 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0c813640 1 -- 172.21.15.120:0/529378343 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7febf80e5b50 con 0x7fec08101cb0 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 -- 172.21.15.120:0/3684982041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 msgr2=0x561d07c9b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec000098b0 tx=0x7fec000098e0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec06ffd640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 0x7febf40b77a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9c0 tx=0x7febf80cf040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 -- 172.21.15.120:0/3684982041 shutdown_connections 2024-09-10T22:55:42.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07ca7d30 0x561d07caa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9bda0 0x561d07ca77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3684982041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9b480 0x561d07c9b860 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 -- 172.21.15.120:0/3684982041 >> 172.21.15.120:0/3684982041 conn(0x561d07c992b0 msgr2=0x561d07c9a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 -- 172.21.15.120:0/3684982041 shutdown_connections 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.650+0000 7fec0fbea740 1 -- 172.21.15.120:0/3684982041 wait complete. 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 -- start start 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 Processor -- start 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9bda0 0x561d07ca4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07ca7d30 0x561d07ca5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d07a62700 con 0x561d07c9bda0 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d07b9c110 con 0x561d07c9b480 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d07caa7d0 con 0x561d07ca7d30 2024-09-10T22:55:42.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec0fbea740 10 client.0 _ll_register_callbacks cb 0x561d07bb5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35208/0 (socket says 172.21.15.120:35208) 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.651+0000 7fec077fe640 1 -- 172.21.15.120:0/1850068970 learned_addr learned my addr 172.21.15.120:0/1850068970 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07ca7d30 0x561d07ca5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7febfffff640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9bda0 0x561d07ca4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 -- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07ca7d30 msgr2=0x561d07ca5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07ca7d30 0x561d07ca5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 -- 172.21.15.120:0/1850068970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9bda0 msgr2=0x561d07ca4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9bda0 0x561d07ca4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 -- 172.21.15.120:0/1850068970 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d07bc9c40 con 0x561d07c9b480 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec077fe640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec00009620 tx=0x7fec00003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98377c5a64e60d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7febe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00031650 con 0x561d07c9b480 2024-09-10T22:55:42.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fec000317f0 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec00036030 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.652+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fec00004b40 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 10 client.25466 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561d07cb6140 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec00036030 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fec00043070 con 0x561d07c9b480 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 10 client.25466 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 20 client.25466 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07cb7490 0x561d07cb9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 4 client.25466 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:42.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d07c13da0 con 0x561d07cb7490 2024-09-10T22:55:42.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.653+0000 7febfffff640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07cb7490 0x561d07cb9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.654+0000 7febe6ffd640 10 client.25466 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:42.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.654+0000 7febfffff640 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07cb7490 0x561d07cb9880 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce11ac0e9c2a8b8e server_cookie=f18b59d8a1458bfb in_seq=0 out_seq=0 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.655+0000 7febe6ffd640 1 -- 172.21.15.120:0/1850068970 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d07c13da0 con 0x561d07cb7490 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.655+0000 7febe6ffd640 10 client.25466 handle_command_reply: tid=0 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.655+0000 7fec0fbea740 1 client.25466 shutdown 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 msgr2=0x561d07ca7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec00009620 tx=0x7fec00003e10 comp rx=0 tx=0).stop 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 shutdown_connections 2024-09-10T22:55:42.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07cb7490 0x561d07cb9880 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07ca7d30 0x561d07ca5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07c9bda0 0x561d07ca4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7febe6ffd640 0 client.25466 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 --2- 172.21.15.120:0/1850068970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9b480 0x561d07ca7680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 >> 172.21.15.120:0/1850068970 conn(0x561d07c992b0 msgr2=0x561d07c9ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 shutdown_connections 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 1 -- 172.21.15.120:0/1850068970 wait complete. 2024-09-10T22:55:42.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.656+0000 7fec0fbea740 20 client.25466 trim_cache size 0 max 16384 2024-09-10T22:55:42.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 Processor -- start 2024-09-10T22:55:42.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 -- start start 2024-09-10T22:55:42.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07c9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07cb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07ca9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec081ac1a0 con 0x561d07ca7d30 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec081aea20 con 0x561d07c9bda0 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec0fbea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec081ac810 con 0x561d07c9b480 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.676+0000 7fec077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07ca9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7fec077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07c9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07cb3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7fec077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07c9aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40774/0 (socket says 172.21.15.120:40774) 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7fec077fe640 1 -- 172.21.15.120:0/4065331066 learned_addr learned my addr 172.21.15.120:0/4065331066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 -- 172.21.15.120:0/4065331066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 msgr2=0x561d07c9aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07c9aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 -- 172.21.15.120:0/4065331066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 msgr2=0x561d07ca9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07ca9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 -- 172.21.15.120:0/4065331066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d07c9bbd0 con 0x561d07c9bda0 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfffff640 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07cb3a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x561d07ca58b0 tx=0x7febf0010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61eaf0e98732ccae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfd7fa640 1 -- 172.21.15.120:0/4065331066 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000fbd0 con 0x561d07c9bda0 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfd7fa640 1 -- 172.21.15.120:0/4065331066 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf000fd70 con 0x561d07c9bda0 2024-09-10T22:55:42.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.677+0000 7febfd7fa640 1 -- 172.21.15.120:0/4065331066 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004da0 con 0x561d07c9bda0 2024-09-10T22:55:42.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- 172.21.15.120:0/4065331066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 msgr2=0x561d07cb3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07cb3a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x561d07ca58b0 tx=0x7febf0010f50 comp rx=0 tx=0).stop 2024-09-10T22:55:42.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- 172.21.15.120:0/4065331066 shutdown_connections 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07ca9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07cb3a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 --2- 172.21.15.120:0/4065331066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07c9aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- 172.21.15.120:0/4065331066 >> 172.21.15.120:0/4065331066 conn(0x561d07c992b0 msgr2=0x561d07c9a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- 172.21.15.120:0/4065331066 shutdown_connections 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- 172.21.15.120:0/4065331066 wait complete. 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.678+0000 7fec0fbea740 1 -- start start 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 Processor -- start 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07caaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec081acee0 con 0x561d07ca7d30 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec081ae350 con 0x561d07c9bda0 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec081ad5b0 con 0x561d07c9b480 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec0fbea740 10 client.0 _ll_register_callbacks cb 0x7febf000aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7febfffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40784/0 (socket says 172.21.15.120:40784) 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 -- 172.21.15.120:0/3155337069 learned_addr learned my addr 172.21.15.120:0/3155337069 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7febfffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35218/0 (socket says 172.21.15.120:35218) 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07caaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 -- 172.21.15.120:0/3155337069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 msgr2=0x561d07caffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 -- 172.21.15.120:0/3155337069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 msgr2=0x561d07caaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07caaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7fec077fe640 1 -- 172.21.15.120:0/3155337069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d07caafe0 con 0x561d07c9b480 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.679+0000 7febfffff640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec077fe640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec00009ca0 tx=0x7fec00033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc5ab47f7c9b9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00031660 con 0x561d07c9b480 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf000fd70 con 0x561d07c9b480 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fec00031800 con 0x561d07c9b480 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fec0002f660 con 0x561d07c9b480 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 10 client.25480 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:42.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561d07d28d00 con 0x561d07c9b480 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf000fd70 con 0x561d07c9b480 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fec00043070 con 0x561d07c9b480 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 10 client.25480 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 20 client.25480 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07d293c0 0x561d07d2b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 4 client.25480 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.680+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d07c0f850 con 0x561d07d293c0 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7febfffff640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07d293c0 0x561d07d2b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:42.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7febfffff640 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07d293c0 0x561d07d2b7e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbadd36d737397fa server_cookie=53b573fa8cce125f in_seq=0 out_seq=0 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7febe6ffd640 10 client.25480 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7febe6ffd640 1 -- 172.21.15.120:0/3155337069 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x561d07c0f850 con 0x561d07d293c0 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7febe6ffd640 10 client.25480 handle_command_reply: tid=0 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.681+0000 7fec0fbea740 1 client.25480 shutdown 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 msgr2=0x561d07cafa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec00009ca0 tx=0x7fec00033040 comp rx=0 tx=0).stop 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 shutdown_connections 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d07d293c0 0x561d07d2b7e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d07ca7d30 0x561d07caaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d07c9bda0 0x561d07caffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 --2- 172.21.15.120:0/3155337069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d07c9b480 0x561d07cafa70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 >> 172.21.15.120:0/3155337069 conn(0x561d07c992b0 msgr2=0x561d07c97bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7febe6ffd640 0 client.25480 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:42.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 shutdown_connections 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 1 -- 172.21.15.120:0/3155337069 wait complete. 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.682+0000 7fec0fbea740 20 client.25480 trim_cache size 0 max 16384 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "13b316f4-2fb8-4178-b82e-6e02b551a316", 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:55:42.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:55:42.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:55:42.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:55:42.695 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:55:42.696 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:55:42.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 msgr2=0x7febf40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 0x7febf40b77a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9c0 tx=0x7febf80cf040 comp rx=0 tx=0).stop 2024-09-10T22:55:42.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 msgr2=0x7fec081991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:42.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec081991a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febf800e9f0 tx=0x7febf800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 shutdown_connections 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7febf40b52e0 0x7febf40b77a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec08108040 0x7fec0819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec081076c0 0x7fec081996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 --2- 172.21.15.120:0/529378343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec08101cb0 0x7fec081991a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 >> 172.21.15.120:0/529378343 conn(0x7fec080fd700 msgr2=0x7fec08100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 shutdown_connections 2024-09-10T22:55:42.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:55:42.683+0000 7fec0d815640 1 -- 172.21.15.120:0/529378343 wait complete. 2024-09-10T22:55:43.592 INFO:tasks.cephfs.filesystem:scrub status for tag:13b316f4-2fb8-4178-b82e-6e02b551a316 - {'path': '/', 'tag': '13b316f4-2fb8-4178-b82e-6e02b551a316', 'options': 'recursive,force'} 2024-09-10T22:55:43.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 509217 17.43 MB/sec execute 38 sec latency 19.881 ms 2024-09-10T22:55:44.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:43 smithi120 ceph-mon[31247]: pgmap v443: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-09-10T22:55:44.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:43 smithi153 ceph-mon[34951]: pgmap v443: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-09-10T22:55:44.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:43 smithi176 ceph-mon[34982]: pgmap v443: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-09-10T22:55:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3405285989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 msgr2=0x7f8b84103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3405285989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84103f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009930 tx=0x7f8b7402a420 comp rx=0 tx=0).stop 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3405285989 shutdown_connections 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3405285989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b841044a0 0x7f8b8410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3405285989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84103f60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3405285989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b84102990 0x7f8b84102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3405285989 >> 172.21.15.120:0/3405285989 conn(0x7f8b840fe250 msgr2=0x7f8b84100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.524+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3405285989 shutdown_connections 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3405285989 wait complete. 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 Processor -- start 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 -- start start 2024-09-10T22:55:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 0x7f8b84198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b841044a0 0x7f8b8419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8410e600 con 0x7f8b84103ae0 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8410e480 con 0x7f8b84102990 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b8b88c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8410e780 con 0x7f8b841044a0 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b88e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b88e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58816/0 (socket says 172.21.15.120:58816) 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b88e27640 1 -- 172.21.15.120:0/3950800393 learned_addr learned my addr 172.21.15.120:0/3950800393 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.525+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b841044a0 0x7f8b8419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b89628640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 0x7f8b84198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 -- 172.21.15.120:0/3950800393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b841044a0 msgr2=0x7f8b8419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b841044a0 0x7f8b8419a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 -- 172.21.15.120:0/3950800393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 msgr2=0x7f8b84198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 0x7f8b84198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 -- 172.21.15.120:0/3950800393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8419a920 con 0x7f8b84103ae0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b89628640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 0x7f8b84198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009a50 tx=0x7f8b740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b7a7fc640 1 -- 172.21.15.120:0/3950800393 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7403f030 con 0x7f8b84103ae0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b7a7fc640 1 -- 172.21.15.120:0/3950800393 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b74003cb0 con 0x7f8b84103ae0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8419abb0 con 0x7f8b84103ae0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b7a7fc640 1 -- 172.21.15.120:0/3950800393 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74035070 con 0x7f8b84103ae0 2024-09-10T22:55:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.526+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841a12e0 con 0x7f8b84103ae0 2024-09-10T22:55:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.537+0000 7f8b8e65f740 1 Processor -- start 2024-09-10T22:55:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.537+0000 7f8b8e65f740 1 -- start start 2024-09-10T22:55:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937350c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b89628640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b89628640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58824/0 (socket says 172.21.15.120:58824) 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b89628640 1 -- 172.21.15.120:0/372672679 learned_addr learned my addr 172.21.15.120:0/372672679 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a9373560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9372af6f0 con 0x55a9373511a0 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a93729eb20 con 0x55a937353be0 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.538+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a9372a2930 con 0x55a93734e870 2024-09-10T22:55:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.539+0000 7f8b7a7fc640 1 -- 172.21.15.120:0/3950800393 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8b7403bb90 con 0x7f8b84103ae0 2024-09-10T22:55:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.540+0000 7f8b89e29640 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937350c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.540+0000 7f8b7a7fc640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 0x7f8b700b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.540+0000 7f8b7a7fc640 1 -- 172.21.15.120:0/3950800393 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8b741067d0 con 0x7f8b84103ae0 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 -- 172.21.15.120:0/372672679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 msgr2=0x55a9373560a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a9373560a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 -- 172.21.15.120:0/372672679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 msgr2=0x55a937353590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 -- 172.21.15.120:0/372672679 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a937264530 con 0x55a93734e870 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 0x7f8b700b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89e29640 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937350c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005fd0 tx=0x7f8b7c0108d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad0435312a8b536 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.541+0000 7f8b89628640 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:55:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.542+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 0x7f8b700b7970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c023000 tx=0x7f8b7c01e730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.544+0000 7f8b6f7fe640 1 -- 172.21.15.120:0/372672679 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c026070 con 0x55a93734e870 2024-09-10T22:55:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.545+0000 7f8b6f7fe640 1 -- 172.21.15.120:0/372672679 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b7c020990 con 0x55a93734e870 2024-09-10T22:55:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.545+0000 7f8b6f7fe640 1 -- 172.21.15.120:0/372672679 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c01fac0 con 0x55a93734e870 2024-09-10T22:55:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.548+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 msgr2=0x55a937350c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.549+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937350c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005fd0 tx=0x7f8b7c0108d0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 shutdown_connections 2024-09-10T22:55:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a9373560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937353590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/372672679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937350c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 >> 172.21.15.120:0/372672679 conn(0x55a9373389b0 msgr2=0x55a937339740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.550+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 shutdown_connections 2024-09-10T22:55:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.551+0000 7f8b8e65f740 1 -- 172.21.15.120:0/372672679 wait complete. 2024-09-10T22:55:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 -- start start 2024-09-10T22:55:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 Processor -- start 2024-09-10T22:55:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a93734e870 0x55a937357fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a9373511a0 0x55a937358660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a9371d6ab0 con 0x55a937353be0 2024-09-10T22:55:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a937244730 con 0x55a93734e870 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a937356bc0 con 0x55a9373511a0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.552+0000 7f8b8e65f740 10 client.0 _ll_register_callbacks cb 0x55a9372986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58826/0 (socket says 172.21.15.120:58826) 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 -- 172.21.15.120:0/3983403927 learned_addr learned my addr 172.21.15.120:0/3983403927 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a9373511a0 0x55a937358660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89628640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a93734e870 0x55a937357fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 -- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a9373511a0 msgr2=0x55a937358660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a9373511a0 0x55a937358660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 -- 172.21.15.120:0/3983403927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a93734e870 msgr2=0x55a937357fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a93734e870 0x55a937357fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.553+0000 7f8b89e29640 1 -- 172.21.15.120:0/3983403927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a937000790 con 0x55a937353be0 2024-09-10T22:55:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c01f400 tx=0x7f8b7c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c294463c1ffc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c038060 con 0x55a937353be0 2024-09-10T22:55:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b7c020990 con 0x55a937353be0 2024-09-10T22:55:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8b7c005b70 con 0x55a937353be0 2024-09-10T22:55:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b7c026040 con 0x55a937353be0 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b8e65f740 10 client.16017 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.554+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a937079620 con 0x55a937353be0 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b7c020990 con 0x55a937353be0 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x55a937353be0 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 10 client.16017 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 20 client.16017 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a937365a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 4 client.16017 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.555+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9372b1640 con 0x55a9373635b0 2024-09-10T22:55:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.557+0000 7f8b89628640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a937365a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.560+0000 7f8b89628640 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a937365a00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fdb9d3876f0b458 server_cookie=f77cac74564d472c in_seq=0 out_seq=0 2024-09-10T22:55:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.560+0000 7f8b6cff9640 10 client.16017 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.562+0000 7f8b6cff9640 1 -- 172.21.15.120:0/3983403927 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9372b1640 con 0x55a9373635b0 2024-09-10T22:55:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.562+0000 7f8b6cff9640 10 client.16017 handle_command_reply: tid=0 2024-09-10T22:55:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.562+0000 7f8b8e65f740 1 client.16017 shutdown 2024-09-10T22:55:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.562+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 msgr2=0x55a937359230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.562+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c01f400 tx=0x7f8b7c00b000 comp rx=0 tx=0).stop 2024-09-10T22:55:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 shutdown_connections 2024-09-10T22:55:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a937365a00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a937359230 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b6cff9640 0 client.16017 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a9373511a0 0x55a937358660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3983403927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a93734e870 0x55a937357fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 >> 172.21.15.120:0/3983403927 conn(0x55a9373389b0 msgr2=0x55a937337bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 shutdown_connections 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3983403927 wait complete. 2024-09-10T22:55:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.563+0000 7f8b8e65f740 20 client.16017 trim_cache size 0 max 16384 2024-09-10T22:55:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 Processor -- start 2024-09-10T22:55:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 -- start start 2024-09-10T22:55:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937354010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a937339210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b840734c0 con 0x55a9373511a0 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84075d40 con 0x55a937353be0 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.583+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84073b30 con 0x55a93734e870 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b89628640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937354010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58842/0 (socket says 172.21.15.120:58842) 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 -- 172.21.15.120:0/3444800996 learned_addr learned my addr 172.21.15.120:0/3444800996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a937339210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 -- 172.21.15.120:0/3444800996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 msgr2=0x55a937354010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937354010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 -- 172.21.15.120:0/3444800996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 msgr2=0x55a937339210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a937339210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 -- 172.21.15.120:0/3444800996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a937366030 con 0x55a9373511a0 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b89e29640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a937339210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b89628640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937354010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b88e27640 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8b740c9530 tx=0x7f8b740cc8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d765b5bf4fdfa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b6effd640 1 -- 172.21.15.120:0/3444800996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74049040 con 0x55a9373511a0 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b6effd640 1 -- 172.21.15.120:0/3444800996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b74003e60 con 0x55a9373511a0 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.584+0000 7f8b6effd640 1 -- 172.21.15.120:0/3444800996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740d1a40 con 0x55a9373511a0 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3444800996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 msgr2=0x55a937354550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8b740c9530 tx=0x7f8b740cc8b0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3444800996 shutdown_connections 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a937353be0 0x55a937339210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9373511a0 0x55a937354550 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/3444800996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a937354010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3444800996 >> 172.21.15.120:0/3444800996 conn(0x55a9373389b0 msgr2=0x55a937356f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3444800996 shutdown_connections 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.585+0000 7f8b8e65f740 1 -- 172.21.15.120:0/3444800996 wait complete. 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 -- start start 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 Processor -- start 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a93735f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9373511a0 0x55a93735e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84074200 con 0x55a937353be0 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84075670 con 0x55a9373511a0 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8b840748d0 con 0x55a93734e870 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b8e65f740 10 client.0 _ll_register_callbacks cb 0x7f8b7c024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b89e29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b89628640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a93735f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b89e29640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58852/0 (socket says 172.21.15.120:58852) 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b89628640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a93735f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53442/0 (socket says 172.21.15.120:53442) 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b88e27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9373511a0 0x55a93735e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.586+0000 7f8b89e29640 1 -- 172.21.15.120:0/362265080 learned_addr learned my addr 172.21.15.120:0/362265080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 -- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 msgr2=0x55a93735f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a93735f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 -- 172.21.15.120:0/362265080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9373511a0 msgr2=0x55a93735e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9373511a0 0x55a93735e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 -- 172.21.15.120:0/362265080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93735ef50 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b89e29640 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c020570 tx=0x7f8b7c004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf603a602fbc1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c043030 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8b7c030040 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9372ad0f0 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b7c038040 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 10 client.16023 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a93735e060 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9372ad0f0 con 0x55a937353be0 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8b7c020c10 con 0x55a937353be0 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 10 client.16023 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.587+0000 7f8b8e65f740 20 client.16023 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a9373d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b8e65f740 4 client.16023 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9372aa520 con 0x55a9373635b0 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b89628640 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a9373d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b6cff9640 10 client.16023 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.588+0000 7f8b89628640 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a9373d2ca0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e741d8378e4c58 server_cookie=d2c6cf322981e325 in_seq=0 out_seq=0 2024-09-10T22:55:44.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 512688 17.47 MB/sec execute 39 sec latency 29.400 ms 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b6cff9640 1 -- 172.21.15.120:0/362265080 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9372aa520 con 0x55a9373635b0 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b6cff9640 10 client.16023 handle_command_reply: tid=0 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 client.16023 shutdown 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 msgr2=0x55a93735ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c020570 tx=0x7f8b7c004a00 comp rx=0 tx=0).stop 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 shutdown_connections 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a9373635b0 0x55a9373d2ca0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b6cff9640 0 client.16023 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a937353be0 0x55a93735ea10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a9373511a0 0x55a93735e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 --2- 172.21.15.120:0/362265080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a93734e870 0x55a93735f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 >> 172.21.15.120:0/362265080 conn(0x55a9373389b0 msgr2=0x55a937339210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 shutdown_connections 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 1 -- 172.21.15.120:0/362265080 wait complete. 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.893+0000 7f8b8e65f740 20 client.16023 trim_cache size 0 max 16384 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:55:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:55:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 msgr2=0x7f8b700b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 0x7f8b700b7970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c023000 tx=0x7f8b7c01e730 comp rx=0 tx=0).stop 2024-09-10T22:55:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 msgr2=0x7f8b84199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009a50 tx=0x7f8b740042f0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 shutdown_connections 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8b700b54b0 0x7f8b700b7970 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b841044a0 0x7f8b8419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b84103ae0 0x7f8b84199430 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 --2- 172.21.15.120:0/3950800393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b84102990 0x7f8b84198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 >> 172.21.15.120:0/3950800393 conn(0x7f8b840fe250 msgr2=0x7f8b84104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 shutdown_connections 2024-09-10T22:55:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:44.894+0000 7f8b8b88c640 1 -- 172.21.15.120:0/3950800393 wait complete. 2024-09-10T22:55:45.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 515551 17.46 MB/sec execute 40 sec latency 119.804 ms 2024-09-10T22:55:46.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:45 smithi120 ceph-mon[31247]: pgmap v444: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T22:55:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:45 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:55:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:45 smithi153 ceph-mon[34951]: pgmap v444: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T22:55:46.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:45 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:55:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:45 smithi176 ceph-mon[34982]: pgmap v444: 97 pgs: 97 active+clean; 33 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 4.2 MiB/s wr, 454 op/s 2024-09-10T22:55:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:45 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:55:46.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 519103 17.50 MB/sec execute 41 sec latency 19.659 ms 2024-09-10T22:55:47.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 522672 17.54 MB/sec execute 42 sec latency 16.675 ms 2024-09-10T22:55:48.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:47 smithi120 ceph-mon[31247]: pgmap v445: 97 pgs: 97 active+clean; 38 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T22:55:48.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:47 smithi153 ceph-mon[34951]: pgmap v445: 97 pgs: 97 active+clean; 38 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T22:55:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:47 smithi176 ceph-mon[34982]: pgmap v445: 97 pgs: 97 active+clean; 38 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-09-10T22:55:48.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 526218 17.58 MB/sec execute 43 sec latency 27.636 ms 2024-09-10T22:55:49.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 529725 17.61 MB/sec execute 44 sec latency 12.578 ms 2024-09-10T22:55:50.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:49 smithi120 ceph-mon[31247]: pgmap v446: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 442 op/s 2024-09-10T22:55:50.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:49 smithi153 ceph-mon[34951]: pgmap v446: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 442 op/s 2024-09-10T22:55:50.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:49 smithi176 ceph-mon[34982]: pgmap v446: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 442 op/s 2024-09-10T22:55:50.619 INFO:tasks.workunit.client.0.smithi120.stdout: 1 532874 17.61 MB/sec execute 45 sec latency 104.945 ms 2024-09-10T22:55:51.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 536075 17.55 MB/sec execute 46 sec latency 22.433 ms 2024-09-10T22:55:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:51 smithi120 ceph-mon[31247]: pgmap v447: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-09-10T22:55:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:51 smithi153 ceph-mon[34951]: pgmap v447: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-09-10T22:55:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:51 smithi176 ceph-mon[34982]: pgmap v447: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-09-10T22:55:52.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 539358 17.57 MB/sec execute 47 sec latency 13.971 ms 2024-09-10T22:55:53.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 542857 17.58 MB/sec execute 48 sec latency 13.897 ms 2024-09-10T22:55:54.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:53 smithi120 ceph-mon[31247]: pgmap v448: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-09-10T22:55:54.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:53 smithi153 ceph-mon[34951]: pgmap v448: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-09-10T22:55:54.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:53 smithi176 ceph-mon[34982]: pgmap v448: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-09-10T22:55:54.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 546011 17.54 MB/sec execute 49 sec latency 22.406 ms 2024-09-10T22:55:55.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 548480 17.51 MB/sec execute 50 sec latency 81.581 ms 2024-09-10T22:55:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:55 smithi120 ceph-mon[31247]: pgmap v449: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:55:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:55 smithi153 ceph-mon[34951]: pgmap v449: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:55:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:55 smithi176 ceph-mon[34982]: pgmap v449: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:55:56.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 551355 17.45 MB/sec execute 51 sec latency 29.258 ms 2024-09-10T22:55:57.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 554750 17.48 MB/sec execute 52 sec latency 28.605 ms 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 -- 172.21.15.120:0/2866209289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9341029b0 msgr2=0x7fe934102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/2866209289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9341029b0 0x7fe934102e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe924009930 tx=0x7fe92402a420 comp rx=0 tx=0).stop 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 -- 172.21.15.120:0/2866209289 shutdown_connections 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/2866209289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 0x7fe934109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/2866209289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9341029b0 0x7fe934102e30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/2866209289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe934101860 0x7fe934101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.987+0000 7fe93bb5d640 1 -- 172.21.15.120:0/2866209289 >> 172.21.15.120:0/2866209289 conn(0x7fe9340fd140 msgr2=0x7fe9340ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- 172.21.15.120:0/2866209289 shutdown_connections 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- 172.21.15.120:0/2866209289 wait complete. 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 Processor -- start 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- start start 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9341029b0 0x7fe9341982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 0x7fe934198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410d4d0 con 0x7fe934101860 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410d350 con 0x7fe9341029b0 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe93bb5d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410d650 con 0x7fe934103370 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe9398f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe9398f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33556/0 (socket says 172.21.15.120:33556) 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.988+0000 7fe9398f9640 1 -- 172.21.15.120:0/3682435371 learned_addr learned my addr 172.21.15.120:0/3682435371 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 0x7fe934198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9390f8640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9341029b0 0x7fe9341982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 -- 172.21.15.120:0/3682435371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 msgr2=0x7fe934198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 0x7fe934198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 -- 172.21.15.120:0/3682435371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9341029b0 msgr2=0x7fe9341982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9341029b0 0x7fe9341982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 -- 172.21.15.120:0/3682435371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9341997d0 con 0x7fe934101860 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe9398f9640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe93000d950 tx=0x7fe93000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe92affd640 1 -- 172.21.15.120:0/3682435371 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930017070 con 0x7fe934101860 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe92affd640 1 -- 172.21.15.120:0/3682435371 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930009c30 con 0x7fe934101860 2024-09-10T22:55:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe934199ac0 con 0x7fe934101860 2024-09-10T22:55:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe92affd640 1 -- 172.21.15.120:0/3682435371 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93000f620 con 0x7fe934101860 2024-09-10T22:55:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.989+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a0190 con 0x7fe934101860 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 Processor -- start 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 -- start start 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e93a0 0x55a8842eb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a8842ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a8842566f0 con 0x55a8842e6a70 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a884245b20 con 0x55a8842ebde0 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a884249930 con 0x55a8842e93a0 2024-09-10T22:55:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe92affd640 1 -- 172.21.15.120:0/3682435371 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe930010460 con 0x7fe934101860 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93a0fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93a0fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33572/0 (socket says 172.21.15.120:33572) 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe93a0fa640 1 -- 172.21.15.120:0/4004777491 learned_addr learned my addr 172.21.15.120:0/4004777491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe9398f9640 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e93a0 0x55a8842eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.992+0000 7fe9390f8640 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a8842ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 -- 172.21.15.120:0/4004777491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e93a0 msgr2=0x55a8842eb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e93a0 0x55a8842eb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 -- 172.21.15.120:0/4004777491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 msgr2=0x55a8842ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a8842ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 -- 172.21.15.120:0/4004777491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88420b530 con 0x55a8842e6a70 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009f90 tx=0x7fe92c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecdf5876c30102a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe91ffff640 1 -- 172.21.15.120:0/4004777491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c004d20 con 0x55a8842e6a70 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe91ffff640 1 -- 172.21.15.120:0/4004777491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c004050 con 0x55a8842e6a70 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe91ffff640 1 -- 172.21.15.120:0/4004777491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c005280 con 0x55a8842e6a70 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe92affd640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe9398f9640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.993+0000 7fe92affd640 1 -- 172.21.15.120:0/3682435371 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fe9300e57a0 con 0x7fe934101860 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- 172.21.15.120:0/4004777491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 msgr2=0x55a8842e8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe9398f9640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe9300058c0 tx=0x7fe9300eb530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009f90 tx=0x7fe92c0118d0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- 172.21.15.120:0/4004777491 shutdown_connections 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a8842ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e93a0 0x55a8842eb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/4004777491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e8e60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- 172.21.15.120:0/4004777491 >> 172.21.15.120:0/4004777491 conn(0x55a8842df9b0 msgr2=0x55a8842e0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- 172.21.15.120:0/4004777491 shutdown_connections 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- 172.21.15.120:0/4004777491 wait complete. 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.994+0000 7fe93ea0c740 1 -- start start 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 Processor -- start 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 0x55a8842f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a88417dab0 con 0x55a8842e93a0 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a8841eb730 con 0x55a8842e6a70 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a8842eedc0 con 0x55a8842ebde0 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93ea0c740 10 client.0 _ll_register_callbacks cb 0x55a88423f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33582/0 (socket says 172.21.15.120:33582) 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 -- 172.21.15.120:0/155460537 learned_addr learned my addr 172.21.15.120:0/155460537 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 0x55a8842f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 -- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 msgr2=0x55a8842f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 -- 172.21.15.120:0/155460537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 msgr2=0x55a8842f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 0x55a8842f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe9390f8640 1 -- 172.21.15.120:0/155460537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a883fa7790 con 0x55a8842e93a0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.995+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 0x55a8842f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe9390f8640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcf4a39f0eaf93f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035040 con 0x55a8842e93a0 2024-09-10T22:55:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fe924004240 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c004050 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe92404d070 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 10 client.16035 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a884020620 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c004050 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fe924044020 con 0x55a8842e93a0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 10 client.16035 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 20 client.16035 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a8842fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 4 client.16035 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.996+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a884258640 con 0x55a8842fb8e0 2024-09-10T22:55:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.997+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a8842fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.997+0000 7fe91d7fa640 10 client.16035 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.997+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a8842fdcf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8c0a1b33aa7231 server_cookie=3eacd1190cf54176 in_seq=0 out_seq=0 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe91d7fa640 1 -- 172.21.15.120:0/155460537 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a884258640 con 0x55a8842fb8e0 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe91d7fa640 10 client.16035 handle_command_reply: tid=0 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 client.16035 shutdown 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 msgr2=0x55a8842f06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924031e80 comp rx=0 tx=0).stop 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 shutdown_connections 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a8842fdcf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe91d7fa640 0 client.16035 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a8842f06f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/155460537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e6a70 0x55a8842f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 >> 172.21.15.120:0/155460537 conn(0x55a8842df9b0 msgr2=0x55a8842debf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 shutdown_connections 2024-09-10T22:55:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 1 -- 172.21.15.120:0/155460537 wait complete. 2024-09-10T22:55:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:57.998+0000 7fe93ea0c740 20 client.16035 trim_cache size 0 max 16384 2024-09-10T22:55:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 Processor -- start 2024-09-10T22:55:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 -- start start 2024-09-10T22:55:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e93a0 0x55a8842f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a5c30 con 0x55a8842e6a70 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a84b0 con 0x55a8842e93a0 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a62a0 con 0x55a8842ebde0 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33594/0 (socket says 172.21.15.120:33594) 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 -- 172.21.15.120:0/1224115878 learned_addr learned my addr 172.21.15.120:0/1224115878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe9390f8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e93a0 0x55a8842f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 -- 172.21.15.120:0/1224115878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 msgr2=0x55a8842f0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 -- 172.21.15.120:0/1224115878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e93a0 msgr2=0x55a8842f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e93a0 0x55a8842f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 -- 172.21.15.120:0/1224115878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8842e91d0 con 0x55a8842e6a70 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009700 tx=0x7fe92c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914d15d3f2762bde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe91f7fe640 1 -- 172.21.15.120:0/1224115878 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c009c10 con 0x55a8842e6a70 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe91f7fe640 1 -- 172.21.15.120:0/1224115878 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c0049c0 con 0x55a8842e6a70 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.013+0000 7fe91f7fe640 1 -- 172.21.15.120:0/1224115878 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c005540 con 0x55a8842e6a70 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- 172.21.15.120:0/1224115878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 msgr2=0x55a8842e6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009700 tx=0x7fe92c003b70 comp rx=0 tx=0).stop 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- 172.21.15.120:0/1224115878 shutdown_connections 2024-09-10T22:55:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842ebde0 0x55a8842f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842e93a0 0x55a8842f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/1224115878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e6a70 0x55a8842e6490 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- 172.21.15.120:0/1224115878 >> 172.21.15.120:0/1224115878 conn(0x55a8842df9b0 msgr2=0x55a8842ec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- 172.21.15.120:0/1224115878 shutdown_connections 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- 172.21.15.120:0/1224115878 wait complete. 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.014+0000 7fe93ea0c740 1 -- start start 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 Processor -- start 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 0x55a884303f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a884368f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a6970 con 0x55a8842e93a0 2024-09-10T22:55:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a7de0 con 0x55a8842ebde0 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a7040 con 0x55a8842e6a70 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93ea0c740 10 client.0 _ll_register_callbacks cb 0x7fe924009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33604/0 (socket says 172.21.15.120:33604) 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 -- 172.21.15.120:0/3483689583 learned_addr learned my addr 172.21.15.120:0/3483689583 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 0x55a884303f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a884368f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 -- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 msgr2=0x55a884303f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 0x55a884303f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 -- 172.21.15.120:0/3483689583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 msgr2=0x55a884368f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a884368f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 -- 172.21.15.120:0/3483689583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a884369760 con 0x55a8842e93a0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 0x55a884303f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe9390f8640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe924003f50 tx=0x7fe924031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3792123ecf903f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.015+0000 7fe91d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924056020 con 0x55a8842e93a0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7fe924046c20 con 0x55a8842e93a0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c0049c0 con 0x55a8842e93a0 2024-09-10T22:55:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe924035070 con 0x55a8842e93a0 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 10 client.16041 fetch_fsmap learned FSMap version 23 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a88436ab10 con 0x55a8842e93a0 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c0049c0 con 0x55a8842e93a0 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fe924044040 con 0x55a8842e93a0 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 10 client.16041 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 20 client.16041 populate_metadata read hostname 'smithi120' 2024-09-10T22:55:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a88436d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 4 client.16041 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8842540f0 con 0x55a8842fb8e0 2024-09-10T22:55:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.016+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a88436d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.017+0000 7fe93a0fa640 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a88436d5b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fdb1e76b5eeca43 server_cookie=964819fb56259ae5 in_seq=0 out_seq=0 2024-09-10T22:55:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.017+0000 7fe91d7fa640 10 client.16041 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:57 smithi120 ceph-mon[31247]: pgmap v450: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 475 op/s 2024-09-10T22:55:58.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:57 smithi153 ceph-mon[34951]: pgmap v450: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 475 op/s 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe91d7fa640 1 -- 172.21.15.120:0/3483689583 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8842540f0 con 0x55a8842fb8e0 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe91d7fa640 10 client.16041 handle_command_reply: tid=0 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 client.16041 shutdown 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 msgr2=0x55a884366830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe924003f50 tx=0x7fe924031b20 comp rx=0 tx=0).stop 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 shutdown_connections 2024-09-10T22:55:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a8842fb8e0 0x55a88436d5b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a8842ebde0 0x55a884368f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe91d7fa640 0 client.16041 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a8842e93a0 0x55a884366830 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 --2- 172.21.15.120:0/3483689583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a8842e6a70 0x55a884303f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 >> 172.21.15.120:0/3483689583 conn(0x55a8842df9b0 msgr2=0x55a8842e5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 shutdown_connections 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.208+0000 7fe93ea0c740 1 -- 172.21.15.120:0/3483689583 wait complete. 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93ea0c740 20 client.16041 trim_cache size 0 max 16384 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:55:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:55:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:55:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 msgr2=0x7fe9200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe9300058c0 tx=0x7fe9300eb530 comp rx=0 tx=0).stop 2024-09-10T22:55:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 msgr2=0x7fe934197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe93000d950 tx=0x7fe93000de10 comp rx=0 tx=0).stop 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 shutdown_connections 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe9200b52e0 0x7fe9200b77a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe934103370 0x7fe934198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9341029b0 0x7fe9341982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 --2- 172.21.15.120:0/3682435371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe934101860 0x7fe934197c90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 >> 172.21.15.120:0/3682435371 conn(0x7fe9340fd140 msgr2=0x7fe934105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 shutdown_connections 2024-09-10T22:55:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:55:58.209+0000 7fe93bb5d640 1 -- 172.21.15.120:0/3682435371 wait complete. 2024-09-10T22:55:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:57 smithi176 ceph-mon[34982]: pgmap v450: 97 pgs: 97 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 475 op/s 2024-09-10T22:55:58.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 557934 17.51 MB/sec execute 53 sec latency 123.979 ms 2024-09-10T22:55:59.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 561343 17.51 MB/sec execute 54 sec latency 16.481 ms 2024-09-10T22:56:00.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:55:59 smithi120 ceph-mon[31247]: pgmap v451: 97 pgs: 97 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-10T22:56:00.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:55:59 smithi153 ceph-mon[34951]: pgmap v451: 97 pgs: 97 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-10T22:56:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:55:59 smithi176 ceph-mon[34982]: pgmap v451: 97 pgs: 97 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-10T22:56:00.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 563755 17.36 MB/sec execute 55 sec latency 57.379 ms 2024-09-10T22:56:01.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 566489 17.35 MB/sec execute 56 sec latency 24.695 ms 2024-09-10T22:56:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:01 smithi120 ceph-mon[31247]: pgmap v452: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 5.3 MiB/s wr, 427 op/s 2024-09-10T22:56:02.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:01 smithi153 ceph-mon[34951]: pgmap v452: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 5.3 MiB/s wr, 427 op/s 2024-09-10T22:56:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:01 smithi176 ceph-mon[34982]: pgmap v452: 97 pgs: 97 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 5.3 MiB/s wr, 427 op/s 2024-09-10T22:56:02.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 569582 17.33 MB/sec execute 57 sec latency 22.427 ms 2024-09-10T22:56:03.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 572831 17.33 MB/sec execute 58 sec latency 12.901 ms 2024-09-10T22:56:04.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:03 smithi120 ceph-mon[31247]: pgmap v453: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-10T22:56:04.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:03 smithi153 ceph-mon[34951]: pgmap v453: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-10T22:56:04.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:03 smithi176 ceph-mon[34982]: pgmap v453: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-09-10T22:56:04.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 576144 17.35 MB/sec execute 59 sec latency 14.573 ms 2024-09-10T22:56:05.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 579257 17.37 MB/sec execute 60 sec latency 75.065 ms 2024-09-10T22:56:06.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:05 smithi120 ceph-mon[31247]: pgmap v454: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-09-10T22:56:06.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:05 smithi153 ceph-mon[34951]: pgmap v454: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-09-10T22:56:06.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:05 smithi176 ceph-mon[34982]: pgmap v454: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-09-10T22:56:06.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 582629 17.39 MB/sec execute 61 sec latency 34.182 ms 2024-09-10T22:56:07.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 585903 17.36 MB/sec execute 62 sec latency 23.001 ms 2024-09-10T22:56:08.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:07 smithi120 ceph-mon[31247]: pgmap v455: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-10T22:56:08.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:07 smithi153 ceph-mon[34951]: pgmap v455: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-10T22:56:08.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:07 smithi176 ceph-mon[34982]: pgmap v455: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-10T22:56:08.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 589282 17.37 MB/sec execute 63 sec latency 12.267 ms 2024-09-10T22:56:09.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 592498 17.37 MB/sec execute 64 sec latency 15.260 ms 2024-09-10T22:56:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:09 smithi120 ceph-mon[31247]: pgmap v456: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 448 op/s 2024-09-10T22:56:10.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:09 smithi153 ceph-mon[34951]: pgmap v456: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 448 op/s 2024-09-10T22:56:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:09 smithi176 ceph-mon[34982]: pgmap v456: 97 pgs: 97 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 448 op/s 2024-09-10T22:56:10.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 594450 17.25 MB/sec execute 65 sec latency 91.117 ms 2024-09-10T22:56:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- 172.21.15.120:0/4122010354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954103370 msgr2=0x7f0954109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 --2- 172.21.15.120:0/4122010354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954103370 0x7f0954109cf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f094c009cd0 tx=0x7f094c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- 172.21.15.120:0/4122010354 shutdown_connections 2024-09-10T22:56:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 --2- 172.21.15.120:0/4122010354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954103370 0x7f0954109cf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 --2- 172.21.15.120:0/4122010354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09541029b0 0x7f0954102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 --2- 172.21.15.120:0/4122010354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0954101860 0x7f0954101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- 172.21.15.120:0/4122010354 >> 172.21.15.120:0/4122010354 conn(0x7f09540fd140 msgr2=0x7f09540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- 172.21.15.120:0/4122010354 shutdown_connections 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- 172.21.15.120:0/4122010354 wait complete. 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 Processor -- start 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.298+0000 7f095a73d640 1 -- start start 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541029b0 0x7f095419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0954103370 0x7f09541a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f095410d4b0 con 0x7f0954101860 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f095410d330 con 0x7f09541029b0 2024-09-10T22:56:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f095a73d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f095410d630 con 0x7f0954103370 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f09537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541029b0 0x7f095419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45882/0 (socket says 172.21.15.120:45882) 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 -- 172.21.15.120:0/2869458460 learned_addr learned my addr 172.21.15.120:0/2869458460 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0958cda640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0954103370 0x7f09541a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 -- 172.21.15.120:0/2869458460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0954103370 msgr2=0x7f09541a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0954103370 0x7f09541a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 -- 172.21.15.120:0/2869458460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541029b0 msgr2=0x7f095419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541029b0 0x7f095419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 -- 172.21.15.120:0/2869458460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09541a08d0 con 0x7f0954101860 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f0953fff640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f094400d950 tx=0x7f094400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.299+0000 7f09517fa640 1 -- 172.21.15.120:0/2869458460 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944017070 con 0x7f0954101860 2024-09-10T22:56:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.300+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09541a0bc0 con 0x7f0954101860 2024-09-10T22:56:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.300+0000 7f09517fa640 1 -- 172.21.15.120:0/2869458460 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0944009c30 con 0x7f0954101860 2024-09-10T22:56:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.300+0000 7f09517fa640 1 -- 172.21.15.120:0/2869458460 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094400f620 con 0x7f0954101860 2024-09-10T22:56:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.300+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541a1160 con 0x7f0954101860 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 Processor -- start 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 -- start start 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x562153750580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x562153753090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f09517fa640 1 -- 172.21.15.120:0/2869458460 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0944010460 con 0x7f0954101860 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621536bd6f0 con 0x56215374b860 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621536acb20 con 0x562153750bd0 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f095d5ec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621536b0930 con 0x56215374e190 2024-09-10T22:56:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f0958cda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f0958cda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45890/0 (socket says 172.21.15.120:45890) 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f0958cda640 1 -- 172.21.15.120:0/2097755989 learned_addr learned my addr 172.21.15.120:0/2097755989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.302+0000 7f0953fff640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x562153750580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f09537fe640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x562153753090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 -- 172.21.15.120:0/2097755989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 msgr2=0x562153750580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x562153750580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 -- 172.21.15.120:0/2097755989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 msgr2=0x562153753090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x562153753090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 -- 172.21.15.120:0/2097755989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562153672530 con 0x56215374b860 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0953fff640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x562153750580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f0958cda640 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f094c009cd0 tx=0x7f094c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6214be50f79dd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f093a7fc640 1 -- 172.21.15.120:0/2097755989 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0314f0 con 0x56215374b860 2024-09-10T22:56:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f093a7fc640 1 -- 172.21.15.120:0/2097755989 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c036040 con 0x56215374b860 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f093a7fc640 1 -- 172.21.15.120:0/2097755989 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0043b0 con 0x56215374b860 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.303+0000 7f09517fa640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 0x7f093c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f0953fff640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 0x7f093c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f09517fa640 1 -- 172.21.15.120:0/2869458460 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f09440e5780 con 0x7f0954101860 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f0953fff640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 0x7f093c0b77a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f094401d000 tx=0x7f094401a750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 -- 172.21.15.120:0/2097755989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 msgr2=0x56215374dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f094c009cd0 tx=0x7f094c009a20 comp rx=0 tx=0).stop 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 -- 172.21.15.120:0/2097755989 shutdown_connections 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x562153753090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x562153750580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2097755989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215374dc50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 -- 172.21.15.120:0/2097755989 >> 172.21.15.120:0/2097755989 conn(0x5621537469b0 msgr2=0x562153747740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 -- 172.21.15.120:0/2097755989 shutdown_connections 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.304+0000 7f095d5ec740 1 -- 172.21.15.120:0/2097755989 wait complete. 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 -- start start 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 Processor -- start 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153754e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153756090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621535e4ab0 con 0x56215374e190 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562153652730 con 0x56215374b860 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562153753af0 con 0x562153750bd0 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f095d5ec740 10 client.0 _ll_register_callbacks cb 0x5621536a66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f09537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f09537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45898/0 (socket says 172.21.15.120:45898) 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f0958cda640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153756090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f09537fe640 1 -- 172.21.15.120:0/1381362497 learned_addr learned my addr 172.21.15.120:0/1381362497 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.305+0000 7f0958cda640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153754e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 -- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 msgr2=0x562153756090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153756090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 -- 172.21.15.120:0/1381362497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 msgr2=0x562153754e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153754e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 -- 172.21.15.120:0/1381362497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215340e790 con 0x56215374e190 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f0958cda640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153756090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09537fe640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0948012680 tx=0x7f0948012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc18c20e4a92dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09337fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094800f920 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094c036040 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f094800fac0 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f094800fcc0 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f095d5ec740 10 client.16053 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562153487620 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.306+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094c036040 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f0948027420 con 0x56215374e190 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 10 client.16053 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 20 client.16053 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537607d0 0x562153762bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 4 client.16053 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621536bf640 con 0x5621537607d0 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f0958cda640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537607d0 0x562153762bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f0958cda640 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537607d0 0x562153762bc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf769a044951e11 server_cookie=7b6c16a87cd88d0d in_seq=0 out_seq=0 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.307+0000 7f09337fe640 10 client.16053 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f09337fe640 1 -- 172.21.15.120:0/1381362497 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5621536bf640 con 0x5621537607d0 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f09337fe640 10 client.16053 handle_command_reply: tid=0 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 client.16053 shutdown 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 msgr2=0x562153755530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0948012680 tx=0x7f0948012b40 comp rx=0 tx=0).stop 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 shutdown_connections 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537607d0 0x562153762bc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153756090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x562153755530 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 --2- 172.21.15.120:0/1381362497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153754e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f09337fe640 0 client.16053 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 >> 172.21.15.120:0/1381362497 conn(0x5621537469b0 msgr2=0x562153745bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 shutdown_connections 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 1 -- 172.21.15.120:0/1381362497 wait complete. 2024-09-10T22:56:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.309+0000 7f095d5ec740 20 client.16053 trim_cache size 0 max 16384 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.324+0000 7f095d5ec740 1 Processor -- start 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.324+0000 7f095d5ec740 1 -- start start 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153750fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153754e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a5be0 con 0x56215374e190 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a8460 con 0x56215374b860 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f095d5ec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a6250 con 0x562153750bd0 2024-09-10T22:56:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45912/0 (socket says 172.21.15.120:45912) 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 -- 172.21.15.120:0/2271538196 learned_addr learned my addr 172.21.15.120:0/2271538196 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0958cda640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153754e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0958cda640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153750fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 -- 172.21.15.120:0/2271538196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 msgr2=0x562153754e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153754e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 -- 172.21.15.120:0/2271538196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 msgr2=0x562153750fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153750fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 -- 172.21.15.120:0/2271538196 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215374dfc0 con 0x56215374e190 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0958cda640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153754e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f09537fe640 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c00 tx=0x7f094800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7216daf42ca7e3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0939ffb640 1 -- 172.21.15.120:0/2271538196 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094800fe60 con 0x56215374e190 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0939ffb640 1 -- 172.21.15.120:0/2271538196 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094802b070 con 0x56215374e190 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.325+0000 7f0939ffb640 1 -- 172.21.15.120:0/2271538196 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948027420 con 0x56215374e190 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- 172.21.15.120:0/2271538196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 msgr2=0x5621537514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c00 tx=0x7f094800f790 comp rx=0 tx=0).stop 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- 172.21.15.120:0/2271538196 shutdown_connections 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562153750bd0 0x562153754e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374e190 0x5621537514f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 --2- 172.21.15.120:0/2271538196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56215374b860 0x562153750fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- 172.21.15.120:0/2271538196 >> 172.21.15.120:0/2271538196 conn(0x5621537469b0 msgr2=0x562153741af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- 172.21.15.120:0/2271538196 shutdown_connections 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- 172.21.15.120:0/2271538196 wait complete. 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.326+0000 7f095d5ec740 1 -- start start 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 Processor -- start 2024-09-10T22:56:11.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x56215375b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x56215375bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a6920 con 0x56215374b860 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a7d90 con 0x562153750bd0 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f09541a6ff0 con 0x56215374e190 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 10 client.0 _ll_register_callbacks cb 0x7f09480097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45928/0 (socket says 172.21.15.120:45928) 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f09537fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x56215375b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 -- 172.21.15.120:0/3499908553 learned_addr learned my addr 172.21.15.120:0/3499908553 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x56215375bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 -- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 msgr2=0x56215375b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x56215375b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 -- 172.21.15.120:0/3499908553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 msgr2=0x56215375bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x56215375bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 -- 172.21.15.120:0/3499908553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562153756400 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f09537fe640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x56215375b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f0958cda640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f094c009960 tx=0x7f0948009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eec8ca77c5ac04a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f09337fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c031950 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f094c031af0 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.327+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094802b070 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f094c003970 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 10 client.16059 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5621537cd6d0 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094802b070 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f094c030070 con 0x56215374b860 2024-09-10T22:56:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 10 client.16059 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 20 client.16059 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537cdc40 0x5621537d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 4 client.16059 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621536bb0f0 con 0x5621537cdc40 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.328+0000 7f09537fe640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537cdc40 0x5621537d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.329+0000 7f09337fe640 10 client.16059 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.329+0000 7f09537fe640 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537cdc40 0x5621537d0030 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89131588e38bacc0 server_cookie=af5947215f37753d in_seq=0 out_seq=0 2024-09-10T22:56:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f09337fe640 1 -- 172.21.15.120:0/3499908553 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621536bb0f0 con 0x5621537cdc40 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f09337fe640 10 client.16059 handle_command_reply: tid=0 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 client.16059 shutdown 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 msgr2=0x56215375ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f094c009960 tx=0x7f0948009de0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 shutdown_connections 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5621537cdc40 0x5621537d0030 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562153750bd0 0x56215375bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f09337fe640 0 client.16059 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56215374e190 0x56215375b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 --2- 172.21.15.120:0/3499908553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56215374b860 0x56215375ca50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 >> 172.21.15.120:0/3499908553 conn(0x5621537469b0 msgr2=0x562153750fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 shutdown_connections 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 1 -- 172.21.15.120:0/3499908553 wait complete. 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.489+0000 7f095d5ec740 20 client.16059 trim_cache size 0 max 16384 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:56:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 msgr2=0x7f093c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 0x7f093c0b77a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f094401d000 tx=0x7f094401a750 comp rx=0 tx=0).stop 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 msgr2=0x7f095406b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f094400d950 tx=0x7f094400de10 comp rx=0 tx=0).stop 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 shutdown_connections 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f093c0b52e0 0x7f093c0b77a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0954103370 0x7f09541a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09541029b0 0x7f095419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 --2- 172.21.15.120:0/2869458460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0954101860 0x7f095406b180 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 >> 172.21.15.120:0/2869458460 conn(0x7f09540fd140 msgr2=0x7f0954105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 shutdown_connections 2024-09-10T22:56:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:11.490+0000 7f095a73d640 1 -- 172.21.15.120:0/2869458460 wait complete. 2024-09-10T22:56:11.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 597112 17.25 MB/sec execute 66 sec latency 68.541 ms 2024-09-10T22:56:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:11 smithi120 ceph-mon[31247]: pgmap v457: 97 pgs: 97 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-09-10T22:56:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:11 smithi153 ceph-mon[34951]: pgmap v457: 97 pgs: 97 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-09-10T22:56:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:11 smithi176 ceph-mon[34982]: pgmap v457: 97 pgs: 97 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-09-10T22:56:12.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 600526 17.27 MB/sec execute 67 sec latency 17.396 ms 2024-09-10T22:56:13.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:56:13.620 INFO:tasks.workunit.client.0.smithi120.stdout: 1 603872 17.28 MB/sec execute 68 sec latency 14.936 ms 2024-09-10T22:56:13.868 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:14.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:13 smithi120 ceph-mon[31247]: pgmap v458: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-09-10T22:56:14.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:13 smithi153 ceph-mon[34951]: pgmap v458: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-09-10T22:56:14.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:13 smithi176 ceph-mon[34982]: pgmap v458: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-09-10T22:56:14.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1241373965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810d2f0 msgr2=0x7f3c1810f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1241373965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810d2f0 0x7f3c1810f6e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ca0 tx=0x7f3c1002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1241373965 shutdown_connections 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1241373965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810d2f0 0x7f3c1810f6e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1241373965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810a9c0 0x7f3c1810cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1241373965 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 0x7f3c18101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1241373965 >> 172.21.15.120:0/1241373965 conn(0x7f3c18076e60 msgr2=0x7f3c18077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1241373965 shutdown_connections 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.481+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1241373965 wait complete. 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 Processor -- start 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 -- start start 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 0x7f3c181992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810d2f0 0x7f3c1819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18112db0 con 0x7f3c1810a9c0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18112c30 con 0x7f3c1810d2f0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.482+0000 7f3c1e6e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18112f30 con 0x7f3c181018b0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42310/0 (socket says 172.21.15.120:42310) 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1853077324 learned_addr learned my addr 172.21.15.120:0/1853077324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810d2f0 0x7f3c1819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c17fff640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 0x7f3c181992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1853077324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 msgr2=0x7f3c181992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 0x7f3c181992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1853077324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810d2f0 msgr2=0x7f3c1819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810d2f0 0x7f3c1819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1853077324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1819f350 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000e9c0 tx=0x7f3c0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c15ffb640 1 -- 172.21.15.120:0/1853077324 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000cd80 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c15ffb640 1 -- 172.21.15.120:0/1853077324 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c000054d0 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1819f640 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.483+0000 7f3c15ffb640 1 -- 172.21.15.120:0/1853077324 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00010430 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.484+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c18102760 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.486+0000 7f3c20abc740 1 Processor -- start 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.486+0000 7f3c20abc740 1 -- start start 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.486+0000 7f3c15ffb640 1 -- 172.21.15.120:0/1853077324 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3c00004fb0 con 0x7f3c1810a9c0 2024-09-10T22:56:14.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b46d30 0x55db15b49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db15ab0e50 con 0x55db15b3ada0 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db15aa0280 con 0x55db15b3a480 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c20abc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55db15aa4090 con 0x55db15b46d30 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42320/0 (socket says 172.21.15.120:42320) 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 -- 172.21.15.120:0/1592691890 learned_addr learned my addr 172.21.15.120:0/1592691890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b46d30 0x55db15b49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 -- 172.21.15.120:0/1592691890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b46d30 msgr2=0x55db15b49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b46d30 0x55db15b49120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 -- 172.21.15.120:0/1592691890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 msgr2=0x55db15b3a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b3a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 -- 172.21.15.120:0/1592691890 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db157fe650 con 0x55db15b3ada0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.487+0000 7f3c17fff640 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009d20 tx=0x7f3c08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d804cab3651fe075 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c0e7fc640 1 -- 172.21.15.120:0/1592691890 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801e070 con 0x55db15b3ada0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c0e7fc640 1 -- 172.21.15.120:0/1592691890 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08019ae0 con 0x55db15b3ada0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c0e7fc640 1 -- 172.21.15.120:0/1592691890 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800eac0 con 0x55db15b3ada0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c15ffb640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 0x7f3c040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 0x7f3c040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.488+0000 7f3c15ffb640 1 -- 172.21.15.120:0/1853077324 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3c000e5ff0 con 0x7f3c1810a9c0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 0x7f3c040b77a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c10005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- 172.21.15.120:0/1592691890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 msgr2=0x55db15b467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009d20 tx=0x7f3c08011810 comp rx=0 tx=0).stop 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- 172.21.15.120:0/1592691890 shutdown_connections 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b46d30 0x55db15b49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b3ada0 0x55db15b467f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1592691890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- 172.21.15.120:0/1592691890 >> 172.21.15.120:0/1592691890 conn(0x55db15b382b0 msgr2=0x55db15b39450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- 172.21.15.120:0/1592691890 shutdown_connections 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- 172.21.15.120:0/1592691890 wait complete. 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 -- start start 2024-09-10T22:56:14.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.489+0000 7f3c20abc740 1 Processor -- start 2024-09-10T22:56:14.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b43f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55db15901700 con 0x55db15b46d30 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55db15a3b110 con 0x55db15b3a480 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55db15b499b0 con 0x55db15b3ada0 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c20abc740 10 client.0 _ll_register_callbacks cb 0x55db15a54990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b43f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b46590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b43f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42326/0 (socket says 172.21.15.120:42326) 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59136/0 (socket says 172.21.15.120:59136) 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c1cc84640 1 -- 172.21.15.120:0/932116263 learned_addr learned my addr 172.21.15.120:0/932116263 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 -- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 msgr2=0x55db15b439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 -- 172.21.15.120:0/932116263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 msgr2=0x55db15b43f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b43f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 -- 172.21.15.120:0/932116263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db15a68c40 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3c17fff640 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b46590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005890 tx=0x7f3c0800e440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ff1aeaa6868f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.490+0000 7f3bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801e040 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3c08004240 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08019ae0 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3c080098f0 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 10 client.25535 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55db15b552c0 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08019ae0 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f3c0800e800 con 0x55db15b3a480 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 10 client.25535 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:14.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 20 client.25535 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15b566c0 0x55db15b58ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 4 client.25535 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.491+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db15ab2da0 con 0x55db15b566c0 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.492+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15b566c0 0x55db15b58ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.492+0000 7f3bf77fe640 10 client.25535 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.492+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15b566c0 0x55db15b58ab0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6aeeb63d106ff7c server_cookie=4ecaf8e6240afbe7 in_seq=0 out_seq=0 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3bf77fe640 1 -- 172.21.15.120:0/932116263 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db15ab2da0 con 0x55db15b566c0 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3bf77fe640 10 client.25535 handle_command_reply: tid=0 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 client.25535 shutdown 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 msgr2=0x55db15b46590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b46590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005890 tx=0x7f3c0800e440 comp rx=0 tx=0).stop 2024-09-10T22:56:14.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 shutdown_connections 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15b566c0 0x55db15b58ab0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b43f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 --2- 172.21.15.120:0/932116263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b46590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3bf77fe640 0 client.25535 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 >> 172.21.15.120:0/932116263 conn(0x55db15b382b0 msgr2=0x55db15b3de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 shutdown_connections 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 1 -- 172.21.15.120:0/932116263 wait complete. 2024-09-10T22:56:14.503 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.493+0000 7f3c20abc740 20 client.25535 trim_cache size 0 max 16384 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 Processor -- start 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 -- start start 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3a480 0x55db15b52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 0x55db15b37bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181aa0c0 con 0x55db15b46d30 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ac940 con 0x55db15b3ada0 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c20abc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181aa730 con 0x55db15b3a480 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 0x55db15b37bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b40c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42328/0 (socket says 172.21.15.120:42328) 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3a480 0x55db15b52c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c1cc84640 1 -- 172.21.15.120:0/1393439482 learned_addr learned my addr 172.21.15.120:0/1393439482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1393439482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3a480 msgr2=0x55db15b52c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3a480 0x55db15b52c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1393439482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 msgr2=0x55db15b40c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b40c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.512+0000 7f3c0ffff640 1 -- 172.21.15.120:0/1393439482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db15b49a00 con 0x55db15b3ada0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 0x55db15b37bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c000e3b70 tx=0x7f3c000e9ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fab23617a94e9e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c0dffb640 1 -- 172.21.15.120:0/1393439482 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000a8e30 con 0x55db15b3ada0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c0dffb640 1 -- 172.21.15.120:0/1393439482 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c000046c0 con 0x55db15b3ada0 2024-09-10T22:56:14.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c0dffb640 1 -- 172.21.15.120:0/1393439482 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000abf50 con 0x55db15b3ada0 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 -- 172.21.15.120:0/1393439482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 msgr2=0x55db15b37bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 0x55db15b37bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c000e3b70 tx=0x7f3c000e9ad0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 -- 172.21.15.120:0/1393439482 shutdown_connections 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b40c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3ada0 0x55db15b37bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 --2- 172.21.15.120:0/1393439482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3a480 0x55db15b52c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.513+0000 7f3c20abc740 1 -- 172.21.15.120:0/1393439482 >> 172.21.15.120:0/1393439482 conn(0x55db15b382b0 msgr2=0x55db15b397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- 172.21.15.120:0/1393439482 shutdown_connections 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- 172.21.15.120:0/1393439482 wait complete. 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- start start 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 Processor -- start 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b4f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b40aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181aae00 con 0x55db15b46d30 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ac270 con 0x55db15b3a480 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ab4d0 con 0x55db15b3ada0 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c20abc740 10 client.0 _ll_register_callbacks cb 0x7f3c0801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.514+0000 7f3c1cc84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42344/0 (socket says 172.21.15.120:42344) 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 -- 172.21.15.120:0/281989451 learned_addr learned my addr 172.21.15.120:0/281989451 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c17fff640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b4f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c0ffff640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b40aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 -- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 msgr2=0x55db15b40aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b40aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 -- 172.21.15.120:0/281989451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 msgr2=0x55db15b4f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b4f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 -- 172.21.15.120:0/281989451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db15bc2490 con 0x55db15b46d30 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c17fff640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b4f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c1cc84640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3c100317a0 tx=0x7f3c1003e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d950358adc990a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:14.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1005a060 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f3c10004040 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db15aae850 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3c10004240 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c20abc740 10 client.16077 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.515+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55db15bc36d0 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55db15aae850 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f3c1002fac0 con 0x55db15b46d30 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 10 client.16077 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 20 client.16077 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15bc3e80 0x55db15bc62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 4 client.16077 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db15aabc80 con 0x55db15bc3e80 2024-09-10T22:56:14.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c17fff640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15bc3e80 0x55db15bc62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3bf77fe640 10 client.16077 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.516+0000 7f3c17fff640 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15bc3e80 0x55db15bc62a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44062bb2d228111 server_cookie=12aa27ebf79da304 in_seq=0 out_seq=0 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3bf77fe640 1 -- 172.21.15.120:0/281989451 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x55db15aabc80 con 0x55db15bc3e80 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3bf77fe640 10 client.16077 handle_command_reply: tid=0 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 client.16077 shutdown 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 msgr2=0x55db15b41230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3c100317a0 tx=0x7f3c1003e050 comp rx=0 tx=0).stop 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 shutdown_connections 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55db15bc3e80 0x55db15bc62a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3bf77fe640 0 client.16077 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55db15b46d30 0x55db15b41230 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55db15b3ada0 0x55db15b40aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 --2- 172.21.15.120:0/281989451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55db15b3a480 0x55db15b4f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 >> 172.21.15.120:0/281989451 conn(0x55db15b382b0 msgr2=0x55db15b42150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 shutdown_connections 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 1 -- 172.21.15.120:0/281989451 wait complete. 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.517+0000 7f3c20abc740 20 client.16077 trim_cache size 0 max 16384 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:56:14.524 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:56:14.527 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:14.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 msgr2=0x7f3c040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 0x7f3c040b77a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c10005f50 comp rx=0 tx=0).stop 2024-09-10T22:56:14.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 msgr2=0x7f3c181997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000e9c0 tx=0x7f3c0000ee80 comp rx=0 tx=0).stop 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 shutdown_connections 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3c040b52e0 0x7f3c040b77a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c1810d2f0 0x7f3c1819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3c1810a9c0 0x7f3c181997e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.530 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 --2- 172.21.15.120:0/1853077324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c181018b0 0x7f3c181992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:14.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 >> 172.21.15.120:0/1853077324 conn(0x7f3c18076e60 msgr2=0x7f3c18100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:14.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 shutdown_connections 2024-09-10T22:56:14.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:14.518+0000 7f3c1e6e7640 1 -- 172.21.15.120:0/1853077324 wait complete. 2024-09-10T22:56:14.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 606718 17.23 MB/sec execute 69 sec latency 39.417 ms 2024-09-10T22:56:15.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 609120 17.16 MB/sec execute 70 sec latency 123.177 ms 2024-09-10T22:56:15.667 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:56:15.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:15.997 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:16.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:15 smithi120 ceph-mon[31247]: pgmap v459: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-10T22:56:16.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:15 smithi153 ceph-mon[34951]: pgmap v459: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-10T22:56:16.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:15 smithi176 ceph-mon[34982]: pgmap v459: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-10T22:56:16.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2131600352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 msgr2=0x7f5d0010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:16.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2131600352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0010e820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009ca0 tx=0x7f5cf802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2131600352 shutdown_connections 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2131600352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0010e820 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2131600352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d001076a0 0x7f5d00107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2131600352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d00101c90 0x7f5d00102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2131600352 >> 172.21.15.120:0/2131600352 conn(0x7f5d000fd700 msgr2=0x7f5d000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2131600352 shutdown_connections 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2131600352 wait complete. 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 Processor -- start 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.517+0000 7f5d06ff8640 1 -- start start 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d00101c90 0x7f5d00199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 0x7f5d00199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111ca0 con 0x7f5d00108020 2024-09-10T22:56:16.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111b20 con 0x7f5d00101c90 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d06ff8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111e20 con 0x7f5d001076a0 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42360/0 (socket says 172.21.15.120:42360) 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 -- 172.21.15.120:0/2690358064 learned_addr learned my addr 172.21.15.120:0/2690358064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5cfffff640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 0x7f5d00199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d04d94640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d00101c90 0x7f5d00199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 -- 172.21.15.120:0/2690358064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 msgr2=0x7f5d00199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 0x7f5d00199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 -- 172.21.15.120:0/2690358064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d00101c90 msgr2=0x7f5d00199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d00101c90 0x7f5d00199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 -- 172.21.15.120:0/2690358064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0019f2d0 con 0x7f5d00108020 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5cfffff640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 0x7f5d00199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:16.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5d05595640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005cc0 tx=0x7f5cf8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8003b70 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.518+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf80318f0 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.519+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0019f560 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.519+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf802f950 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.519+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001a5c80 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.520+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d00102070 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.521+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5cf802fb30 con 0x7f5d00108020 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.522+0000 7f5cfdffb640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 0x7f5cf00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:16.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.522+0000 7f5d04d94640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 0x7f5cf00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.522+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f5cf8106ca0 con 0x7f5d00108020 2024-09-10T22:56:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.522+0000 7f5d04d94640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 0x7f5cf00b77a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009bd0 tx=0x7f5cf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:16.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.526+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5cf80cdb90 con 0x7f5d00108020 2024-09-10T22:56:16.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 611802 17.12 MB/sec execute 71 sec latency 38.929 ms 2024-09-10T22:56:16.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.760+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d0010a880 con 0x7f5d00108020 2024-09-10T22:56:16.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.760+0000 7f5cfdffb640 1 -- 172.21.15.120:0/2690358064 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f5cf80cd490 con 0x7f5d00108020 2024-09-10T22:56:16.762 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:56:16.762 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":23,"btime":"2024-09-10T22:54:39:370770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:54:39.370760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25252":{"gid":25252,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2789135593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2789135593},{"type":"v1","addr":"172.21.15.176:6837","nonce":2789135593}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:56:16.762 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 23 2024-09-10T22:56:16.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 msgr2=0x7f5cf00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 0x7f5cf00b77a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009bd0 tx=0x7f5cf4005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 msgr2=0x7f5d0019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005cc0 tx=0x7f5cf8031700 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 shutdown_connections 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cf00b52e0 0x7f5cf00b77a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d00108020 0x7f5d0019eb00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5d001076a0 0x7f5d00199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 --2- 172.21.15.120:0/2690358064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d00101c90 0x7f5d00199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.762+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 >> 172.21.15.120:0/2690358064 conn(0x7f5d000fd700 msgr2=0x7f5d0006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.763+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 shutdown_connections 2024-09-10T22:56:16.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:16.763+0000 7f5d06ff8640 1 -- 172.21.15.120:0/2690358064 wait complete. 2024-09-10T22:56:17.199 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:16 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2690358064' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:17.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:16 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2690358064' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:17.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:16 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2690358064' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:17.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:56:17.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 615033 17.14 MB/sec execute 72 sec latency 49.973 ms 2024-09-10T22:56:17.851 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:18.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:17 smithi120 ceph-mon[31247]: pgmap v460: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-09-10T22:56:18.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:17 smithi153 ceph-mon[34951]: pgmap v460: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-09-10T22:56:18.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:17 smithi176 ceph-mon[34982]: pgmap v460: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 -- 172.21.15.120:0/3847335187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e70103b30 msgr2=0x7f4e70103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 --2- 172.21.15.120:0/3847335187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e70103b30 0x7f4e70103f10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e6002a410 comp rx=0 tx=0).stop 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 -- 172.21.15.120:0/3847335187 shutdown_connections 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 --2- 172.21.15.120:0/3847335187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7010d0e0 0x7f4e7010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 --2- 172.21.15.120:0/3847335187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70104530 0x7f4e7010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 --2- 172.21.15.120:0/3847335187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e70103b30 0x7f4e70103f10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 -- 172.21.15.120:0/3847335187 >> 172.21.15.120:0/3847335187 conn(0x7f4e700fd700 msgr2=0x7f4e700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 -- 172.21.15.120:0/3847335187 shutdown_connections 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.353+0000 7f4e754be640 1 -- 172.21.15.120:0/3847335187 wait complete. 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 Processor -- start 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 -- start start 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70103b30 0x7f4e7019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e70104530 0x7f4e7019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112d60 con 0x7f4e7010d0e0 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112be0 con 0x7f4e70104530 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e754be640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112ee0 con 0x7f4e70103b30 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42370/0 (socket says 172.21.15.120:42370) 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3265422482 learned_addr learned my addr 172.21.15.120:0/3265422482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e70104530 0x7f4e7019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.354+0000 7f4e6effd640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70103b30 0x7f4e7019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3265422482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70103b30 msgr2=0x7f4e7019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70103b30 0x7f4e7019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3265422482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e70104530 msgr2=0x7f4e7019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e70104530 0x7f4e7019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3265422482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701a1520 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e67fff640 1 -- 172.21.15.120:0/3265422482 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017070 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701a1810 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e67fff640 1 -- 172.21.15.120:0/3265422482 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e68003e20 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a7ed0 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.355+0000 7f4e67fff640 1 -- 172.21.15.120:0/3265422482 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68010460 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.357+0000 7f4e77893740 1 Processor -- start 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.357+0000 7f4e67fff640 1 -- 172.21.15.120:0/3265422482 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4e68003970 con 0x7f4e7010d0e0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 -- start start 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 0x561d343408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d3434f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d342b6ea0 con 0x561d34340df0 2024-09-10T22:56:18.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d342a62d0 con 0x561d343404d0 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e77893740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d342aa0e0 con 0x561d3434cd80 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 0x561d343408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42384/0 (socket says 172.21.15.120:42384) 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 -- 172.21.15.120:0/2387535814 learned_addr learned my addr 172.21.15.120:0/2387535814 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d3434f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 -- 172.21.15.120:0/2387535814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 msgr2=0x561d3434f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d3434f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 -- 172.21.15.120:0/2387535814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 msgr2=0x561d343408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 0x561d343408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6effd640 1 -- 172.21.15.120:0/2387535814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d34004650 con 0x561d34340df0 2024-09-10T22:56:18.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 0x561d343408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.358+0000 7f4e67fff640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 0x7f4e5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e6effd640 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e60032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3698203fa07a8c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 0x7f4e5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e67fff640 1 -- 172.21.15.120:0/3265422482 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f4e680e5880 con 0x7f4e7010d0e0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e657fa640 1 -- 172.21.15.120:0/2387535814 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60040070 con 0x561d34340df0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e657fa640 1 -- 172.21.15.120:0/2387535814 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e60004030 con 0x561d34340df0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.359+0000 7f4e657fa640 1 -- 172.21.15.120:0/2387535814 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60035040 con 0x561d34340df0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- 172.21.15.120:0/2387535814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 msgr2=0x561d3434c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 0x7f4e5c0b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009fd0 tx=0x7f4e580096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e60032bd0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- 172.21.15.120:0/2387535814 shutdown_connections 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d3434f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d3434c840 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 --2- 172.21.15.120:0/2387535814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d343404d0 0x561d343408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- 172.21.15.120:0/2387535814 >> 172.21.15.120:0/2387535814 conn(0x561d3433e300 msgr2=0x561d3433f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- 172.21.15.120:0/2387535814 shutdown_connections 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- 172.21.15.120:0/2387535814 wait complete. 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 -- start start 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.360+0000 7f4e77893740 1 Processor -- start 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d3434c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 0x561d3434a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561d34107700 con 0x561d34340df0 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561d34241160 con 0x561d3434cd80 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561d3434f9a0 con 0x561d343404d0 2024-09-10T22:56:18.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e77893740 10 client.0 _ll_register_callbacks cb 0x561d3425a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d3434c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42394/0 (socket says 172.21.15.120:42394) 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6e7fc640 1 -- 172.21.15.120:0/3651610552 learned_addr learned my addr 172.21.15.120:0/3651610552 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 0x561d3434a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 -- 172.21.15.120:0/3651610552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 msgr2=0x561d3434a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 0x561d3434a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 -- 172.21.15.120:0/3651610552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 msgr2=0x561d34349aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 -- 172.21.15.120:0/3651610552 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3426ec90 con 0x561d343404d0 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 0x561d3434a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:18.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e6effd640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d3434c650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e6002a930 tx=0x7f4e6002ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc752df7b08b014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e567fc640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.361+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60040070 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f4e600043f0 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e60004030 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e60035040 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 10 client.25570 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561d3435b2b0 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e60004030 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f4e60004870 con 0x561d343404d0 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 10 client.25570 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 20 client.25570 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d3435c600 0x561d3435e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 4 client.25570 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d342b8df0 con 0x561d3435c600 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.362+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d3435c600 0x561d3435e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.363+0000 7f4e567fc640 10 client.25570 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.363+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d3435c600 0x561d3435e9f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e483db93077d6e07 server_cookie=2d38a3e20e3cb703 in_seq=0 out_seq=0 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.363+0000 7f4e567fc640 1 -- 172.21.15.120:0/3651610552 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d342b8df0 con 0x561d3435c600 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.363+0000 7f4e567fc640 10 client.25570 handle_command_reply: tid=0 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 client.25570 shutdown 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 msgr2=0x561d3434c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d3434c650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e6002a930 tx=0x7f4e6002ab80 comp rx=0 tx=0).stop 2024-09-10T22:56:18.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 shutdown_connections 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d3435c600 0x561d3435e9f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d3434cd80 0x561d3434a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d34340df0 0x561d34349aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e567fc640 0 client.25570 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 --2- 172.21.15.120:0/3651610552 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d3434c650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 >> 172.21.15.120:0/3651610552 conn(0x561d3433e300 msgr2=0x561d34343ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 shutdown_connections 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 1 -- 172.21.15.120:0/3651610552 wait complete. 2024-09-10T22:56:18.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.364+0000 7f4e77893740 20 client.25570 trim_cache size 0 max 16384 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 Processor -- start 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 -- start start 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d34348f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3433dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701ae460 con 0x561d3434cd80 2024-09-10T22:56:18.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b0ce0 con 0x561d34340df0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e77893740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701aead0 con 0x561d343404d0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42402/0 (socket says 172.21.15.120:42402) 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3500184807 learned_addr learned my addr 172.21.15.120:0/3500184807 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6effd640 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d34348f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3433dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3500184807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 msgr2=0x561d34348f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d34348f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3500184807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 msgr2=0x561d3433dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3433dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.379+0000 7f4e6f7fe640 1 -- 172.21.15.120:0/3500184807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d34339bb0 con 0x561d3434cd80 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.380+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005c40 tx=0x7f4e68004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a95a0344073710 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.380+0000 7f4e64ff9640 1 -- 172.21.15.120:0/3500184807 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68020040 con 0x561d3434cd80 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.380+0000 7f4e64ff9640 1 -- 172.21.15.120:0/3500184807 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e680abc30 con 0x561d3434cd80 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.380+0000 7f4e64ff9640 1 -- 172.21.15.120:0/3500184807 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680a7a70 con 0x561d3434cd80 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.380+0000 7f4e77893740 1 -- 172.21.15.120:0/3500184807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 msgr2=0x561d3434e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005c40 tx=0x7f4e68004020 comp rx=0 tx=0).stop 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- 172.21.15.120:0/3500184807 shutdown_connections 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d3434cd80 0x561d3434e170 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3433dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- 172.21.15.120:0/3500184807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d343404d0 0x561d34348f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- 172.21.15.120:0/3500184807 >> 172.21.15.120:0/3500184807 conn(0x561d3433e300 msgr2=0x561d34348320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- 172.21.15.120:0/3500184807 shutdown_connections 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- 172.21.15.120:0/3500184807 wait complete. 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- start start 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 Processor -- start 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 0x561d3435e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3435e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d343c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701af1a0 con 0x561d343404d0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701b0610 con 0x561d34340df0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701af870 con 0x561d3434cd80 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e77893740 10 client.0 _ll_register_callbacks cb 0x7f4e58009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 0x561d3435e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e6e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3435e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e6effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 0x561d3435e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42408/0 (socket says 172.21.15.120:42408) 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e6effd640 1 -- 172.21.15.120:0/2289063268 learned_addr learned my addr 172.21.15.120:0/2289063268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.381+0000 7f4e6f7fe640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d343c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 -- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 msgr2=0x561d343c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d343c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 -- 172.21.15.120:0/2289063268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 msgr2=0x561d3435e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:18.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 0x561d3435e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 -- 172.21.15.120:0/2289063268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d343c86a0 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e6e7fc640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3435e5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x561d342411c0 tx=0x7f4e58005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfeabfabd9f3e1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e567fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58021190 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e680abc30 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f4e5801a070 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e58038070 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e77893740 10 client.25565 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561d343c9a00 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.382+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e680abc30 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f4e5801d070 con 0x561d34340df0 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 10 client.25565 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:18.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 20 client.25565 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d343ca130 0x561d343cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 4 client.25565 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d342b48a0 con 0x561d343ca130 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e6effd640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d343ca130 0x561d343cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e567fc640 10 client.25565 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.383+0000 7f4e6effd640 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d343ca130 0x561d343cc550 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907153c5bea2e8f1 server_cookie=8c8fce5c382d1651 in_seq=0 out_seq=0 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e567fc640 1 -- 172.21.15.120:0/2289063268 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d342b48a0 con 0x561d343ca130 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e567fc640 10 client.25565 handle_command_reply: tid=0 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 client.25565 shutdown 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 msgr2=0x561d3435e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3435e5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x561d342411c0 tx=0x7f4e58005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:18.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 shutdown_connections 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561d343ca130 0x561d343cc550 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561d3434cd80 0x561d343c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561d34340df0 0x561d3435e5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 --2- 172.21.15.120:0/2289063268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561d343404d0 0x561d3435e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e567fc640 0 client.25565 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:18.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 >> 172.21.15.120:0/2289063268 conn(0x561d3433e300 msgr2=0x561d34343530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 shutdown_connections 2024-09-10T22:56:18.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 1 -- 172.21.15.120:0/2289063268 wait complete. 2024-09-10T22:56:18.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e77893740 20 client.25565 trim_cache size 0 max 16384 2024-09-10T22:56:18.401 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 msgr2=0x7f4e5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 0x7f4e5c0b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009fd0 tx=0x7f4e580096c0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 msgr2=0x7f4e701a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.384+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6800b260 comp rx=0 tx=0).stop 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 shutdown_connections 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4e5c0b52e0 0x7f4e5c0b77a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e7010d0e0 0x7f4e701a0d50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e70104530 0x7f4e7019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 --2- 172.21.15.120:0/3265422482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e70103b30 0x7f4e7019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:18.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 >> 172.21.15.120:0/3265422482 conn(0x7f4e700fd700 msgr2=0x7f4e700ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:18.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 shutdown_connections 2024-09-10T22:56:18.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:18.385+0000 7f4e754be640 1 -- 172.21.15.120:0/3265422482 wait complete. 2024-09-10T22:56:18.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 618384 17.17 MB/sec execute 73 sec latency 34.487 ms 2024-09-10T22:56:19.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 621772 17.19 MB/sec execute 74 sec latency 18.487 ms 2024-09-10T22:56:20.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:56:20.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:56:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:19 smithi120 ceph-mon[31247]: pgmap v461: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-10T22:56:20.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:19 smithi153 ceph-mon[34951]: pgmap v461: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-10T22:56:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:19 smithi176 ceph-mon[34982]: pgmap v461: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 398 op/s 2024-09-10T22:56:20.478 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:20.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 624535 17.13 MB/sec execute 75 sec latency 69.501 ms 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2993054843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f8108020 msgr2=0x7f32f810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2993054843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f8108020 0x7f32f810e820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009ca0 tx=0x7f32f002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2993054843 shutdown_connections 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2993054843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f8108020 0x7f32f810e820 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2993054843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f81076a0 0x7f32f8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2993054843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2993054843 >> 172.21.15.120:0/2993054843 conn(0x7f32f80fd700 msgr2=0x7f32f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2993054843 shutdown_connections 2024-09-10T22:56:20.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.984+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2993054843 wait complete. 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 Processor -- start 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 -- start start 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f8108020 0x7f32f819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112de0 con 0x7f32f81076a0 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112c60 con 0x7f32f8101c90 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32fd6fe640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112f60 con 0x7f32f8108020 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42420/0 (socket says 172.21.15.120:42420) 2024-09-10T22:56:20.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37262/0 (socket says 172.21.15.120:37262) 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 -- 172.21.15.120:0/2034906143 learned_addr learned my addr 172.21.15.120:0/2034906143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32f77fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f8108020 0x7f32f819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 -- 172.21.15.120:0/2034906143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f8108020 msgr2=0x7f32f819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f8108020 0x7f32f819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 -- 172.21.15.120:0/2034906143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 msgr2=0x7f32f8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.985+0000 7f32effff640 1 -- 172.21.15.120:0/2034906143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f320 con 0x7f32f81076a0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32effff640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32f4ff9640 1 -- 172.21.15.120:0/2034906143 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32f81076a0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32f4ff9640 1 -- 172.21.15.120:0/2034906143 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e00054d0 con 0x7f32f81076a0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32f4ff9640 1 -- 172.21.15.120:0/2034906143 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0010690 con 0x7f32f81076a0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819f610 con 0x7f32f81076a0 2024-09-10T22:56:20.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.986+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f810f060 con 0x7f32f81076a0 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 Processor -- start 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32f4ff9640 1 -- 172.21.15.120:0/2034906143 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f32e0004e90 con 0x7f32f81076a0 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 -- start start 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d832e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d8330b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5649d8298e60 con 0x5649d8321ee0 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5649d8288290 con 0x5649d8322800 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32ffad3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5649d828c0a0 con 0x5649d832e790 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.989+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42436/0 (socket says 172.21.15.120:42436) 2024-09-10T22:56:20.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 -- 172.21.15.120:0/2474235849 learned_addr learned my addr 172.21.15.120:0/2474235849 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32effff640 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d8330b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d832e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 -- 172.21.15.120:0/2474235849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 msgr2=0x5649d8330b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d8330b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 -- 172.21.15.120:0/2474235849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 msgr2=0x5649d832e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d832e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 -- 172.21.15.120:0/2474235849 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d7fe6930 con 0x5649d8321ee0 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f77fe640 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005ef0 tx=0x7f32f0005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727057c29163a35a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f4ff9640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 0x7f32e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 0x7f32e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32ed7fa640 1 -- 172.21.15.120:0/2474235849 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0031cf0 con 0x5649d8321ee0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32ed7fa640 1 -- 172.21.15.120:0/2474235849 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f0036070 con 0x5649d8321ee0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32ed7fa640 1 -- 172.21.15.120:0/2474235849 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f00042a0 con 0x5649d8321ee0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.990+0000 7f32f4ff9640 1 -- 172.21.15.120:0/2034906143 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f32e00e5ff0 con 0x7f32f81076a0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 -- 172.21.15.120:0/2474235849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 msgr2=0x5649d83222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 0x7f32e40b77a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5649d8325060 tx=0x7f32e8012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005ef0 tx=0x7f32f0005f20 comp rx=0 tx=0).stop 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 -- 172.21.15.120:0/2474235849 shutdown_connections 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d8330b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d832e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2474235849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d83222c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 -- 172.21.15.120:0/2474235849 >> 172.21.15.120:0/2474235849 conn(0x5649d831fd10 msgr2=0x5649d8320eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:20.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 -- 172.21.15.120:0/2474235849 shutdown_connections 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.991+0000 7f32ffad3740 1 -- 172.21.15.120:0/2474235849 wait complete. 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 -- start start 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 Processor -- start 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8321ee0 0x5649d832b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d832c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5649d80e9890 con 0x5649d8322800 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5649d8222d40 con 0x5649d8321ee0 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5649d8331260 con 0x5649d832e790 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32ffad3740 10 client.0 _ll_register_callbacks cb 0x5649d8286c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d832c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:20.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32effff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42448/0 (socket says 172.21.15.120:42448) 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32effff640 1 -- 172.21.15.120:0/3346827578 learned_addr learned my addr 172.21.15.120:0/3346827578 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.992+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8321ee0 0x5649d832b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 -- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 msgr2=0x5649d832c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d832c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 -- 172.21.15.120:0/3346827578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8321ee0 msgr2=0x5649d832b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8321ee0 0x5649d832b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 -- 172.21.15.120:0/3346827578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d833a6f0 con 0x5649d8322800 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32effff640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f32e00a8d70 tx=0x7f32e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0d6dd64f1953a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00abf10 con 0x5649d8322800 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f32e00b58a0 con 0x5649d8322800 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f0036070 con 0x5649d8322800 2024-09-10T22:56:21.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f32e00ac2b0 con 0x5649d8322800 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 10 client.16113 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5649d833d0e0 con 0x5649d8322800 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f0036070 con 0x5649d8322800 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f32e00cfaf0 con 0x5649d8322800 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 10 client.16113 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 20 client.16113 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.993+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d833e4f0 0x5649d83408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.994+0000 7f32ffad3740 4 client.16113 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.994+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d829adb0 con 0x5649d833e4f0 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.994+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d833e4f0 0x5649d83408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.994+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d833e4f0 0x5649d83408e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f842a41fdd43a20a server_cookie=96b78f2eff56d6ba in_seq=0 out_seq=0 2024-09-10T22:56:21.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.994+0000 7f32de7fc640 10 client.16113 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:21.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.996+0000 7f32de7fc640 1 -- 172.21.15.120:0/3346827578 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649d829adb0 con 0x5649d833e4f0 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.996+0000 7f32de7fc640 10 client.16113 handle_command_reply: tid=0 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 client.16113 shutdown 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 msgr2=0x5649d832bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f32e00a8d70 tx=0x7f32e00048f0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 shutdown_connections 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d833e4f0 0x5649d83408e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d832c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8322800 0x5649d832bd00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 --2- 172.21.15.120:0/3346827578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8321ee0 0x5649d832b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 >> 172.21.15.120:0/3346827578 conn(0x5649d831fd10 msgr2=0x5649d831ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32de7fc640 0 client.16113 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 shutdown_connections 2024-09-10T22:56:21.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 1 -- 172.21.15.120:0/3346827578 wait complete. 2024-09-10T22:56:21.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:20.997+0000 7f32ffad3740 20 client.16113 trim_cache size 0 max 16384 2024-09-10T22:56:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.014+0000 7f32ffad3740 1 Processor -- start 2024-09-10T22:56:21.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.014+0000 7f32ffad3740 1 -- start start 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.014+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.014+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d8322800 0x5649d83257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 0x5649d83289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ffad3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81a9fa0 con 0x5649d8321ee0 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ffad3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81ac820 con 0x5649d832e790 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ffad3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81aa610 con 0x5649d8322800 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42450/0 (socket says 172.21.15.120:42450) 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 -- 172.21.15.120:0/1873208683 learned_addr learned my addr 172.21.15.120:0/1873208683 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f77fe640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 0x5649d83289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32effff640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d8322800 0x5649d83257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 -- 172.21.15.120:0/1873208683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d8322800 msgr2=0x5649d83257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d8322800 0x5649d83257a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 -- 172.21.15.120:0/1873208683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 msgr2=0x5649d83289d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 0x5649d83289d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 -- 172.21.15.120:0/1873208683 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d831b640 con 0x5649d8321ee0 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f77fe640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 0x5649d83289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5649d8341130 tx=0x7f32e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f8af9809bed32e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ecff9640 1 -- 172.21.15.120:0/1873208683 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e801d460 con 0x5649d8321ee0 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ecff9640 1 -- 172.21.15.120:0/1873208683 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e801dab0 con 0x5649d8321ee0 2024-09-10T22:56:21.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.015+0000 7f32ecff9640 1 -- 172.21.15.120:0/1873208683 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e802ba20 con 0x5649d8321ee0 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 -- 172.21.15.120:0/1873208683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 msgr2=0x5649d832b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5649d8341130 tx=0x7f32e80046c0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 -- 172.21.15.120:0/1873208683 shutdown_connections 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d832e790 0x5649d83289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d8322800 0x5649d83257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 --2- 172.21.15.120:0/1873208683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d832b050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 -- 172.21.15.120:0/1873208683 >> 172.21.15.120:0/1873208683 conn(0x5649d831fd10 msgr2=0x5649d8325090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 -- 172.21.15.120:0/1873208683 shutdown_connections 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.016+0000 7f32ffad3740 1 -- 172.21.15.120:0/1873208683 wait complete. 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 -- start start 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 Processor -- start 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d8332eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d83aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81aace0 con 0x5649d8321ee0 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81ac150 con 0x5649d8322800 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81ab3b0 con 0x5649d832e790 2024-09-10T22:56:21.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32ffad3740 10 client.0 _ll_register_callbacks cb 0x7f32e000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42462/0 (socket says 172.21.15.120:42462) 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 -- 172.21.15.120:0/2030527863 learned_addr learned my addr 172.21.15.120:0/2030527863 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32effff640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d8332eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f77fe640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d83aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 -- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 msgr2=0x5649d83aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d83aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 -- 172.21.15.120:0/2030527863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 msgr2=0x5649d8332eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d8332eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.017+0000 7f32f6ffd640 1 -- 172.21.15.120:0/2030527863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d83aa8d0 con 0x5649d8321ee0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32f6ffd640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005940 tx=0x7f32e80040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eedad61d06e082d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e803f060 con 0x5649d8321ee0 2024-09-10T22:56:21.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f32e802b450 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e801dab0 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f32e801d460 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 10 client.16119 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5649d83abbe0 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e801dab0 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f32e00051b0 con 0x5649d8321ee0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 10 client.16119 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 20 client.16119 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d83ac1f0 0x5649d83ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 4 client.16119 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649d8296860 con 0x5649d83ac1f0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.018+0000 7f32effff640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d83ac1f0 0x5649d83ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:21.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.019+0000 7f32effff640 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d83ac1f0 0x5649d83ae5e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5ca6a9e6118ae9 server_cookie=d4c9c123402a6985 in_seq=0 out_seq=0 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.019+0000 7f32de7fc640 10 client.16119 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32de7fc640 1 -- 172.21.15.120:0/2030527863 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649d8296860 con 0x5649d83ac1f0 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32de7fc640 10 client.16119 handle_command_reply: tid=0 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 client.16119 shutdown 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 msgr2=0x5649d8332950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005940 tx=0x7f32e80040b0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 shutdown_connections 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649d83ac1f0 0x5649d83ae5e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649d832e790 0x5649d83aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649d8322800 0x5649d8332eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 --2- 172.21.15.120:0/2030527863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649d8321ee0 0x5649d8332950 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 >> 172.21.15.120:0/2030527863 conn(0x5649d831fd10 msgr2=0x5649d83258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32de7fc640 0 client.16119 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.020+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 shutdown_connections 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32ffad3740 1 -- 172.21.15.120:0/2030527863 wait complete. 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32ffad3740 20 client.16119 trim_cache size 0 max 16384 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "4777f81f-6a19-4258-a6b1-74ce729177eb", 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:56:21.028 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 msgr2=0x7f32e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 0x7f32e40b77a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5649d8325060 tx=0x7f32e8012040 comp rx=0 tx=0).stop 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 msgr2=0x7f32f81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000edf0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 shutdown_connections 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f32e40b52e0 0x7f32e40b77a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f8108020 0x7f32f819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32f81076a0 0x7f32f81997b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 --2- 172.21.15.120:0/2034906143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32f8101c90 0x7f32f8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:21.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 >> 172.21.15.120:0/2034906143 conn(0x7f32f80fd700 msgr2=0x7f32f80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:21.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 shutdown_connections 2024-09-10T22:56:21.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:21.021+0000 7f32fd6fe640 1 -- 172.21.15.120:0/2034906143 wait complete. 2024-09-10T22:56:21.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 626903 17.06 MB/sec execute 76 sec latency 33.114 ms 2024-09-10T22:56:21.964 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:56:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:21 smithi120 ceph-mon[31247]: pgmap v462: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-09-10T22:56:22.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:21 smithi153 ceph-mon[34951]: pgmap v462: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-09-10T22:56:22.218 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:21 smithi176 ceph-mon[34982]: pgmap v462: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-09-10T22:56:22.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 629799 17.05 MB/sec execute 77 sec latency 56.385 ms 2024-09-10T22:56:22.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- 172.21.15.120:0/819468776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d641076a0 msgr2=0x7f8d64107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/819468776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d641076a0 0x7f8d64107ae0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d5402a410 comp rx=0 tx=0).stop 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- 172.21.15.120:0/819468776 shutdown_connections 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/819468776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 0x7f8d6410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/819468776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d641076a0 0x7f8d64107ae0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/819468776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d64101c90 0x7f8d64102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- 172.21.15.120:0/819468776 >> 172.21.15.120:0/819468776 conn(0x7f8d640fd700 msgr2=0x7f8d640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- 172.21.15.120:0/819468776 shutdown_connections 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- 172.21.15.120:0/819468776 wait complete. 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 Processor -- start 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.670+0000 7f8d6bb25640 1 -- start start 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d641076a0 0x7f8d6419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 0x7f8d6419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64111ce0 con 0x7f8d64101c90 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64111b60 con 0x7f8d641076a0 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64111e60 con 0x7f8d64108020 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42480/0 (socket says 172.21.15.120:42480) 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 -- 172.21.15.120:0/15115866 learned_addr learned my addr 172.21.15.120:0/15115866 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 0x7f8d6419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d690c0640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d641076a0 0x7f8d6419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 -- 172.21.15.120:0/15115866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 msgr2=0x7f8d6419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 0x7f8d6419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 -- 172.21.15.120:0/15115866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d641076a0 msgr2=0x7f8d6419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d641076a0 0x7f8d6419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 -- 172.21.15.120:0/15115866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d641a54d0 con 0x7f8d64101c90 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d698c1640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000e9f0 tx=0x7f8d6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d5affd640 1 -- 172.21.15.120:0/15115866 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000cd80 con 0x7f8d64101c90 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d5affd640 1 -- 172.21.15.120:0/15115866 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600108e0 con 0x7f8d64101c90 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d641a57c0 con 0x7f8d64101c90 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d5affd640 1 -- 172.21.15.120:0/15115866 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60018980 con 0x7f8d64101c90 2024-09-10T22:56:22.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.671+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d641a5cd0 con 0x7f8d64101c90 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 Processor -- start 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 -- start start 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d5affd640 1 -- 172.21.15.120:0/15115866 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8d60010430 con 0x7f8d64101c90 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b167023e50 con 0x55b1670adda0 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b167013280 con 0x55b1670b9d30 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6defa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b167017090 con 0x55b1670ad480 2024-09-10T22:56:22.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d6a0c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42484/0 (socket says 172.21.15.120:42484) 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d698c1640 1 -- 172.21.15.120:0/3031011917 learned_addr learned my addr 172.21.15.120:0/3031011917 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.674+0000 7f8d690c0640 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 -- 172.21.15.120:0/3031011917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 msgr2=0x55b1670ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 -- 172.21.15.120:0/3031011917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 msgr2=0x55b1670bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 -- 172.21.15.120:0/3031011917 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b166d71650 con 0x55b1670adda0 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d690c0640 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:22.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d698c1640 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8d600e9fa0 tx=0x7f8d600ee570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa56e1e2b2b6eee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d4ffff640 1 -- 172.21.15.120:0/3031011917 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600f9070 con 0x55b1670adda0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d4ffff640 1 -- 172.21.15.120:0/3031011917 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600f4ab0 con 0x55b1670adda0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d4ffff640 1 -- 172.21.15.120:0/3031011917 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600f3a50 con 0x55b1670adda0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d5affd640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 0x7f8d500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 0x7f8d500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.675+0000 7f8d5affd640 1 -- 172.21.15.120:0/15115866 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8d600e5e90 con 0x7f8d64101c90 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- 172.21.15.120:0/3031011917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 msgr2=0x55b1670b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 0x7f8d500b77a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d6419b0d0 tx=0x7f8d5c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8d600e9fa0 tx=0x7f8d600ee570 comp rx=0 tx=0).stop 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- 172.21.15.120:0/3031011917 shutdown_connections 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b97f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 --2- 172.21.15.120:0/3031011917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- 172.21.15.120:0/3031011917 >> 172.21.15.120:0/3031011917 conn(0x55b1670ab2b0 msgr2=0x55b1670ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- 172.21.15.120:0/3031011917 shutdown_connections 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- 172.21.15.120:0/3031011917 wait complete. 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 -- start start 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.676+0000 7f8d6defa740 1 Processor -- start 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b166e74700 con 0x55b1670adda0 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b166fae110 con 0x55b1670b9d30 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b1670bc9b0 con 0x55b1670ad480 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6defa740 10 client.0 _ll_register_callbacks cb 0x55b166fc7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6a0c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6a0c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37358/0 (socket says 172.21.15.120:37358) 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59312/0 (socket says 172.21.15.120:59312) 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6a0c2640 1 -- 172.21.15.120:0/2312684701 learned_addr learned my addr 172.21.15.120:0/2312684701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42498/0 (socket says 172.21.15.120:42498) 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 -- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 msgr2=0x55b1670b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 -- 172.21.15.120:0/2312684701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 msgr2=0x55b1670b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 -- 172.21.15.120:0/2312684701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b166fdbc40 con 0x55b1670adda0 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d698c1640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d690c0640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009230 tx=0x7f8d54032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1952614cf99ae2e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.677+0000 7f8d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:22.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54035040 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8d540317a0 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d600f4ab0 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8d5404d070 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 10 client.16131 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b1670c82e0 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600f4ab0 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8d54004750 con 0x55b1670adda0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 10 client.16131 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 20 client.16131 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b1670cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 4 client.16131 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b167025da0 con 0x55b1670c97f0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.678+0000 7f8d698c1640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b1670cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.679+0000 7f8d4d7fa640 10 client.16131 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.679+0000 7f8d698c1640 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b1670cbc00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f1e5afaa794451 server_cookie=918819f9b6a48f71 in_seq=0 out_seq=0 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/2312684701 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b167025da0 con 0x55b1670c97f0 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d4d7fa640 10 client.16131 handle_command_reply: tid=0 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 client.16131 shutdown 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 msgr2=0x55b1670b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009230 tx=0x7f8d54032f40 comp rx=0 tx=0).stop 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 shutdown_connections 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b1670cbc00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b6a00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d4d7fa640 0 client.16131 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 --2- 172.21.15.120:0/2312684701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670ad480 0x55b1670b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 >> 172.21.15.120:0/2312684701 conn(0x55b1670ab2b0 msgr2=0x55b1670b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 shutdown_connections 2024-09-10T22:56:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 1 -- 172.21.15.120:0/2312684701 wait complete. 2024-09-10T22:56:22.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.680+0000 7f8d6defa740 20 client.16131 trim_cache size 0 max 16384 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.699+0000 7f8d6defa740 1 Processor -- start 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 -- start start 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670b9d30 0x55b1670b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641ac240 con 0x55b1670adda0 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641aeac0 con 0x55b1670ad480 2024-09-10T22:56:22.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6defa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641ac8b0 con 0x55b1670b9d30 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d690c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37372/0 (socket says 172.21.15.120:37372) 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 -- 172.21.15.120:0/918667581 learned_addr learned my addr 172.21.15.120:0/918667581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670b9d30 0x55b1670b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 -- 172.21.15.120:0/918667581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670b9d30 msgr2=0x55b1670b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670b9d30 0x55b1670b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 -- 172.21.15.120:0/918667581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 msgr2=0x55b1670b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.700+0000 7f8d698c1640 1 -- 172.21.15.120:0/918667581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1670b7310 con 0x55b1670ad480 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d698c1640 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d600a8cb0 tx=0x7f8d600f36b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ceb2877b1af7a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d4f7fe640 1 -- 172.21.15.120:0/918667581 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600b5e60 con 0x55b1670ad480 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d4f7fe640 1 -- 172.21.15.120:0/918667581 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600e9740 con 0x55b1670ad480 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d4f7fe640 1 -- 172.21.15.120:0/918667581 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600ac500 con 0x55b1670ad480 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 -- 172.21.15.120:0/918667581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 msgr2=0x55b1670aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d600a8cb0 tx=0x7f8d600f36b0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 -- 172.21.15.120:0/918667581 shutdown_connections 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670b9d30 0x55b1670b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670adda0 0x55b1670b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.701+0000 7f8d6defa740 1 --2- 172.21.15.120:0/918667581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670ad480 0x55b1670aa2c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.702+0000 7f8d6defa740 1 -- 172.21.15.120:0/918667581 >> 172.21.15.120:0/918667581 conn(0x55b1670ab2b0 msgr2=0x55b1670a9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.702+0000 7f8d6defa740 1 -- 172.21.15.120:0/918667581 shutdown_connections 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.702+0000 7f8d6defa740 1 -- 172.21.15.120:0/918667581 wait complete. 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.702+0000 7f8d6defa740 1 -- start start 2024-09-10T22:56:22.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.702+0000 7f8d6defa740 1 Processor -- start 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641acf80 con 0x55b1670ad480 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641ae3f0 con 0x55b1670b9d30 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641ad650 con 0x55b1670adda0 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6defa740 10 client.0 _ll_register_callbacks cb 0x7f8d54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59334/0 (socket says 172.21.15.120:59334) 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d698c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42518/0 (socket says 172.21.15.120:42518) 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 -- 172.21.15.120:0/4290682758 learned_addr learned my addr 172.21.15.120:0/4290682758 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d6a0c2640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 -- 172.21.15.120:0/4290682758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 msgr2=0x55b1670be640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670be640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 -- 172.21.15.120:0/4290682758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 msgr2=0x55b1670bdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 -- 172.21.15.120:0/4290682758 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1671356d0 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d698c1640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d690c0640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d54000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f730a6df59786acc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.703+0000 7f8d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54035050 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f8d540052b0 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d600e9740 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8d540054f0 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 10 client.25612 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b167136a10 con 0x55b1670adda0 2024-09-10T22:56:22.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600e9740 con 0x55b1670adda0 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8d54052580 con 0x55b1670adda0 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 10 client.25612 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 20 client.25612 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b167139200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 4 client.25612 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b167021850 con 0x55b1670c97f0 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.704+0000 7f8d698c1640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b167139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:22.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d698c1640 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b167139200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf56386c6dcea651 server_cookie=9f6691bf9370b in_seq=0 out_seq=0 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d4d7fa640 10 client.25612 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d4d7fa640 1 -- 172.21.15.120:0/4290682758 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55b167021850 con 0x55b1670c97f0 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d4d7fa640 10 client.25612 handle_command_reply: tid=0 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 client.25612 shutdown 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 msgr2=0x55b1670be100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d54000bc0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 shutdown_connections 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b1670c97f0 0x55b167139200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b1670b9d30 0x55b1670be640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b1670adda0 0x55b1670be100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d4d7fa640 0 client.25612 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 --2- 172.21.15.120:0/4290682758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b1670ad480 0x55b1670bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 >> 172.21.15.120:0/4290682758 conn(0x55b1670ab2b0 msgr2=0x55b1670b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 shutdown_connections 2024-09-10T22:56:22.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 1 -- 172.21.15.120:0/4290682758 wait complete. 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.705+0000 7f8d6defa740 20 client.25612 trim_cache size 0 max 16384 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:56:22.715 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "4777f81f-6a19-4258-a6b1-74ce729177eb", 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.716 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.717 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.718 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.719 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:22.720 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:22.721 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 msgr2=0x7f8d500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 0x7f8d500b77a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d6419b0d0 tx=0x7f8d5c009500 comp rx=0 tx=0).stop 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 msgr2=0x7f8d6419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000e9f0 tx=0x7f8d6000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 shutdown_connections 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d500b52e0 0x7f8d500b77a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d64108020 0x7f8d6419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d641076a0 0x7f8d6419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 --2- 172.21.15.120:0/15115866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d64101c90 0x7f8d6419a0f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 >> 172.21.15.120:0/15115866 conn(0x7f8d640fd700 msgr2=0x7f8d640ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 shutdown_connections 2024-09-10T22:56:22.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:22.706+0000 7f8d6bb25640 1 -- 172.21.15.120:0/15115866 wait complete. 2024-09-10T22:56:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:22 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:56:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:22 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:22 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:56:23.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:22 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:22 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:56:23.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:22 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:22 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:56:23.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:22 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:22 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:56:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:22 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:22 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:56:23.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:22 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:23.541 INFO:tasks.cephfs.filesystem:scrub status for tag:4777f81f-6a19-4258-a6b1-74ce729177eb - {'path': '/', 'tag': '4777f81f-6a19-4258-a6b1-74ce729177eb', 'options': 'recursive,force'} 2024-09-10T22:56:23.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 632967 17.05 MB/sec execute 78 sec latency 18.630 ms 2024-09-10T22:56:24.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:23 smithi120 ceph-mon[31247]: pgmap v463: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-09-10T22:56:24.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:23 smithi153 ceph-mon[34951]: pgmap v463: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-09-10T22:56:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:23 smithi176 ceph-mon[34982]: pgmap v463: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 -- 172.21.15.120:0/781930565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 msgr2=0x7f5aec106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 --2- 172.21.15.120:0/781930565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec106920 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009ca0 tx=0x7f5ae402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 -- 172.21.15.120:0/781930565 shutdown_connections 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 --2- 172.21.15.120:0/781930565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec106920 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 --2- 172.21.15.120:0/781930565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec1038c0 0x7f5aec105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 --2- 172.21.15.120:0/781930565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec06bc90 0x7f5aec103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 -- 172.21.15.120:0/781930565 >> 172.21.15.120:0/781930565 conn(0x7f5aec0fd130 msgr2=0x7f5aec0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 -- 172.21.15.120:0/781930565 shutdown_connections 2024-09-10T22:56:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 -- 172.21.15.120:0/781930565 wait complete. 2024-09-10T22:56:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.577+0000 7f5af22be640 1 Processor -- start 2024-09-10T22:56:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 -- start start 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec06bc90 0x7f5aec19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec1038c0 0x7f5aec19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec114bf0 con 0x7f5aec106470 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec114a70 con 0x7f5aec1038c0 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af22be640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec114d70 con 0x7f5aec06bc90 2024-09-10T22:56:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52392/0 (socket says 172.21.15.120:52392) 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 -- 172.21.15.120:0/1981042630 learned_addr learned my addr 172.21.15.120:0/1981042630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec06bc90 0x7f5aec19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec1038c0 0x7f5aec19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 -- 172.21.15.120:0/1981042630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec06bc90 msgr2=0x7f5aec19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec06bc90 0x7f5aec19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 -- 172.21.15.120:0/1981042630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec1038c0 msgr2=0x7f5aec19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec1038c0 0x7f5aec19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.578+0000 7f5af085b640 1 -- 172.21.15.120:0/1981042630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec1a0e30 con 0x7f5aec106470 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5af085b640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40097b0 tx=0x7f5ae4031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5ae97fa640 1 -- 172.21.15.120:0/1981042630 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4003b70 con 0x7f5aec106470 2024-09-10T22:56:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5ae97fa640 1 -- 172.21.15.120:0/1981042630 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4003d10 con 0x7f5aec106470 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec1a10c0 con 0x7f5aec106470 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5ae97fa640 1 -- 172.21.15.120:0/1981042630 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae402fc10 con 0x7f5aec106470 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.579+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec1a77f0 con 0x7f5aec106470 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 Processor -- start 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5ae97fa640 1 -- 172.21.15.120:0/1981042630 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5ae4004040 con 0x7f5aec106470 2024-09-10T22:56:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 -- start start 2024-09-10T22:56:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555d246276f0 con 0x555d246b8190 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555d24616b20 con 0x555d246b5860 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.581+0000 7f5af5089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555d2461a930 con 0x555d246babd0 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52402/0 (socket says 172.21.15.120:52402) 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 -- 172.21.15.120:0/1657223176 learned_addr learned my addr 172.21.15.120:0/1657223176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 -- 172.21.15.120:0/1657223176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 msgr2=0x555d246bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 -- 172.21.15.120:0/1657223176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 msgr2=0x555d246b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 -- 172.21.15.120:0/1657223176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d245dc530 con 0x555d246b8190 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5ae97fa640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 0x7f5ad40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009cf0 tx=0x7f5adc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b82466bfde5781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 0x7f5ad40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5ad27fc640 1 -- 172.21.15.120:0/1657223176 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc004230 con 0x555d246b8190 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5ae97fa640 1 -- 172.21.15.120:0/1981042630 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f5ae4106ef0 con 0x7f5aec106470 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5ad27fc640 1 -- 172.21.15.120:0/1657223176 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5adc0043d0 con 0x555d246b8190 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.582+0000 7f5ad27fc640 1 -- 172.21.15.120:0/1657223176 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc019880 con 0x555d246b8190 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- 172.21.15.120:0/1657223176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 msgr2=0x555d246ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009cf0 tx=0x7f5adc011810 comp rx=0 tx=0).stop 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 0x7f5ad40b77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009fc0 tx=0x7f5ae0013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- 172.21.15.120:0/1657223176 shutdown_connections 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246ba580 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 --2- 172.21.15.120:0/1657223176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- 172.21.15.120:0/1657223176 >> 172.21.15.120:0/1657223176 conn(0x555d246b09b0 msgr2=0x555d246b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- 172.21.15.120:0/1657223176 shutdown_connections 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- 172.21.15.120:0/1657223176 wait complete. 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.583+0000 7f5af5089740 1 -- start start 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 Processor -- start 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555d2454eab0 con 0x555d246b5860 2024-09-10T22:56:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555d245bc730 con 0x555d246b8190 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555d246bdbb0 con 0x555d246babd0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af5089740 10 client.0 _ll_register_callbacks cb 0x555d246106b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52404/0 (socket says 172.21.15.120:52404) 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 -- 172.21.15.120:0/1205423389 learned_addr learned my addr 172.21.15.120:0/1205423389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5af085b640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 -- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 msgr2=0x555d246c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 -- 172.21.15.120:0/1205423389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 msgr2=0x555d246bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 -- 172.21.15.120:0/1205423389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d24378790 con 0x555d246b5860 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5aebfff640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000c00 tx=0x7f5adc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275806a76ada4e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5acb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.584+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc027070 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f5adc005040 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5adc0043d0 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5adc033050 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 10 client.16149 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555d243f1620 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5adc0043d0 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f5adc004bb0 con 0x555d246b5860 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 10 client.16149 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 20 client.16149 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d246ca500 0x555d246cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 4 client.16149 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d24629640 con 0x555d246ca500 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.585+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d246ca500 0x555d246cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.586+0000 7f5acb7fe640 10 client.16149 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.586+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d246ca500 0x555d246cc8f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49382d9de9965b60 server_cookie=af72c882f3cbc006 in_seq=0 out_seq=0 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5acb7fe640 1 -- 172.21.15.120:0/1205423389 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d24629640 con 0x555d246ca500 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5acb7fe640 10 client.16149 handle_command_reply: tid=0 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 client.16149 shutdown 2024-09-10T22:56:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 msgr2=0x555d246beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000c00 tx=0x7f5adc003820 comp rx=0 tx=0).stop 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 shutdown_connections 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d246ca500 0x555d246cc8f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 --2- 172.21.15.120:0/1205423389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b5860 0x555d246beec0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 >> 172.21.15.120:0/1205423389 conn(0x555d246b09b0 msgr2=0x555d246afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5acb7fe640 0 client.16149 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 shutdown_connections 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 1 -- 172.21.15.120:0/1205423389 wait complete. 2024-09-10T22:56:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.587+0000 7f5af5089740 20 client.16149 trim_cache size 0 max 16384 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 Processor -- start 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 -- start start 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246b5860 0x555d246afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1ae320 con 0x555d246babd0 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1b0ba0 con 0x555d246b8190 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af5089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1ae990 con 0x555d246b5860 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af085b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af085b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52412/0 (socket says 172.21.15.120:52412) 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5af085b640 1 -- 172.21.15.120:0/3539695926 learned_addr learned my addr 172.21.15.120:0/3539695926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.603+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246b5860 0x555d246afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 -- 172.21.15.120:0/3539695926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246b5860 msgr2=0x555d246afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246b5860 0x555d246afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 -- 172.21.15.120:0/3539695926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 msgr2=0x555d246b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 -- 172.21.15.120:0/3539695926 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d246b1dc0 con 0x555d246babd0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af085b640 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ca950 tx=0x7f5ae4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf854ae531f1b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5ad1ffb640 1 -- 172.21.15.120:0/3539695926 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40cd1f0 con 0x555d246babd0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5ad1ffb640 1 -- 172.21.15.120:0/3539695926 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4036040 con 0x555d246babd0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5ad1ffb640 1 -- 172.21.15.120:0/3539695926 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae402fdb0 con 0x555d246babd0 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 -- 172.21.15.120:0/3539695926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 msgr2=0x555d246b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ca950 tx=0x7f5ae4004580 comp rx=0 tx=0).stop 2024-09-10T22:56:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 -- 172.21.15.120:0/3539695926 shutdown_connections 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246babd0 0x555d246b2d20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b8190 0x555d246b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 --2- 172.21.15.120:0/3539695926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246b5860 0x555d246afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.604+0000 7f5af5089740 1 -- 172.21.15.120:0/3539695926 >> 172.21.15.120:0/3539695926 conn(0x555d246b09b0 msgr2=0x555d246af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- 172.21.15.120:0/3539695926 shutdown_connections 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- 172.21.15.120:0/3539695926 wait complete. 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- start start 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 Processor -- start 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1af060 con 0x555d246b8190 2024-09-10T22:56:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1b04d0 con 0x555d246b5860 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1af730 con 0x555d246babd0 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5aeb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af5089740 10 client.0 _ll_register_callbacks cb 0x7f5ae00097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5aeb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52420/0 (socket says 172.21.15.120:52420) 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5aeb7fe640 1 -- 172.21.15.120:0/2943900570 learned_addr learned my addr 172.21.15.120:0/2943900570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.605+0000 7f5af085b640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aebfff640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 -- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 msgr2=0x555d246c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 -- 172.21.15.120:0/2943900570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 msgr2=0x555d246c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 -- 172.21.15.120:0/2943900570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d246b7fc0 con 0x555d246b8190 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af085b640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5aeb7fe640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ae001e9e0 tx=0x7f5ae0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612ca79df0afb7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0037060 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f5ae002c430 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae4036040 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ae001be10 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 10 client.16155 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555d246c4bb0 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4036040 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f5ae00327f0 con 0x555d246b8190 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 10 client.16155 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 20 client.16155 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d247379f0 0x555d24739e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 4 client.16155 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.606+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d246250f0 con 0x555d247379f0 2024-09-10T22:56:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.607+0000 7f5aebfff640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d247379f0 0x555d24739e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.607+0000 7f5acb7fe640 10 client.16155 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.607+0000 7f5aebfff640 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d247379f0 0x555d24739e10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11010d77f4329fc8 server_cookie=541987d8dfe6ac97 in_seq=0 out_seq=0 2024-09-10T22:56:24.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 636306 17.07 MB/sec execute 79 sec latency 14.080 ms 2024-09-10T22:56:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5acb7fe640 1 -- 172.21.15.120:0/2943900570 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d246250f0 con 0x555d247379f0 2024-09-10T22:56:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5acb7fe640 10 client.16155 handle_command_reply: tid=0 2024-09-10T22:56:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 client.16155 shutdown 2024-09-10T22:56:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 msgr2=0x555d246c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ae001e9e0 tx=0x7f5ae0009940 comp rx=0 tx=0).stop 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 shutdown_connections 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555d247379f0 0x555d24739e10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d246babd0 0x555d246c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555d246b8190 0x555d246c53f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5acb7fe640 0 client.16155 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.851+0000 7f5af5089740 1 --2- 172.21.15.120:0/2943900570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d246b5860 0x555d246c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 >> 172.21.15.120:0/2943900570 conn(0x555d246b09b0 msgr2=0x555d246b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 shutdown_connections 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af5089740 1 -- 172.21.15.120:0/2943900570 wait complete. 2024-09-10T22:56:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af5089740 20 client.16155 trim_cache size 0 max 16384 2024-09-10T22:56:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:56:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:56:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:56:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:56:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 msgr2=0x7f5ad40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 0x7f5ad40b77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009fc0 tx=0x7f5ae0013040 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 msgr2=0x7f5aec1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40097b0 tx=0x7f5ae4031770 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 shutdown_connections 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ad40b52e0 0x7f5ad40b77a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5aec106470 0x7f5aec1a0570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aec1038c0 0x7f5aec19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 --2- 172.21.15.120:0/1981042630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5aec06bc90 0x7f5aec19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 >> 172.21.15.120:0/1981042630 conn(0x7f5aec0fd130 msgr2=0x7f5aec101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 shutdown_connections 2024-09-10T22:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:24.852+0000 7f5af22be640 1 -- 172.21.15.120:0/1981042630 wait complete. 2024-09-10T22:56:25.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 639245 17.08 MB/sec execute 80 sec latency 80.809 ms 2024-09-10T22:56:26.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:25 smithi120 ceph-mon[31247]: pgmap v464: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 397 op/s 2024-09-10T22:56:26.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:25 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:56:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:25 smithi153 ceph-mon[34951]: pgmap v464: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 397 op/s 2024-09-10T22:56:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:25 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:56:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:25 smithi176 ceph-mon[34982]: pgmap v464: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 397 op/s 2024-09-10T22:56:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:25 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:56:26.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 641864 16.99 MB/sec execute 81 sec latency 30.443 ms 2024-09-10T22:56:27.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 645229 17.00 MB/sec execute 82 sec latency 22.354 ms 2024-09-10T22:56:28.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:27 smithi120 ceph-mon[31247]: pgmap v465: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 571 op/s 2024-09-10T22:56:28.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:27 smithi153 ceph-mon[34951]: pgmap v465: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 571 op/s 2024-09-10T22:56:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:27 smithi176 ceph-mon[34982]: pgmap v465: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 571 op/s 2024-09-10T22:56:28.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 648752 17.03 MB/sec execute 83 sec latency 11.612 ms 2024-09-10T22:56:29.621 INFO:tasks.workunit.client.0.smithi120.stdout: 1 651885 17.03 MB/sec execute 84 sec latency 12.481 ms 2024-09-10T22:56:30.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:29 smithi153 ceph-mon[34951]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-09-10T22:56:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:29 smithi176 ceph-mon[34982]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-09-10T22:56:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:29 smithi120 ceph-mon[31247]: pgmap v466: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-09-10T22:56:30.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 654946 17.05 MB/sec execute 85 sec latency 19.488 ms 2024-09-10T22:56:31.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 657576 17.02 MB/sec execute 86 sec latency 75.236 ms 2024-09-10T22:56:32.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:31 smithi176 ceph-mon[34982]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2024-09-10T22:56:32.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:31 smithi120 ceph-mon[31247]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2024-09-10T22:56:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:31 smithi153 ceph-mon[34951]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 520 op/s 2024-09-10T22:56:32.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 660970 17.04 MB/sec execute 87 sec latency 11.389 ms 2024-09-10T22:56:33.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 664401 17.06 MB/sec execute 88 sec latency 16.671 ms 2024-09-10T22:56:34.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:33 smithi176 ceph-mon[34982]: pgmap v468: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-09-10T22:56:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:33 smithi120 ceph-mon[31247]: pgmap v468: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-09-10T22:56:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:33 smithi153 ceph-mon[34951]: pgmap v468: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-09-10T22:56:34.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 667854 17.08 MB/sec execute 89 sec latency 12.131 ms 2024-09-10T22:56:35.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 671237 17.09 MB/sec execute 90 sec latency 21.584 ms 2024-09-10T22:56:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:35 smithi176 ceph-mon[34982]: pgmap v469: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-10T22:56:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:35 smithi120 ceph-mon[31247]: pgmap v469: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-10T22:56:36.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:35 smithi153 ceph-mon[34951]: pgmap v469: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-10T22:56:36.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 673673 17.00 MB/sec execute 91 sec latency 55.387 ms 2024-09-10T22:56:37.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 676470 17.00 MB/sec execute 92 sec latency 16.062 ms 2024-09-10T22:56:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 -- 172.21.15.120:0/1300164980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4640105ad0 msgr2=0x7f4640107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 --2- 172.21.15.120:0/1300164980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4640105ad0 0x7f4640107f40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4634009900 tx=0x7f463402a410 comp rx=0 tx=0).stop 2024-09-10T22:56:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 -- 172.21.15.120:0/1300164980 shutdown_connections 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 --2- 172.21.15.120:0/1300164980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 0x7f464010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 --2- 172.21.15.120:0/1300164980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4640105ad0 0x7f4640107f40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 --2- 172.21.15.120:0/1300164980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f464006bc90 0x7f4640105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.949+0000 7f46463f3640 1 -- 172.21.15.120:0/1300164980 >> 172.21.15.120:0/1300164980 conn(0x7f46400fd130 msgr2=0x7f46400ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.950+0000 7f46463f3640 1 -- 172.21.15.120:0/1300164980 shutdown_connections 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.950+0000 7f46463f3640 1 -- 172.21.15.120:0/1300164980 wait complete. 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.950+0000 7f46463f3640 1 Processor -- start 2024-09-10T22:56:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.950+0000 7f46463f3640 1 -- start start 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4640105ad0 0x7f4640199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 0x7f464019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f464010e600 con 0x7f464006bc90 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f464010e480 con 0x7f4640108680 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f46463f3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f464010e780 con 0x7f4640105ad0 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57482/0 (socket says 172.21.15.120:57482) 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 -- 172.21.15.120:0/1710964109 learned_addr learned my addr 172.21.15.120:0/1710964109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463f7fe640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4640105ad0 0x7f4640199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f4644990640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 0x7f464019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 -- 172.21.15.120:0/1710964109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4640105ad0 msgr2=0x7f4640199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4640105ad0 0x7f4640199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 -- 172.21.15.120:0/1710964109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 msgr2=0x7f464019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 0x7f464019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.951+0000 7f463ffff640 1 -- 172.21.15.120:0/1710964109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464019a920 con 0x7f464006bc90 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f463ffff640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f463000d950 tx=0x7f463000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f463d7fa640 1 -- 172.21.15.120:0/1710964109 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630017070 con 0x7f464006bc90 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f463d7fa640 1 -- 172.21.15.120:0/1710964109 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4630009c30 con 0x7f464006bc90 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f463d7fa640 1 -- 172.21.15.120:0/1710964109 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463000f620 con 0x7f464006bc90 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464019ac10 con 0x7f464006bc90 2024-09-10T22:56:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.952+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4640101500 con 0x7f464006bc90 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 Processor -- start 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 -- start start 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f463d7fa640 1 -- 172.21.15.120:0/1710964109 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4630010460 con 0x7f464006bc90 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563da33906f0 con 0x563da34299a0 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563da337fb20 con 0x563da3427070 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.955+0000 7f46491c6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563da3383930 con 0x563da342c3e0 2024-09-10T22:56:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37080/0 (socket says 172.21.15.120:37080) 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 -- 172.21.15.120:0/1280270465 learned_addr learned my addr 172.21.15.120:0/1280270465 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f463ffff640 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 -- 172.21.15.120:0/1280270465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 msgr2=0x563da342e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 -- 172.21.15.120:0/1280270465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 msgr2=0x563da342bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 -- 172.21.15.120:0/1280270465 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da3345530 con 0x563da3427070 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.956+0000 7f4644990640 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46380095d0 tx=0x7f46380099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e3d4ef49d3285f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.957+0000 7f463d7fa640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 0x7f46280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.957+0000 7f463ffff640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 0x7f46280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.957+0000 7f463d7fa640 1 -- 172.21.15.120:0/1710964109 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f46300e5790 con 0x7f464006bc90 2024-09-10T22:56:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.957+0000 7f463ffff640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 0x7f46280b77a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f46300a8600 tx=0x7f4630010cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.960+0000 7f46267fc640 1 -- 172.21.15.120:0/1280270465 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638004d20 con 0x563da3427070 2024-09-10T22:56:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.960+0000 7f46267fc640 1 -- 172.21.15.120:0/1280270465 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46380054d0 con 0x563da3427070 2024-09-10T22:56:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.960+0000 7f46267fc640 1 -- 172.21.15.120:0/1280270465 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463801a8c0 con 0x563da3427070 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- 172.21.15.120:0/1280270465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 msgr2=0x563da3429460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46380095d0 tx=0x7f46380099d0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- 172.21.15.120:0/1280270465 shutdown_connections 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- 172.21.15.120:0/1280270465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da3429460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- 172.21.15.120:0/1280270465 >> 172.21.15.120:0/1280270465 conn(0x563da34199b0 msgr2=0x563da341a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- 172.21.15.120:0/1280270465 shutdown_connections 2024-09-10T22:56:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- 172.21.15.120:0/1280270465 wait complete. 2024-09-10T22:56:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 -- start start 2024-09-10T22:56:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 Processor -- start 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.961+0000 7f46491c6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da3431950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563da32b7ab0 con 0x563da34299a0 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563da3325730 con 0x563da3427070 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563da342f3c0 con 0x563da342c3e0 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 10 client.0 _ll_register_callbacks cb 0x563da33796b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37082/0 (socket says 172.21.15.120:37082) 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 -- 172.21.15.120:0/2010676162 learned_addr learned my addr 172.21.15.120:0/2010676162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57504/0 (socket says 172.21.15.120:57504) 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da3431950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 -- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 msgr2=0x563da3431950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da3431950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 -- 172.21.15.120:0/2010676162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 msgr2=0x563da3430d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 -- 172.21.15.120:0/2010676162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da30e1790 con 0x563da3427070 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f463f7fe640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f4644990640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4638010650 tx=0x7f4638010520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e391515d24aaa9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f461f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463801aaa0 con 0x563da3427070 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46380054d0 con 0x563da3427070 2024-09-10T22:56:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.962+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f463801ac40 con 0x563da3427070 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f463801ae40 con 0x563da3427070 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 10 client.25622 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563da315a620 con 0x563da3427070 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46380054d0 con 0x563da3427070 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f4638004030 con 0x563da3427070 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 10 client.25622 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 20 client.25622 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da343bca0 0x563da343e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 4 client.25622 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da3392640 con 0x563da343bca0 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.963+0000 7f463f7fe640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da343bca0 0x563da343e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.964+0000 7f463f7fe640 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da343bca0 0x563da343e090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43f8e2b2022e61a server_cookie=2e04209769dfa4cb in_seq=0 out_seq=0 2024-09-10T22:56:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.964+0000 7f461f7fe640 10 client.25622 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f461f7fe640 1 -- 172.21.15.120:0/2010676162 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563da3392640 con 0x563da343bca0 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f461f7fe640 10 client.25622 handle_command_reply: tid=0 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 client.25622 shutdown 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 msgr2=0x563da34306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4638010650 tx=0x7f4638010520 comp rx=0 tx=0).stop 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 shutdown_connections 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da343bca0 0x563da343e090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da3431950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da3430d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 --2- 172.21.15.120:0/2010676162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da34306e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 >> 172.21.15.120:0/2010676162 conn(0x563da34199b0 msgr2=0x563da3418bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f461f7fe640 0 client.25622 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 shutdown_connections 2024-09-10T22:56:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 1 -- 172.21.15.120:0/2010676162 wait complete. 2024-09-10T22:56:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.966+0000 7f46491c6740 20 client.25622 trim_cache size 0 max 16384 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 Processor -- start 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 -- start start 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da342f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a5cc0 con 0x563da34299a0 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a8540 con 0x563da3427070 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f46491c6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a6330 con 0x563da342c3e0 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57506/0 (socket says 172.21.15.120:57506) 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 -- 172.21.15.120:0/1373118905 learned_addr learned my addr 172.21.15.120:0/1373118905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f4644990640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f4644990640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da342f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 -- 172.21.15.120:0/1373118905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 msgr2=0x563da342cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 -- 172.21.15.120:0/1373118905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 msgr2=0x563da342f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da342f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 -- 172.21.15.120:0/1373118905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da34297d0 con 0x563da34299a0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f4644990640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.982+0000 7f463f7fe640 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f46340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495a5e70f2747181 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f4625ffb640 1 -- 172.21.15.120:0/1373118905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634035040 con 0x563da34299a0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f4625ffb640 1 -- 172.21.15.120:0/1373118905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634003ca0 con 0x563da34299a0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f4625ffb640 1 -- 172.21.15.120:0/1373118905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634031dd0 con 0x563da34299a0 2024-09-10T22:56:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 -- 172.21.15.120:0/1373118905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 msgr2=0x563da342c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f46340042f0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 -- 172.21.15.120:0/1373118905 shutdown_connections 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da342c3e0 0x563da342cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da34299a0 0x563da342c7c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 --2- 172.21.15.120:0/1373118905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da3427070 0x563da342f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 -- 172.21.15.120:0/1373118905 >> 172.21.15.120:0/1373118905 conn(0x563da34199b0 msgr2=0x563da3426560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 -- 172.21.15.120:0/1373118905 shutdown_connections 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.983+0000 7f46491c6740 1 -- 172.21.15.120:0/1373118905 wait complete. 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 -- start start 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 Processor -- start 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da3427070 0x563da3437c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da34299a0 0x563da34381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a6a00 con 0x563da342c3e0 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a7e70 con 0x563da34299a0 2024-09-10T22:56:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a70d0 con 0x563da3427070 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f46491c6740 10 client.0 _ll_register_callbacks cb 0x7f46380094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f4644990640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f4644990640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57518/0 (socket says 172.21.15.120:57518) 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f4644990640 1 -- 172.21.15.120:0/720220850 learned_addr learned my addr 172.21.15.120:0/720220850 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f463f7fe640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da34299a0 0x563da34381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.984+0000 7f4644990640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da3427070 0x563da3437c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 -- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da3427070 msgr2=0x563da3437c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da3427070 0x563da3437c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 -- 172.21.15.120:0/720220850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da34299a0 msgr2=0x563da34381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da34299a0 0x563da34381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 -- 172.21.15.120:0/720220850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da34a7280 con 0x563da342c3e0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f4644990640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4638004450 tx=0x7f4638012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d4862a9e17e14b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f461f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803f040 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4634003ca0 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f463801abf0 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f46380057f0 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f46491c6740 10 client.16173 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563da34a8590 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.985+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634003ca0 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f4638046020 con 0x563da342c3e0 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 10 client.16173 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 20 client.16173 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da34a8e20 0x563da34ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 4 client.16173 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563da338e0f0 con 0x563da34a8e20 2024-09-10T22:56:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f463f7fe640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da34a8e20 0x563da34ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f461f7fe640 10 client.16173 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:37.986+0000 7f463f7fe640 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da34a8e20 0x563da34ab240 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e436a511eef43f server_cookie=672eb7be3b890901 in_seq=0 out_seq=0 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f461f7fe640 1 -- 172.21.15.120:0/720220850 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563da338e0f0 con 0x563da34a8e20 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f461f7fe640 10 client.16173 handle_command_reply: tid=0 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 client.16173 shutdown 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 msgr2=0x563da34a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4638004450 tx=0x7f4638012040 comp rx=0 tx=0).stop 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 shutdown_connections 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563da34a8e20 0x563da34ab240 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563da342c3e0 0x563da34a6be0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563da34299a0 0x563da34381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 --2- 172.21.15.120:0/720220850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563da3427070 0x563da3437c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 >> 172.21.15.120:0/720220850 conn(0x563da34199b0 msgr2=0x563da34263f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f461f7fe640 0 client.16173 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 shutdown_connections 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 1 -- 172.21.15.120:0/720220850 wait complete. 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.188+0000 7f46491c6740 20 client.16173 trim_cache size 0 max 16384 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:56:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 msgr2=0x7f46280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 0x7f46280b77a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f46300a8600 tx=0x7f4630010cb0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 msgr2=0x7f4640198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f463000d950 tx=0x7f463000de10 comp rx=0 tx=0).stop 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 shutdown_connections 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f46280b52e0 0x7f46280b77a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4640108680 0x7f464019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4640105ad0 0x7f4640199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 --2- 172.21.15.120:0/1710964109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f464006bc90 0x7f4640198de0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 >> 172.21.15.120:0/1710964109 conn(0x7f46400fd130 msgr2=0x7f4640103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 shutdown_connections 2024-09-10T22:56:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:38.189+0000 7f46463f3640 1 -- 172.21.15.120:0/1710964109 wait complete. 2024-09-10T22:56:38.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:37 smithi176 ceph-mon[34982]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-09-10T22:56:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:37 smithi120 ceph-mon[31247]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-09-10T22:56:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:37 smithi153 ceph-mon[34951]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-09-10T22:56:38.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 679031 16.97 MB/sec execute 93 sec latency 92.886 ms 2024-09-10T22:56:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:39.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:38 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:56:39.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:38 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:56:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:38 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:56:39.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 682394 16.99 MB/sec execute 94 sec latency 27.529 ms 2024-09-10T22:56:40.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:39 smithi176 ceph-mon[34982]: pgmap v471: 97 pgs: 97 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-10T22:56:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:39 smithi120 ceph-mon[31247]: pgmap v471: 97 pgs: 97 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-10T22:56:40.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:39 smithi153 ceph-mon[34951]: pgmap v471: 97 pgs: 97 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-10T22:56:40.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 685648 17.01 MB/sec execute 95 sec latency 17.077 ms 2024-09-10T22:56:41.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 688586 16.98 MB/sec execute 96 sec latency 87.109 ms 2024-09-10T22:56:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:41 smithi176 ceph-mon[34982]: pgmap v472: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-09-10T22:56:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:41 smithi120 ceph-mon[31247]: pgmap v472: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-09-10T22:56:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:41 smithi153 ceph-mon[34951]: pgmap v472: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 460 op/s 2024-09-10T22:56:42.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 691439 16.94 MB/sec execute 97 sec latency 30.876 ms 2024-09-10T22:56:43.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 694921 16.96 MB/sec execute 98 sec latency 21.785 ms 2024-09-10T22:56:44.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:43 smithi176 ceph-mon[34982]: pgmap v473: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T22:56:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:43 smithi120 ceph-mon[31247]: pgmap v473: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T22:56:44.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:43 smithi153 ceph-mon[34951]: pgmap v473: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 411 op/s 2024-09-10T22:56:44.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 697996 16.96 MB/sec execute 99 sec latency 12.719 ms 2024-09-10T22:56:45.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 701007 16.98 MB/sec execute 100 sec latency 14.750 ms 2024-09-10T22:56:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:45 smithi176 ceph-mon[34982]: pgmap v474: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T22:56:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:45 smithi120 ceph-mon[31247]: pgmap v474: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T22:56:46.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:45 smithi153 ceph-mon[34951]: pgmap v474: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T22:56:46.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 703797 16.95 MB/sec execute 101 sec latency 98.170 ms 2024-09-10T22:56:47.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 707064 16.97 MB/sec execute 102 sec latency 18.572 ms 2024-09-10T22:56:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:47 smithi176 ceph-mon[34982]: pgmap v475: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-09-10T22:56:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:47 smithi120 ceph-mon[31247]: pgmap v475: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-09-10T22:56:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:47 smithi153 ceph-mon[34951]: pgmap v475: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 516 op/s 2024-09-10T22:56:48.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 710463 16.99 MB/sec execute 103 sec latency 22.719 ms 2024-09-10T22:56:49.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 713855 17.00 MB/sec execute 104 sec latency 16.426 ms 2024-09-10T22:56:50.248 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:49 smithi120 ceph-mon[31247]: pgmap v476: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-09-10T22:56:50.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:49 smithi176 ceph-mon[34982]: pgmap v476: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-09-10T22:56:50.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:49 smithi153 ceph-mon[34951]: pgmap v476: 97 pgs: 97 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-09-10T22:56:50.622 INFO:tasks.workunit.client.0.smithi120.stdout: 1 716960 16.98 MB/sec execute 105 sec latency 16.378 ms 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 -- 172.21.15.120:0/4096540199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b20101890 msgr2=0x7f6b20101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 --2- 172.21.15.120:0/4096540199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b20101890 0x7f6b20101c90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009930 tx=0x7f6b1002a420 comp rx=0 tx=0).stop 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 -- 172.21.15.120:0/4096540199 shutdown_connections 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 --2- 172.21.15.120:0/4096540199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b20109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 --2- 172.21.15.120:0/4096540199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b201029e0 0x7f6b20102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 --2- 172.21.15.120:0/4096540199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b20101890 0x7f6b20101c90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 -- 172.21.15.120:0/4096540199 >> 172.21.15.120:0/4096540199 conn(0x7f6b200fd150 msgr2=0x7f6b200ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 -- 172.21.15.120:0/4096540199 shutdown_connections 2024-09-10T22:56:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.283+0000 7f6b267bb640 1 -- 172.21.15.120:0/4096540199 wait complete. 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 Processor -- start 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 -- start start 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b20101890 0x7f6b20198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b2019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d4e0 con 0x7f6b201029e0 2024-09-10T22:56:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d360 con 0x7f6b201033a0 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.284+0000 7f6b267bb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010d660 con 0x7f6b20101890 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33380/0 (socket says 172.21.15.120:33380) 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b17fff640 1 -- 172.21.15.120:0/3567106238 learned_addr learned my addr 172.21.15.120:0/3567106238 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b20101890 0x7f6b20198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b2019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 -- 172.21.15.120:0/3567106238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b20101890 msgr2=0x7f6b20198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b20101890 0x7f6b20198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 -- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 msgr2=0x7f6b20199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 -- 172.21.15.120:0/3567106238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2019a900 con 0x7f6b201033a0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b17fff640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b24d58640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b2019a040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ec50 tx=0x7f6b1800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b1dffb640 1 -- 172.21.15.120:0/3567106238 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18003b70 con 0x7f6b201033a0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b1dffb640 1 -- 172.21.15.120:0/3567106238 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b18003d10 con 0x7f6b201033a0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b1dffb640 1 -- 172.21.15.120:0/3567106238 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18010830 con 0x7f6b201033a0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.285+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2019abf0 con 0x7f6b201033a0 2024-09-10T22:56:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.286+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a12c0 con 0x7f6b201033a0 2024-09-10T22:56:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 Processor -- start 2024-09-10T22:56:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b1dffb640 1 -- 172.21.15.120:0/3567106238 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6b180109d0 con 0x7f6b201033a0 2024-09-10T22:56:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 -- start start 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926543060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c9265484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9264ae6f0 con 0x55c926545fe0 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c92649db20 con 0x55c926540c70 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b29665740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c9264a1930 con 0x55c9265435a0 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.288+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41776/0 (socket says 172.21.15.120:41776) 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926543060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 -- 172.21.15.120:0/1742285961 learned_addr learned my addr 172.21.15.120:0/1742285961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c9265484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 -- 172.21.15.120:0/1742285961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 msgr2=0x55c926543060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926543060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 -- 172.21.15.120:0/1742285961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 msgr2=0x55c9265484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c9265484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 -- 172.21.15.120:0/1742285961 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c926463530 con 0x55c9265435a0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b1002a940 tx=0x7f6b10005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1ae79b244c4667 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1dffb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b167fc640 1 -- 172.21.15.120:0/1742285961 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10035030 con 0x55c9265435a0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b167fc640 1 -- 172.21.15.120:0/1742285961 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b10040070 con 0x55c9265435a0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b167fc640 1 -- 172.21.15.120:0/1742285961 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10003910 con 0x55c9265435a0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b17fff640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.289+0000 7f6b1dffb640 1 -- 172.21.15.120:0/3567106238 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6b180e6120 con 0x7f6b201033a0 2024-09-10T22:56:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 -- 172.21.15.120:0/1742285961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 msgr2=0x55c926545990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b17fff640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009620 tx=0x7f6b08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b1002a940 tx=0x7f6b10005de0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 -- 172.21.15.120:0/1742285961 shutdown_connections 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c9265484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c926545990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 --2- 172.21.15.120:0/1742285961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926543060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 -- 172.21.15.120:0/1742285961 >> 172.21.15.120:0/1742285961 conn(0x55c9265379b0 msgr2=0x55c926538740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 -- 172.21.15.120:0/1742285961 shutdown_connections 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.290+0000 7f6b29665740 1 -- 172.21.15.120:0/1742285961 wait complete. 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 -- start start 2024-09-10T22:56:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 Processor -- start 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c92654a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c92654b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9263d5ab0 con 0x55c926545fe0 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c926443730 con 0x55c926540c70 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c926548f60 con 0x55c9265435a0 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b29665740 10 client.0 _ll_register_callbacks cb 0x55c9264976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c92654a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41792/0 (socket says 172.21.15.120:41792) 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c92654a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36492/0 (socket says 172.21.15.120:36492) 2024-09-10T22:56:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b17fff640 1 -- 172.21.15.120:0/2338318599 learned_addr learned my addr 172.21.15.120:0/2338318599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.291+0000 7f6b24d58640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c92654b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 -- 172.21.15.120:0/2338318599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 msgr2=0x55c92654a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c92654a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 -- 172.21.15.120:0/2338318599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 msgr2=0x55c92654b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c92654b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 -- 172.21.15.120:0/2338318599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9261ff790 con 0x55c9265435a0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b17fff640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b08011a20 tx=0x7f6b08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68315efb6209db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08030060 con 0x55c9265435a0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b10040070 con 0x55c9265435a0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6b0801a050 con 0x55c9265435a0 2024-09-10T22:56:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6b08023b50 con 0x55c9265435a0 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b29665740 10 client.25660 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.292+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c926278620 con 0x55c9265435a0 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b10040070 con 0x55c9265435a0 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6b08010430 con 0x55c9265435a0 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 10 client.25660 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 20 client.25660 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c926555b90 0x55c926557fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 4 client.25660 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9264b0640 con 0x55c926555b90 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c926555b90 0x55c926557fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6aff7fe640 10 client.25660 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.293+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c926555b90 0x55c926557fc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371451eb51cbec5f server_cookie=f2a1b21660e2c905 in_seq=0 out_seq=0 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.294+0000 7f6aff7fe640 1 -- 172.21.15.120:0/2338318599 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9264b0640 con 0x55c926555b90 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.294+0000 7f6aff7fe640 10 client.25660 handle_command_reply: tid=0 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.294+0000 7f6b29665740 1 client.25660 shutdown 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 msgr2=0x55c92654a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b08011a20 tx=0x7f6b08004730 comp rx=0 tx=0).stop 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 shutdown_connections 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c926555b90 0x55c926557fc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926545fe0 0x55c92654b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6aff7fe640 0 client.25660 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9265435a0 0x55c92654a940 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 --2- 172.21.15.120:0/2338318599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c92654a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 >> 172.21.15.120:0/2338318599 conn(0x55c9265379b0 msgr2=0x55c926536bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 shutdown_connections 2024-09-10T22:56:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 1 -- 172.21.15.120:0/2338318599 wait complete. 2024-09-10T22:56:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.295+0000 7f6b29665740 20 client.25660 trim_cache size 0 max 16384 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 Processor -- start 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 -- start start 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926549360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c92655c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a5c50 con 0x55c9265435a0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a84b0 con 0x55c926540c70 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b29665740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a62a0 con 0x55c926545fe0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33414/0 (socket says 172.21.15.120:33414) 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b17fff640 1 -- 172.21.15.120:0/1795254416 learned_addr learned my addr 172.21.15.120:0/1795254416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b24d58640 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c92655c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.312+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926549360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 -- 172.21.15.120:0/1795254416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 msgr2=0x55c92655c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c92655c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 -- 172.21.15.120:0/1795254416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 msgr2=0x55c926549360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926549360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 -- 172.21.15.120:0/1795254416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c926538210 con 0x55c9265435a0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b08010800 tx=0x7f6b08009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b0dd5928674b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b15ffb640 1 -- 172.21.15.120:0/1795254416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08010430 con 0x55c9265435a0 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b15ffb640 1 -- 172.21.15.120:0/1795254416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b080105d0 con 0x55c9265435a0 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.313+0000 7f6b15ffb640 1 -- 172.21.15.120:0/1795254416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08037900 con 0x55c9265435a0 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- 172.21.15.120:0/1795254416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 msgr2=0x55c9265498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b08010800 tx=0x7f6b08009f50 comp rx=0 tx=0).stop 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- 172.21.15.120:0/1795254416 shutdown_connections 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c92655c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9265435a0 0x55c9265498a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 --2- 172.21.15.120:0/1795254416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c926540c70 0x55c926549360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- 172.21.15.120:0/1795254416 >> 172.21.15.120:0/1795254416 conn(0x55c9265379b0 msgr2=0x55c92653aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- 172.21.15.120:0/1795254416 shutdown_connections 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- 172.21.15.120:0/1795254416 wait complete. 2024-09-10T22:56:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 -- start start 2024-09-10T22:56:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.314+0000 7f6b29665740 1 Processor -- start 2024-09-10T22:56:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 0x55c926550c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c9265512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a6970 con 0x55c926540c70 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a7de0 con 0x55c9265435a0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a7040 con 0x55c926545fe0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b29665740 10 client.0 _ll_register_callbacks cb 0x7f6b08011970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33424/0 (socket says 172.21.15.120:33424) 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c9265512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b1ffff640 1 -- 172.21.15.120:0/1338501464 learned_addr learned my addr 172.21.15.120:0/1338501464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 0x55c926550c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 -- 172.21.15.120:0/1338501464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 msgr2=0x55c926550c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 0x55c926550c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 -- 172.21.15.120:0/1338501464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 msgr2=0x55c926551f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 -- 172.21.15.120:0/1338501464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92654f720 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b17fff640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 0x55c926550c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6b24d58640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c9265512d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6b180e3bf0 tx=0x7f6b180a8dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99c1cf609f50cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.315+0000 7f6aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18004660 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8182+0+0 (secure 0 0 0) 0x7f6b180b5bc0 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b080105d0 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6b180ac3c0 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 10 client.25666 fetch_fsmap learned FSMap version 23 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c9265c2d30 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b080105d0 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6b180a8390 con 0x55c926545fe0 2024-09-10T22:56:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 10 client.25666 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 20 client.25666 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9265c3710 0x55c9265c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 4 client.25666 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9264ac0f0 con 0x55c9265c3710 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.316+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9265c3710 0x55c9265c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.317+0000 7f6aff7fe640 10 client.25666 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.317+0000 7f6b1ffff640 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9265c3710 0x55c9265c5b30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a790eeff9adf16 server_cookie=58143a20cf99117c in_seq=0 out_seq=0 2024-09-10T22:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6aff7fe640 1 -- 172.21.15.120:0/1338501464 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9264ac0f0 con 0x55c9265c3710 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6aff7fe640 10 client.25666 handle_command_reply: tid=0 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 client.25666 shutdown 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 msgr2=0x55c9265512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c9265512d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6b180e3bf0 tx=0x7f6b180a8dd0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 shutdown_connections 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9265c3710 0x55c9265c5b30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c926545fe0 0x55c9265512d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6aff7fe640 0 client.25666 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9265435a0 0x55c926550c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.473+0000 7f6b29665740 1 --2- 172.21.15.120:0/1338501464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c926540c70 0x55c926551f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 >> 172.21.15.120:0/1338501464 conn(0x55c9265379b0 msgr2=0x55c926549360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 shutdown_connections 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b29665740 1 -- 172.21.15.120:0/1338501464 wait complete. 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b29665740 20 client.25666 trim_cache size 0 max 16384 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:56:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 msgr2=0x7f6b0c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009620 tx=0x7f6b08005f90 comp rx=0 tx=0).stop 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 msgr2=0x7f6b2019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.474+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b2019a040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ec50 tx=0x7f6b1800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 shutdown_connections 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b201033a0 0x7f6b2019a040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b201029e0 0x7f6b20199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 --2- 172.21.15.120:0/3567106238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b20101890 0x7f6b20198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 >> 172.21.15.120:0/3567106238 conn(0x7f6b200fd150 msgr2=0x7f6b200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 shutdown_connections 2024-09-10T22:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:56:51.475+0000 7f6b267bb640 1 -- 172.21.15.120:0/3567106238 wait complete. 2024-09-10T22:56:51.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 718564 16.89 MB/sec execute 106 sec latency 91.049 ms 2024-09-10T22:56:51.989 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setuser ceph since I am not root 2024-09-10T22:56:51.990 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setgroup ceph since I am not root 2024-09-10T22:56:51.990 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/2874996333 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558af9e69860 con 0x558af9e47400 2024-09-10T22:56:51.990 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T22:56:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-mon[34982]: pgmap v477: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 479 op/s 2024-09-10T22:56:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:51 smithi176 ceph-mon[34982]: daemon mds.c restarted 2024-09-10T22:56:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:51 smithi120 ceph-mon[31247]: pgmap v477: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 479 op/s 2024-09-10T22:56:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:51 smithi120 ceph-mon[31247]: daemon mds.c restarted 2024-09-10T22:56:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:51 smithi153 ceph-mon[34951]: pgmap v477: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 479 op/s 2024-09-10T22:56:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:51 smithi153 ceph-mon[34951]: daemon mds.c restarted 2024-09-10T22:56:52.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 721423 16.89 MB/sec execute 107 sec latency 43.304 ms 2024-09-10T22:56:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:52 smithi176 ceph-mon[34982]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:52 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/1259604772,v1:172.21.15.176:6837/1259604772] up:boot 2024-09-10T22:56:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:52 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:56:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:56:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:52 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:56:53.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:52 smithi120 ceph-mon[31247]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:53.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:52 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/1259604772,v1:172.21.15.176:6837/1259604772] up:boot 2024-09-10T22:56:53.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:52 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:56:53.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:56:53.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:52 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:56:53.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:52 smithi153 ceph-mon[34951]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:56:53.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:52 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/1259604772,v1:172.21.15.176:6837/1259604772] up:boot 2024-09-10T22:56:53.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:52 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T22:56:53.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:56:53.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:52 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:56:53.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:56:53.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 724664 16.91 MB/sec execute 108 sec latency 32.232 ms 2024-09-10T22:56:53.802 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:54.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:53 smithi120 ceph-mon[31247]: pgmap v479: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 325 op/s 2024-09-10T22:56:54.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:53 smithi176 ceph-mon[34982]: pgmap v479: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 325 op/s 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 -- 172.21.15.120:0/1468441432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 msgr2=0x7ff5301040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 --2- 172.21.15.120:0/1468441432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff5301040c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff520009900 tx=0x7ff52002a410 comp rx=0 tx=0).stop 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 -- 172.21.15.120:0/1468441432 shutdown_connections 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 --2- 172.21.15.120:0/1468441432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 0x7ff53010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 --2- 172.21.15.120:0/1468441432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff53010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 --2- 172.21.15.120:0/1468441432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff5301040c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 -- 172.21.15.120:0/1468441432 >> 172.21.15.120:0/1468441432 conn(0x7ff5300fd700 msgr2=0x7ff5300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 -- 172.21.15.120:0/1468441432 shutdown_connections 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.310+0000 7ff534e72640 1 -- 172.21.15.120:0/1468441432 wait complete. 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 Processor -- start 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 -- start start 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff530199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 0x7ff53019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff530112db0 con 0x7ff530103ce0 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff530112c30 con 0x7ff53010d140 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff534e72640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff530112f30 con 0x7ff530104600 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52748/0 (socket says 172.21.15.120:52748) 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff52e59c640 1 -- 172.21.15.120:0/1278376954 learned_addr learned my addr 172.21.15.120:0/1278376954 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.311+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff530199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 0x7ff53019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 -- 172.21.15.120:0/1278376954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 msgr2=0x7ff53019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 0x7ff53019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 -- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 msgr2=0x7ff530199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 -- 172.21.15.120:0/1278376954 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53019f3b0 con 0x7ff530104600 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff52e59c640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff530199840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e790 tx=0x7ff51800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff527fff640 1 -- 172.21.15.120:0/1278376954 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800eea0 con 0x7ff530104600 2024-09-10T22:56:54.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff527fff640 1 -- 172.21.15.120:0/1278376954 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff51800ccc0 con 0x7ff530104600 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53019f6a0 con 0x7ff530104600 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff527fff640 1 -- 172.21.15.120:0/1278376954 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518010430 con 0x7ff530104600 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.312+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301a5d60 con 0x7ff530104600 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.315+0000 7ff527fff640 1 -- 172.21.15.120:0/1278376954 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff518003f20 con 0x7ff530104600 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.315+0000 7ff537247740 1 Processor -- start 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.315+0000 7ff537247740 1 -- start start 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb50a0 0x56329ffb5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffc3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56329ff2be50 con 0x56329ffc1950 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56329ff1b280 con 0x56329ffb50a0 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff537247740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56329ff1f090 con 0x56329ffb59c0 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffc3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55968/0 (socket says 172.21.15.120:55968) 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 -- 172.21.15.120:0/2661277948 learned_addr learned my addr 172.21.15.120:0/2661277948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb50a0 0x56329ffb5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff527fff640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 0x7ff51c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 -- 172.21.15.120:0/2661277948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb50a0 msgr2=0x56329ffb5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb50a0 0x56329ffb5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 -- 172.21.15.120:0/2661277948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 msgr2=0x56329ffc3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffc3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 -- 172.21.15.120:0/2661277948 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329fc79650 con 0x56329ffb59c0 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 0x7ff51c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff527fff640 1 -- 172.21.15.120:0/1278376954 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5180e5f20 con 0x7ff530104600 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.316+0000 7ff52e59c640 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=527745605efbca95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 0x7ff51c0b77a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff528005fd0 tx=0x7ff528004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff524d99640 1 -- 172.21.15.120:0/2661277948 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520035040 con 0x56329ffb59c0 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff524d99640 1 -- 172.21.15.120:0/2661277948 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff520031820 con 0x56329ffb59c0 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff524d99640 1 -- 172.21.15.120:0/2661277948 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520004a00 con 0x56329ffb59c0 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 -- 172.21.15.120:0/2661277948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 msgr2=0x56329ffc1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520005be0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 -- 172.21.15.120:0/2661277948 shutdown_connections 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffc3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffc1410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 --2- 172.21.15.120:0/2661277948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb50a0 0x56329ffb5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.317+0000 7ff537247740 1 -- 172.21.15.120:0/2661277948 >> 172.21.15.120:0/2661277948 conn(0x56329ffb2ed0 msgr2=0x56329ffb4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- 172.21.15.120:0/2661277948 shutdown_connections 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- 172.21.15.120:0/2661277948 wait complete. 2024-09-10T22:56:54.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- start start 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 Processor -- start 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb50a0 0x56329ffc1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56329fd7c700 con 0x56329ffc1950 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56329feb6110 con 0x56329ffb59c0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56329ffc4570 con 0x56329ffb50a0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff537247740 10 client.0 _ll_register_callbacks cb 0x56329fecf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb50a0 0x56329ffc1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52756/0 (socket says 172.21.15.120:52756) 2024-09-10T22:56:54.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.318+0000 7ff52ed9d640 1 -- 172.21.15.120:0/2227838928 learned_addr learned my addr 172.21.15.120:0/2227838928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 -- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb50a0 msgr2=0x56329ffc1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb50a0 0x56329ffc1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 -- 172.21.15.120:0/2227838928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 msgr2=0x56329ffbe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffbe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 -- 172.21.15.120:0/2227838928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329fee3c40 con 0x56329ffc1950 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff525d9b640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff528012880 tx=0x7ff528004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8d29cd733404e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52801db70 con 0x56329ffc1950 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff520031820 con 0x56329ffc1950 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff52801dd10 con 0x56329ffc1950 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff528027470 con 0x56329ffc1950 2024-09-10T22:56:54.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff537247740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56329ffcff40 con 0x56329ffc1950 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff520031820 con 0x56329ffc1950 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.319+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7ff52801e720 con 0x56329ffc1950 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 20 client.16209 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x56329ffd3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 4 client.16209 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56329ff2dda0 con 0x56329ffd1760 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff52e59c640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x56329ffd3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff515ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:54.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.320+0000 7ff52e59c640 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x56329ffd3be0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a197ab7b535569 server_cookie=71c75b1de7426da0 in_seq=0 out_seq=0 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff515ffb640 1 -- 172.21.15.120:0/2227838928 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56329ff2dda0 con 0x56329ffd1760 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff515ffb640 10 client.16209 handle_command_reply: tid=0 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 client.16209 shutdown 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 msgr2=0x56329ffbecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff528012880 tx=0x7ff528004ab0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 shutdown_connections 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x56329ffd3be0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffc1950 0x56329ffbecb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff515ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 --2- 172.21.15.120:0/2227838928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb50a0 0x56329ffc1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.321+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 >> 172.21.15.120:0/2227838928 conn(0x56329ffb2ed0 msgr2=0x56329ffb8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.322+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 shutdown_connections 2024-09-10T22:56:54.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.322+0000 7ff537247740 1 -- 172.21.15.120:0/2227838928 wait complete. 2024-09-10T22:56:54.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.323+0000 7ff537247740 20 client.16209 trim_cache size 0 max 16384 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.340+0000 7ff537247740 1 Processor -- start 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.340+0000 7ff537247740 1 -- start start 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffcd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 0x56329ffb44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ac2f0 con 0x56329ffb50a0 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301aeb70 con 0x56329ffb59c0 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff537247740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ac960 con 0x56329ffc1950 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffcd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 0x56329ffb44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52744/0 (socket says 172.21.15.120:52744) 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 0x56329ffb44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55992/0 (socket says 172.21.15.120:55992) 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 -- 172.21.15.120:0/1456893472 learned_addr learned my addr 172.21.15.120:0/1456893472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 -- 172.21.15.120:0/1456893472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 msgr2=0x56329ffb44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 0x56329ffb44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 -- 172.21.15.120:0/1456893472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 msgr2=0x56329ffcd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffcd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 -- 172.21.15.120:0/1456893472 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329ffbd5e0 con 0x56329ffb59c0 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.341+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c6a0 tx=0x7ff5180e9490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb04406e1c36ad00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.342+0000 7ff517fff640 1 -- 172.21.15.120:0/1456893472 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180ac000 con 0x56329ffb59c0 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.342+0000 7ff517fff640 1 -- 172.21.15.120:0/1456893472 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5180ac1c0 con 0x56329ffb59c0 2024-09-10T22:56:54.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.342+0000 7ff517fff640 1 -- 172.21.15.120:0/1456893472 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518005ab0 con 0x56329ffb59c0 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- 172.21.15.120:0/1456893472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 msgr2=0x56329ffade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c6a0 tx=0x7ff5180e9490 comp rx=0 tx=0).stop 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- 172.21.15.120:0/1456893472 shutdown_connections 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffc1950 0x56329ffb44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffb59c0 0x56329ffade20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 --2- 172.21.15.120:0/1456893472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffcd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- 172.21.15.120:0/1456893472 >> 172.21.15.120:0/1456893472 conn(0x56329ffb2ed0 msgr2=0x56329ffb3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- 172.21.15.120:0/1456893472 shutdown_connections 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- 172.21.15.120:0/1456893472 wait complete. 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 -- start start 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.343+0000 7ff537247740 1 Processor -- start 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffc9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffc1950 0x56329ffc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ad030 con 0x56329ffb50a0 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ae4a0 con 0x56329ffc1950 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ad700 con 0x56329ffb59c0 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff537247740 10 client.0 _ll_register_callbacks cb 0x7ff528009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:54.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffc9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55994/0 (socket says 172.21.15.120:55994) 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffc9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52778/0 (socket says 172.21.15.120:52778) 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 -- 172.21.15.120:0/1325915092 learned_addr learned my addr 172.21.15.120:0/1325915092 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff52ed9d640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffc1950 0x56329ffc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 -- 172.21.15.120:0/1325915092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffc1950 msgr2=0x56329ffc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffc1950 0x56329ffc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 -- 172.21.15.120:0/1325915092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 msgr2=0x56329ffc9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffc9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 -- 172.21.15.120:0/1325915092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329ffc15a0 con 0x56329ffb59c0 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff525d9b640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a8e70 tx=0x7ff5180b5db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925f393689cfcef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:54.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.344+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180abe20 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5180b59b0 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5180ac1c0 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5180ac000 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 10 client.25690 fetch_fsmap learned FSMap version 25 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632a003f080 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5180ac1c0 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7ff5180ab380 con 0x56329ffb59c0 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 10 client.25690 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 20 client.25690 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x5632a0041890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 4 client.25690 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:56:54.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56329ff29850 con 0x56329ffd1760 2024-09-10T22:56:54.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.345+0000 7ff52e59c640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x5632a0041890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:54.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.346+0000 7ff52e59c640 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x5632a0041890 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71191e408bbf7c66 server_cookie=d6a5d8dce9f0476 in_seq=0 out_seq=0 2024-09-10T22:56:54.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.346+0000 7ff515ffb640 10 client.25690 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff515ffb640 1 -- 172.21.15.120:0/1325915092 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x56329ff29850 con 0x56329ffd1760 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff515ffb640 10 client.25690 handle_command_reply: tid=0 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 client.25690 shutdown 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 msgr2=0x56329ffca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a8e70 tx=0x7ff5180b5db0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 shutdown_connections 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56329ffd1760 0x5632a0041890 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56329ffc1950 0x56329ffc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56329ffb59c0 0x56329ffca000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 --2- 172.21.15.120:0/1325915092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56329ffb50a0 0x56329ffc9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff515ffb640 0 client.25690 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 >> 172.21.15.120:0/1325915092 conn(0x56329ffb2ed0 msgr2=0x56329ffb1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 shutdown_connections 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 1 -- 172.21.15.120:0/1325915092 wait complete. 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.347+0000 7ff537247740 20 client.25690 trim_cache size 0 max 16384 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:56:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:56:54.355 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:56:54.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:56:54.359 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 msgr2=0x7ff51c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 0x7ff51c0b77a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff528005fd0 tx=0x7ff528004a40 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 msgr2=0x7ff530199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff530199840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e790 tx=0x7ff51800ec50 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 shutdown_connections 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff51c0b52e0 0x7ff51c0b77a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff53010d140 0x7ff53019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff530104600 0x7ff530199840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 --2- 172.21.15.120:0/1278376954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530103ce0 0x7ff530199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 >> 172.21.15.120:0/1278376954 conn(0x7ff5300fd700 msgr2=0x7ff5300fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:54.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 shutdown_connections 2024-09-10T22:56:54.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:54.348+0000 7ff534e72640 1 -- 172.21.15.120:0/1278376954 wait complete. 2024-09-10T22:56:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:53 smithi153 ceph-mon[34951]: pgmap v479: 97 pgs: 97 active+clean; 59 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 325 op/s 2024-09-10T22:56:54.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 727939 16.92 MB/sec execute 109 sec latency 14.478 ms 2024-09-10T22:56:55.293 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:56:55.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:55.555 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:55.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 730959 16.90 MB/sec execute 110 sec latency 14.503 ms 2024-09-10T22:56:56.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 -- 172.21.15.120:0/1239923291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c102de0 msgr2=0x7fc06c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:56.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 --2- 172.21.15.120:0/1239923291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c102de0 0x7fc06c1031c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002a410 comp rx=0 tx=0).stop 2024-09-10T22:56:56.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 -- 172.21.15.120:0/1239923291 shutdown_connections 2024-09-10T22:56:56.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 --2- 172.21.15.120:0/1239923291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c109170 0x7fc06c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 --2- 172.21.15.120:0/1239923291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c1087f0 0x7fc06c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 --2- 172.21.15.120:0/1239923291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c102de0 0x7fc06c1031c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 -- 172.21.15.120:0/1239923291 >> 172.21.15.120:0/1239923291 conn(0x7fc06c0fe830 msgr2=0x7fc06c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 -- 172.21.15.120:0/1239923291 shutdown_connections 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.073+0000 7fc072b9a640 1 -- 172.21.15.120:0/1239923291 wait complete. 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 Processor -- start 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 -- start start 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c102de0 0x7fc06c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c1087f0 0x7fc06c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c112e80 con 0x7fc06c109170 2024-09-10T22:56:56.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c112d00 con 0x7fc06c1087f0 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc072b9a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c113000 con 0x7fc06c102de0 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc071137640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc071137640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52794/0 (socket says 172.21.15.120:52794) 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc071137640 1 -- 172.21.15.120:0/3277843840 learned_addr learned my addr 172.21.15.120:0/3277843840 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.074+0000 7fc070936640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c102de0 0x7fc06c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc06bfff640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c1087f0 0x7fc06c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 -- 172.21.15.120:0/3277843840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c102de0 msgr2=0x7fc06c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c102de0 0x7fc06c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 -- 172.21.15.120:0/3277843840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c1087f0 msgr2=0x7fc06c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c1087f0 0x7fc06c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 -- 172.21.15.120:0/3277843840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c19f340 con 0x7fc06c109170 2024-09-10T22:56:56.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc071137640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc06400eb10 tx=0x7fc06400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004b20 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc064004cc0 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064010670 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c19f630 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.075+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1a5cf0 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.076+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06c1031c0 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.078+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc064005020 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.078+0000 7fc069ffb640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 0x7fc05c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.079+0000 7fc070936640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 0x7fc05c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.079+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc0640e60e0 con 0x7fc06c109170 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.079+0000 7fc070936640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 0x7fc05c0b7820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc060005ab0 tx=0x7fc060005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:56.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.083+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc0640acfd0 con 0x7fc06c109170 2024-09-10T22:56:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:55 smithi176 ceph-mon[34982]: pgmap v480: 97 pgs: 97 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-09-10T22:56:56.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.338+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc06c1a6080 con 0x7fc06c109170 2024-09-10T22:56:56.340 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:55 smithi120 ceph-mon[31247]: pgmap v480: 97 pgs: 97 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-09-10T22:56:56.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.339+0000 7fc069ffb640 1 -- 172.21.15.120:0/3277843840 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7fc0640ac8d0 con 0x7fc06c109170 2024-09-10T22:56:56.341 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:56:56.341 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":25,"btime":"2024-09-10T22:56:51:983742+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:56:51.983723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1259604772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1259604772},{"type":"v1","addr":"172.21.15.176:6837","nonce":1259604772}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:56:56.341 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 25 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 msgr2=0x7fc05c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 0x7fc05c0b7820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc060005ab0 tx=0x7fc060005a20 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 msgr2=0x7fc06c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc06400eb10 tx=0x7fc06400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 shutdown_connections 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc05c0b5360 0x7fc05c0b7820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc06c109170 0x7fc06c19eb70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc06c1087f0 0x7fc06c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 --2- 172.21.15.120:0/3277843840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc06c102de0 0x7fc06c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.341+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 >> 172.21.15.120:0/3277843840 conn(0x7fc06c0fe830 msgr2=0x7fc06c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.342+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 shutdown_connections 2024-09-10T22:56:56.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:56.342+0000 7fc072b9a640 1 -- 172.21.15.120:0/3277843840 wait complete. 2024-09-10T22:56:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:55 smithi153 ceph-mon[34951]: pgmap v480: 97 pgs: 97 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 309 op/s 2024-09-10T22:56:56.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 733475 16.85 MB/sec execute 111 sec latency 58.850 ms 2024-09-10T22:56:57.162 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:56:57.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:57 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3277843840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:57.412 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:57.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:57 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3277843840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:57.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:57 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3277843840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:57.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 736105 16.84 MB/sec execute 112 sec latency 31.739 ms 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.977+0000 7f36df59e640 1 -- 172.21.15.120:0/4068668500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 msgr2=0x7f36d80a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.977+0000 7f36df59e640 1 --2- 172.21.15.120:0/4068668500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d80a4590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f36e0068c00 tx=0x7f36e0097b70 comp rx=0 tx=0).stop 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 -- 172.21.15.120:0/4068668500 shutdown_connections 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 --2- 172.21.15.120:0/4068668500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d80a4590 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 --2- 172.21.15.120:0/4068668500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d809d410 0x7f36d809d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 --2- 172.21.15.120:0/4068668500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d8097de0 0x7f36d80981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 -- 172.21.15.120:0/4068668500 >> 172.21.15.120:0/4068668500 conn(0x7f36d800e5b0 msgr2=0x7f36d800e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 -- 172.21.15.120:0/4068668500 shutdown_connections 2024-09-10T22:56:57.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 -- 172.21.15.120:0/4068668500 wait complete. 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 Processor -- start 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.978+0000 7f36df59e640 1 -- start start 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 0x7f36d812e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80a7990 con 0x7f36d809dd90 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80a7810 con 0x7f36d8097de0 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36df59e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36d80a7b10 con 0x7f36d809d410 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 0x7f36d812e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52806/0 (socket says 172.21.15.120:52806) 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 -- 172.21.15.120:0/1675077366 learned_addr learned my addr 172.21.15.120:0/1675077366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52818/0 (socket says 172.21.15.120:52818) 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 -- 172.21.15.120:0/1675077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 msgr2=0x7f36d812e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 0x7f36d812e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 -- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 msgr2=0x7f36d8133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 -- 172.21.15.120:0/1675077366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d8133ed0 con 0x7f36d8097de0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 0x7f36d812e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36ded9d640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36de59c640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e9f0 tx=0x7f36d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.979+0000 7f36cf7fe640 1 -- 172.21.15.120:0/1675077366 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d000cd80 con 0x7f36d8097de0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.980+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d81341c0 con 0x7f36d8097de0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.980+0000 7f36cf7fe640 1 -- 172.21.15.120:0/1675077366 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d00108e0 con 0x7f36d8097de0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.980+0000 7f36cf7fe640 1 -- 172.21.15.120:0/1675077366 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0018980 con 0x7f36d8097de0 2024-09-10T22:56:57.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.980+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d813aa50 con 0x7f36d8097de0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.982+0000 7f36cf7fe640 1 -- 172.21.15.120:0/1675077366 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f36d0010430 con 0x7f36d8097de0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.982+0000 7f36e6e27740 1 Processor -- start 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.982+0000 7f36e6e27740 1 -- start start 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ada676ea0 con 0x564ada700fc0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ada6662d0 con 0x564ada7006a0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36e6e27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ada66a0e0 con 0x564ada70cf50 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36de59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:57.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52814/0 (socket says 172.21.15.120:52814) 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 -- 172.21.15.120:0/2886145527 learned_addr learned my addr 172.21.15.120:0/2886145527 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36de59c640 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52822/0 (socket says 172.21.15.120:52822) 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 -- 172.21.15.120:0/2886145527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 msgr2=0x564ada70f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 -- 172.21.15.120:0/2886145527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 msgr2=0x564ada70ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 -- 172.21.15.120:0/2886145527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ada3c4650 con 0x564ada7006a0 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36de59c640 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36cf7fe640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 0x7f36c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36cf7fe640 1 -- 172.21.15.120:0/1675077366 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36d00e5f50 con 0x7f36d8097de0 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36de59c640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 0x7f36c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.983+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36e0052150 tx=0x7f36e0051ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0438012fe0f807c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:58.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36ccff9640 1 -- 172.21.15.120:0/2886145527 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0099540 con 0x564ada7006a0 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36ccff9640 1 -- 172.21.15.120:0/2886145527 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0099e90 con 0x564ada7006a0 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36ccff9640 1 -- 172.21.15.120:0/2886145527 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0077d30 con 0x564ada7006a0 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 -- 172.21.15.120:0/2886145527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 msgr2=0x564ada700a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36de59c640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 0x7f36c80b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f36d00ac720 tx=0x7f36d00e9990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36e0052150 tx=0x7f36e0051ee0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 -- 172.21.15.120:0/2886145527 shutdown_connections 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada70ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2886145527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada700a80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.984+0000 7f36e6e27740 1 -- 172.21.15.120:0/2886145527 >> 172.21.15.120:0/2886145527 conn(0x564ada6fe4d0 msgr2=0x564ada6ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- 172.21.15.120:0/2886145527 shutdown_connections 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- 172.21.15.120:0/2886145527 wait complete. 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- start start 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 Processor -- start 2024-09-10T22:56:58.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada70c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ada4c7700 con 0x564ada700fc0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ada601160 con 0x564ada7006a0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ada70fb10 con 0x564ada70cf50 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36e6e27740 10 client.0 _ll_register_callbacks cb 0x564ada61a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36ddd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52838/0 (socket says 172.21.15.120:52838) 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36ddd9b640 1 -- 172.21.15.120:0/541411685 learned_addr learned my addr 172.21.15.120:0/541411685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada70c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.985+0000 7f36ded9d640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 -- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 msgr2=0x564ada70a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 -- 172.21.15.120:0/541411685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 msgr2=0x564ada70c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada70c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 -- 172.21.15.120:0/541411685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ada62ec90 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36d4012d20 tx=0x7f36d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f976913fc8bfa9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4012110 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36d4004050 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e0099e90 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36d4023070 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36e6e27740 10 client.16233 fetch_fsmap learned FSMap version 25 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ada71b4e0 con 0x564ada700fc0 2024-09-10T22:56:58.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0099e90 con 0x564ada700fc0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f36d4010430 con 0x564ada700fc0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.986+0000 7f36e6e27740 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36e6e27740 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36e6e27740 20 client.16233 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada71c9a0 0x564ada71edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36e6e27740 4 client.16233 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ada678df0 con 0x564ada71c9a0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36ded9d640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada71c9a0 0x564ada71edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36ded9d640 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada71c9a0 0x564ada71edb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbffa4f36951797 server_cookie=7105922185651e5c in_seq=0 out_seq=0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.987+0000 7f36c5ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36c5ffb640 1 -- 172.21.15.120:0/541411685 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ada678df0 con 0x564ada71c9a0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36c5ffb640 10 client.16233 handle_command_reply: tid=0 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 client.16233 shutdown 2024-09-10T22:56:58.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 msgr2=0x564ada709c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36d4012d20 tx=0x7f36d400f790 comp rx=0 tx=0).stop 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 shutdown_connections 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada71c9a0 0x564ada71edb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada70cf50 0x564ada70a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada700fc0 0x564ada709c70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 --2- 172.21.15.120:0/541411685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada7006a0 0x564ada70c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 >> 172.21.15.120:0/541411685 conn(0x564ada6fe4d0 msgr2=0x564ada704040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36c5ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 shutdown_connections 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 1 -- 172.21.15.120:0/541411685 wait complete. 2024-09-10T22:56:58.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:57.989+0000 7f36e6e27740 20 client.16233 trim_cache size 0 max 16384 2024-09-10T22:56:58.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.005+0000 7f36e6e27740 1 Processor -- start 2024-09-10T22:56:58.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 -- start start 2024-09-10T22:56:58.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada6ff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada6ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8140e60 con 0x564ada7006a0 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81436e0 con 0x564ada70cf50 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36e6e27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81414d0 con 0x564ada700fc0 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52844/0 (socket says 172.21.15.120:52844) 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 -- 172.21.15.120:0/2471643968 learned_addr learned my addr 172.21.15.120:0/2471643968 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada6ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada6ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 -- 172.21.15.120:0/2471643968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 msgr2=0x564ada6ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada6ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 -- 172.21.15.120:0/2471643968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 msgr2=0x564ada6ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada6ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.006+0000 7f36ded9d640 1 -- 172.21.15.120:0/2471643968 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ada700df0 con 0x564ada7006a0 2024-09-10T22:56:58.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.007+0000 7f36ded9d640 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f36e0064650 tx=0x7f36e0097b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000b703bddc660c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:58.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.007+0000 7f36c7fff640 1 -- 172.21.15.120:0/2471643968 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00995f0 con 0x564ada7006a0 2024-09-10T22:56:58.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.007+0000 7f36c7fff640 1 -- 172.21.15.120:0/2471643968 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0099790 con 0x564ada7006a0 2024-09-10T22:56:58.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.007+0000 7f36c7fff640 1 -- 172.21.15.120:0/2471643968 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00bb730 con 0x564ada7006a0 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- 172.21.15.120:0/2471643968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 msgr2=0x564ada718da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f36e0064650 tx=0x7f36e0097b70 comp rx=0 tx=0).stop 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- 172.21.15.120:0/2471643968 shutdown_connections 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada6ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada6ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- 172.21.15.120:0/2471643968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada718da0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- 172.21.15.120:0/2471643968 >> 172.21.15.120:0/2471643968 conn(0x564ada6fe4d0 msgr2=0x564ada704e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- 172.21.15.120:0/2471643968 shutdown_connections 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- 172.21.15.120:0/2471643968 wait complete. 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 -- start start 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 Processor -- start 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.008+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada711570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada788910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8141ba0 con 0x564ada7006a0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8143010 con 0x564ada70cf50 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8142270 con 0x564ada700fc0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 10 client.0 _ll_register_callbacks cb 0x7f36d4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52852/0 (socket says 172.21.15.120:52852) 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 -- 172.21.15.120:0/3655801519 learned_addr learned my addr 172.21.15.120:0/3655801519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada711570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada788910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 -- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 msgr2=0x564ada711570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada711570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 -- 172.21.15.120:0/3655801519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 msgr2=0x564ada788910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada788910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 -- 172.21.15.120:0/3655801519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ada711e20 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada711570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36ded9d640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36e0068e50 tx=0x7f36e0069a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8667ed260ee87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0077900 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36e00aa040 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e0099790 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36e00999e0 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 10 client.16239 fetch_fsmap learned FSMap version 25 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.009+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ada789f30 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0099790 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f36e00bbb80 con 0x564ada7006a0 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:56:58.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 20 client.16239 populate_metadata read hostname 'smithi120' 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada78a100 0x564ada78c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 4 client.16239 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ada6748a0 con 0x564ada78a100 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada78a100 0x564ada78c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36ddd9b640 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada78a100 0x564ada78c4f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c4dab604d40e52 server_cookie=990b6cb115ee66b1 in_seq=0 out_seq=0 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.010+0000 7f36c5ffb640 10 client.16239 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36c5ffb640 1 -- 172.21.15.120:0/3655801519 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ada6748a0 con 0x564ada78a100 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36c5ffb640 10 client.16239 handle_command_reply: tid=0 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 client.16239 shutdown 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 msgr2=0x564ada711030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36e0068e50 tx=0x7f36e0069a60 comp rx=0 tx=0).stop 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 shutdown_connections 2024-09-10T22:56:58.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ada78a100 0x564ada78c4f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ada70cf50 0x564ada788910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36c5ffb640 0 client.16239 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ada700fc0 0x564ada711570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 --2- 172.21.15.120:0/3655801519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ada7006a0 0x564ada711030 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 >> 172.21.15.120:0/3655801519 conn(0x564ada6fe4d0 msgr2=0x564ada7080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 shutdown_connections 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 1 -- 172.21.15.120:0/3655801519 wait complete. 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36e6e27740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T22:56:58.017 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 msgr2=0x7f36c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 0x7f36c80b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f36d00ac720 tx=0x7f36d00e9990 comp rx=0 tx=0).stop 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 msgr2=0x7f36d812de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.011+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e9f0 tx=0x7f36d000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 shutdown_connections 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f36c80b52e0 0x7f36c80b77a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36d809dd90 0x7f36d8133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36d809d410 0x7f36d812e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 --2- 172.21.15.120:0/1675077366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36d8097de0 0x7f36d812de20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:58.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 >> 172.21.15.120:0/1675077366 conn(0x7f36d800e5b0 msgr2=0x7f36d80958d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:58.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 shutdown_connections 2024-09-10T22:56:58.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:56:58.012+0000 7f36df59e640 1 -- 172.21.15.120:0/1675077366 wait complete. 2024-09-10T22:56:58.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:56:58 smithi120 ceph-mon[31247]: pgmap v481: 97 pgs: 97 active+clean; 42 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-09-10T22:56:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:56:58 smithi176 ceph-mon[34982]: pgmap v481: 97 pgs: 97 active+clean; 42 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-09-10T22:56:58.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:56:58 smithi153 ceph-mon[34951]: pgmap v481: 97 pgs: 97 active+clean; 42 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-09-10T22:56:58.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 739392 16.85 MB/sec execute 113 sec latency 12.546 ms 2024-09-10T22:56:59.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 742599 16.86 MB/sec execute 114 sec latency 17.565 ms 2024-09-10T22:56:59.867 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:56:59.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:57:00.142 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:00.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:00 smithi120 ceph-mon[31247]: pgmap v482: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T22:57:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:00 smithi176 ceph-mon[34982]: pgmap v482: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T22:57:00.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:00 smithi153 ceph-mon[34951]: pgmap v482: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T22:57:00.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 745941 16.88 MB/sec execute 115 sec latency 21.931 ms 2024-09-10T22:57:00.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.654+0000 7f146c6b6640 1 -- 172.21.15.120:0/4248498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f146410d0e0 msgr2=0x7f146410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.654+0000 7f146c6b6640 1 --2- 172.21.15.120:0/4248498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f146410d0e0 0x7f146410f550 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f145c009ca0 tx=0x7f145c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- 172.21.15.120:0/4248498479 shutdown_connections 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- 172.21.15.120:0/4248498479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f146410d0e0 0x7f146410f550 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- 172.21.15.120:0/4248498479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 0x7f146410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- 172.21.15.120:0/4248498479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1464103b30 0x7f1464103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- 172.21.15.120:0/4248498479 >> 172.21.15.120:0/4248498479 conn(0x7f14640fd700 msgr2=0x7f14640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- 172.21.15.120:0/4248498479 shutdown_connections 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- 172.21.15.120:0/4248498479 wait complete. 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 Processor -- start 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- start start 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 0x7f1464199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146410d0e0 0x7f146419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1464112d60 con 0x7f1464103b30 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1464112be0 con 0x7f1464104530 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.655+0000 7f146c6b6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1464112ee0 con 0x7f146410d0e0 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52874/0 (socket says 172.21.15.120:52874) 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 -- 172.21.15.120:0/2105003102 learned_addr learned my addr 172.21.15.120:0/2105003102 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146ac53640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146410d0e0 0x7f146419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f1469c51640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 0x7f1464199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 -- 172.21.15.120:0/2105003102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146410d0e0 msgr2=0x7f146419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146410d0e0 0x7f146419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 -- 172.21.15.120:0/2105003102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 msgr2=0x7f1464199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 0x7f1464199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 -- 172.21.15.120:0/2105003102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146419f280 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f146a452640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9f0 tx=0x7f146000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f145b7fe640 1 -- 172.21.15.120:0/2105003102 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146000cd80 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f145b7fe640 1 -- 172.21.15.120:0/2105003102 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600108e0 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.656+0000 7f145b7fe640 1 -- 172.21.15.120:0/2105003102 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460018980 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.657+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146419f570 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.657+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14641a5c30 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.659+0000 7f145b7fe640 1 -- 172.21.15.120:0/2105003102 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1460010430 con 0x7f1464103b30 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.659+0000 7f146ea8b740 1 Processor -- start 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 -- start start 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbae564e60 con 0x55fbae5ee800 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fbae554290 con 0x55fbae5fa790 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ea8b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fbae5580a0 con 0x55fbae5edee0 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52882/0 (socket says 172.21.15.120:52882) 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146a452640 1 -- 172.21.15.120:0/1842393701 learned_addr learned my addr 172.21.15.120:0/1842393701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f145b7fe640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 0x7f14500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f1469c51640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ac53640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.660+0000 7f146ac53640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 0x7f14500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 -- 172.21.15.120:0/1842393701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 msgr2=0x55fbae5ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f145b7fe640 1 -- 172.21.15.120:0/2105003102 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f14600e5ef0 con 0x7f1464103b30 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 -- 172.21.15.120:0/1842393701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 msgr2=0x55fbae5fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 -- 172.21.15.120:0/1842393701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbae2b2930 con 0x55fbae5fa790 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f146a452640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1469c51640 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5fcb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1454009bf0 tx=0x7f1454011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62f88d05cc3ba5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1458ff9640 1 -- 172.21.15.120:0/1842393701 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454004050 con 0x55fbae5fa790 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f146ac53640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 0x7f14500b77a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f145c02f500 tx=0x7f145c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1458ff9640 1 -- 172.21.15.120:0/1842393701 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14540054d0 con 0x55fbae5fa790 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.661+0000 7f1458ff9640 1 -- 172.21.15.120:0/1842393701 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454004d00 con 0x55fbae5fa790 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- 172.21.15.120:0/1842393701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 msgr2=0x55fbae5fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5fcb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1454009bf0 tx=0x7f1454011810 comp rx=0 tx=0).stop 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- 172.21.15.120:0/1842393701 shutdown_connections 2024-09-10T22:57:00.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5fcb80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae5fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 --2- 172.21.15.120:0/1842393701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- 172.21.15.120:0/1842393701 >> 172.21.15.120:0/1842393701 conn(0x55fbae5ebd10 msgr2=0x55fbae5eceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- 172.21.15.120:0/1842393701 shutdown_connections 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- 172.21.15.120:0/1842393701 wait complete. 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.662+0000 7f146ea8b740 1 -- start start 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 Processor -- start 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 0x55fbae5f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5fa790 0x55fbae5f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbae3b5890 con 0x55fbae5edee0 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fbae4eed40 con 0x55fbae5ee800 2024-09-10T22:57:00.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fbae5fd440 con 0x55fbae5fa790 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146ea8b740 10 client.0 _ll_register_callbacks cb 0x55fbae552c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52884/0 (socket says 172.21.15.120:52884) 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 -- 172.21.15.120:0/3217255826 learned_addr learned my addr 172.21.15.120:0/3217255826 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f1469c51640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 0x55fbae5f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 -- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5fa790 msgr2=0x55fbae5f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5fa790 0x55fbae5f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 -- 172.21.15.120:0/3217255826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 msgr2=0x55fbae5f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 0x55fbae5f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f146a452640 1 -- 172.21.15.120:0/3217255826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbae6067d0 con 0x55fbae5edee0 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.663+0000 7f1469c51640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 0x55fbae5f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146a452640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f1460009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a3cb172045cc5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600abf00 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f14600ab9f0 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14540054d0 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14600ac2a0 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 10 client.16251 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbae609180 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14540054d0 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f14600a80e0 con 0x55fbae5edee0 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 20 client.16251 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae60a6e0 0x55fbae60cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 4 client.16251 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.664+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbae566db0 con 0x55fbae60a6e0 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.665+0000 7f1469c51640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae60a6e0 0x55fbae60cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.665+0000 7f1469c51640 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae60a6e0 0x55fbae60cb10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11106f6557c176ff server_cookie=80641984d6007d0a in_seq=0 out_seq=0 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.665+0000 7f144dffb640 10 client.16251 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f144dffb640 1 -- 172.21.15.120:0/3217255826 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fbae566db0 con 0x55fbae60a6e0 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f144dffb640 10 client.16251 handle_command_reply: tid=0 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 client.16251 shutdown 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 msgr2=0x55fbae5f76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f1460009310 comp rx=0 tx=0).stop 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 shutdown_connections 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae60a6e0 0x55fbae60cb10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5fa790 0x55fbae5f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5ee800 0x55fbae5f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f144dffb640 0 client.16251 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3217255826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5edee0 0x55fbae5f76d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 >> 172.21.15.120:0/3217255826 conn(0x55fbae5ebd10 msgr2=0x55fbae5eaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 shutdown_connections 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 1 -- 172.21.15.120:0/3217255826 wait complete. 2024-09-10T22:57:00.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.666+0000 7f146ea8b740 20 client.16251 trim_cache size 0 max 16384 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 Processor -- start 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 -- start start 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14641ac2d0 con 0x55fbae5ee800 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14641aeb50 con 0x55fbae5fa790 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ea8b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f14641ac940 con 0x55fbae5edee0 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f1469c51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146a452640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146a452640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56114/0 (socket says 172.21.15.120:56114) 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f1469c51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52888/0 (socket says 172.21.15.120:52888) 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146a452640 1 -- 172.21.15.120:0/3172621725 learned_addr learned my addr 172.21.15.120:0/3172621725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.687+0000 7f146ac53640 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 -- 172.21.15.120:0/3172621725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 msgr2=0x55fbae5f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 -- 172.21.15.120:0/3172621725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 msgr2=0x55fbae605da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 -- 172.21.15.120:0/3172621725 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbae5ec790 con 0x55fbae5edee0 2024-09-10T22:57:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f1469c51640 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f146a452640 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14600a8d10 tx=0x7f146000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31a65e107c6ff11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f144ffff640 1 -- 172.21.15.120:0/3172621725 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600ab7d0 con 0x55fbae5edee0 2024-09-10T22:57:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f144ffff640 1 -- 172.21.15.120:0/3172621725 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600b11c0 con 0x55fbae5edee0 2024-09-10T22:57:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.688+0000 7f144ffff640 1 -- 172.21.15.120:0/3172621725 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600ada10 con 0x55fbae5edee0 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 -- 172.21.15.120:0/3172621725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 msgr2=0x55fbae5f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14600a8d10 tx=0x7f146000cf20 comp rx=0 tx=0).stop 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 -- 172.21.15.120:0/3172621725 shutdown_connections 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae605da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 --2- 172.21.15.120:0/3172621725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae5f1f50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 -- 172.21.15.120:0/3172621725 >> 172.21.15.120:0/3172621725 conn(0x55fbae5ebd10 msgr2=0x55fbae605690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 -- 172.21.15.120:0/3172621725 shutdown_connections 2024-09-10T22:57:00.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.689+0000 7f146ea8b740 1 -- 172.21.15.120:0/3172621725 wait complete. 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 -- start start 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 Processor -- start 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14641ad010 con 0x55fbae5ee800 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14641ae480 con 0x55fbae5fa790 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f14641ad6e0 con 0x55fbae5edee0 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ea8b740 10 client.0 _ll_register_callbacks cb 0x7f1460010750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f1469c51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56122/0 (socket says 172.21.15.120:56122) 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f1469c51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52894/0 (socket says 172.21.15.120:52894) 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 -- 172.21.15.120:0/928464243 learned_addr learned my addr 172.21.15.120:0/928464243 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146ac53640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 -- 172.21.15.120:0/928464243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 msgr2=0x55fbae5ff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5ff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 -- 172.21.15.120:0/928464243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 msgr2=0x55fbae603010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f146a452640 1 -- 172.21.15.120:0/928464243 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbae5ebb40 con 0x55fbae5edee0 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.690+0000 7f1469c51640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146a452640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f14600a8390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1028c055865baca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:57:00.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600abbf0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14600b11c0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f14600c73f0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f14600cc050 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 10 client.25729 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbae6780b0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600b11c0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f14600cfaa0 con 0x55fbae5edee0 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 10 client.25729 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 20 client.25729 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae678590 0x55fbae67a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 4 client.25729 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:57:00.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.691+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fbae562860 con 0x55fbae678590 2024-09-10T22:57:00.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.692+0000 7f1469c51640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae678590 0x55fbae67a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:00.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.692+0000 7f144dffb640 10 client.25729 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:00.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.692+0000 7f1469c51640 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae678590 0x55fbae67a9b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0e1c6ef2a2cfa8 server_cookie=88843affa06c23ee in_seq=0 out_seq=0 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f144dffb640 1 -- 172.21.15.120:0/928464243 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fbae562860 con 0x55fbae678590 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f144dffb640 10 client.25729 handle_command_reply: tid=0 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 client.25729 shutdown 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 msgr2=0x55fbae602a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f14600a8390 comp rx=0 tx=0).stop 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 shutdown_connections 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbae678590 0x55fbae67a9b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbae5fa790 0x55fbae5ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbae5ee800 0x55fbae603010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f144dffb640 0 client.25729 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 --2- 172.21.15.120:0/928464243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbae5edee0 0x55fbae602a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.693+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 >> 172.21.15.120:0/928464243 conn(0x55fbae5ebd10 msgr2=0x55fbae605eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 shutdown_connections 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146ea8b740 1 -- 172.21.15.120:0/928464243 wait complete. 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146ea8b740 20 client.25729 trim_cache size 0 max 16384 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "374bdd63-bd62-4996-bb70-48d17f31fda6", 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:57:00.700 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 msgr2=0x7f14500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 0x7f14500b77a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f145c02f500 tx=0x7f145c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 msgr2=0x7f14641991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.694+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9f0 tx=0x7f146000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 shutdown_connections 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f14500b52e0 0x7f14500b77a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f146410d0e0 0x7f146419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1464104530 0x7f1464199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 --2- 172.21.15.120:0/2105003102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1464103b30 0x7f14641991d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 >> 172.21.15.120:0/2105003102 conn(0x7f14640fd700 msgr2=0x7f14640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.695+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 shutdown_connections 2024-09-10T22:57:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:00.696+0000 7f146c6b6640 1 -- 172.21.15.120:0/2105003102 wait complete. 2024-09-10T22:57:01.519 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:01.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 748258 16.81 MB/sec execute 116 sec latency 86.845 ms 2024-09-10T22:57:01.792 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:02 smithi120 ceph-mon[31247]: pgmap v483: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 420 op/s 2024-09-10T22:57:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:02 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:57:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:02 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:02 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:57:02.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:02 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:02 smithi176 ceph-mon[34982]: pgmap v483: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 420 op/s 2024-09-10T22:57:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:02 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:57:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:02 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:02 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:57:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:02 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2166776496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 msgr2=0x7f5f9410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2166776496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9410e820 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ca0 tx=0x7f5f8c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2166776496 shutdown_connections 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2166776496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9410e820 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2166776496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 0x7f5f94107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2166776496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f94102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2166776496 >> 172.21.15.120:0/2166776496 conn(0x7f5f940fd700 msgr2=0x7f5f940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2166776496 shutdown_connections 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.322+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2166776496 wait complete. 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 Processor -- start 2024-09-10T22:57:02.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 -- start start 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 0x7f5f9419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111ce0 con 0x7f5f94108020 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111b60 con 0x7f5f941076a0 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f9a3c8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111e60 con 0x7f5f94101c90 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f93fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f93fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56142/0 (socket says 172.21.15.120:56142) 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f93fff640 1 -- 172.21.15.120:0/2352499337 learned_addr learned my addr 172.21.15.120:0/2352499337 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 0x7f5f9419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.323+0000 7f5f98965640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 -- 172.21.15.120:0/2352499337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 msgr2=0x7f5f9419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 0x7f5f9419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 -- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 msgr2=0x7f5f9419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 -- 172.21.15.120:0/2352499337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a5410 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f93fff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400e9f0 tx=0x7f5f8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f91ffb640 1 -- 172.21.15.120:0/2352499337 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400cd80 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f91ffb640 1 -- 172.21.15.120:0/2352499337 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f840108e0 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a5700 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f91ffb640 1 -- 172.21.15.120:0/2352499337 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84018980 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.324+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941a5c90 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 Processor -- start 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 -- start start 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f91ffb640 1 -- 172.21.15.120:0/2352499337 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5f84010430 con 0x7f5f94101c90 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 0x55624ef44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef46d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55624eeaee50 con 0x55624ef380a0 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55624ee9e280 con 0x55624ef389c0 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55624eea2090 con 0x55624ef44950 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f93fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 0x55624ef44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.327+0000 7f5f8bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef46d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52920/0 (socket says 172.21.15.120:52920) 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 -- 172.21.15.120:0/1108122349 learned_addr learned my addr 172.21.15.120:0/1108122349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 -- 172.21.15.120:0/1108122349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 msgr2=0x55624ef46d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef46d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 -- 172.21.15.120:0/1108122349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 msgr2=0x55624ef44410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 0x55624ef44410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 -- 172.21.15.120:0/1108122349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624ebfc650 con 0x55624ef380a0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f93fff640 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 0x55624ef44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.328+0000 7f5f98965640 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c035000 tx=0x7f5f8c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824d4b0f37c2b089 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f91ffb640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 0x7f5f800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/1108122349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c031db0 con 0x55624ef380a0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/1108122349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c036030 con 0x55624ef380a0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/1108122349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0042a0 con 0x55624ef380a0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f93fff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 0x7f5f800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.329+0000 7f5f91ffb640 1 -- 172.21.15.120:0/2352499337 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5f840e5e00 con 0x7f5f94101c90 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1108122349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 msgr2=0x55624ef38480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c035000 tx=0x7f5f8c005fa0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1108122349 shutdown_connections 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef46d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef389c0 0x55624ef44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1108122349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef380a0 0x55624ef38480 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f93fff640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 0x7f5f800b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f840e3ca0 tx=0x7f5f84093ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1108122349 >> 172.21.15.120:0/1108122349 conn(0x55624ef35ed0 msgr2=0x55624ef37070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.330+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1108122349 shutdown_connections 2024-09-10T22:57:02.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.331+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1108122349 wait complete. 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.331+0000 7f5f9c79d740 1 -- start start 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.331+0000 7f5f9c79d740 1 Processor -- start 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef44950 0x55624ef41d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55624ecff700 con 0x55624ef389c0 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55624ee39110 con 0x55624ef44950 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55624ef473f0 con 0x55624ef380a0 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f9c79d740 10 client.0 _ll_register_callbacks cb 0x55624ee52990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef44950 0x55624ef41d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56166/0 (socket says 172.21.15.120:56166) 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 -- 172.21.15.120:0/4276947413 learned_addr learned my addr 172.21.15.120:0/4276947413 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 -- 172.21.15.120:0/4276947413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef44950 msgr2=0x55624ef41d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef44950 0x55624ef41d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 -- 172.21.15.120:0/4276947413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 msgr2=0x55624ef416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.332+0000 7f5f98965640 1 -- 172.21.15.120:0/4276947413 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624ee66c40 con 0x55624ef380a0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.333+0000 7f5f98965640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c003eb0 tx=0x7f5f8c003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b9c53915a3928 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.333+0000 7f5f737fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.333+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c059060 con 0x55624ef380a0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5f8c031450 con 0x55624ef380a0 2024-09-10T22:57:02.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8c036030 con 0x55624ef380a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f8c04a040 con 0x55624ef380a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 10 client.25741 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55624ef51f90 con 0x55624ef380a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c036030 con 0x55624ef380a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f5f8c043070 con 0x55624ef380a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 10 client.25741 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 20 client.25741 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624ef533a0 0x55624ef537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 4 client.25741 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.334+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55624eeb0da0 con 0x55624ef533a0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.335+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624ef533a0 0x55624ef537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.335+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624ef533a0 0x55624ef537f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c83901010c17bc4 server_cookie=46081b3164433d98 in_seq=0 out_seq=0 2024-09-10T22:57:02.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.335+0000 7f5f737fe640 10 client.25741 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.336+0000 7f5f737fe640 1 -- 172.21.15.120:0/4276947413 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55624eeb0da0 con 0x55624ef533a0 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.336+0000 7f5f737fe640 10 client.25741 handle_command_reply: tid=0 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.336+0000 7f5f9c79d740 1 client.25741 shutdown 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.336+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 msgr2=0x55624ef44290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.336+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c003eb0 tx=0x7f5f8c003db0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 shutdown_connections 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624ef533a0 0x55624ef537f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef44950 0x55624ef41d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/4276947413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef380a0 0x55624ef44290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f737fe640 0 client.25741 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 >> 172.21.15.120:0/4276947413 conn(0x55624ef35ed0 msgr2=0x55624ef3ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 shutdown_connections 2024-09-10T22:57:02.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 1 -- 172.21.15.120:0/4276947413 wait complete. 2024-09-10T22:57:02.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.337+0000 7f5f9c79d740 20 client.25741 trim_cache size 0 max 16384 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 Processor -- start 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 -- start start 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef45d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941aa0a0 con 0x55624ef389c0 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941ac920 con 0x55624ef380a0 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941aa710 con 0x55624ef44950 2024-09-10T22:57:02.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f8bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f8bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52932/0 (socket says 172.21.15.120:52932) 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f8bfff640 1 -- 172.21.15.120:0/1501993660 learned_addr learned my addr 172.21.15.120:0/1501993660 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f98965640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.356+0000 7f5f98965640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef45d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 -- 172.21.15.120:0/1501993660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 msgr2=0x55624ef45d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef45d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 -- 172.21.15.120:0/1501993660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 msgr2=0x55624ef547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 -- 172.21.15.120:0/1501993660 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624ef36950 con 0x55624ef389c0 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f98965640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:02.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55624ee9e310 tx=0x7f5f7c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4084026c36521 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f89ffb640 1 -- 172.21.15.120:0/1501993660 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c005220 con 0x55624ef389c0 2024-09-10T22:57:02.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f89ffb640 1 -- 172.21.15.120:0/1501993660 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c004c50 con 0x55624ef389c0 2024-09-10T22:57:02.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.357+0000 7f5f89ffb640 1 -- 172.21.15.120:0/1501993660 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c01d850 con 0x55624ef389c0 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1501993660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 msgr2=0x55624ef4fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55624ee9e310 tx=0x7f5f7c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1501993660 shutdown_connections 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef45d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef4fd90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/1501993660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1501993660 >> 172.21.15.120:0/1501993660 conn(0x55624ef35ed0 msgr2=0x55624ef4f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1501993660 shutdown_connections 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.358+0000 7f5f9c79d740 1 -- 172.21.15.120:0/1501993660 wait complete. 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 -- start start 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 Processor -- start 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef5fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941aade0 con 0x55624ef389c0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941ac250 con 0x55624ef380a0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941ab4b0 con 0x55624ef44950 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.359+0000 7f5f9c79d740 10 client.0 _ll_register_callbacks cb 0x7f5f7c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f98965640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef5fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f98965640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef5fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56184/0 (socket says 172.21.15.120:56184) 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f98965640 1 -- 172.21.15.120:0/236137176 learned_addr learned my addr 172.21.15.120:0/236137176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f98965640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 -- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 msgr2=0x55624ef5fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef5fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 -- 172.21.15.120:0/236137176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 msgr2=0x55624ef5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef5f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 -- 172.21.15.120:0/236137176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624ef64990 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f8bfff640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef5f690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55624ee9e310 tx=0x7f5f7c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d65ce3ef7fa37cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.360+0000 7f5f737fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c032020 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5f7c005020 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c004c50 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f7c005220 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f9c79d740 10 client.16275 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55624efca750 con 0x55624ef389c0 2024-09-10T22:57:02.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.361+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c004c50 con 0x55624ef389c0 2024-09-10T22:57:02.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f5f7c02e460 con 0x55624ef389c0 2024-09-10T22:57:02.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 10 client.16275 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:02.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:02.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 20 client.16275 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624efcaf00 0x55624efcd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 4 client.16275 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55624eeac850 con 0x55624efcaf00 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.362+0000 7f5f98965640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624efcaf00 0x55624efcd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f98965640 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624efcaf00 0x55624efcd320 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7ef8aaa9f68516 server_cookie=1a1c6da41854c57 in_seq=0 out_seq=0 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f737fe640 10 client.16275 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f737fe640 1 -- 172.21.15.120:0/236137176 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55624eeac850 con 0x55624efcaf00 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f737fe640 10 client.16275 handle_command_reply: tid=0 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 client.16275 shutdown 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 msgr2=0x55624ef5f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef5f690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55624ee9e310 tx=0x7f5f7c004e70 comp rx=0 tx=0).stop 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 shutdown_connections 2024-09-10T22:57:02.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55624efcaf00 0x55624efcd320 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55624ef44950 0x55624ef5fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55624ef389c0 0x55624ef5f690 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 --2- 172.21.15.120:0/236137176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55624ef380a0 0x55624ef5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 >> 172.21.15.120:0/236137176 conn(0x55624ef35ed0 msgr2=0x55624ef4fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "374bdd63-bd62-4996-bb70-48d17f31fda6", 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.370 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.373 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:02.375 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.363+0000 7f5f737fe640 0 client.16275 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 shutdown_connections 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9c79d740 1 -- 172.21.15.120:0/236137176 wait complete. 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9c79d740 20 client.16275 trim_cache size 0 max 16384 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 msgr2=0x7f5f800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.377 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 0x7f5f800b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f840e3ca0 tx=0x7f5f84093ca0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 msgr2=0x7f5f9419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400e9f0 tx=0x7f5f8400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 shutdown_connections 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f800b5360 0x7f5f800b7820 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f94108020 0x7f5f9419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f941076a0 0x7f5f9419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 --2- 172.21.15.120:0/2352499337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f94101c90 0x7f5f9419a030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.364+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 >> 172.21.15.120:0/2352499337 conn(0x7f5f940fd700 msgr2=0x7f5f940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.365+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 shutdown_connections 2024-09-10T22:57:02.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:02.365+0000 7f5f9a3c8640 1 -- 172.21.15.120:0/2352499337 wait complete. 2024-09-10T22:57:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:02 smithi153 ceph-mon[34951]: pgmap v483: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 420 op/s 2024-09-10T22:57:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:02 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:57:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:02 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:02 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:57:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:02 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:02.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 750738 16.80 MB/sec execute 117 sec latency 44.911 ms 2024-09-10T22:57:03.370 INFO:tasks.cephfs.filesystem:scrub status for tag:374bdd63-bd62-4996-bb70-48d17f31fda6 - {'path': '/', 'tag': '374bdd63-bd62-4996-bb70-48d17f31fda6', 'options': 'recursive,force'} 2024-09-10T22:57:03.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 753692 16.78 MB/sec execute 118 sec latency 25.490 ms 2024-09-10T22:57:04.275 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:04 smithi120 ceph-mon[31247]: pgmap v484: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-09-10T22:57:04.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:04 smithi176 ceph-mon[34982]: pgmap v484: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-09-10T22:57:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:04 smithi153 ceph-mon[34951]: pgmap v484: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.564+0000 7f10c1d9b640 1 -- 172.21.15.120:0/64593126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc103370 msgr2=0x7f10bc109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.564+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/64593126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc103370 0x7f10bc109cf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009cd0 tx=0x7f10b402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- 172.21.15.120:0/64593126 shutdown_connections 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/64593126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc103370 0x7f10bc109cf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/64593126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc1029b0 0x7f10bc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/64593126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 0x7f10bc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- 172.21.15.120:0/64593126 >> 172.21.15.120:0/64593126 conn(0x7f10bc0fd140 msgr2=0x7f10bc0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- 172.21.15.120:0/64593126 shutdown_connections 2024-09-10T22:57:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- 172.21.15.120:0/64593126 wait complete. 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 Processor -- start 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- start start 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 0x7f10bc06b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc103370 0x7f10bc1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d4d0 con 0x7f10bc1029b0 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d350 con 0x7f10bc103370 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.565+0000 7f10c1d9b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d650 con 0x7f10bc101860 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 0x7f10bc06b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44644/0 (socket says 172.21.15.120:44644) 2024-09-10T22:57:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 -- 172.21.15.120:0/2157765304 learned_addr learned my addr 172.21.15.120:0/2157765304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc103370 0x7f10bc1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 -- 172.21.15.120:0/2157765304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 msgr2=0x7f10bc06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 0x7f10bc06b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 -- 172.21.15.120:0/2157765304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc103370 msgr2=0x7f10bc1a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc103370 0x7f10bc1a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 -- 172.21.15.120:0/2157765304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc1a0940 con 0x7f10bc1029b0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10baffd640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9f0 tx=0x7f10b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10b8ff9640 1 -- 172.21.15.120:0/2157765304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000cd80 con 0x7f10bc1029b0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10b8ff9640 1 -- 172.21.15.120:0/2157765304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b00108e0 con 0x7f10bc1029b0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10b8ff9640 1 -- 172.21.15.120:0/2157765304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0018980 con 0x7f10bc1029b0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc1a0bd0 con 0x7f10bc1029b0 2024-09-10T22:57:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.566+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1a1120 con 0x7f10bc1029b0 2024-09-10T22:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 Processor -- start 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 -- start start 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10b8ff9640 1 -- 172.21.15.120:0/2157765304 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f10b0010430 con 0x7f10bc1029b0 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291dbd0 0x559e62920090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e6288a6f0 con 0x559e6291b190 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559e62879b20 con 0x559e62918860 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.569+0000 7f10c4c46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559e6287d930 con 0x559e6291dbd0 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10baffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291dbd0 0x559e62920090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44646/0 (socket says 172.21.15.120:44646) 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 -- 172.21.15.120:0/1462874452 learned_addr learned my addr 172.21.15.120:0/1462874452 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bbfff640 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 -- 172.21.15.120:0/1462874452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291dbd0 msgr2=0x559e62920090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291dbd0 0x559e62920090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 -- 172.21.15.120:0/1462874452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 msgr2=0x559e6291ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 -- 172.21.15.120:0/1462874452 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6283f530 con 0x559e6291b190 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009d20 tx=0x7f10ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e5612a2b0fc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10b8ff9640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 0x7f10a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10a1ffb640 1 -- 172.21.15.120:0/1462874452 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac01e070 con 0x559e6291b190 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10a1ffb640 1 -- 172.21.15.120:0/1462874452 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac019ae0 con 0x559e6291b190 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10a1ffb640 1 -- 172.21.15.120:0/1462874452 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00eac0 con 0x559e6291b190 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 0x7f10a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.570+0000 7f10b8ff9640 1 -- 172.21.15.120:0/2157765304 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f10b00e5ef0 con 0x7f10bc1029b0 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- 172.21.15.120:0/1462874452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 msgr2=0x559e6291d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 0x7f10a40b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005e30 tx=0x7f10b403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009d20 tx=0x7f10ac011810 comp rx=0 tx=0).stop 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- 172.21.15.120:0/1462874452 shutdown_connections 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291dbd0 0x559e62920090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291b190 0x559e6291d580 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 --2- 172.21.15.120:0/1462874452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- 172.21.15.120:0/1462874452 >> 172.21.15.120:0/1462874452 conn(0x559e629139b0 msgr2=0x559e62914740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- 172.21.15.120:0/1462874452 shutdown_connections 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- 172.21.15.120:0/1462874452 wait complete. 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.571+0000 7f10c4c46740 1 -- start start 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 Processor -- start 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e629262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e62926980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559e627b1ab0 con 0x559e6291dbd0 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559e6281f730 con 0x559e62918860 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559e62920bb0 con 0x559e6291b190 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10c4c46740 10 client.0 _ll_register_callbacks cb 0x559e628736b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44656/0 (socket says 172.21.15.120:44656) 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 -- 172.21.15.120:0/2792293182 learned_addr learned my addr 172.21.15.120:0/2792293182 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10baffd640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e62926980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e629262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 -- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 msgr2=0x559e62926980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e62926980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 -- 172.21.15.120:0/2792293182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 msgr2=0x559e629262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e629262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.572+0000 7f10bbfff640 1 -- 172.21.15.120:0/2792293182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e625db790 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10b4035000 tx=0x7f10b402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b327d379aabd1294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4004720 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f10b40305a0 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac019ae0 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10b40307a0 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e62654620 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac019ae0 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f10b4030070 con 0x559e6291dbd0 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 20 client.16287 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e62931b90 0x559e62933fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 4 client.16287 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6288c640 con 0x559e62931b90 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.573+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e62931b90 0x559e62933fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.574+0000 7f109affd640 10 client.16287 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.574+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e62931b90 0x559e62933fe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97cfaa39a6254a08 server_cookie=6b2989d5bb8bce61 in_seq=0 out_seq=0 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f109affd640 1 -- 172.21.15.120:0/2792293182 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e6288c640 con 0x559e62931b90 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f109affd640 10 client.16287 handle_command_reply: tid=0 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 client.16287 shutdown 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 msgr2=0x559e62927550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10b4035000 tx=0x7f10b402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 shutdown_connections 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e62931b90 0x559e62933fe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62927550 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f109affd640 0 client.16287 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e62926980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2792293182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e629262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 >> 172.21.15.120:0/2792293182 conn(0x559e629139b0 msgr2=0x559e62912bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 shutdown_connections 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 1 -- 172.21.15.120:0/2792293182 wait complete. 2024-09-10T22:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.576+0000 7f10c4c46740 20 client.16287 trim_cache size 0 max 16384 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 Processor -- start 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 -- start start 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6291e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a5bb0 con 0x559e6291dbd0 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a8430 con 0x559e62918860 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10c4c46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a6220 con 0x559e6291b190 2024-09-10T22:57:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35260/0 (socket says 172.21.15.120:35260) 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44664/0 (socket says 172.21.15.120:44664) 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 -- 172.21.15.120:0/914147854 learned_addr learned my addr 172.21.15.120:0/914147854 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10baffd640 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6291e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 -- 172.21.15.120:0/914147854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 msgr2=0x559e6291e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6291e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 -- 172.21.15.120:0/914147854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 msgr2=0x559e62917cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.592+0000 7f10bb7fe640 1 -- 172.21.15.120:0/914147854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6291d8f0 con 0x559e62918860 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.593+0000 7f10bbfff640 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.593+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ab0 tx=0x7f10ac005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3ed7ffe6b72cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.593+0000 7f10a17fa640 1 -- 172.21.15.120:0/914147854 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac01e040 con 0x559e62918860 2024-09-10T22:57:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.593+0000 7f10a17fa640 1 -- 172.21.15.120:0/914147854 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac01f040 con 0x559e62918860 2024-09-10T22:57:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.593+0000 7f10a17fa640 1 -- 172.21.15.120:0/914147854 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0035f0 con 0x559e62918860 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- 172.21.15.120:0/914147854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 msgr2=0x559e6291dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ab0 tx=0x7f10ac005d90 comp rx=0 tx=0).stop 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- 172.21.15.120:0/914147854 shutdown_connections 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e62917cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6291e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 --2- 172.21.15.120:0/914147854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6291dfb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- 172.21.15.120:0/914147854 >> 172.21.15.120:0/914147854 conn(0x559e629139b0 msgr2=0x559e62914450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- 172.21.15.120:0/914147854 shutdown_connections 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- 172.21.15.120:0/914147854 wait complete. 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 -- start start 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.594+0000 7f10c4c46740 1 Processor -- start 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6292de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6292cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a68f0 con 0x559e6291dbd0 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a7d60 con 0x559e62918860 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1a6fc0 con 0x559e6291b190 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 10 client.0 _ll_register_callbacks cb 0x7f10b4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6292de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44676/0 (socket says 172.21.15.120:44676) 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 -- 172.21.15.120:0/2672237500 learned_addr learned my addr 172.21.15.120:0/2672237500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10baffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6292cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 -- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 msgr2=0x559e6292cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6292cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 -- 172.21.15.120:0/2672237500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 msgr2=0x559e6292de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6292de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 -- 172.21.15.120:0/2672237500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6292b690 con 0x559e6291dbd0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6292de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10bbfff640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009bf0 tx=0x7f10b4031d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7433d3ee55752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f109affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40305a0 con 0x559e6291dbd0 2024-09-10T22:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f10b405d480 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.595+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac01f040 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10b406d050 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 10 client.16293 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e6299ec10 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac01f040 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f10b4030040 con 0x559e6291dbd0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 10 client.16293 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 20 client.16293 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e6299f5f0 0x559e629a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 4 client.16293 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e628880f0 con 0x559e6299f5f0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.596+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e6299f5f0 0x559e629a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.597+0000 7f109affd640 10 client.16293 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.597+0000 7f10bb7fe640 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e6299f5f0 0x559e629a1a10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737e5ceb8c151d49 server_cookie=5933e590577aaf5e in_seq=0 out_seq=0 2024-09-10T22:57:04.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 757035 16.80 MB/sec execute 119 sec latency 48.034 ms 2024-09-10T22:57:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f109affd640 1 -- 172.21.15.120:0/2672237500 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e628880f0 con 0x559e6299f5f0 2024-09-10T22:57:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f109affd640 10 client.16293 handle_command_reply: tid=0 2024-09-10T22:57:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 client.16293 shutdown 2024-09-10T22:57:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 msgr2=0x559e6292d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009bf0 tx=0x7f10b4031d70 comp rx=0 tx=0).stop 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 shutdown_connections 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559e6299f5f0 0x559e629a1a10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559e6291dbd0 0x559e6292d150 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f109affd640 0 client.16293 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e6291b190 0x559e6292cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 --2- 172.21.15.120:0/2672237500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559e62918860 0x559e6292de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 >> 172.21.15.120:0/2672237500 conn(0x559e629139b0 msgr2=0x559e6291dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 shutdown_connections 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.757+0000 7f10c4c46740 1 -- 172.21.15.120:0/2672237500 wait complete. 2024-09-10T22:57:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c4c46740 20 client.16293 trim_cache size 0 max 16384 2024-09-10T22:57:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:57:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:57:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:57:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:57:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 msgr2=0x7f10a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 0x7f10a40b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005e30 tx=0x7f10b403d040 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 msgr2=0x7f10bc19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9f0 tx=0x7f10b000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 shutdown_connections 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f10a40b52e0 0x7f10a40b77a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10bc103370 0x7f10bc1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10bc1029b0 0x7f10bc19da10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 --2- 172.21.15.120:0/2157765304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10bc101860 0x7f10bc06b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.758+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 >> 172.21.15.120:0/2157765304 conn(0x7f10bc0fd140 msgr2=0x7f10bc105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.759+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 shutdown_connections 2024-09-10T22:57:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:04.759+0000 7f10c1d9b640 1 -- 172.21.15.120:0/2157765304 wait complete. 2024-09-10T22:57:05.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:05 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:57:05.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:05 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:57:05.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:05 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:57:05.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 760167 16.81 MB/sec execute 120 sec latency 57.310 ms 2024-09-10T22:57:06.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:06 smithi176 ceph-mon[34982]: pgmap v485: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 411 op/s 2024-09-10T22:57:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:06 smithi120 ceph-mon[31247]: pgmap v485: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 411 op/s 2024-09-10T22:57:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:06 smithi153 ceph-mon[34951]: pgmap v485: 97 pgs: 97 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 411 op/s 2024-09-10T22:57:06.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 763222 16.80 MB/sec execute 121 sec latency 89.700 ms 2024-09-10T22:57:07.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 766005 16.76 MB/sec execute 122 sec latency 43.418 ms 2024-09-10T22:57:08.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:08 smithi176 ceph-mon[34982]: pgmap v486: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 569 op/s 2024-09-10T22:57:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:08 smithi120 ceph-mon[31247]: pgmap v486: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 569 op/s 2024-09-10T22:57:08.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:08 smithi153 ceph-mon[34951]: pgmap v486: 97 pgs: 97 active+clean; 49 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 569 op/s 2024-09-10T22:57:08.623 INFO:tasks.workunit.client.0.smithi120.stdout: 1 769304 16.77 MB/sec execute 123 sec latency 12.873 ms 2024-09-10T22:57:09.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 772108 16.78 MB/sec execute 124 sec latency 12.771 ms 2024-09-10T22:57:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:10 smithi176 ceph-mon[34982]: pgmap v487: 97 pgs: 97 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-09-10T22:57:10.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:10 smithi120 ceph-mon[31247]: pgmap v487: 97 pgs: 97 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-09-10T22:57:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:10 smithi153 ceph-mon[34951]: pgmap v487: 97 pgs: 97 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-09-10T22:57:10.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 774972 16.76 MB/sec execute 125 sec latency 15.168 ms 2024-09-10T22:57:11.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 778252 16.78 MB/sec execute 126 sec latency 29.391 ms 2024-09-10T22:57:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:12 smithi176 ceph-mon[34982]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-09-10T22:57:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:12 smithi120 ceph-mon[31247]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-09-10T22:57:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:12 smithi153 ceph-mon[34951]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-09-10T22:57:12.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 781371 16.79 MB/sec execute 127 sec latency 80.920 ms 2024-09-10T22:57:13.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 784689 16.80 MB/sec execute 128 sec latency 15.319 ms 2024-09-10T22:57:14.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:14 smithi176 ceph-mon[34982]: pgmap v489: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-10T22:57:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:14 smithi120 ceph-mon[31247]: pgmap v489: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-10T22:57:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:14 smithi153 ceph-mon[34951]: pgmap v489: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-09-10T22:57:14.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 788018 16.79 MB/sec execute 129 sec latency 19.764 ms 2024-09-10T22:57:15.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 791161 16.80 MB/sec execute 130 sec latency 16.781 ms 2024-09-10T22:57:16.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:16 smithi176 ceph-mon[34982]: pgmap v490: 97 pgs: 97 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 421 op/s 2024-09-10T22:57:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:16 smithi120 ceph-mon[31247]: pgmap v490: 97 pgs: 97 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 421 op/s 2024-09-10T22:57:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:16 smithi153 ceph-mon[34951]: pgmap v490: 97 pgs: 97 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 421 op/s 2024-09-10T22:57:16.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 794167 16.77 MB/sec execute 131 sec latency 15.416 ms 2024-09-10T22:57:17.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 796475 16.75 MB/sec execute 132 sec latency 55.948 ms 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 -- 172.21.15.120:0/1073668298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 msgr2=0x7f046410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 --2- 172.21.15.120:0/1073668298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f046410ab40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f045c009ca0 tx=0x7f045c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 -- 172.21.15.120:0/1073668298 shutdown_connections 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 --2- 172.21.15.120:0/1073668298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f046410ab40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 --2- 172.21.15.120:0/1073668298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0464105b10 0x7f0464107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 --2- 172.21.15.120:0/1073668298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f046406b3d0 0x7f04641055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 -- 172.21.15.120:0/1073668298 >> 172.21.15.120:0/1073668298 conn(0x7f04640fd140 msgr2=0x7f04640ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 -- 172.21.15.120:0/1073668298 shutdown_connections 2024-09-10T22:57:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.852+0000 7f046a6c6640 1 -- 172.21.15.120:0/1073668298 wait complete. 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 Processor -- start 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 -- start start 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 0x7f0464198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0464105b10 0x7f0464199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f046410e5d0 con 0x7f0464108680 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f046410e450 con 0x7f0464105b10 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f046a6c6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f046410e750 con 0x7f046406b3d0 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48296/0 (socket says 172.21.15.120:48296) 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f0468c63640 1 -- 172.21.15.120:0/3921052239 learned_addr learned my addr 172.21.15.120:0/3921052239 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.853+0000 7f0463fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 0x7f0464198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f04637fe640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0464105b10 0x7f0464199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 -- 172.21.15.120:0/3921052239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0464105b10 msgr2=0x7f0464199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0464105b10 0x7f0464199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 -- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 msgr2=0x7f0464199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 -- 172.21.15.120:0/3921052239 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046419a870 con 0x7f046406b3d0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0468c63640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f0463fff640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 0x7f0464198d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f045400d6d0 tx=0x7f045400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f04617fa640 1 -- 172.21.15.120:0/3921052239 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454017070 con 0x7f046406b3d0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f04617fa640 1 -- 172.21.15.120:0/3921052239 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0454009820 con 0x7f046406b3d0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f04617fa640 1 -- 172.21.15.120:0/3921052239 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004c40 con 0x7f046406b3d0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046419ab60 con 0x7f046406b3d0 2024-09-10T22:57:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.854+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641a1230 con 0x7f046406b3d0 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.857+0000 7f04617fa640 1 -- 172.21.15.120:0/3921052239 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0454005220 con 0x7f046406b3d0 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.857+0000 7f046d497740 1 Processor -- start 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.857+0000 7f046d497740 1 -- start start 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.857+0000 7f046d497740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.857+0000 7f046d497740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1b5a0 0x558109a1d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f046d497740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1dfe0 0x558109a204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f046d497740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581099866f0 con 0x558109a18c70 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f046d497740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558109975b20 con 0x558109a1b5a0 2024-09-10T22:57:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f046d497740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558109979930 con 0x558109a1dfe0 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48308/0 (socket says 172.21.15.120:48308) 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 -- 172.21.15.120:0/2430195126 learned_addr learned my addr 172.21.15.120:0/2430195126 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f04637fe640 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1dfe0 0x558109a204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0463fff640 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1b5a0 0x558109a1d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f04617fa640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 0x7f044c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 -- 172.21.15.120:0/2430195126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1dfe0 msgr2=0x558109a204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1dfe0 0x558109a204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 -- 172.21.15.120:0/2430195126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1b5a0 msgr2=0x558109a1d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1b5a0 0x558109a1d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 -- 172.21.15.120:0/2430195126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810993b530 con 0x558109a18c70 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f04617fa640 1 -- 172.21.15.120:0/3921052239 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f04540e5620 con 0x7f046406b3d0 2024-09-10T22:57:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 0x7f044c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f0468c63640 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f045c005f70 tx=0x7f045c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c540f5d9446d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.858+0000 7f044a7fc640 1 -- 172.21.15.120:0/2430195126 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c0317b0 con 0x558109a18c70 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f044a7fc640 1 -- 172.21.15.120:0/2430195126 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045c036040 con 0x558109a18c70 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f044a7fc640 1 -- 172.21.15.120:0/2430195126 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c02faf0 con 0x558109a18c70 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 -- 172.21.15.120:0/2430195126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 msgr2=0x558109a1b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f0468c63640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 0x7f044c0b7820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f045c04b000 tx=0x7f045c04c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f045c005f70 tx=0x7f045c005fa0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 -- 172.21.15.120:0/2430195126 shutdown_connections 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1dfe0 0x558109a204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1b5a0 0x558109a1d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 --2- 172.21.15.120:0/2430195126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1b060 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 -- 172.21.15.120:0/2430195126 >> 172.21.15.120:0/2430195126 conn(0x558109a0f9b0 msgr2=0x558109a10740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 -- 172.21.15.120:0/2430195126 shutdown_connections 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.859+0000 7f046d497740 1 -- 172.21.15.120:0/2430195126 wait complete. 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 -- start start 2024-09-10T22:57:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 Processor -- start 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581098adab0 con 0x558109a1dfe0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55810991b730 con 0x558109a18c70 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558109a20e40 con 0x558109a1b5a0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f046d497740 10 client.0 _ll_register_callbacks cb 0x55810996f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48322/0 (socket says 172.21.15.120:48322) 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f04637fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f0468c63640 1 -- 172.21.15.120:0/1383832969 learned_addr learned my addr 172.21.15.120:0/1383832969 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.860+0000 7f0463fff640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 -- 172.21.15.120:0/1383832969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 msgr2=0x558109a22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 -- 172.21.15.120:0/1383832969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 msgr2=0x558109a23390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 -- 172.21.15.120:0/1383832969 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581096d7790 con 0x558109a1b5a0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f0468c63640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04637fe640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a227c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f04580126d0 tx=0x7f0458012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eaf93f117766df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04437fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458004320 con 0x558109a1b5a0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045c036040 con 0x558109a1b5a0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f04580119a0 con 0x558109a1b5a0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04580044c0 con 0x558109a1b5a0 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f046d497740 10 client.25777 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558109750620 con 0x558109a1b5a0 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.861+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045c036040 con 0x558109a1b5a0 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f045800f440 con 0x558109a1b5a0 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 10 client.25777 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 20 client.25777 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a2dea0 0x558109a30320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 4 client.25777 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558109988640 con 0x558109a2dea0 2024-09-10T22:57:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f0463fff640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a2dea0 0x558109a30320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f0463fff640 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a2dea0 0x558109a30320 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f87b1bc2d45636 server_cookie=4a5c7401ce179fce in_seq=0 out_seq=0 2024-09-10T22:57:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.862+0000 7f04437fe640 10 client.25777 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f04437fe640 1 -- 172.21.15.120:0/1383832969 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558109988640 con 0x558109a2dea0 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f04437fe640 10 client.25777 handle_command_reply: tid=0 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 client.25777 shutdown 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 msgr2=0x558109a227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a227c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f04580126d0 tx=0x7f0458012b90 comp rx=0 tx=0).stop 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 shutdown_connections 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a2dea0 0x558109a30320 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a23390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a227c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f04437fe640 0 client.25777 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 --2- 172.21.15.120:0/1383832969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 >> 172.21.15.120:0/1383832969 conn(0x558109a0f9b0 msgr2=0x558109a0ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 shutdown_connections 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.864+0000 7f046d497740 1 -- 172.21.15.120:0/1383832969 wait complete. 2024-09-10T22:57:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.865+0000 7f046d497740 20 client.25777 trim_cache size 0 max 16384 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.880+0000 7f046d497740 1 Processor -- start 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.880+0000 7f046d497740 1 -- start start 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a1e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1dfe0 0x558109a22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a5c00 con 0x558109a18c70 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a8480 con 0x558109a1dfe0 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f046d497740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a6270 con 0x558109a1b5a0 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f0463fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f0463fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48330/0 (socket says 172.21.15.120:48330) 2024-09-10T22:57:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a1e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f0463fff640 1 -- 172.21.15.120:0/2305474884 learned_addr learned my addr 172.21.15.120:0/2305474884 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f0468c63640 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1dfe0 0x558109a22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 -- 172.21.15.120:0/2305474884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1dfe0 msgr2=0x558109a22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1dfe0 0x558109a22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 -- 172.21.15.120:0/2305474884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 msgr2=0x558109a1e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 -- 172.21.15.120:0/2305474884 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558109a1dd00 con 0x558109a1b5a0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f0463fff640 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.881+0000 7f04637fe640 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a1e900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f045800f8e0 tx=0x7f0458009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae3219bf7dab2fba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f0449ffb640 1 -- 172.21.15.120:0/2305474884 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800fea0 con 0x558109a1b5a0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f0449ffb640 1 -- 172.21.15.120:0/2305474884 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0458011e50 con 0x558109a1b5a0 2024-09-10T22:57:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f0449ffb640 1 -- 172.21.15.120:0/2305474884 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458005040 con 0x558109a1b5a0 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 -- 172.21.15.120:0/2305474884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 msgr2=0x558109a1e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a1e900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f045800f8e0 tx=0x7f0458009e10 comp rx=0 tx=0).stop 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 -- 172.21.15.120:0/2305474884 shutdown_connections 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a1dfe0 0x558109a22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a1e900 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 --2- 172.21.15.120:0/2305474884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a18c70 0x558109a1e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 -- 172.21.15.120:0/2305474884 >> 172.21.15.120:0/2305474884 conn(0x558109a0f9b0 msgr2=0x558109a10610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 -- 172.21.15.120:0/2305474884 shutdown_connections 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.882+0000 7f046d497740 1 -- 172.21.15.120:0/2305474884 wait complete. 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 -- start start 2024-09-10T22:57:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 Processor -- start 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a26ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a27180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a6940 con 0x558109a1dfe0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a7db0 con 0x558109a18c70 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f04641a7010 con 0x558109a1b5a0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f046d497740 10 client.0 _ll_register_callbacks cb 0x7f04580097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f0468c63640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48340/0 (socket says 172.21.15.120:48340) 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f0468c63640 1 -- 172.21.15.120:0/484726386 learned_addr learned my addr 172.21.15.120:0/484726386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f04637fe640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.883+0000 7f0463fff640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a26ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 -- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 msgr2=0x558109a27180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a27180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 -- 172.21.15.120:0/484726386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 msgr2=0x558109a26ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a26ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 -- 172.21.15.120:0/484726386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558109a28490 con 0x558109a1dfe0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0463fff640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a26ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04637fe640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a27180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f0468c63640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f045c005e30 tx=0x7f045c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd1bdffa8de5eff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04437fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c064060 con 0x558109a1dfe0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f045c04d040 con 0x558109a1dfe0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0458011e50 con 0x558109a1dfe0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f045c004100 con 0x558109a1dfe0 2024-09-10T22:57:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f046d497740 10 client.16311 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558109a29830 con 0x558109a1dfe0 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.884+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0458011e50 con 0x558109a1dfe0 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f045c030830 con 0x558109a1dfe0 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 10 client.16311 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 20 client.16311 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a29f50 0x558109a9ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 4 client.16311 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581099840f0 con 0x558109a29f50 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f0463fff640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a29f50 0x558109a9ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f04437fe640 10 client.16311 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:17.885+0000 7f0463fff640 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a29f50 0x558109a9ce00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5763293f0361cb server_cookie=dd013c8c045a97a7 in_seq=0 out_seq=0 2024-09-10T22:57:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f04437fe640 1 -- 172.21.15.120:0/484726386 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581099840f0 con 0x558109a29f50 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f04437fe640 10 client.16311 handle_command_reply: tid=0 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 client.16311 shutdown 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 msgr2=0x558109a27c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f045c005e30 tx=0x7f045c003970 comp rx=0 tx=0).stop 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 shutdown_connections 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558109a29f50 0x558109a9ce00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558109a1dfe0 0x558109a27c60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558109a1b5a0 0x558109a27180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f04437fe640 0 client.16311 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 --2- 172.21.15.120:0/484726386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558109a18c70 0x558109a26ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 >> 172.21.15.120:0/484726386 conn(0x558109a0f9b0 msgr2=0x558109a1e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 shutdown_connections 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 1 -- 172.21.15.120:0/484726386 wait complete. 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.052+0000 7f046d497740 20 client.16311 trim_cache size 0 max 16384 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:57:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:57:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 msgr2=0x7f044c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 0x7f044c0b7820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f045c04b000 tx=0x7f045c04c000 comp rx=0 tx=0).stop 2024-09-10T22:57:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 msgr2=0x7f0464198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 0x7f0464198d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f045400d6d0 tx=0x7f045400db90 comp rx=0 tx=0).stop 2024-09-10T22:57:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 shutdown_connections 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f044c0b5360 0x7f044c0b7820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0464108680 0x7f0464199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0464105b10 0x7f0464199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 --2- 172.21.15.120:0/3921052239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f046406b3d0 0x7f0464198d30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 >> 172.21.15.120:0/3921052239 conn(0x7f04640fd140 msgr2=0x7f0464103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 shutdown_connections 2024-09-10T22:57:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:18.053+0000 7f046a6c6640 1 -- 172.21.15.120:0/3921052239 wait complete. 2024-09-10T22:57:18.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:18 smithi176 ceph-mon[34982]: pgmap v491: 97 pgs: 97 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T22:57:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:18 smithi120 ceph-mon[31247]: pgmap v491: 97 pgs: 97 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T22:57:18.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:18 smithi153 ceph-mon[34951]: pgmap v491: 97 pgs: 97 active+clean; 49 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T22:57:18.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 799482 16.75 MB/sec execute 133 sec latency 97.636 ms 2024-09-10T22:57:19.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 802937 16.77 MB/sec execute 134 sec latency 17.979 ms 2024-09-10T22:57:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:20 smithi176 ceph-mon[34982]: pgmap v492: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 299 op/s 2024-09-10T22:57:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:20 smithi120 ceph-mon[31247]: pgmap v492: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 299 op/s 2024-09-10T22:57:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:20 smithi153 ceph-mon[34951]: pgmap v492: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 299 op/s 2024-09-10T22:57:20.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 806194 16.79 MB/sec execute 135 sec latency 91.789 ms 2024-09-10T22:57:21.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 809463 16.79 MB/sec execute 136 sec latency 16.417 ms 2024-09-10T22:57:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:22 smithi176 ceph-mon[34982]: pgmap v493: 97 pgs: 97 active+clean; 40 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 420 op/s 2024-09-10T22:57:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:22 smithi120 ceph-mon[31247]: pgmap v493: 97 pgs: 97 active+clean; 40 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 420 op/s 2024-09-10T22:57:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:22 smithi153 ceph-mon[34951]: pgmap v493: 97 pgs: 97 active+clean; 40 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 420 op/s 2024-09-10T22:57:22.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 811831 16.73 MB/sec execute 137 sec latency 47.772 ms 2024-09-10T22:57:23.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 814574 16.74 MB/sec execute 138 sec latency 11.952 ms 2024-09-10T22:57:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:24 smithi176 ceph-mon[34982]: pgmap v494: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T22:57:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:24 smithi120 ceph-mon[31247]: pgmap v494: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T22:57:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:24 smithi153 ceph-mon[34951]: pgmap v494: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 434 op/s 2024-09-10T22:57:24.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 817844 16.73 MB/sec execute 139 sec latency 12.601 ms 2024-09-10T22:57:25.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 821080 16.74 MB/sec execute 140 sec latency 14.587 ms 2024-09-10T22:57:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:26 smithi176 ceph-mon[34982]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T22:57:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:26 smithi120 ceph-mon[31247]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T22:57:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:26 smithi153 ceph-mon[34951]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T22:57:26.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 824382 16.75 MB/sec execute 141 sec latency 21.645 ms 2024-09-10T22:57:27.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 827368 16.76 MB/sec execute 142 sec latency 84.555 ms 2024-09-10T22:57:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:28 smithi176 ceph-mon[34982]: pgmap v496: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-09-10T22:57:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:28 smithi120 ceph-mon[31247]: pgmap v496: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-09-10T22:57:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:28 smithi153 ceph-mon[34951]: pgmap v496: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 487 op/s 2024-09-10T22:57:28.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 830597 16.75 MB/sec execute 143 sec latency 40.150 ms 2024-09-10T22:57:29.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 833997 16.76 MB/sec execute 144 sec latency 41.008 ms 2024-09-10T22:57:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:30 smithi176 ceph-mon[34982]: pgmap v497: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-09-10T22:57:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:30 smithi120 ceph-mon[31247]: pgmap v497: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-09-10T22:57:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:30 smithi153 ceph-mon[34951]: pgmap v497: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 420 op/s 2024-09-10T22:57:30.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 837152 16.76 MB/sec execute 145 sec latency 17.640 ms 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 -- 172.21.15.120:0/2607510352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560094450 msgr2=0x7f7560094850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 --2- 172.21.15.120:0/2607510352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560094450 0x7f7560094850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f755800d910 tx=0x7f755802c370 comp rx=0 tx=0).stop 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 -- 172.21.15.120:0/2607510352 shutdown_connections 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 --2- 172.21.15.120:0/2607510352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7560095f60 0x7f756009c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 --2- 172.21.15.120:0/2607510352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75600955a0 0x7f7560095a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 --2- 172.21.15.120:0/2607510352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560094450 0x7f7560094850 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 -- 172.21.15.120:0/2607510352 >> 172.21.15.120:0/2607510352 conn(0x7f756008fcf0 msgr2=0x7f7560092130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 -- 172.21.15.120:0/2607510352 shutdown_connections 2024-09-10T22:57:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 -- 172.21.15.120:0/2607510352 wait complete. 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.153+0000 7f756d423640 1 Processor -- start 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 -- start start 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 0x7f7560129760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75600955a0 0x7f7560129db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f756009efe0 con 0x7f7560094450 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f756009ee60 con 0x7f7560095f60 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f756d423640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f756009f160 con 0x7f75600955a0 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 0x7f7560129760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75667fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75600955a0 0x7f7560129db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47258/0 (socket says 172.21.15.120:47258) 2024-09-10T22:57:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 0x7f7560129760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46266/0 (socket says 172.21.15.120:46266) 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 -- 172.21.15.120:0/2825583115 learned_addr learned my addr 172.21.15.120:0/2825583115 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 -- 172.21.15.120:0/2825583115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75600955a0 msgr2=0x7f7560129db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75600955a0 0x7f7560129db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 -- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 msgr2=0x7f7560129760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 0x7f7560129760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.154+0000 7f75677fe640 1 -- 172.21.15.120:0/2825583115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756012b2a0 con 0x7f7560095f60 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f75677fe640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75680695f0 tx=0x7f7568075750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f754ffff640 1 -- 172.21.15.120:0/2825583115 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568069d50 con 0x7f7560095f60 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756012b590 con 0x7f7560095f60 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f754ffff640 1 -- 172.21.15.120:0/2825583115 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f756806c620 con 0x7f7560095f60 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f754ffff640 1 -- 172.21.15.120:0/2825583115 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680848c0 con 0x7f7560095f60 2024-09-10T22:57:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.155+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7560131c60 con 0x7f7560095f60 2024-09-10T22:57:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 Processor -- start 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f754ffff640 1 -- 172.21.15.120:0/2825583115 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f756808c030 con 0x7f7560095f60 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 -- start start 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7dc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5570bb7466f0 con 0x5570bb7d7390 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5570bb735b20 con 0x5570bb7d4a60 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.157+0000 7f75701ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5570bb739930 con 0x5570bb7d9dd0 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46268/0 (socket says 172.21.15.120:46268) 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 -- 172.21.15.120:0/3408665881 learned_addr learned my addr 172.21.15.120:0/3408665881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f75667fc640 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f75667fc640 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 -- 172.21.15.120:0/3408665881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 msgr2=0x5570bb7dc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7dc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 -- 172.21.15.120:0/3408665881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 msgr2=0x5570bb7d6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7d6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 -- 172.21.15.120:0/3408665881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bb6fb530 con 0x5570bb7d7390 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f754ffff640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 0x7f75500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f7566ffd640 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f755800d910 tx=0x7f7558005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a9fdbec7fe2d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f75667fc640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 0x7f75500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f754d7fa640 1 -- 172.21.15.120:0/3408665881 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558042070 con 0x5570bb7d7390 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f754ffff640 1 -- 172.21.15.120:0/2825583115 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f75681523d0 con 0x7f7560095f60 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f754d7fa640 1 -- 172.21.15.120:0/3408665881 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558030070 con 0x5570bb7d7390 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.158+0000 7f754d7fa640 1 -- 172.21.15.120:0/3408665881 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558003af0 con 0x5570bb7d7390 2024-09-10T22:57:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- 172.21.15.120:0/3408665881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 msgr2=0x5570bb7d9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75667fc640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 0x7f75500b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bfd0 tx=0x7f755c00b650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f755800d910 tx=0x7f7558005c00 comp rx=0 tx=0).stop 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- 172.21.15.120:0/3408665881 shutdown_connections 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7dc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7d9780 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 --2- 172.21.15.120:0/3408665881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7d6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- 172.21.15.120:0/3408665881 >> 172.21.15.120:0/3408665881 conn(0x5570bb7cfbb0 msgr2=0x5570bb7d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- 172.21.15.120:0/3408665881 shutdown_connections 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- 172.21.15.120:0/3408665881 wait complete. 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 -- start start 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.159+0000 7f75701ef740 1 Processor -- start 2024-09-10T22:57:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7df290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5570bb66dab0 con 0x5570bb7d4a60 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5570bb6db730 con 0x5570bb7d7390 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5570bb7dcd50 con 0x5570bb7d9dd0 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 10 client.0 _ll_register_callbacks cb 0x5570bb72f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46276/0 (socket says 172.21.15.120:46276) 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 -- 172.21.15.120:0/2627186862 learned_addr learned my addr 172.21.15.120:0/2627186862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75667fc640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75677fe640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 -- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 msgr2=0x5570bb7df290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7df290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 -- 172.21.15.120:0/2627186862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 msgr2=0x5570bb7de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7de730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 -- 172.21.15.120:0/2627186862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bb497790 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f7566ffd640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f755802c890 tx=0x7f7558004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57960254948f4a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f754aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558039040 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7558030070 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.160+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7558003ab0 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7558042040 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 10 client.16323 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570bb510620 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558030070 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f755803e430 con 0x5570bb7d4a60 2024-09-10T22:57:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 10 client.16323 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 20 client.16323 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb7e9510 0x5570bb7eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 4 client.16323 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570bb748640 con 0x5570bb7e9510 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.161+0000 7f75667fc640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb7e9510 0x5570bb7eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.162+0000 7f754aff5640 10 client.16323 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.162+0000 7f75667fc640 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb7e9510 0x5570bb7eb9b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ffd49d11c325eb server_cookie=bd62d286504e7dd5 in_seq=0 out_seq=0 2024-09-10T22:57:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f754aff5640 1 -- 172.21.15.120:0/2627186862 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570bb748640 con 0x5570bb7e9510 2024-09-10T22:57:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f754aff5640 10 client.16323 handle_command_reply: tid=0 2024-09-10T22:57:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 client.16323 shutdown 2024-09-10T22:57:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 msgr2=0x5570bb7de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f755802c890 tx=0x7f7558004050 comp rx=0 tx=0).stop 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 shutdown_connections 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb7e9510 0x5570bb7eb9b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7df290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f754aff5640 0 client.16323 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 --2- 172.21.15.120:0/2627186862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d4a60 0x5570bb7de090 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 >> 172.21.15.120:0/2627186862 conn(0x5570bb7cfbb0 msgr2=0x5570bb7cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 shutdown_connections 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 1 -- 172.21.15.120:0/2627186862 wait complete. 2024-09-10T22:57:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.163+0000 7f75701ef740 20 client.16323 trim_cache size 0 max 16384 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 Processor -- start 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 -- start start 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d4a60 0x5570bb7da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75601386a0 con 0x5570bb7d9dd0 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f756013af20 con 0x5570bb7d7390 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75701ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7560138d10 con 0x5570bb7d4a60 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46280/0 (socket says 172.21.15.120:46280) 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f75677fe640 1 -- 172.21.15.120:0/2182723323 learned_addr learned my addr 172.21.15.120:0/2182723323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.178+0000 7f7566ffd640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d4a60 0x5570bb7da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75667fc640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 -- 172.21.15.120:0/2182723323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d4a60 msgr2=0x5570bb7da8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d4a60 0x5570bb7da8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 -- 172.21.15.120:0/2182723323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 msgr2=0x5570bb7ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 -- 172.21.15.120:0/2182723323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bb7d71c0 con 0x5570bb7d9dd0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75667fc640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f75677fe640 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7568078fd0 tx=0x7f756807b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa2654fccca9cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f754cff9640 1 -- 172.21.15.120:0/2182723323 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75681186d0 con 0x5570bb7d9dd0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f754cff9640 1 -- 172.21.15.120:0/2182723323 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75681192c0 con 0x5570bb7d9dd0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.179+0000 7f754cff9640 1 -- 172.21.15.120:0/2182723323 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756811d450 con 0x5570bb7d9dd0 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- 172.21.15.120:0/2182723323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 msgr2=0x5570bb7ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7568078fd0 tx=0x7f756807b040 comp rx=0 tx=0).stop 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- 172.21.15.120:0/2182723323 shutdown_connections 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d9dd0 0x5570bb7ce9b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d7390 0x5570bb7ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- 172.21.15.120:0/2182723323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d4a60 0x5570bb7da8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- 172.21.15.120:0/2182723323 >> 172.21.15.120:0/2182723323 conn(0x5570bb7cfbb0 msgr2=0x5570bb7da1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- 172.21.15.120:0/2182723323 shutdown_connections 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- 172.21.15.120:0/2182723323 wait complete. 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- start start 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 Processor -- start 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75601393e0 con 0x5570bb7d7390 2024-09-10T22:57:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f756013a850 con 0x5570bb7d4a60 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7560139ab0 con 0x5570bb7d9dd0 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75701ef740 10 client.0 _ll_register_callbacks cb 0x7f755c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46288/0 (socket says 172.21.15.120:46288) 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 -- 172.21.15.120:0/487087822 learned_addr learned my addr 172.21.15.120:0/487087822 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75677fe640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f7566ffd640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 -- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 msgr2=0x5570bb7e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 -- 172.21.15.120:0/487087822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 msgr2=0x5570bb7e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.180+0000 7f75667fc640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75667fc640 1 -- 172.21.15.120:0/487087822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bb7e3810 con 0x5570bb7d7390 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f7566ffd640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75667fc640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f755c00ea90 tx=0x7f755c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c47449824b43c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c023a40 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f755c01b030 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75681192c0 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f755c023be0 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 10 client.16329 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570bb8568b0 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75681192c0 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f755c004a40 con 0x5570bb7d7390 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 10 client.16329 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 20 client.16329 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb857290 0x5570bb8596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 4 client.16329 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.181+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570bb7440f0 con 0x5570bb857290 2024-09-10T22:57:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.182+0000 7f7566ffd640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb857290 0x5570bb8596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.182+0000 7f754aff5640 10 client.16329 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.182+0000 7f7566ffd640 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb857290 0x5570bb8596b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4263fdd15bca9c1c server_cookie=ee54e64ed668c671 in_seq=0 out_seq=0 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f754aff5640 1 -- 172.21.15.120:0/487087822 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570bb7440f0 con 0x5570bb857290 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f754aff5640 10 client.16329 handle_command_reply: tid=0 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 client.16329 shutdown 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 msgr2=0x5570bb7e50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f755c00ea90 tx=0x7f755c00b410 comp rx=0 tx=0).stop 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 shutdown_connections 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5570bb857290 0x5570bb8596b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5570bb7d9dd0 0x5570bb7e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5570bb7d7390 0x5570bb7e50c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f754aff5640 0 client.16329 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 --2- 172.21.15.120:0/487087822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5570bb7d4a60 0x5570bb7e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.336+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 >> 172.21.15.120:0/487087822 conn(0x5570bb7cfbb0 msgr2=0x5570bb7d3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.337+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 shutdown_connections 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.337+0000 7f75701ef740 1 -- 172.21.15.120:0/487087822 wait complete. 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.337+0000 7f75701ef740 20 client.16329 trim_cache size 0 max 16384 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:57:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:57:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.337+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 msgr2=0x7f75500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.337+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 0x7f75500b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bfd0 tx=0x7f755c00b650 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 msgr2=0x7f756012a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75680695f0 tx=0x7f7568075750 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 shutdown_connections 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75500b52e0 0x7f75500b77a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7560095f60 0x7f756012a9e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75600955a0 0x7f7560129db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 --2- 172.21.15.120:0/2825583115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7560094450 0x7f7560129760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.338+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 >> 172.21.15.120:0/2825583115 conn(0x7f756008fcf0 msgr2=0x7f7560098990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.339+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 shutdown_connections 2024-09-10T22:57:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:31.339+0000 7f756d423640 1 -- 172.21.15.120:0/2825583115 wait complete. 2024-09-10T22:57:31.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 839556 16.74 MB/sec execute 146 sec latency 73.037 ms 2024-09-10T22:57:32.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:32 smithi176 ceph-mon[34982]: pgmap v498: 97 pgs: 97 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-09-10T22:57:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:32 smithi120 ceph-mon[31247]: pgmap v498: 97 pgs: 97 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-09-10T22:57:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:32 smithi153 ceph-mon[34951]: pgmap v498: 97 pgs: 97 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-09-10T22:57:32.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 842193 16.72 MB/sec execute 147 sec latency 57.628 ms 2024-09-10T22:57:33.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:33.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 845460 16.74 MB/sec execute 148 sec latency 29.858 ms 2024-09-10T22:57:33.640 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:34.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- 172.21.15.120:0/3213879106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 msgr2=0x7f27c010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 --2- 172.21.15.120:0/3213879106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c010cab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009900 tx=0x7f27ac02a410 comp rx=0 tx=0).stop 2024-09-10T22:57:34.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- 172.21.15.120:0/3213879106 shutdown_connections 2024-09-10T22:57:34.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 --2- 172.21.15.120:0/3213879106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 --2- 172.21.15.120:0/3213879106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c010cab0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 --2- 172.21.15.120:0/3213879106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- 172.21.15.120:0/3213879106 >> 172.21.15.120:0/3213879106 conn(0x7f27c00fd700 msgr2=0x7f27c00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- 172.21.15.120:0/3213879106 shutdown_connections 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- 172.21.15.120:0/3213879106 wait complete. 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 Processor -- start 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.189+0000 7f27c70e7640 1 -- start start 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0112d70 con 0x7f27c01045c0 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0112bf0 con 0x7f27c010d100 2024-09-10T22:57:34.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c70e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0112ef0 con 0x7f27c0103ca0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34460/0 (socket says 172.21.15.120:34460) 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 -- 172.21.15.120:0/2401314887 learned_addr learned my addr 172.21.15.120:0/2401314887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c5684640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 -- 172.21.15.120:0/2401314887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 msgr2=0x7f27c01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 -- 172.21.15.120:0/2401314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 msgr2=0x7f27c019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 -- 172.21.15.120:0/2401314887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c019f370 con 0x7f27c01045c0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27c5684640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.190+0000 7f27bffff640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f27ac032fd0 tx=0x7f27ac031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.191+0000 7f27bdffb640 1 -- 172.21.15.120:0/2401314887 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac03f030 con 0x7f27c01045c0 2024-09-10T22:57:34.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.191+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c019f600 con 0x7f27c01045c0 2024-09-10T22:57:34.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.191+0000 7f27bdffb640 1 -- 172.21.15.120:0/2401314887 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27ac004490 con 0x7f27c01045c0 2024-09-10T22:57:34.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.191+0000 7f27bdffb640 1 -- 172.21.15.120:0/2401314887 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac035070 con 0x7f27c01045c0 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.192+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c0101830 con 0x7f27c01045c0 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.193+0000 7f27bdffb640 1 -- 172.21.15.120:0/2401314887 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f27ac04c060 con 0x7f27c01045c0 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.194+0000 7f27bdffb640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 0x7f27b00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.194+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 0x7f27b00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.194+0000 7f27bdffb640 1 -- 172.21.15.120:0/2401314887 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f27ac105ad0 con 0x7f27c01045c0 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.194+0000 7f27c94bc740 1 Processor -- start 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 -- start start 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d747a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d747d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620d73e4e50 con 0x5620d746eda0 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620d73d4280 con 0x5620d746e480 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c94bc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5620d73d8090 con 0x5620d747ad30 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d747a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60376/0 (socket says 172.21.15.120:60376) 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d747a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34476/0 (socket says 172.21.15.120:34476) 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 -- 172.21.15.120:0/2589117642 learned_addr learned my addr 172.21.15.120:0/2589117642 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 0x7f27b00b7820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27b40096a0 tx=0x7f27b4012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c5684640 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d747d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 -- 172.21.15.120:0/2589117642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 msgr2=0x5620d747d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d747d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 -- 172.21.15.120:0/2589117642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 msgr2=0x5620d747a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d747a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c4e83640 1 -- 172.21.15.120:0/2589117642 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620d7132650 con 0x5620d746e480 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.195+0000 7f27c5684640 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d747d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.196+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f27b4004420 tx=0x7f27b4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd371fe0e7432fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.196+0000 7f27aaffd640 1 -- 172.21.15.120:0/2589117642 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402c070 con 0x5620d746e480 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.196+0000 7f27aaffd640 1 -- 172.21.15.120:0/2589117642 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b400bbf0 con 0x5620d746e480 2024-09-10T22:57:34.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.196+0000 7f27aaffd640 1 -- 172.21.15.120:0/2589117642 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4027bc0 con 0x5620d746e480 2024-09-10T22:57:34.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- 172.21.15.120:0/2589117642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 msgr2=0x5620d746e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f27b4004420 tx=0x7f27b4018000 comp rx=0 tx=0).stop 2024-09-10T22:57:34.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- 172.21.15.120:0/2589117642 shutdown_connections 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d747d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d747a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2589117642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746e860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- 172.21.15.120:0/2589117642 >> 172.21.15.120:0/2589117642 conn(0x5620d746c2b0 msgr2=0x5620d746d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- 172.21.15.120:0/2589117642 shutdown_connections 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- 172.21.15.120:0/2589117642 wait complete. 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 -- start start 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.197+0000 7f27c94bc740 1 Processor -- start 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d7477a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d7477fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620d7235700 con 0x5620d746eda0 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5620d736f110 con 0x5620d746e480 2024-09-10T22:57:34.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5620d747d9b0 con 0x5620d747ad30 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c94bc740 10 client.0 _ll_register_callbacks cb 0x5620d7388990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60388/0 (socket says 172.21.15.120:60388) 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d7477a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 -- 172.21.15.120:0/3281787683 learned_addr learned my addr 172.21.15.120:0/3281787683 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c5684640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d7477fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 -- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 msgr2=0x5620d7477fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d7477fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 -- 172.21.15.120:0/3281787683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 msgr2=0x5620d7477a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d7477a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.198+0000 7f27c4e83640 1 -- 172.21.15.120:0/3281787683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620d739cc40 con 0x5620d746e480 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f27c4e83640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005890 tx=0x7f27b400b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b113d36c54ee65b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402c070 con 0x5620d746e480 2024-09-10T22:57:34.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f27b4027900 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b400bbf0 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27b400bda0 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f27c94bc740 10 client.25787 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620d7491ab0 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b400bbf0 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.199+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f27b4030a50 con 0x5620d746e480 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 10 client.25787 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 20 client.25787 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d7492f10 0x5620d7495300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 4 client.25787 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620d73e6da0 con 0x5620d7492f10 2024-09-10T22:57:34.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27bffff640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d7492f10 0x5620d7495300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f279ffff640 10 client.25787 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:34.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.200+0000 7f27bffff640 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d7492f10 0x5620d7495300 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3c196f5e65c62c server_cookie=1b352f9794bae780 in_seq=0 out_seq=0 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f279ffff640 1 -- 172.21.15.120:0/3281787683 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620d73e6da0 con 0x5620d7492f10 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f279ffff640 10 client.25787 handle_command_reply: tid=0 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 client.25787 shutdown 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 msgr2=0x5620d747a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005890 tx=0x7f27b400b750 comp rx=0 tx=0).stop 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 shutdown_connections 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d7492f10 0x5620d7495300 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d7477fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d7477a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3281787683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d747a5d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 >> 172.21.15.120:0/3281787683 conn(0x5620d746c2b0 msgr2=0x5620d7471e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f279ffff640 0 client.25787 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 shutdown_connections 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 1 -- 172.21.15.120:0/3281787683 wait complete. 2024-09-10T22:57:34.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.202+0000 7f27c94bc740 20 client.25787 trim_cache size 0 max 16384 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 Processor -- start 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 -- start start 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d74767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01aa0e0 con 0x5620d746eda0 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01ac960 con 0x5620d746e480 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c94bc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01aa750 con 0x5620d747ad30 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34486/0 (socket says 172.21.15.120:34486) 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27bffff640 1 -- 172.21.15.120:0/2169886336 learned_addr learned my addr 172.21.15.120:0/2169886336 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.222+0000 7f27c5684640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d74767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 -- 172.21.15.120:0/2169886336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 msgr2=0x5620d74767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d74767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 -- 172.21.15.120:0/2169886336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 msgr2=0x5620d74762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 -- 172.21.15.120:0/2169886336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620d746cd30 con 0x5620d746e480 2024-09-10T22:57:34.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27bffff640 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:34.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27c4e83640 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746bbc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009f30 tx=0x7f27b4027710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f3ceba5d449d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27aa7fc640 1 -- 172.21.15.120:0/2169886336 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402c070 con 0x5620d746e480 2024-09-10T22:57:34.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27aa7fc640 1 -- 172.21.15.120:0/2169886336 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b40054d0 con 0x5620d746e480 2024-09-10T22:57:34.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.223+0000 7f27aa7fc640 1 -- 172.21.15.120:0/2169886336 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4027a60 con 0x5620d746e480 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- 172.21.15.120:0/2169886336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 msgr2=0x5620d746bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746bbc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009f30 tx=0x7f27b4027710 comp rx=0 tx=0).stop 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- 172.21.15.120:0/2169886336 shutdown_connections 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d747ad30 0x5620d74767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746eda0 0x5620d74762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 --2- 172.21.15.120:0/2169886336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d746e480 0x5620d746bbc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- 172.21.15.120:0/2169886336 >> 172.21.15.120:0/2169886336 conn(0x5620d746c2b0 msgr2=0x5620d746d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- 172.21.15.120:0/2169886336 shutdown_connections 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- 172.21.15.120:0/2169886336 wait complete. 2024-09-10T22:57:34.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 -- start start 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.224+0000 7f27c94bc740 1 Processor -- start 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 0x5620d748bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 0x5620d749a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01aae20 con 0x5620d746e480 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01ac290 con 0x5620d747ad30 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01ab4f0 con 0x5620d746eda0 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c94bc740 10 client.0 _ll_register_callbacks cb 0x7f27b402a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c5684640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27bffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 0x5620d749a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 0x5620d748bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c5684640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60400/0 (socket says 172.21.15.120:60400) 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c4e83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 0x5620d748bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34496/0 (socket says 172.21.15.120:34496) 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27bffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 0x5620d749a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44618/0 (socket says 172.21.15.120:44618) 2024-09-10T22:57:34.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.225+0000 7f27c5684640 1 -- 172.21.15.120:0/3663178251 learned_addr learned my addr 172.21.15.120:0/3663178251 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 -- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 msgr2=0x5620d749a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 0x5620d749a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 -- 172.21.15.120:0/3663178251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 msgr2=0x5620d748bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 0x5620d748bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 -- 172.21.15.120:0/3663178251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620d749b1e0 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c5684640 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27b8012c40 tx=0x7f27b80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c597a6a7b9caad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f279ffff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004d20 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f27b8005020 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b40054d0 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27b8005260 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c94bc740 10 client.25793 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.226+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620d74ffc70 con 0x5620d747ad30 2024-09-10T22:57:34.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b40054d0 con 0x5620d747ad30 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f27b801b070 con 0x5620d747ad30 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 10 client.25793 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 20 client.25793 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d75003e0 0x5620d7502800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 4 client.25793 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620d73e2850 con 0x5620d75003e0 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.227+0000 7f27c4e83640 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d75003e0 0x5620d7502800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.228+0000 7f27c4e83640 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d75003e0 0x5620d7502800 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89baa078503c738 server_cookie=b788ec529e8550cc in_seq=0 out_seq=0 2024-09-10T22:57:34.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.228+0000 7f279ffff640 10 client.25793 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:34.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f279ffff640 1 -- 172.21.15.120:0/3663178251 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5620d73e2850 con 0x5620d75003e0 2024-09-10T22:57:34.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f279ffff640 10 client.25793 handle_command_reply: tid=0 2024-09-10T22:57:34.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 client.25793 shutdown 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 msgr2=0x5620d749aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27b8012c40 tx=0x7f27b80117b0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 shutdown_connections 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5620d75003e0 0x5620d7502800 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5620d747ad30 0x5620d749aaa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620d746eda0 0x5620d749a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 --2- 172.21.15.120:0/3663178251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620d746e480 0x5620d748bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 >> 172.21.15.120:0/3663178251 conn(0x5620d746c2b0 msgr2=0x5620d746d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f279ffff640 0 client.25793 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 shutdown_connections 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 1 -- 172.21.15.120:0/3663178251 wait complete. 2024-09-10T22:57:34.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.231+0000 7f27c94bc740 20 client.25793 trim_cache size 0 max 16384 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.235 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.236 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:57:34.237 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:34.240 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 msgr2=0x7f27b00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 0x7f27b00b7820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27b40096a0 tx=0x7f27b4012040 comp rx=0 tx=0).stop 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 msgr2=0x7f27c0199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f27ac032fd0 tx=0x7f27ac031a20 comp rx=0 tx=0).stop 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 shutdown_connections 2024-09-10T22:57:34.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27b00b5360 0x7f27b00b7820 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27c010d100 0x7f27c019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c01045c0 0x7f27c0199800 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 --2- 172.21.15.120:0/2401314887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27c0103ca0 0x7f27c01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 >> 172.21.15.120:0/2401314887 conn(0x7f27c00fd700 msgr2=0x7f27c00fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 shutdown_connections 2024-09-10T22:57:34.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:34.232+0000 7f27c70e7640 1 -- 172.21.15.120:0/2401314887 wait complete. 2024-09-10T22:57:34.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:34 smithi176 ceph-mon[34982]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-09-10T22:57:34.388 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:34 smithi120 ceph-mon[31247]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-09-10T22:57:34.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:34 smithi153 ceph-mon[34951]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-09-10T22:57:34.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 848811 16.75 MB/sec execute 149 sec latency 31.391 ms 2024-09-10T22:57:35.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:57:35.197 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:57:35.470 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:35.624 INFO:tasks.workunit.client.0.smithi120.stdout: 1 852204 16.76 MB/sec execute 150 sec latency 13.212 ms 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 -- 172.21.15.120:0/1107796696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 msgr2=0x7f4b98101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1107796696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98101630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009900 tx=0x7f4b8802a410 comp rx=0 tx=0).stop 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 -- 172.21.15.120:0/1107796696 shutdown_connections 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1107796696 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1107796696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b98105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1107796696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98101630 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 -- 172.21.15.120:0/1107796696 >> 172.21.15.120:0/1107796696 conn(0x7f4b98078140 msgr2=0x7f4b980ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 -- 172.21.15.120:0/1107796696 shutdown_connections 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 -- 172.21.15.120:0/1107796696 wait complete. 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.994+0000 7f4b9de94640 1 Processor -- start 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 -- start start 2024-09-10T22:57:36.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b981997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b98112db0 con 0x7f4b98101250 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b98112c30 con 0x7f4b98105f40 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b9de94640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b98112f30 con 0x7f4b981055a0 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34510/0 (socket says 172.21.15.120:34510) 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 -- 172.21.15.120:0/1780690800 learned_addr learned my addr 172.21.15.120:0/1780690800 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b97fff640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:36.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b96ffd640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b981997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 -- 172.21.15.120:0/1780690800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 msgr2=0x7f4b981997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b981997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.995+0000 7f4b977fe640 1 -- 172.21.15.120:0/1780690800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 msgr2=0x7f4b9819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b977fe640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b977fe640 1 -- 172.21.15.120:0/1780690800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9819f310 con 0x7f4b98101250 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b96ffd640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b981997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b97fff640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:36.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b977fe640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005940 tx=0x7f4b880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8803f030 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b880044c0 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88035070 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9819f5a0 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.996+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b981a5cc0 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.997+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b98101630 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:35.999+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4b8803b9d0 con 0x7f4b98101250 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.000+0000 7f4b94ff9640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 0x7f4b800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.000+0000 7f4b96ffd640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 0x7f4b800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:36.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.000+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4b88106490 con 0x7f4b98101250 2024-09-10T22:57:36.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.000+0000 7f4b96ffd640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 0x7f4b800b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4b98107260 tx=0x7f4b8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:36.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.003+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b880cd380 con 0x7f4b98101250 2024-09-10T22:57:36.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.248+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9802e5c0 con 0x7f4b98101250 2024-09-10T22:57:36.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.249+0000 7f4b94ff9640 1 -- 172.21.15.120:0/1780690800 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7f4b880ccc80 con 0x7f4b98101250 2024-09-10T22:57:36.250 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:57:36.250 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":25,"btime":"2024-09-10T22:56:51:983742+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:56:51.983723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/1259604772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":1259604772},{"type":"v1","addr":"172.21.15.176:6837","nonce":1259604772}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:57:36.251 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 25 2024-09-10T22:57:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 msgr2=0x7f4b800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 0x7f4b800b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4b98107260 tx=0x7f4b8c005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 msgr2=0x7f4b98199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005940 tx=0x7f4b880040c0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 shutdown_connections 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4b800b5360 0x7f4b800b7820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b98105f40 0x7f4b9819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b981055a0 0x7f4b981997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 --2- 172.21.15.120:0/1780690800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b98101250 0x7f4b98199260 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 >> 172.21.15.120:0/1780690800 conn(0x7f4b98078140 msgr2=0x7f4b98108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 shutdown_connections 2024-09-10T22:57:36.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:36.251+0000 7f4b9de94640 1 -- 172.21.15.120:0/1780690800 wait complete. 2024-09-10T22:57:36.262 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:36 smithi120 ceph-mon[31247]: pgmap v500: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-09-10T22:57:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:36 smithi176 ceph-mon[34982]: pgmap v500: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-09-10T22:57:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:36 smithi153 ceph-mon[34951]: pgmap v500: 97 pgs: 97 active+clean; 51 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 307 op/s 2024-09-10T22:57:36.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 854989 16.73 MB/sec execute 151 sec latency 39.947 ms 2024-09-10T22:57:36.980 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:57:37.326 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:37.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:37 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1780690800' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:37.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:37 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1780690800' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:37.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:37 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1780690800' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:37.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 856933 16.69 MB/sec execute 152 sec latency 65.848 ms 2024-09-10T22:57:37.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 -- 172.21.15.120:0/462397046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 msgr2=0x7f64ec10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 --2- 172.21.15.120:0/462397046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec10cb30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009900 tx=0x7f64d402a410 comp rx=0 tx=0).stop 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 -- 172.21.15.120:0/462397046 shutdown_connections 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 --2- 172.21.15.120:0/462397046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 0x7f64ec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 --2- 172.21.15.120:0/462397046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec10cb30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 --2- 172.21.15.120:0/462397046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 -- 172.21.15.120:0/462397046 >> 172.21.15.120:0/462397046 conn(0x7f64ec0fd700 msgr2=0x7f64ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 -- 172.21.15.120:0/462397046 shutdown_connections 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.872+0000 7f64f17f1640 1 -- 172.21.15.120:0/462397046 wait complete. 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 Processor -- start 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 -- start start 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 0x7f64ec19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec112d60 con 0x7f64ec104550 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec112be0 con 0x7f64ec103b50 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.873+0000 7f64f17f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec112ee0 con 0x7f64ec10d100 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34528/0 (socket says 172.21.15.120:34528) 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 -- 172.21.15.120:0/3035792325 learned_addr learned my addr 172.21.15.120:0/3035792325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64eaffd640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 0x7f64ec19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 -- 172.21.15.120:0/3035792325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 msgr2=0x7f64ec19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 0x7f64ec19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 -- 172.21.15.120:0/3035792325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 msgr2=0x7f64ec1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 -- 172.21.15.120:0/3035792325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec19f350 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64eaffd640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e27fc640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005940 tx=0x7f64d40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e8ff9640 1 -- 172.21.15.120:0/3035792325 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d403f030 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e8ff9640 1 -- 172.21.15.120:0/3035792325 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d40044c0 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec19f5e0 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64e8ff9640 1 -- 172.21.15.120:0/3035792325 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4035070 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.874+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1a5d00 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 Processor -- start 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64e8ff9640 1 -- 172.21.15.120:0/3035792325 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f64d403ba20 con 0x7f64ec104550 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 -- start start 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e47999ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e47999d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e479900ea0 con 0x55e47998e8e0 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e4798f02d0 con 0x55e47998f200 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.877+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e4798f40e0 con 0x55e47999b190 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34542/0 (socket says 172.21.15.120:34542) 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 -- 172.21.15.120:0/352663623 learned_addr learned my addr 172.21.15.120:0/352663623 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e27fc640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e47999d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eaffd640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e47999ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 -- 172.21.15.120:0/352663623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 msgr2=0x55e47999d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e47999d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 -- 172.21.15.120:0/352663623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 msgr2=0x55e47999ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e47999ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 -- 172.21.15.120:0/352663623 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47964e650 con 0x55e47998e8e0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eaffd640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e47999ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e8ff9640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 0x7f64d80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19086d568c94f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64eaffd640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 0x7f64d80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e17fa640 1 -- 172.21.15.120:0/352663623 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004d20 con 0x55e47998e8e0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e8ff9640 1 -- 172.21.15.120:0/3035792325 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64d4106180 con 0x7f64ec104550 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e17fa640 1 -- 172.21.15.120:0/352663623 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4004050 con 0x55e47998e8e0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.878+0000 7f64e17fa640 1 -- 172.21.15.120:0/352663623 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4005260 con 0x55e47998e8e0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 -- 172.21.15.120:0/352663623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 msgr2=0x55e47998ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64eaffd640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 0x7f64d80b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009fd0 tx=0x7f64e40118d0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 -- 172.21.15.120:0/352663623 shutdown_connections 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e47999d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e47999ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/352663623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998e8e0 0x55e47998ecc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 -- 172.21.15.120:0/352663623 >> 172.21.15.120:0/352663623 conn(0x55e47998c710 msgr2=0x55e47998d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 -- 172.21.15.120:0/352663623 shutdown_connections 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.879+0000 7f64f3bc6740 1 -- 172.21.15.120:0/352663623 wait complete. 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 -- start start 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 Processor -- start 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e47999aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e479998490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e479751700 con 0x55e47998f200 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e47988b160 con 0x55e47998e8e0 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e47999de10 con 0x55e47999b190 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64f3bc6740 10 client.0 _ll_register_callbacks cb 0x55e4798a49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34558/0 (socket says 172.21.15.120:34558) 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 -- 172.21.15.120:0/1991424969 learned_addr learned my addr 172.21.15.120:0/1991424969 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e47999aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e479998490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 -- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 msgr2=0x55e479998490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e479998490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 -- 172.21.15.120:0/1991424969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 msgr2=0x55e47999aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e47999aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.880+0000 7f64e27fc640 1 -- 172.21.15.120:0/1991424969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4798b8c90 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64e27fc640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f64d4006010 tx=0x7f64d4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1373a5fbcd966d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4049040 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64d40c8c30 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4004050 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64d40c8860 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 10 client.16365 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4799a9780 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4004050 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f64d40ef070 con 0x55e47998f200 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 10 client.16365 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 20 client.16365 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:37.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e4799aab20 0x55e4799acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 4 client.16365 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.881+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e479902df0 con 0x55e4799aab20 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.882+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e4799aab20 0x55e4799acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.882+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e4799aab20 0x55e4799acf30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799ce93672b7a96 server_cookie=be5e0227a249e806 in_seq=0 out_seq=0 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.882+0000 7f64d27fc640 10 client.16365 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64d27fc640 1 -- 172.21.15.120:0/1991424969 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e479902df0 con 0x55e4799aab20 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64d27fc640 10 client.16365 handle_command_reply: tid=0 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 client.16365 shutdown 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 msgr2=0x55e479997eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f64d4006010 tx=0x7f64d4005c00 comp rx=0 tx=0).stop 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 shutdown_connections 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e4799aab20 0x55e4799acf30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47999b190 0x55e479998490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47998f200 0x55e479997eb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64d27fc640 0 client.16365 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1991424969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e47999aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 >> 172.21.15.120:0/1991424969 conn(0x55e47998c710 msgr2=0x55e479987420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 shutdown_connections 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1991424969 wait complete. 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.883+0000 7f64f3bc6740 20 client.16365 trim_cache size 0 max 16384 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 Processor -- start 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 -- start start 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e479996730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1aa070 con 0x55e47999b190 2024-09-10T22:57:37.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ac8f0 con 0x55e47998f200 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.896+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1aa6e0 con 0x55e47998e8e0 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64e27fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e479996730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44646/0 (socket says 172.21.15.120:44646) 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34570/0 (socket says 172.21.15.120:34570) 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 -- 172.21.15.120:0/1694371928 learned_addr learned my addr 172.21.15.120:0/1694371928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 -- 172.21.15.120:0/1694371928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 msgr2=0x55e479996730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e479996730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 -- 172.21.15.120:0/1694371928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 msgr2=0x55e479996c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 -- 172.21.15.120:0/1694371928 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e479998800 con 0x55e47998e8e0 2024-09-10T22:57:37.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:37.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64e27fc640 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e479996730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:37.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64eaffd640 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e4799ad710 tx=0x7f64dc01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33578fcaa61fa18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64e0ff9640 1 -- 172.21.15.120:0/1694371928 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004030 con 0x55e47998e8e0 2024-09-10T22:57:37.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64e0ff9640 1 -- 172.21.15.120:0/1694371928 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0035f0 con 0x55e47998e8e0 2024-09-10T22:57:37.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.897+0000 7f64e0ff9640 1 -- 172.21.15.120:0/1694371928 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc02b660 con 0x55e47998e8e0 2024-09-10T22:57:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1694371928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 msgr2=0x55e4799973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e4799ad710 tx=0x7f64dc01a000 comp rx=0 tx=0).stop 2024-09-10T22:57:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1694371928 shutdown_connections 2024-09-10T22:57:37.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e479996c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998f200 0x55e479996730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/1694371928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998e8e0 0x55e4799973a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1694371928 >> 172.21.15.120:0/1694371928 conn(0x55e47998c710 msgr2=0x55e479993160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1694371928 shutdown_connections 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- 172.21.15.120:0/1694371928 wait complete. 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.898+0000 7f64f3bc6740 1 -- start start 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 Processor -- start 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e4799a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 0x55e4799a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1aadb0 con 0x55e47999b190 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ac220 con 0x55e47998e8e0 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ab480 con 0x55e47998f200 2024-09-10T22:57:37.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64f3bc6740 10 client.0 _ll_register_callbacks cb 0x7f64d41098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34580/0 (socket says 172.21.15.120:34580) 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 -- 172.21.15.120:0/2664124862 learned_addr learned my addr 172.21.15.120:0/2664124862 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64e27fc640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 0x55e4799a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eaffd640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e4799a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 -- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 msgr2=0x55e4799a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 0x55e4799a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 -- 172.21.15.120:0/2664124862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 msgr2=0x55e4799a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e4799a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 -- 172.21.15.120:0/2664124862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47999e8f0 con 0x55e47999b190 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64e27fc640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 0x55e4799a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eaffd640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e4799a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:37.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64eb7fe640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f64e4014040 tx=0x7f64e4009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79cad18ffe141e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.899+0000 7f64d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004b60 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64e4004d00 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64dc0035f0 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f64e4004000 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 10 client.16371 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e479a18440 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0035f0 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f64e4005530 con 0x55e47999b190 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 10 client.16371 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 20 client.16371 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e479a189c0 0x55e479a1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:37.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 4 client.16371 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4798fe8a0 con 0x55e479a189c0 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.900+0000 7f64eaffd640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e479a189c0 0x55e479a1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64d27fc640 10 client.16371 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64eaffd640 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e479a189c0 0x55e479a1ade0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9cc06e65caeab7 server_cookie=e471145f66203ceb in_seq=0 out_seq=0 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64d27fc640 1 -- 172.21.15.120:0/2664124862 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4798fe8a0 con 0x55e479a189c0 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64d27fc640 10 client.16371 handle_command_reply: tid=0 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 client.16371 shutdown 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 msgr2=0x55e47999e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f64e4014040 tx=0x7f64e4009d20 comp rx=0 tx=0).stop 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 shutdown_connections 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e479a189c0 0x55e479a1ade0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e47999b190 0x55e47999e0c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e47998f200 0x55e4799a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 --2- 172.21.15.120:0/2664124862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e47998e8e0 0x55e4799a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 >> 172.21.15.120:0/2664124862 conn(0x55e47998c710 msgr2=0x55e4799966b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64d27fc640 0 client.16371 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 shutdown_connections 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 1 -- 172.21.15.120:0/2664124862 wait complete. 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.901+0000 7f64f3bc6740 20 client.16371 trim_cache size 0 max 16384 2024-09-10T22:57:37.918 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 msgr2=0x7f64d80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 0x7f64d80b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc005f90 comp rx=0 tx=0).stop 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 msgr2=0x7f64ec1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005940 tx=0x7f64d40040c0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 shutdown_connections 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f64d80b5360 0x7f64d80b7820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64ec10d100 0x7f64ec19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64ec104550 0x7f64ec1997e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 --2- 172.21.15.120:0/3035792325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64ec103b50 0x7f64ec1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:37.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 >> 172.21.15.120:0/3035792325 conn(0x7f64ec0fd700 msgr2=0x7f64ec0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:37.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 shutdown_connections 2024-09-10T22:57:37.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:37.902+0000 7f64f17f1640 1 -- 172.21.15.120:0/3035792325 wait complete. 2024-09-10T22:57:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:38 smithi120 ceph-mon[31247]: pgmap v501: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2024-09-10T22:57:38.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:38 smithi153 ceph-mon[34951]: pgmap v501: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2024-09-10T22:57:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:38 smithi176 ceph-mon[34982]: pgmap v501: 97 pgs: 97 active+clean; 46 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2024-09-10T22:57:38.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 859869 16.69 MB/sec execute 153 sec latency 27.206 ms 2024-09-10T22:57:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:39 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:57:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:39 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:57:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:39 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:57:39.607 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:57:39.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:57:39.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 863127 16.70 MB/sec execute 154 sec latency 21.202 ms 2024-09-10T22:57:39.864 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:40.300 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:40 smithi120 ceph-mon[31247]: pgmap v502: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-10T22:57:40.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- 172.21.15.120:0/1472570222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33381045f0 msgr2=0x7f333810cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 --2- 172.21.15.120:0/1472570222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33381045f0 0x7f333810cbd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f332002a410 comp rx=0 tx=0).stop 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- 172.21.15.120:0/1472570222 shutdown_connections 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 --2- 172.21.15.120:0/1472570222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f333810d1a0 0x7f333810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 --2- 172.21.15.120:0/1472570222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33381045f0 0x7f333810cbd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 --2- 172.21.15.120:0/1472570222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 0x7f3338103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- 172.21.15.120:0/1472570222 >> 172.21.15.120:0/1472570222 conn(0x7f33380fd7c0 msgr2=0x7f33380ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- 172.21.15.120:0/1472570222 shutdown_connections 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- 172.21.15.120:0/1472570222 wait complete. 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 Processor -- start 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.384+0000 7f333e307640 1 -- start start 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 0x7f3338101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33381045f0 0x7f3338101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112de0 con 0x7f333810d1a0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112c60 con 0x7f33381045f0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112f60 con 0x7f3338103bf0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f3337fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 0x7f3338101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34610/0 (socket says 172.21.15.120:34610) 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 -- 172.21.15.120:0/1594919701 learned_addr learned my addr 172.21.15.120:0/1594919701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f332ffff640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33381045f0 0x7f3338101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 -- 172.21.15.120:0/1594919701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 msgr2=0x7f3338101a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 0x7f3338101a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 -- 172.21.15.120:0/1594919701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33381045f0 msgr2=0x7f3338101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33381045f0 0x7f3338101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 -- 172.21.15.120:0/1594919701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a76f0 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333c8a4640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f333000eaa0 tx=0x7f333000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f3335ffb640 1 -- 172.21.15.120:0/1594919701 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333000cdb0 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a79e0 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.385+0000 7f3335ffb640 1 -- 172.21.15.120:0/1594919701 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3330005020 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.386+0000 7f3335ffb640 1 -- 172.21.15.120:0/1594919701 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33300116d0 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.386+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a7ea0 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.388+0000 7f33406dc740 1 Processor -- start 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3335ffb640 1 -- 172.21.15.120:0/1594919701 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f333000f040 con 0x7f333810d1a0 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 -- start start 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfea0790 0x5614dfea2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5614dfe0ae60 con 0x5614dfe94800 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614dfdfa290 con 0x5614dfea0790 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f33406dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614dfdfe0a0 con 0x5614dfe93ee0 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34614/0 (socket says 172.21.15.120:34614) 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 -- 172.21.15.120:0/754660283 learned_addr learned my addr 172.21.15.120:0/754660283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f332ffff640 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfea0790 0x5614dfea2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f332ffff640 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 -- 172.21.15.120:0/754660283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 msgr2=0x5614dfe942c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe942c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 -- 172.21.15.120:0/754660283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfea0790 msgr2=0x5614dfea2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfea0790 0x5614dfea2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.389+0000 7f3337fff640 1 -- 172.21.15.120:0/754660283 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dfb58930 con 0x5614dfe94800 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f3335ffb640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 0x7f33240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f3337fff640 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3328009d20 tx=0x7f3328011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db196ff359b26f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f332ffff640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 0x7f33240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f3335ffb640 1 -- 172.21.15.120:0/1594919701 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f33300e5960 con 0x7f333810d1a0 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f332e7fc640 1 -- 172.21.15.120:0/754660283 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328004230 con 0x5614dfe94800 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f332e7fc640 1 -- 172.21.15.120:0/754660283 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3328009e10 con 0x5614dfe94800 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.390+0000 7f332e7fc640 1 -- 172.21.15.120:0/754660283 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328019720 con 0x5614dfe94800 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- 172.21.15.120:0/754660283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 msgr2=0x5614dfea0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3328009d20 tx=0x7f3328011810 comp rx=0 tx=0).stop 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f332ffff640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 0x7f33240b77a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3320031440 tx=0x7f3320041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- 172.21.15.120:0/754660283 shutdown_connections 2024-09-10T22:57:40.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfea0790 0x5614dfea2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe94800 0x5614dfea0250 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- 172.21.15.120:0/754660283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe942c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- 172.21.15.120:0/754660283 >> 172.21.15.120:0/754660283 conn(0x5614dfe91d10 msgr2=0x5614dfe92eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- 172.21.15.120:0/754660283 shutdown_connections 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- 172.21.15.120:0/754660283 wait complete. 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- start start 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 Processor -- start 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfe9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5614dfc5b890 con 0x5614dfea0790 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5614dfd94d40 con 0x5614dfe94800 2024-09-10T22:57:40.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.391+0000 7f33406dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5614dfea3440 con 0x5614dfe93ee0 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33406dc740 10 client.0 _ll_register_callbacks cb 0x5614dfdf8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f3337fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34626/0 (socket says 172.21.15.120:34626) 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f3337fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe9d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44688/0 (socket says 172.21.15.120:44688) 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 -- 172.21.15.120:0/3185925259 learned_addr learned my addr 172.21.15.120:0/3185925259 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f332ffff640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfe9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 -- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 msgr2=0x5614dfe9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 -- 172.21.15.120:0/3185925259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 msgr2=0x5614dfe9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfe9dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 -- 172.21.15.120:0/3185925259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dfeac820 con 0x5614dfea0790 2024-09-10T22:57:40.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f333c8a4640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f33300a87d0 tx=0x7f33300abe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19d6ec70735319d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33177fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333001f030 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3328009e10 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f33300054f0 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33300b56e0 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33406dc740 10 client.16383 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.392+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614dfeaf210 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3328009e10 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f33300a7b50 con 0x5614dfea0790 2024-09-10T22:57:40.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 10 client.16383 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 20 client.16383 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dfeb0770 0x5614dfeb2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 4 client.16383 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614dfe0cdb0 con 0x5614dfeb0770 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f3337fff640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dfeb0770 0x5614dfeb2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f3337fff640 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dfeb0770 0x5614dfeb2b80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d98c2b45a02fd8 server_cookie=8e01ec9f08a6202b in_seq=0 out_seq=0 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.393+0000 7f33177fe640 10 client.16383 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33177fe640 1 -- 172.21.15.120:0/3185925259 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614dfe0cdb0 con 0x5614dfeb0770 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33177fe640 10 client.16383 handle_command_reply: tid=0 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 client.16383 shutdown 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 msgr2=0x5614dfe9e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f33300a87d0 tx=0x7f33300abe50 comp rx=0 tx=0).stop 2024-09-10T22:57:40.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 shutdown_connections 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dfeb0770 0x5614dfeb2b80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfe9e1c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfe9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33177fe640 0 client.16383 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 --2- 172.21.15.120:0/3185925259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe93ee0 0x5614dfe9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 >> 172.21.15.120:0/3185925259 conn(0x5614dfe91d10 msgr2=0x5614dfe90f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 shutdown_connections 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 1 -- 172.21.15.120:0/3185925259 wait complete. 2024-09-10T22:57:40.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.394+0000 7f33406dc740 20 client.16383 trim_cache size 0 max 16384 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 Processor -- start 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 -- start start 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe94800 0x5614dfe9f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfeb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33381ae480 con 0x5614dfea0790 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33381b0d00 con 0x5614dfe93ee0 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f33406dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f33381aeaf0 con 0x5614dfe94800 2024-09-10T22:57:40.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f3337fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f3337fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60492/0 (socket says 172.21.15.120:60492) 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfeb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f3337fff640 1 -- 172.21.15.120:0/1179341582 learned_addr learned my addr 172.21.15.120:0/1179341582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.413+0000 7f332ffff640 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe94800 0x5614dfe9f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 -- 172.21.15.120:0/1179341582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe94800 msgr2=0x5614dfe9f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe94800 0x5614dfe9f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 -- 172.21.15.120:0/1179341582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 msgr2=0x5614dfeb8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfeb8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 -- 172.21.15.120:0/1179341582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dfeb3200 con 0x5614dfe93ee0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f333c8a4640 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfeb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f3337fff640 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3328014040 tx=0x7f3328011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31465abcbec5a77a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f332dffb640 1 -- 172.21.15.120:0/1179341582 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328031070 con 0x5614dfe93ee0 2024-09-10T22:57:40.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f332dffb640 1 -- 172.21.15.120:0/1179341582 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33280035f0 con 0x5614dfe93ee0 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.414+0000 7f332dffb640 1 -- 172.21.15.120:0/1179341582 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280196f0 con 0x5614dfe93ee0 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- 172.21.15.120:0/1179341582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 msgr2=0x5614dfe9b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3328014040 tx=0x7f3328011810 comp rx=0 tx=0).stop 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- 172.21.15.120:0/1179341582 shutdown_connections 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfea0790 0x5614dfeb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfe94800 0x5614dfe9f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 --2- 172.21.15.120:0/1179341582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe93ee0 0x5614dfe9b1d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- 172.21.15.120:0/1179341582 >> 172.21.15.120:0/1179341582 conn(0x5614dfe91d10 msgr2=0x5614dfe9ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- 172.21.15.120:0/1179341582 shutdown_connections 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- 172.21.15.120:0/1179341582 wait complete. 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 -- start start 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.415+0000 7f33406dc740 1 Processor -- start 2024-09-10T22:57:40.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 0x5614dfea4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfea0790 0x5614dff1c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33381af1c0 con 0x5614dfe93ee0 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33381b0630 con 0x5614dfe94800 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f33381af890 con 0x5614dfea0790 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33406dc740 10 client.0 _ll_register_callbacks cb 0x7f33300a8850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f3337fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 0x5614dfea4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60506/0 (socket says 172.21.15.120:60506) 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 -- 172.21.15.120:0/1623224795 learned_addr learned my addr 172.21.15.120:0/1623224795 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f333c8a4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfea0790 0x5614dff1c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 -- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfea0790 msgr2=0x5614dff1c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfea0790 0x5614dff1c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 -- 172.21.15.120:0/1623224795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 msgr2=0x5614dfea4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 0x5614dfea4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 -- 172.21.15.120:0/1623224795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dff1cbe0 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f3337fff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 0x5614dfea4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f332ffff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3320033000 tx=0x7f3320009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=788c54b227e996ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.416+0000 7f33177fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332005b060 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f33200035f0 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33280035f0 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33200037f0 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 10 client.25832 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614dff1df50 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33280035f0 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f3320056c20 con 0x5614dfe94800 2024-09-10T22:57:40.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 10 client.25832 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 20 client.25832 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dff1e730 0x5614dff20b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 4 client.25832 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5614dfe08860 con 0x5614dff1e730 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.417+0000 7f3337fff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dff1e730 0x5614dff20b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.418+0000 7f3337fff640 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dff1e730 0x5614dff20b50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7ab7cdbd87e963 server_cookie=ee157bb550876951 in_seq=0 out_seq=0 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.418+0000 7f33177fe640 10 client.25832 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.419+0000 7f33177fe640 1 -- 172.21.15.120:0/1623224795 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614dfe08860 con 0x5614dff1e730 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.419+0000 7f33177fe640 10 client.25832 handle_command_reply: tid=0 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.419+0000 7f33406dc740 1 client.25832 shutdown 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 msgr2=0x5614dfea5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3320033000 tx=0x7f3320009260 comp rx=0 tx=0).stop 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 shutdown_connections 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5614dff1e730 0x5614dff20b50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5614dfea0790 0x5614dff1c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dfe94800 0x5614dfea5290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 --2- 172.21.15.120:0/1623224795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5614dfe93ee0 0x5614dfea4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 >> 172.21.15.120:0/1623224795 conn(0x5614dfe91d10 msgr2=0x5614dfe9f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33177fe640 0 client.25832 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 shutdown_connections 2024-09-10T22:57:40.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 1 -- 172.21.15.120:0/1623224795 wait complete. 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f33406dc740 20 client.25832 trim_cache size 0 max 16384 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "291470de-ce79-4054-bccb-d0d766773893", 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:57:40.430 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:57:40.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 msgr2=0x7f33240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 0x7f33240b77a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3320031440 tx=0x7f3320041040 comp rx=0 tx=0).stop 2024-09-10T22:57:40.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 msgr2=0x7f33381024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f333000eaa0 tx=0x7f333000ef60 comp rx=0 tx=0).stop 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 shutdown_connections 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33240b52e0 0x7f33240b77a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f333810d1a0 0x7f33381024f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33381045f0 0x7f3338101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 --2- 172.21.15.120:0/1594919701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3338103bf0 0x7f3338101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.420+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 >> 172.21.15.120:0/1594919701 conn(0x7f33380fd7c0 msgr2=0x7f33380fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.421+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 shutdown_connections 2024-09-10T22:57:40.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:40.421+0000 7f333e307640 1 -- 172.21.15.120:0/1594919701 wait complete. 2024-09-10T22:57:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:40 smithi153 ceph-mon[34951]: pgmap v502: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-10T22:57:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:40 smithi176 ceph-mon[34982]: pgmap v502: 97 pgs: 97 active+clean; 45 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 381 op/s 2024-09-10T22:57:40.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 866083 16.69 MB/sec execute 155 sec latency 31.197 ms 2024-09-10T22:57:41.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:41.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:41 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:57:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:41 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:57:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:41 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:41 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:57:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:41 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:57:41.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:41 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.476 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:41 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:57:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:41 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:57:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:41 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:41.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 868899 16.66 MB/sec execute 156 sec latency 11.387 ms 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- 172.21.15.120:0/280654538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 msgr2=0x7f686010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 --2- 172.21.15.120:0/280654538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686010f550 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6858009ca0 tx=0x7f685802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- 172.21.15.120:0/280654538 shutdown_connections 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 --2- 172.21.15.120:0/280654538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686010f550 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 --2- 172.21.15.120:0/280654538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860104530 0x7f686010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 --2- 172.21.15.120:0/280654538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860103b30 0x7f6860103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- 172.21.15.120:0/280654538 >> 172.21.15.120:0/280654538 conn(0x7f68600fd700 msgr2=0x7f68600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- 172.21.15.120:0/280654538 shutdown_connections 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- 172.21.15.120:0/280654538 wait complete. 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 Processor -- start 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.986+0000 7f6867ddf640 1 -- start start 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 0x7f6860199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860104530 0x7f68601997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112d20 con 0x7f686010d0e0 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112ba0 con 0x7f6860103b30 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112ea0 con 0x7f6860104530 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34662/0 (socket says 172.21.15.120:34662) 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 -- 172.21.15.120:0/1462104042 learned_addr learned my addr 172.21.15.120:0/1462104042 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6865b7b640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 0x7f6860199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686537a640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860104530 0x7f68601997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 -- 172.21.15.120:0/1462104042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860104530 msgr2=0x7f68601997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860104530 0x7f68601997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 -- 172.21.15.120:0/1462104042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 msgr2=0x7f6860199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 0x7f6860199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 -- 172.21.15.120:0/1462104042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686019f320 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6865b7b640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 0x7f6860199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f686637c640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6858005cc0 tx=0x7f6858031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6856ffd640 1 -- 172.21.15.120:0/1462104042 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858003b70 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686019f5b0 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6856ffd640 1 -- 172.21.15.120:0/1462104042 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68580318f0 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.987+0000 7f6856ffd640 1 -- 172.21.15.120:0/1462104042 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685802f950 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.988+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a5cd0 con 0x7f686010d0e0 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.990+0000 7f686a1b4740 1 Processor -- start 2024-09-10T22:57:42.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.990+0000 7f6856ffd640 1 -- 172.21.15.120:0/1462104042 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f685802fb50 con 0x7f686010d0e0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 -- start start 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdcbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c67fd33e50 con 0x55c67fdbd0a0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c67fd23280 con 0x55c67fdbd9c0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686a1b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c67fd27090 con 0x55c67fdc9950 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34666/0 (socket says 172.21.15.120:34666) 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 -- 172.21.15.120:0/2292045388 learned_addr learned my addr 172.21.15.120:0/2292045388 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdcbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f6865b7b640 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f6856ffd640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 0x7f684c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 -- 172.21.15.120:0/2292045388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 msgr2=0x55c67fdcbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdcbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 -- 172.21.15.120:0/2292045388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 msgr2=0x55c67fdc9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.991+0000 7f686537a640 1 -- 172.21.15.120:0/2292045388 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67fa81650 con 0x55c67fdbd0a0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f686637c640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 0x7f684c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f686537a640 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6850005bc0 tx=0x7f6850005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5ce3703b62612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f6856ffd640 1 -- 172.21.15.120:0/1462104042 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6858106cc0 con 0x7f686010d0e0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f684bfff640 1 -- 172.21.15.120:0/2292045388 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850013df0 con 0x55c67fdbd0a0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f684bfff640 1 -- 172.21.15.120:0/2292045388 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68500054d0 con 0x55c67fdbd0a0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.992+0000 7f684bfff640 1 -- 172.21.15.120:0/2292045388 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850004b00 con 0x55c67fdbd0a0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- 172.21.15.120:0/2292045388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 msgr2=0x55c67fdbd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686637c640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 0x7f684c0b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f68580c9a80 tx=0x7f68580921d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6850005bc0 tx=0x7f6850005850 comp rx=0 tx=0).stop 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- 172.21.15.120:0/2292045388 shutdown_connections 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdcbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2292045388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdbd480 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- 172.21.15.120:0/2292045388 >> 172.21.15.120:0/2292045388 conn(0x55c67fdbaed0 msgr2=0x55c67fdbc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- 172.21.15.120:0/2292045388 shutdown_connections 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- 172.21.15.120:0/2292045388 wait complete. 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 -- start start 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.993+0000 7f686a1b4740 1 Processor -- start 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 0x55c67fdc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c67fb84700 con 0x55c67fdbd9c0 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c67fcbe110 con 0x55c67fdbd0a0 2024-09-10T22:57:42.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c67fdcc570 con 0x55c67fdc9950 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 10 client.0 _ll_register_callbacks cb 0x55c67fcd7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34670/0 (socket says 172.21.15.120:34670) 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 -- 172.21.15.120:0/1183907978 learned_addr learned my addr 172.21.15.120:0/1183907978 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686537a640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686537a640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 0x55c67fdc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 -- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 msgr2=0x55c67fdc6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdc6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 -- 172.21.15.120:0/1183907978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 msgr2=0x55c67fdc91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 0x55c67fdc91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 -- 172.21.15.120:0/1183907978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67fcebc40 con 0x55c67fdbd9c0 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686537a640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 0x55c67fdc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:42.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f68497fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f6865b7b640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f685c012d20 tx=0x7f685c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f89283c2d3dad86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c012110 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68500054d0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.994+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f685c004050 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f685c023070 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 10 client.16401 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c67fdd7ef0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68500054d0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f685c010430 con 0x55c67fdbd9c0 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 10 client.16401 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 20 client.16401 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fdd9630 0x55c67fddbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 4 client.16401 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67fd35da0 con 0x55c67fdd9630 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.995+0000 7f686537a640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fdd9630 0x55c67fddbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.996+0000 7f686537a640 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fdd9630 0x55c67fddbaf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e837e348b9798cca server_cookie=5bfe653d4559213d in_seq=0 out_seq=0 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.996+0000 7f68497fa640 10 client.16401 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.996+0000 7f68497fa640 1 -- 172.21.15.120:0/1183907978 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c67fd35da0 con 0x55c67fdd9630 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.996+0000 7f68497fa640 10 client.16401 handle_command_reply: tid=0 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.996+0000 7f686a1b4740 1 client.16401 shutdown 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 msgr2=0x55c67fdc6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f685c012d20 tx=0x7f685c00f790 comp rx=0 tx=0).stop 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 shutdown_connections 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fdd9630 0x55c67fddbaf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdc9950 0x55c67fdc6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd9c0 0x55c67fdc6640 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 --2- 172.21.15.120:0/1183907978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd0a0 0x55c67fdc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f68497fa640 0 client.16401 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:42.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 >> 172.21.15.120:0/1183907978 conn(0x55c67fdbaed0 msgr2=0x55c67fdc0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 shutdown_connections 2024-09-10T22:57:42.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 1 -- 172.21.15.120:0/1183907978 wait complete. 2024-09-10T22:57:42.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:41.997+0000 7f686a1b4740 20 client.16401 trim_cache size 0 max 16384 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 Processor -- start 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 -- start start 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdcad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdc9950 0x55c67fdc3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68601ac310 con 0x55c67fdbd0a0 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68601aeb70 con 0x55c67fdc9950 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686a1b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f68601ac960 con 0x55c67fdbd9c0 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686537a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdcad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686537a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdcad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34678/0 (socket says 172.21.15.120:34678) 2024-09-10T22:57:42.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44762/0 (socket says 172.21.15.120:44762) 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686537a640 1 -- 172.21.15.120:0/4078381426 learned_addr learned my addr 172.21.15.120:0/4078381426 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f686537a640 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdc9950 0x55c67fdc3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 -- 172.21.15.120:0/4078381426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdc9950 msgr2=0x55c67fdc3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdc9950 0x55c67fdc3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 -- 172.21.15.120:0/4078381426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 msgr2=0x55c67fdcad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdcad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.017+0000 7f6865b7b640 1 -- 172.21.15.120:0/4078381426 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67fdb6790 con 0x55c67fdbd9c0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.018+0000 7f6865b7b640 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f685c010a40 tx=0x7f685c012320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a4a7f9bf8d6762f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.018+0000 7f684b7fe640 1 -- 172.21.15.120:0/4078381426 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c02c070 con 0x55c67fdbd9c0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.018+0000 7f684b7fe640 1 -- 172.21.15.120:0/4078381426 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685c005560 con 0x55c67fdbd9c0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.018+0000 7f684b7fe640 1 -- 172.21.15.120:0/4078381426 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c003810 con 0x55c67fdbd9c0 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- 172.21.15.120:0/4078381426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 msgr2=0x55c67fdcb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f685c010a40 tx=0x7f685c012320 comp rx=0 tx=0).stop 2024-09-10T22:57:42.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- 172.21.15.120:0/4078381426 shutdown_connections 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdc9950 0x55c67fdc3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd9c0 0x55c67fdcb280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 --2- 172.21.15.120:0/4078381426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdbd0a0 0x55c67fdcad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- 172.21.15.120:0/4078381426 >> 172.21.15.120:0/4078381426 conn(0x55c67fdbaed0 msgr2=0x55c67fdc4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- 172.21.15.120:0/4078381426 shutdown_connections 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- 172.21.15.120:0/4078381426 wait complete. 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 -- start start 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.019+0000 7f686a1b4740 1 Processor -- start 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd0a0 0x55c67fe405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdc9950 0x55c67fdd1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68601ad030 con 0x55c67fdc9950 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f68601ae4a0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f68601ad700 con 0x55c67fdbd0a0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686a1b4740 10 client.0 _ll_register_callbacks cb 0x7f685c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686537a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd0a0 0x55c67fe405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60562/0 (socket says 172.21.15.120:60562) 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 -- 172.21.15.120:0/2676880968 learned_addr learned my addr 172.21.15.120:0/2676880968 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f686537a640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdc9950 0x55c67fdd1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 -- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd0a0 msgr2=0x55c67fe405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd0a0 0x55c67fe405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 -- 172.21.15.120:0/2676880968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdc9950 msgr2=0x55c67fdd1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdc9950 0x55c67fdd1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 -- 172.21.15.120:0/2676880968 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67fe413c0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f6865b7b640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f685c012cf0 tx=0x7f685c010d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b150cfad1ba45214 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:42.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.020+0000 7f68497fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c035020 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f685c0052c0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f685c005560 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f685c02c040 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 10 client.25847 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c67fe4f8a0 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685c005560 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f685c028480 con 0x55c67fdbd9c0 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 10 client.25847 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 20 client.25847 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fe4ff00 0x55c67fe522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 4 client.25847 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:42.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c67fd31850 con 0x55c67fe4ff00 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.021+0000 7f686537a640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fe4ff00 0x55c67fe522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.022+0000 7f68497fa640 10 client.25847 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.022+0000 7f686537a640 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fe4ff00 0x55c67fe522f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2c33d1014c4539 server_cookie=9c91caad5cddcacc in_seq=0 out_seq=0 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f68497fa640 1 -- 172.21.15.120:0/2676880968 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55c67fd31850 con 0x55c67fe4ff00 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f68497fa640 10 client.25847 handle_command_reply: tid=0 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 client.25847 shutdown 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 msgr2=0x55c67fe40b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f685c012cf0 tx=0x7f685c010d20 comp rx=0 tx=0).stop 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 shutdown_connections 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c67fe4ff00 0x55c67fe522f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c67fdc9950 0x55c67fdd1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c67fdbd9c0 0x55c67fe40b10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f68497fa640 0 client.25847 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:42.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 --2- 172.21.15.120:0/2676880968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c67fdbd0a0 0x55c67fe405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 >> 172.21.15.120:0/2676880968 conn(0x55c67fdbaed0 msgr2=0x55c67fdc8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 shutdown_connections 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 1 -- 172.21.15.120:0/2676880968 wait complete. 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.023+0000 7f686a1b4740 20 client.25847 trim_cache size 0 max 16384 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.031 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "291470de-ce79-4054-bccb-d0d766773893", 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.032 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.034 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.035 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:57:42.036 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:42.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 msgr2=0x7f684c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 0x7f684c0b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f68580c9a80 tx=0x7f68580921d0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 msgr2=0x7f686019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6858005cc0 tx=0x7f6858031700 comp rx=0 tx=0).stop 2024-09-10T22:57:42.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 shutdown_connections 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f684c0b52e0 0x7f684c0b77a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f686010d0e0 0x7f686019eb50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6860104530 0x7f68601997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 --2- 172.21.15.120:0/1462104042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860103b30 0x7f6860199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 >> 172.21.15.120:0/1462104042 conn(0x7f68600fd700 msgr2=0x7f68600fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 shutdown_connections 2024-09-10T22:57:42.039 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:57:42.024+0000 7f6867ddf640 1 -- 172.21.15.120:0/1462104042 wait complete. 2024-09-10T22:57:42.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:42 smithi120 ceph-mon[31247]: pgmap v503: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-10T22:57:42.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:42 smithi153 ceph-mon[34951]: pgmap v503: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-10T22:57:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:42 smithi176 ceph-mon[34982]: pgmap v503: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-10T22:57:42.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 871323 16.66 MB/sec execute 157 sec latency 116.819 ms 2024-09-10T22:57:42.779 INFO:tasks.cephfs.filesystem:scrub status for tag:291470de-ce79-4054-bccb-d0d766773893 - {'path': '/', 'tag': '291470de-ce79-4054-bccb-d0d766773893', 'options': 'recursive,force'} 2024-09-10T22:57:43.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 874197 16.65 MB/sec execute 158 sec latency 27.880 ms 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 -- 172.21.15.120:0/1192696966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec101860 msgr2=0x7f74ec101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 --2- 172.21.15.120:0/1192696966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec101860 0x7f74ec101c60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009930 tx=0x7f74e802a420 comp rx=0 tx=0).stop 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 -- 172.21.15.120:0/1192696966 shutdown_connections 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 --2- 172.21.15.120:0/1192696966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 0x7f74ec109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 --2- 172.21.15.120:0/1192696966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec1029b0 0x7f74ec102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 --2- 172.21.15.120:0/1192696966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec101860 0x7f74ec101c60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 -- 172.21.15.120:0/1192696966 >> 172.21.15.120:0/1192696966 conn(0x7f74ec0fd140 msgr2=0x7f74ec0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 -- 172.21.15.120:0/1192696966 shutdown_connections 2024-09-10T22:57:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.429+0000 7f74f43c2640 1 -- 172.21.15.120:0/1192696966 wait complete. 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 Processor -- start 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 -- start start 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 0x7f74ec06b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 0x7f74ec1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d4d0 con 0x7f74ec1029b0 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d350 con 0x7f74ec101860 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f43c2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d650 con 0x7f74ec103370 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60070/0 (socket says 172.21.15.120:60070) 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 -- 172.21.15.120:0/18581378 learned_addr learned my addr 172.21.15.120:0/18581378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f295f640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 0x7f74ec1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f215e640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 0x7f74ec06b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 -- 172.21.15.120:0/18581378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 msgr2=0x7f74ec1a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 0x7f74ec1a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 -- 172.21.15.120:0/18581378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 msgr2=0x7f74ec06b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 0x7f74ec06b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f195d640 1 -- 172.21.15.120:0/18581378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec1a0950 con 0x7f74ec1029b0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.430+0000 7f74f215e640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 0x7f74ec06b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74f195d640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00d950 tx=0x7f74dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74e37fe640 1 -- 172.21.15.120:0/18581378 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc017030 con 0x7f74ec1029b0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74e37fe640 1 -- 172.21.15.120:0/18581378 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74dc004020 con 0x7f74ec1029b0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec1a0c40 con 0x7f74ec1029b0 2024-09-10T22:57:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74e37fe640 1 -- 172.21.15.120:0/18581378 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0047a0 con 0x7f74ec1029b0 2024-09-10T22:57:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.431+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec1a1190 con 0x7f74ec1029b0 2024-09-10T22:57:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 Processor -- start 2024-09-10T22:57:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 -- start start 2024-09-10T22:57:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74e37fe640 1 -- 172.21.15.120:0/18581378 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f74dc009a30 con 0x7f74ec1029b0 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f960580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f963090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c32f8cd6f0 con 0x55c32f95b860 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c32f8bcb20 con 0x55c32f95e190 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f7274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c32f8c0930 con 0x55c32f960bd0 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f295f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f195d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f963090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f295f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60074/0 (socket says 172.21.15.120:60074) 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f295f640 1 -- 172.21.15.120:0/1036056566 learned_addr learned my addr 172.21.15.120:0/1036056566 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.434+0000 7f74f215e640 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f960580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 -- 172.21.15.120:0/1036056566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 msgr2=0x55c32f963090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f963090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 -- 172.21.15.120:0/1036056566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 msgr2=0x55c32f960580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f960580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 -- 172.21.15.120:0/1036056566 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32f882530 con 0x55c32f95b860 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f295f640 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009f50 tx=0x7f74e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a45e748bd98ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74e37fe640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 0x7f74d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74e0bfe640 1 -- 172.21.15.120:0/1036056566 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4004d20 con 0x55c32f95b860 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74e0bfe640 1 -- 172.21.15.120:0/1036056566 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e4004050 con 0x55c32f95b860 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74e0bfe640 1 -- 172.21.15.120:0/1036056566 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4005280 con 0x55c32f95b860 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74f215e640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 0x7f74d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.435+0000 7f74e37fe640 1 -- 172.21.15.120:0/18581378 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f74dc0e5680 con 0x7f74ec1029b0 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- 172.21.15.120:0/1036056566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 msgr2=0x55c32f95dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f215e640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 0x7f74d80b77a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74e8000c00 tx=0x7f74e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009f50 tx=0x7f74e40118d0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- 172.21.15.120:0/1036056566 shutdown_connections 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f963090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f960580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 --2- 172.21.15.120:0/1036056566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95b860 0x55c32f95dc50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- 172.21.15.120:0/1036056566 >> 172.21.15.120:0/1036056566 conn(0x55c32f9569b0 msgr2=0x55c32f957740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- 172.21.15.120:0/1036056566 shutdown_connections 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- 172.21.15.120:0/1036056566 wait complete. 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 -- start start 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.436+0000 7f74f7274740 1 Processor -- start 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f95b860 0x55c32f964ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f965570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c32f7f4ab0 con 0x55c32f960bd0 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c32f862730 con 0x55c32f95e190 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c32f963bb0 con 0x55c32f95b860 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 10 client.0 _ll_register_callbacks cb 0x55c32f8b66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60082/0 (socket says 172.21.15.120:60082) 2024-09-10T22:57:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 -- 172.21.15.120:0/3359226659 learned_addr learned my addr 172.21.15.120:0/3359226659 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f215e640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f95b860 0x55c32f964ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f195d640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f965570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 -- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f95b860 msgr2=0x55c32f964ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f95b860 0x55c32f964ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 -- 172.21.15.120:0/3359226659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 msgr2=0x55c32f965570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f965570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 -- 172.21.15.120:0/3359226659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32f61e790 con 0x55c32f960bd0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f295f640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009730 tx=0x7f74e4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c08eee60d0c30f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4028070 con 0x55c32f960bd0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f74e4005040 con 0x55c32f960bd0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.437+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e4004050 con 0x55c32f960bd0 2024-09-10T22:57:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74e4005240 con 0x55c32f960bd0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 10 client.16419 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c32f697620 con 0x55c32f960bd0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e4004050 con 0x55c32f960bd0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f74e40247b0 con 0x55c32f960bd0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 10 client.16419 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 20 client.16419 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f970610 0x55c32f972a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 4 client.16419 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c32f8cf640 con 0x55c32f970610 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.438+0000 7f74f215e640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f970610 0x55c32f972a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f215e640 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f970610 0x55c32f972a60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7671086613fa1fa server_cookie=2698bfead25d242c in_seq=0 out_seq=0 2024-09-10T22:57:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74d5ffb640 10 client.16419 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74d5ffb640 1 -- 172.21.15.120:0/3359226659 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c32f8cf640 con 0x55c32f970610 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74d5ffb640 10 client.16419 handle_command_reply: tid=0 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 client.16419 shutdown 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 msgr2=0x55c32f966140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009730 tx=0x7f74e4004530 comp rx=0 tx=0).stop 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 shutdown_connections 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f970610 0x55c32f972a60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f960bd0 0x55c32f966140 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95e190 0x55c32f965570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74d5ffb640 0 client.16419 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 --2- 172.21.15.120:0/3359226659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f95b860 0x55c32f964ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.439+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 >> 172.21.15.120:0/3359226659 conn(0x55c32f9569b0 msgr2=0x55c32f955bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.440+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 shutdown_connections 2024-09-10T22:57:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.440+0000 7f74f7274740 1 -- 172.21.15.120:0/3359226659 wait complete. 2024-09-10T22:57:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.440+0000 7f74f7274740 20 client.16419 trim_cache size 0 max 16384 2024-09-10T22:57:44.452 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:44 smithi120 ceph-mon[31247]: pgmap v504: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2024-09-10T22:57:44.452 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:44 smithi153 ceph-mon[34951]: pgmap v504: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 Processor -- start 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 -- start start 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f956410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f960fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1a7ed0 con 0x55c32f95e190 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1aa750 con 0x55c32f95b860 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f7274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1a8540 con 0x55c32f960bd0 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60086/0 (socket says 172.21.15.120:60086) 2024-09-10T22:57:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 -- 172.21.15.120:0/1526007807 learned_addr learned my addr 172.21.15.120:0/1526007807 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f215e640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f956410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f295f640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f960fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 -- 172.21.15.120:0/1526007807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 msgr2=0x55c32f960fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f960fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 -- 172.21.15.120:0/1526007807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 msgr2=0x55c32f956410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f956410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f195d640 1 -- 172.21.15.120:0/1526007807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32f95dfc0 con 0x55c32f95e190 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.456+0000 7f74f295f640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f960fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74f195d640 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0a8440 tx=0x7f74dc0eb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcddc89b55d2096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74d7fff640 1 -- 172.21.15.120:0/1526007807 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0abc40 con 0x55c32f95e190 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74d7fff640 1 -- 172.21.15.120:0/1526007807 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74dc0ab980 con 0x55c32f95e190 2024-09-10T22:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74d7fff640 1 -- 172.21.15.120:0/1526007807 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0ab5e0 con 0x55c32f95e190 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74f7274740 1 -- 172.21.15.120:0/1526007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 msgr2=0x55c32f957670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.457+0000 7f74f7274740 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0a8440 tx=0x7f74dc0eb550 comp rx=0 tx=0).stop 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- 172.21.15.120:0/1526007807 shutdown_connections 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f960fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f957670 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- 172.21.15.120:0/1526007807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f956410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- 172.21.15.120:0/1526007807 >> 172.21.15.120:0/1526007807 conn(0x55c32f9569b0 msgr2=0x55c32f966a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- 172.21.15.120:0/1526007807 shutdown_connections 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- 172.21.15.120:0/1526007807 wait complete. 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- start start 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 Processor -- start 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f96c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f9db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1a8c10 con 0x55c32f95e190 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1aa080 con 0x55c32f95b860 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1a92e0 con 0x55c32f960bd0 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f7274740 10 client.0 _ll_register_callbacks cb 0x7f74e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f195d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60092/0 (socket says 172.21.15.120:60092) 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f195d640 1 -- 172.21.15.120:0/888102853 learned_addr learned my addr 172.21.15.120:0/888102853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f215e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f96c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.458+0000 7f74f295f640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f9db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 -- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 msgr2=0x55c32f9db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f9db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 -- 172.21.15.120:0/888102853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 msgr2=0x55c32f96c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f96c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 -- 172.21.15.120:0/888102853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32f9608f0 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f195d640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0b55c0 tx=0x7f74dc0eb320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436d021dc59900c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0ab7c0 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f74dc0a80e0 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74dc0ab980 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74dc0abe70 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f7274740 10 client.16425 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c32f9dcc30 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74dc0ab980 con 0x55c32f95e190 2024-09-10T22:57:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f74dc0e7b90 con 0x55c32f95e190 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.459+0000 7f74f7274740 10 client.16425 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f7274740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f7274740 20 client.16425 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f9dd610 0x55c32f9dfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f7274740 4 client.16425 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c32f8cb0f0 con 0x55c32f9dd610 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f215e640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f9dd610 0x55c32f9dfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74d5ffb640 10 client.16425 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.460+0000 7f74f215e640 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f9dd610 0x55c32f9dfa30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4389ba2f80176c1b server_cookie=b6ef5b0bdcca3e4e in_seq=0 out_seq=0 2024-09-10T22:57:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:44 smithi176 ceph-mon[34982]: pgmap v504: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74d5ffb640 1 -- 172.21.15.120:0/888102853 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c32f8cb0f0 con 0x55c32f9dd610 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74d5ffb640 10 client.16425 handle_command_reply: tid=0 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 client.16425 shutdown 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 msgr2=0x55c32f96c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0b55c0 tx=0x7f74dc0eb320 comp rx=0 tx=0).stop 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 shutdown_connections 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c32f9dd610 0x55c32f9dfa30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c32f960bd0 0x55c32f9db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74d5ffb640 0 client.16425 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c32f95e190 0x55c32f96c880 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 --2- 172.21.15.120:0/888102853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c32f95b860 0x55c32f96c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 >> 172.21.15.120:0/888102853 conn(0x55c32f9569b0 msgr2=0x55c32f957450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 shutdown_connections 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 1 -- 172.21.15.120:0/888102853 wait complete. 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.620+0000 7f74f7274740 20 client.16425 trim_cache size 0 max 16384 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:57:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:57:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 msgr2=0x7f74d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 0x7f74d80b77a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74e8000c00 tx=0x7f74e80058d0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 msgr2=0x7f74ec19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00d950 tx=0x7f74dc00de10 comp rx=0 tx=0).stop 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 shutdown_connections 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f74d80b52e0 0x7f74d80b77a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f74ec103370 0x7f74ec1a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74ec1029b0 0x7f74ec19daa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 --2- 172.21.15.120:0/18581378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec101860 0x7f74ec06b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 >> 172.21.15.120:0/18581378 conn(0x7f74ec0fd140 msgr2=0x7f74ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 shutdown_connections 2024-09-10T22:57:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:44.621+0000 7f74f43c2640 1 -- 172.21.15.120:0/18581378 wait complete. 2024-09-10T22:57:44.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 877319 16.66 MB/sec execute 159 sec latency 106.697 ms 2024-09-10T22:57:45.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:45 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:57:45.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:45 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:57:45.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:45 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:57:45.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 880612 16.67 MB/sec execute 160 sec latency 51.233 ms 2024-09-10T22:57:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:46 smithi120 ceph-mon[31247]: pgmap v505: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-10T22:57:46.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:46 smithi153 ceph-mon[34951]: pgmap v505: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-10T22:57:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:46 smithi176 ceph-mon[34982]: pgmap v505: 97 pgs: 97 active+clean; 51 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-10T22:57:46.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 883801 16.66 MB/sec execute 161 sec latency 18.906 ms 2024-09-10T22:57:47.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 886408 16.63 MB/sec execute 162 sec latency 64.792 ms 2024-09-10T22:57:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:48 smithi120 ceph-mon[31247]: pgmap v506: 97 pgs: 97 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-09-10T22:57:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:48 smithi153 ceph-mon[34951]: pgmap v506: 97 pgs: 97 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-09-10T22:57:48.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:48 smithi176 ceph-mon[34982]: pgmap v506: 97 pgs: 97 active+clean; 47 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 548 op/s 2024-09-10T22:57:48.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 888991 16.63 MB/sec execute 163 sec latency 29.855 ms 2024-09-10T22:57:49.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 892217 16.63 MB/sec execute 164 sec latency 13.322 ms 2024-09-10T22:57:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:50 smithi120 ceph-mon[31247]: pgmap v507: 97 pgs: 97 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-10T22:57:50.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:50 smithi153 ceph-mon[34951]: pgmap v507: 97 pgs: 97 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-10T22:57:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:50 smithi176 ceph-mon[34982]: pgmap v507: 97 pgs: 97 active+clean; 48 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-10T22:57:50.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 895532 16.64 MB/sec execute 165 sec latency 14.208 ms 2024-09-10T22:57:51.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 898732 16.65 MB/sec execute 166 sec latency 10.496 ms 2024-09-10T22:57:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:52 smithi120 ceph-mon[31247]: pgmap v508: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-09-10T22:57:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:52 smithi153 ceph-mon[34951]: pgmap v508: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-09-10T22:57:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:52 smithi176 ceph-mon[34982]: pgmap v508: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2024-09-10T22:57:52.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 902049 16.66 MB/sec execute 167 sec latency 47.110 ms 2024-09-10T22:57:53.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 905096 16.65 MB/sec execute 168 sec latency 61.450 ms 2024-09-10T22:57:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:54 smithi120 ceph-mon[31247]: pgmap v509: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-09-10T22:57:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:54 smithi153 ceph-mon[34951]: pgmap v509: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-09-10T22:57:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:54 smithi176 ceph-mon[34982]: pgmap v509: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-09-10T22:57:54.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 908219 16.65 MB/sec execute 169 sec latency 12.036 ms 2024-09-10T22:57:55.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 911268 16.64 MB/sec execute 170 sec latency 22.220 ms 2024-09-10T22:57:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:56 smithi120 ceph-mon[31247]: pgmap v510: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:57:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:56 smithi153 ceph-mon[34951]: pgmap v510: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:57:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:56 smithi176 ceph-mon[34982]: pgmap v510: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:57:56.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 914068 16.64 MB/sec execute 171 sec latency 50.053 ms 2024-09-10T22:57:57.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 916853 16.66 MB/sec execute 172 sec latency 39.102 ms 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- 172.21.15.120:0/3112751445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 msgr2=0x7f73301055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- 172.21.15.120:0/3112751445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f73301055a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7320009ca0 tx=0x7f732002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- 172.21.15.120:0/3112751445 shutdown_connections 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- 172.21.15.120:0/3112751445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- 172.21.15.120:0/3112751445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 0x7f7330107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- 172.21.15.120:0/3112751445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f73301055a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- 172.21.15.120:0/3112751445 >> 172.21.15.120:0/3112751445 conn(0x7f73300fd150 msgr2=0x7f73300ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- 172.21.15.120:0/3112751445 shutdown_connections 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- 172.21.15.120:0/3112751445 wait complete. 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 Processor -- start 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- start start 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 0x7f7330199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e690 con 0x7f733006bf50 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e510 con 0x7f7330108690 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.731+0000 7f7336a79640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f733010e810 con 0x7f7330105ae0 2024-09-10T22:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36724/0 (socket says 172.21.15.120:36724) 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 -- 172.21.15.120:0/907516056 learned_addr learned my addr 172.21.15.120:0/907516056 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f732ffff640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 0x7f7330199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7335016640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 -- 172.21.15.120:0/907516056 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 msgr2=0x7f7330199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 0x7f7330199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 -- 172.21.15.120:0/907516056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 msgr2=0x7f733019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 -- 172.21.15.120:0/907516056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733019a950 con 0x7f733006bf50 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7335016640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7334815640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7320006010 tx=0x7f7320003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f732dffb640 1 -- 172.21.15.120:0/907516056 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732002f380 con 0x7f733006bf50 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f732dffb640 1 -- 172.21.15.120:0/907516056 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73200317c0 con 0x7f733006bf50 2024-09-10T22:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f732dffb640 1 -- 172.21.15.120:0/907516056 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732002fc60 con 0x7f733006bf50 2024-09-10T22:57:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733019abe0 con 0x7f733006bf50 2024-09-10T22:57:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.732+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73301a1310 con 0x7f733006bf50 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 Processor -- start 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 -- start start 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f732dffb640 1 -- 172.21.15.120:0/907516056 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7320004040 con 0x7f733006bf50 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 0x55f590f5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f590eca6f0 con 0x55f590f5f5a0 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f590eb9b20 con 0x55f590f5cc70 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7339848740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f590ebd930 con 0x55f590f61fe0 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7335016640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 0x55f590f5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36734/0 (socket says 172.21.15.120:36734) 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 -- 172.21.15.120:0/2949763248 learned_addr learned my addr 172.21.15.120:0/2949763248 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7335016640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 0x55f590f5f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49036/0 (socket says 172.21.15.120:49036) 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f732ffff640 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 -- 172.21.15.120:0/2949763248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 msgr2=0x55f590f644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 -- 172.21.15.120:0/2949763248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 msgr2=0x55f590f5f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 0x55f590f5f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.735+0000 7f7334815640 1 -- 172.21.15.120:0/2949763248 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f590e7f530 con 0x55f590f5f5a0 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f7334815640 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f732010ae50 tx=0x7f732010af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea7220122ba7048 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f732dffb640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 0x7f731c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f7326ffd640 1 -- 172.21.15.120:0/2949763248 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200c9d10 con 0x55f590f5f5a0 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f7335016640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 0x7f731c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f7326ffd640 1 -- 172.21.15.120:0/2949763248 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7320114070 con 0x55f590f5f5a0 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f7326ffd640 1 -- 172.21.15.120:0/2949763248 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200cd010 con 0x55f590f5f5a0 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.736+0000 7f732dffb640 1 -- 172.21.15.120:0/907516056 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7320106ed0 con 0x7f733006bf50 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- 172.21.15.120:0/2949763248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 msgr2=0x55f590f61990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7335016640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 0x7f731c0b79c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7330199b90 tx=0x7f7328008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f732010ae50 tx=0x7f732010af40 comp rx=0 tx=0).stop 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- 172.21.15.120:0/2949763248 shutdown_connections 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5f5a0 0x55f590f61990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 --2- 172.21.15.120:0/2949763248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5cc70 0x55f590f5f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- 172.21.15.120:0/2949763248 >> 172.21.15.120:0/2949763248 conn(0x55f590f539b0 msgr2=0x55f590f54740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- 172.21.15.120:0/2949763248 shutdown_connections 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- 172.21.15.120:0/2949763248 wait complete. 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 -- start start 2024-09-10T22:57:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 Processor -- start 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.737+0000 7f7339848740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f66510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f590df1ab0 con 0x55f590f5cc70 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f590e5f730 con 0x55f590f5f5a0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f590eca6f0 con 0x55f590f61fe0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36748/0 (socket says 172.21.15.120:36748) 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 -- 172.21.15.120:0/294831766 learned_addr learned my addr 172.21.15.120:0/294831766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 10 client.0 _ll_register_callbacks cb 0x55f590eb36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7335016640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f732ffff640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f66510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 -- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 msgr2=0x55f590f670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 -- 172.21.15.120:0/294831766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 msgr2=0x55f590f66510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f66510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 -- 172.21.15.120:0/294831766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f590c1b790 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7334815640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7320104a50 tx=0x7f73200095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78bdfbae220073 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f730ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200cd200 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f73200f60a0 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7320114070 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73200cd490 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 10 client.16437 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.738+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f590c94620 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7320114070 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f73200c9930 con 0x55f590f5cc70 2024-09-10T22:57:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 20 client.16437 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590f73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 4 client.16437 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f590ecc640 con 0x55f590f71630 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.739+0000 7f732ffff640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590f73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.740+0000 7f732ffff640 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590f73a50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326c201c67b094c2 server_cookie=c4bcf612908df324 in_seq=0 out_seq=0 2024-09-10T22:57:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.740+0000 7f730ffff640 10 client.16437 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f730ffff640 1 -- 172.21.15.120:0/294831766 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f590ecc640 con 0x55f590f71630 2024-09-10T22:57:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f730ffff640 10 client.16437 handle_command_reply: tid=0 2024-09-10T22:57:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 client.16437 shutdown 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 msgr2=0x55f590f65e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7320104a50 tx=0x7f73200095e0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 shutdown_connections 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590f73a50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f730ffff640 0 client.16437 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f66510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 --2- 172.21.15.120:0/294831766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f65e70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 >> 172.21.15.120:0/294831766 conn(0x55f590f539b0 msgr2=0x55f590f52bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 shutdown_connections 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 1 -- 172.21.15.120:0/294831766 wait complete. 2024-09-10T22:57:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.741+0000 7f7339848740 20 client.16437 trim_cache size 0 max 16384 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 Processor -- start 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 -- start start 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f5cc70 0x55f590f623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f62900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a5d30 con 0x55f590f61fe0 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a85b0 con 0x55f590f5f5a0 2024-09-10T22:57:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7339848740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a63a0 con 0x55f590f5cc70 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36756/0 (socket says 172.21.15.120:36756) 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 -- 172.21.15.120:0/4264274217 learned_addr learned my addr 172.21.15.120:0/4264274217 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7334815640 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f5cc70 0x55f590f623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 -- 172.21.15.120:0/4264274217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f5cc70 msgr2=0x55f590f623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f5cc70 0x55f590f623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f732ffff640 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 -- 172.21.15.120:0/4264274217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 msgr2=0x55f590f62900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f62900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 -- 172.21.15.120:0/4264274217 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f590f5f3d0 con 0x55f590f61fe0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f7335016640 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7328010a90 tx=0x7f7328013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43142b65360a8599 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f73267fc640 1 -- 172.21.15.120:0/4264274217 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328026070 con 0x55f590f61fe0 2024-09-10T22:57:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f73267fc640 1 -- 172.21.15.120:0/4264274217 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732801d040 con 0x55f590f61fe0 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.756+0000 7f73267fc640 1 -- 172.21.15.120:0/4264274217 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732801c040 con 0x55f590f61fe0 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- 172.21.15.120:0/4264274217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 msgr2=0x55f590f6d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7328010a90 tx=0x7f7328013000 comp rx=0 tx=0).stop 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- 172.21.15.120:0/4264274217 shutdown_connections 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f61fe0 0x55f590f6d9e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 --2- 172.21.15.120:0/4264274217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f5cc70 0x55f590f623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- 172.21.15.120:0/4264274217 >> 172.21.15.120:0/4264274217 conn(0x55f590f539b0 msgr2=0x55f590f57cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- 172.21.15.120:0/4264274217 shutdown_connections 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- 172.21.15.120:0/4264274217 wait complete. 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.757+0000 7f7339848740 1 -- start start 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 Processor -- start 2024-09-10T22:57:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a6a70 con 0x55f590f5cc70 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a7ee0 con 0x55f590f5f5a0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73301a7140 con 0x55f590f61fe0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7339848740 10 client.0 _ll_register_callbacks cb 0x7f732800b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36764/0 (socket says 172.21.15.120:36764) 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 -- 172.21.15.120:0/853500347 learned_addr learned my addr 172.21.15.120:0/853500347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f732ffff640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7335016640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 -- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 msgr2=0x55f590f6cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f6cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 -- 172.21.15.120:0/853500347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 msgr2=0x55f590f6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 -- 172.21.15.120:0/853500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f590f6d740 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f7334815640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f73200d0040 tx=0x7f7320004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288d0613e2e234a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.758+0000 7f730ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200c8e30 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f73200d67c0 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732801d040 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73200c9010 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 10 client.16443 fetch_fsmap learned FSMap version 25 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f590f6d740 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732801d040 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8330+0+0 (secure 0 0 0) 0x7f73200cd920 con 0x55f590f5cc70 2024-09-10T22:57:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 10 client.16443 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 20 client.16443 populate_metadata read hostname 'smithi120' 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590fe0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 4 client.16443 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f590ec80f0 con 0x55f590f71630 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.759+0000 7f732ffff640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590fe0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.760+0000 7f730ffff640 10 client.16443 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.760+0000 7f732ffff640 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590fe0bb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0745b73ec8707d server_cookie=48f8ec4ef8e20d49 in_seq=0 out_seq=0 2024-09-10T22:57:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f730ffff640 1 -- 172.21.15.120:0/853500347 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f590ec80f0 con 0x55f590f71630 2024-09-10T22:57:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f730ffff640 10 client.16443 handle_command_reply: tid=0 2024-09-10T22:57:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 client.16443 shutdown 2024-09-10T22:57:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 msgr2=0x55f590f6dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f73200d0040 tx=0x7f7320004940 comp rx=0 tx=0).stop 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 shutdown_connections 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f590f71630 0x55f590fe0bb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f730ffff640 0 client.16443 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f590f61fe0 0x55f590f6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.982+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f590f5f5a0 0x55f590f6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.983+0000 7f7339848740 1 --2- 172.21.15.120:0/853500347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f590f5cc70 0x55f590f6dd90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.983+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 >> 172.21.15.120:0/853500347 conn(0x55f590f539b0 msgr2=0x55f590f57890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.983+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 shutdown_connections 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.983+0000 7f7339848740 1 -- 172.21.15.120:0/853500347 wait complete. 2024-09-10T22:57:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.983+0000 7f7339848740 20 client.16443 trim_cache size 0 max 16384 2024-09-10T22:57:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:57:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:57:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:57:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 msgr2=0x7f731c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 0x7f731c0b79c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7330199b90 tx=0x7f7328008040 comp rx=0 tx=0).stop 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 msgr2=0x7f7330198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7320006010 tx=0x7f7320003ba0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 shutdown_connections 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.984+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f731c0b5500 0x7f731c0b79c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7330108690 0x7f733019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7330105ae0 0x7f7330199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 --2- 172.21.15.120:0/907516056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f733006bf50 0x7f7330198e10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 >> 172.21.15.120:0/907516056 conn(0x7f73300fd150 msgr2=0x7f7330103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 shutdown_connections 2024-09-10T22:57:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:57:57.985+0000 7f7336a79640 1 -- 172.21.15.120:0/907516056 wait complete. 2024-09-10T22:57:58.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:58 smithi176 ceph-mon[34982]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-10T22:57:58.374 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:57:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setuser ceph since I am not root 2024-09-10T22:57:58.375 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:57:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setgroup ceph since I am not root 2024-09-10T22:57:58.375 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:57:58 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/1886992157 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c2be91860 con 0x562c2be6f000 2024-09-10T22:57:58.375 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 22:57:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T22:57:58.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:58 smithi120 ceph-mon[31247]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-10T22:57:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:58 smithi153 ceph-mon[34951]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-10T22:57:58.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 918374 16.68 MB/sec execute 173 sec latency 96.528 ms 2024-09-10T22:57:59.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:59 smithi120 ceph-mon[31247]: daemon mds.c restarted 2024-09-10T22:57:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:59 smithi120 ceph-mon[31247]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:57:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:59 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/458342810,v1:172.21.15.176:6837/458342810] up:boot 2024-09-10T22:57:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:59 smithi120 ceph-mon[31247]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:57:59.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:57:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:57:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:59 smithi153 ceph-mon[34951]: daemon mds.c restarted 2024-09-10T22:57:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:59 smithi153 ceph-mon[34951]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:57:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:59 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/458342810,v1:172.21.15.176:6837/458342810] up:boot 2024-09-10T22:57:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:59 smithi153 ceph-mon[34951]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:57:59.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:57:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:57:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:59 smithi176 ceph-mon[34982]: daemon mds.c restarted 2024-09-10T22:57:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:59 smithi176 ceph-mon[34982]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:57:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:59 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/458342810,v1:172.21.15.176:6837/458342810] up:boot 2024-09-10T22:57:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:59 smithi176 ceph-mon[34982]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T22:57:59.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:57:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:57:59.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 921778 16.69 MB/sec execute 174 sec latency 12.097 ms 2024-09-10T22:58:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:00 smithi120 ceph-mon[31247]: pgmap v512: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 321 op/s 2024-09-10T22:58:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:00 smithi153 ceph-mon[34951]: pgmap v512: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 321 op/s 2024-09-10T22:58:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:00 smithi176 ceph-mon[34982]: pgmap v512: 97 pgs: 97 active+clean; 54 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 321 op/s 2024-09-10T22:58:00.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 925278 16.71 MB/sec execute 175 sec latency 11.637 ms 2024-09-10T22:58:01.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 928280 16.70 MB/sec execute 176 sec latency 21.856 ms 2024-09-10T22:58:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:02 smithi120 ceph-mon[31247]: pgmap v514: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 389 op/s 2024-09-10T22:58:02.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:02 smithi153 ceph-mon[34951]: pgmap v514: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 389 op/s 2024-09-10T22:58:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:02 smithi176 ceph-mon[34982]: pgmap v514: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 389 op/s 2024-09-10T22:58:02.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 931796 16.71 MB/sec execute 177 sec latency 22.144 ms 2024-09-10T22:58:03.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 934830 16.72 MB/sec execute 178 sec latency 99.781 ms 2024-09-10T22:58:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:04 smithi120 ceph-mon[31247]: pgmap v515: 97 pgs: 97 active+clean; 28 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:58:04.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:04 smithi153 ceph-mon[34951]: pgmap v515: 97 pgs: 97 active+clean; 28 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:58:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:04 smithi176 ceph-mon[34982]: pgmap v515: 97 pgs: 97 active+clean; 28 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 479 op/s 2024-09-10T22:58:04.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 938404 16.73 MB/sec execute 179 sec latency 14.250 ms 2024-09-10T22:58:05.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 941942 16.75 MB/sec execute 180 sec latency 14.049 ms 2024-09-10T22:58:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:06 smithi120 ceph-mon[31247]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T22:58:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:06 smithi153 ceph-mon[34951]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T22:58:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:06 smithi176 ceph-mon[34982]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T22:58:06.625 INFO:tasks.workunit.client.0.smithi120.stdout: 1 945367 16.76 MB/sec execute 181 sec latency 25.831 ms 2024-09-10T22:58:07.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 948879 16.77 MB/sec execute 182 sec latency 12.019 ms 2024-09-10T22:58:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:08 smithi120 ceph-mon[31247]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T22:58:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:08 smithi153 ceph-mon[34951]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T22:58:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:08 smithi176 ceph-mon[34982]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T22:58:08.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 951053 16.72 MB/sec execute 183 sec latency 83.695 ms 2024-09-10T22:58:09.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 954287 16.73 MB/sec execute 184 sec latency 14.307 ms 2024-09-10T22:58:10.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:10 smithi120 ceph-mon[31247]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-10T22:58:10.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:10 smithi153 ceph-mon[34951]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-10T22:58:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:10 smithi176 ceph-mon[34982]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 450 op/s 2024-09-10T22:58:10.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 957179 16.73 MB/sec execute 185 sec latency 14.969 ms 2024-09-10T22:58:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 -- 172.21.15.120:0/1310866775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 msgr2=0x7eff38105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/1310866775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff38105590 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7eff34009900 tx=0x7eff3402a410 comp rx=0 tx=0).stop 2024-09-10T22:58:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 -- 172.21.15.120:0/1310866775 shutdown_connections 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/1310866775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 0x7eff3810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/1310866775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 0x7eff38107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/1310866775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff38105590 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 -- 172.21.15.120:0/1310866775 >> 172.21.15.120:0/1310866775 conn(0x7eff380fd140 msgr2=0x7eff380ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 -- 172.21.15.120:0/1310866775 shutdown_connections 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.078+0000 7eff3fc2a640 1 -- 172.21.15.120:0/1310866775 wait complete. 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 Processor -- start 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 -- start start 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 0x7eff3819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 0x7eff3819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810e5d0 con 0x7eff3806b410 2024-09-10T22:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810e450 con 0x7eff38105ad0 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3fc2a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7eff3810e750 con 0x7eff38108680 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45266/0 (socket says 172.21.15.120:45266) 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 -- 172.21.15.120:0/740897310 learned_addr learned my addr 172.21.15.120:0/740897310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 0x7eff3819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 0x7eff3819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 -- 172.21.15.120:0/740897310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 msgr2=0x7eff3819c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 0x7eff3819c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 -- 172.21.15.120:0/740897310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 msgr2=0x7eff3819b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 0x7eff3819b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 -- 172.21.15.120:0/740897310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3819cac0 con 0x7eff3806b410 2024-09-10T22:58:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.079+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7eff34005940 tx=0x7eff34004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.080+0000 7eff2effd640 1 -- 172.21.15.120:0/740897310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3403f030 con 0x7eff3806b410 2024-09-10T22:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.080+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3819cd50 con 0x7eff3806b410 2024-09-10T22:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.080+0000 7eff2effd640 1 -- 172.21.15.120:0/740897310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff34003cb0 con 0x7eff3806b410 2024-09-10T22:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.080+0000 7eff2effd640 1 -- 172.21.15.120:0/740897310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34035070 con 0x7eff3806b410 2024-09-10T22:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.080+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a3480 con 0x7eff3806b410 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 Processor -- start 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 -- start start 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec628c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 0x5557ec62b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.082+0000 7eff429fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5557ec5986f0 con 0x5557ec62bbd0 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff429fe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5557ec587b20 con 0x5557ec626860 2024-09-10T22:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff429fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557ec58b930 con 0x5557ec629190 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff2effd640 1 -- 172.21.15.120:0/740897310 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7eff3403b460 con 0x7eff3806b410 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45268/0 (socket says 172.21.15.120:45268) 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 -- 172.21.15.120:0/4011828617 learned_addr learned my addr 172.21.15.120:0/4011828617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 0x5557ec62b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec628c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 -- 172.21.15.120:0/4011828617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 msgr2=0x5557ec62b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 0x5557ec62b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 -- 172.21.15.120:0/4011828617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 msgr2=0x5557ec628c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec628c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 -- 172.21.15.120:0/4011828617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ec54d530 con 0x5557ec62bbd0 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec628c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 0x5557ec62b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7eff2800bcb0 tx=0x7eff280107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc7121a79d23f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff23fff640 1 -- 172.21.15.120:0/4011828617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28004030 con 0x5557ec62bbd0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff23fff640 1 -- 172.21.15.120:0/4011828617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff280041d0 con 0x5557ec62bbd0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.083+0000 7eff23fff640 1 -- 172.21.15.120:0/4011828617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28011680 con 0x5557ec62bbd0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff2effd640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 0x7eff240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 0x7eff240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff2effd640 1 -- 172.21.15.120:0/740897310 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7eff34106710 con 0x7eff3806b410 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 -- 172.21.15.120:0/4011828617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 msgr2=0x5557ec62e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7eff2800bcb0 tx=0x7eff280107a0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 0x7eff240b77a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff3819bd00 tx=0x7eff30009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 -- 172.21.15.120:0/4011828617 shutdown_connections 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec62bbd0 0x5557ec62e090 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec629190 0x5557ec62b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 --2- 172.21.15.120:0/4011828617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec628c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 -- 172.21.15.120:0/4011828617 >> 172.21.15.120:0/4011828617 conn(0x5557ec6219b0 msgr2=0x5557ec622740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 -- 172.21.15.120:0/4011828617 shutdown_connections 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.084+0000 7eff429fe740 1 -- 172.21.15.120:0/4011828617 wait complete. 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 -- start start 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 Processor -- start 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 0x5557ec62fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 0x5557ec631130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5557ec4bfab0 con 0x5557ec629190 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5557ec52d730 con 0x5557ec62bbd0 2024-09-10T22:58:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5557ec62ebb0 con 0x5557ec626860 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff429fe740 10 client.0 _ll_register_callbacks cb 0x5557ec5816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45272/0 (socket says 172.21.15.120:45272) 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff3d1c5640 1 -- 172.21.15.120:0/870718130 learned_addr learned my addr 172.21.15.120:0/870718130 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 0x5557ec62fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.085+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 0x5557ec631130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 -- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 msgr2=0x5557ec62fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 0x5557ec62fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 -- 172.21.15.120:0/870718130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 msgr2=0x5557ec631130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 0x5557ec631130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 -- 172.21.15.120:0/870718130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ec2e9790 con 0x5557ec629190 2024-09-10T22:58:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 0x5557ec62fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 0x5557ec631130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7eff28011010 tx=0x7eff2800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece32e5eba6987f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff217fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2802a060 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7eff28003800 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff280041d0 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff28003a00 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff429fe740 10 client.16458 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557ec362620 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.086+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff280041d0 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7eff28004a60 con 0x5557ec629190 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 10 client.16458 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 20 client.16458 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec63b390 0x5557ec63d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 4 client.16458 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ec59a640 con 0x5557ec63b390 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec63b390 0x5557ec63d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff217fa640 10 client.16458 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.087+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec63b390 0x5557ec63d780 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6144475a0ea185 server_cookie=ec20288d360fa419 in_seq=0 out_seq=0 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff217fa640 1 -- 172.21.15.120:0/870718130 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557ec59a640 con 0x5557ec63b390 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff217fa640 10 client.16458 handle_command_reply: tid=0 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 client.16458 shutdown 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 msgr2=0x5557ec630560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7eff28011010 tx=0x7eff2800bfd0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 shutdown_connections 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec63b390 0x5557ec63d780 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec62bbd0 0x5557ec631130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec630560 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 --2- 172.21.15.120:0/870718130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec626860 0x5557ec62fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff217fa640 0 client.16458 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.088+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 >> 172.21.15.120:0/870718130 conn(0x5557ec6219b0 msgr2=0x5557ec620bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.089+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 shutdown_connections 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.089+0000 7eff429fe740 1 -- 172.21.15.120:0/870718130 wait complete. 2024-09-10T22:58:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.089+0000 7eff429fe740 20 client.16458 trim_cache size 0 max 16384 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 Processor -- start 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 -- start start 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 0x5557ec624d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec62f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff381aa040 con 0x5557ec626860 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ac8c0 con 0x5557ec629190 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff429fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7eff381aa6b0 con 0x5557ec62bbd0 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45282/0 (socket says 172.21.15.120:45282) 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff3e1c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec62f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff3d9c6640 1 -- 172.21.15.120:0/819652029 learned_addr learned my addr 172.21.15.120:0/819652029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.103+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 0x5557ec624d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 -- 172.21.15.120:0/819652029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 msgr2=0x5557ec62f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec62f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 -- 172.21.15.120:0/819652029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 msgr2=0x5557ec624d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 0x5557ec624d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 -- 172.21.15.120:0/819652029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ec63dd40 con 0x5557ec626860 2024-09-10T22:58:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 0x5557ec624d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7eff340ccf40 tx=0x7eff34005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8898e665bae785 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff237fe640 1 -- 172.21.15.120:0/819652029 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34049040 con 0x5557ec626860 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff237fe640 1 -- 172.21.15.120:0/819652029 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff340ccac0 con 0x5557ec626860 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.104+0000 7eff237fe640 1 -- 172.21.15.120:0/819652029 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340c9220 con 0x5557ec626860 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- 172.21.15.120:0/819652029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 msgr2=0x5557ec6247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7eff340ccf40 tx=0x7eff34005e10 comp rx=0 tx=0).stop 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- 172.21.15.120:0/819652029 shutdown_connections 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec62f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec629190 0x5557ec624d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- 172.21.15.120:0/819652029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec626860 0x5557ec6247f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- 172.21.15.120:0/819652029 >> 172.21.15.120:0/819652029 conn(0x5557ec6219b0 msgr2=0x5557ec62fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- 172.21.15.120:0/819652029 shutdown_connections 2024-09-10T22:58:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- 172.21.15.120:0/819652029 wait complete. 2024-09-10T22:58:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 -- start start 2024-09-10T22:58:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 Processor -- start 2024-09-10T22:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.105+0000 7eff429fe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec636cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff381aad80 con 0x5557ec629190 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ac1f0 con 0x5557ec626860 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7eff381ab450 con 0x5557ec62bbd0 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 10 client.0 _ll_register_callbacks cb 0x7eff2800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45286/0 (socket says 172.21.15.120:45286) 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d9c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60834/0 (socket says 172.21.15.120:60834) 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 -- 172.21.15.120:0/367478402 learned_addr learned my addr 172.21.15.120:0/367478402 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3e1c7640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec636cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 -- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 msgr2=0x5557ec636cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec636cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 -- 172.21.15.120:0/367478402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 msgr2=0x5557ec637a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 -- 172.21.15.120:0/367478402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ec6374f0 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff3d1c5640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7eff28011010 tx=0x7eff28004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dfb491ac12fb6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff217fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28035070 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7eff28003800 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff340ccac0 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff28041070 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 10 client.16464 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.106+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557ec6a8290 con 0x5557ec629190 2024-09-10T22:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff340ccac0 con 0x5557ec629190 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7eff280051d0 con 0x5557ec629190 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 10 client.16464 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 20 client.16464 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec6a8c70 0x5557ec6ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 4 client.16464 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557ec5960f0 con 0x5557ec6a8c70 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec6a8c70 0x5557ec6ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff3d9c6640 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec6a8c70 0x5557ec6ab090 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119ab72c42750c13 server_cookie=25a94572017812e2 in_seq=0 out_seq=0 2024-09-10T22:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.107+0000 7eff217fa640 10 client.16464 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff217fa640 1 -- 172.21.15.120:0/367478402 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557ec5960f0 con 0x5557ec6a8c70 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff217fa640 10 client.16464 handle_command_reply: tid=0 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 client.16464 shutdown 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 msgr2=0x5557ec636720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7eff28011010 tx=0x7eff28004cc0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 shutdown_connections 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5557ec6a8c70 0x5557ec6ab090 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff217fa640 0 client.16464 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5557ec62bbd0 0x5557ec636cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5557ec629190 0x5557ec636720 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 --2- 172.21.15.120:0/367478402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557ec626860 0x5557ec637a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.303+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 >> 172.21.15.120:0/367478402 conn(0x5557ec6219b0 msgr2=0x5557ec625890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 shutdown_connections 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff429fe740 1 -- 172.21.15.120:0/367478402 wait complete. 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff429fe740 20 client.16464 trim_cache size 0 max 16384 2024-09-10T22:58:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:58:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:58:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:58:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:58:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 msgr2=0x7eff240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 0x7eff240b77a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff3819bd00 tx=0x7eff30009600 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 msgr2=0x7eff3819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7eff34005940 tx=0x7eff34004430 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 shutdown_connections 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7eff240b52e0 0x7eff240b77a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff38108680 0x7eff3819c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff38105ad0 0x7eff3819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.304+0000 7eff3fc2a640 1 --2- 172.21.15.120:0/740897310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff3806b410 0x7eff3819af80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.305+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 >> 172.21.15.120:0/740897310 conn(0x7eff380fd140 msgr2=0x7eff380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.305+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 shutdown_connections 2024-09-10T22:58:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:11.305+0000 7eff3fc2a640 1 -- 172.21.15.120:0/740897310 wait complete. 2024-09-10T22:58:11.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 959879 16.72 MB/sec execute 186 sec latency 130.534 ms 2024-09-10T22:58:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:12 smithi120 ceph-mon[31247]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-10T22:58:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:12 smithi153 ceph-mon[34951]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-10T22:58:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:12 smithi176 ceph-mon[34982]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-10T22:58:12.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 963160 16.73 MB/sec execute 187 sec latency 20.858 ms 2024-09-10T22:58:12.780 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:58:13.074 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:13.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 965430 16.69 MB/sec execute 188 sec latency 44.948 ms 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 -- 172.21.15.120:0/2616908779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f205010cf60 msgr2=0x7f205010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 --2- 172.21.15.120:0/2616908779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f205010cf60 0x7f205010f3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2048009ca0 tx=0x7f204802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 -- 172.21.15.120:0/2616908779 shutdown_connections 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 --2- 172.21.15.120:0/2616908779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f205010cf60 0x7f205010f3c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 --2- 172.21.15.120:0/2616908779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050104420 0x7f205010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 --2- 172.21.15.120:0/2616908779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 -- 172.21.15.120:0/2616908779 >> 172.21.15.120:0/2616908779 conn(0x7f20500fd560 msgr2=0x7f20500ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 -- 172.21.15.120:0/2616908779 shutdown_connections 2024-09-10T22:58:13.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.618+0000 7f205559f640 1 -- 172.21.15.120:0/2616908779 wait complete. 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 Processor -- start 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 -- start start 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f205010cf60 0x7f205019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2050112bd0 con 0x7f2050104420 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2050112a50 con 0x7f2050103b00 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f205559f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2050112d50 con 0x7f205010cf60 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54526/0 (socket says 172.21.15.120:54526) 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 -- 172.21.15.120:0/190878470 learned_addr learned my addr 172.21.15.120:0/190878470 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f204ffff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2054d9e640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f205010cf60 0x7f205019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 -- 172.21.15.120:0/190878470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f205010cf60 msgr2=0x7f205019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f205010cf60 0x7f205019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 -- 172.21.15.120:0/190878470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 msgr2=0x7f2050199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.619+0000 7f2047fff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f2047fff640 1 -- 172.21.15.120:0/190878470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205019f0b0 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f204ffff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f2047fff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f203800d6b0 tx=0x7f203800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f204dffb640 1 -- 172.21.15.120:0/190878470 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038017070 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205019f3a0 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f204dffb640 1 -- 172.21.15.120:0/190878470 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2038010460 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f204dffb640 1 -- 172.21.15.120:0/190878470 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800f620 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.620+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20501a5a60 con 0x7f2050104420 2024-09-10T22:58:13.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 Processor -- start 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 -- start start 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x56064115b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641169f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606410d1e50 con 0x56064115bba0 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606410c1280 con 0x560641167b30 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2058bd6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606410c5090 con 0x56064115b280 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54536/0 (socket says 172.21.15.120:54536) 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 -- 172.21.15.120:0/3780439208 learned_addr learned my addr 172.21.15.120:0/3780439208 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2054d9e640 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x56064115b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 -- 172.21.15.120:0/3780439208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 msgr2=0x56064115b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x56064115b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 -- 172.21.15.120:0/3780439208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 msgr2=0x560641169f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641169f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f204ffff640 1 -- 172.21.15.120:0/3780439208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560640e1f650 con 0x56064115bba0 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.623+0000 7f2054d9e640 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x56064115b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f204ffff640 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f20400115f0 tx=0x7f2040011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282753523b01b830 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f204dffb640 1 -- 172.21.15.120:0/190878470 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2038010930 con 0x7f2050104420 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f20467fc640 1 -- 172.21.15.120:0/3780439208 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204001e070 con 0x56064115bba0 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f20467fc640 1 -- 172.21.15.120:0/3780439208 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204000faf0 con 0x56064115bba0 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f20467fc640 1 -- 172.21.15.120:0/3780439208 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040019b50 con 0x56064115bba0 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f2058bd6740 1 -- 172.21.15.120:0/3780439208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 msgr2=0x5606411675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.624+0000 7f2058bd6740 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f20400115f0 tx=0x7f2040011ab0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 -- 172.21.15.120:0/3780439208 shutdown_connections 2024-09-10T22:58:13.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641169f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115bba0 0x5606411675f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 --2- 172.21.15.120:0/3780439208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x56064115b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 -- 172.21.15.120:0/3780439208 >> 172.21.15.120:0/3780439208 conn(0x5606411590b0 msgr2=0x56064115a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 -- 172.21.15.120:0/3780439208 shutdown_connections 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 -- 172.21.15.120:0/3780439208 wait complete. 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f204dffb640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 0x7f203c0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f204ffff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 0x7f203c0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 -- start start 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f204dffb640 1 -- 172.21.15.120:0/190878470 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f20380e57c0 con 0x7f2050104420 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.625+0000 7f2058bd6740 1 Processor -- start 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f204ffff640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 0x7f203c0b7770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20400115c0 tx=0x7f2040005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115bba0 0x560641164860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641164e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560640f22700 con 0x56064115b280 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56064105c110 con 0x560641167b30 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56064116a7b0 con 0x56064115bba0 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2058bd6740 10 client.0 _ll_register_callbacks cb 0x560641075990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54540/0 (socket says 172.21.15.120:54540) 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f204ffff640 1 -- 172.21.15.120:0/1572628842 learned_addr learned my addr 172.21.15.120:0/1572628842 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 -- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115bba0 msgr2=0x560641164860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2047fff640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115bba0 0x560641164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115bba0 0x560641164860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 -- 172.21.15.120:0/1572628842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 msgr2=0x560641167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f2054d9e640 1 -- 172.21.15.120:0/1572628842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560641089c40 con 0x560641167b30 2024-09-10T22:58:13.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.626+0000 7f204ffff640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:13.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f2054d9e640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641164e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2048005e30 tx=0x7f204802f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5022e34f8468bc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048031660 con 0x560641167b30 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2048031800 con 0x560641167b30 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f204000faf0 con 0x560641167b30 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2048004b40 con 0x560641167b30 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f2058bd6740 10 client.25904 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56064117a560 con 0x560641167b30 2024-09-10T22:58:13.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204000faf0 con 0x560641167b30 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.627+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f204804f030 con 0x560641167b30 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 10 client.25904 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 20 client.25904 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56064117b9c0 0x56064117de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 4 client.25904 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:13.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.628+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606410d3da0 con 0x56064117b9c0 2024-09-10T22:58:13.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.655+0000 7f204ffff640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56064117b9c0 0x56064117de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.656+0000 7f202f7fe640 10 client.25904 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:13.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.656+0000 7f204ffff640 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56064117b9c0 0x56064117de20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6a87885a27be69 server_cookie=48cf216b89fafcdb in_seq=0 out_seq=0 2024-09-10T22:58:13.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.656+0000 7f202f7fe640 1 -- 172.21.15.120:0/1572628842 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606410d3da0 con 0x56064117b9c0 2024-09-10T22:58:13.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.656+0000 7f202f7fe640 10 client.25904 handle_command_reply: tid=0 2024-09-10T22:58:13.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.656+0000 7f2058bd6740 1 client.25904 shutdown 2024-09-10T22:58:13.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 msgr2=0x560641164e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641164e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2048005e30 tx=0x7f204802f4c0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 shutdown_connections 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56064117b9c0 0x56064117de20 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560641167b30 0x560641164e90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115bba0 0x560641164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f202f7fe640 0 client.25904 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 --2- 172.21.15.120:0/1572628842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x560641167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 >> 172.21.15.120:0/1572628842 conn(0x5606411590b0 msgr2=0x56064115ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 shutdown_connections 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 1 -- 172.21.15.120:0/1572628842 wait complete. 2024-09-10T22:58:13.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.657+0000 7f2058bd6740 20 client.25904 trim_cache size 0 max 16384 2024-09-10T22:58:13.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.674+0000 7f2058bd6740 1 Processor -- start 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.674+0000 7f2058bd6740 1 -- start start 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x5606411589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x5606411665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 0x560641168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20501a9e80 con 0x560641167b30 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20501ac6b0 con 0x56064115bba0 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2058bd6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20501aa4a0 con 0x56064115b280 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f204ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x5606411589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2054d9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 0x560641168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f204ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x5606411589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60138/0 (socket says 172.21.15.120:60138) 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x5606411665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f204ffff640 1 -- 172.21.15.120:0/2832488136 learned_addr learned my addr 172.21.15.120:0/2832488136 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 -- 172.21.15.120:0/2832488136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 msgr2=0x5606411589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x5606411589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 -- 172.21.15.120:0/2832488136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 msgr2=0x560641168f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 0x560641168f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 -- 172.21.15.120:0/2832488136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560641159b30 con 0x56064115bba0 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2054d9e640 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 0x560641168f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.675+0000 7f2047fff640 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x5606411665a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560641154d10 tx=0x7f20380abb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5977e43a3963f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2045ffb640 1 -- 172.21.15.120:0/2832488136 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038021070 con 0x56064115bba0 2024-09-10T22:58:13.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2045ffb640 1 -- 172.21.15.120:0/2832488136 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20380b0a90 con 0x56064115bba0 2024-09-10T22:58:13.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2045ffb640 1 -- 172.21.15.120:0/2832488136 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380aad30 con 0x56064115bba0 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 -- 172.21.15.120:0/2832488136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 msgr2=0x5606411665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x5606411665a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560641154d10 tx=0x7f20380abb70 comp rx=0 tx=0).stop 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 -- 172.21.15.120:0/2832488136 shutdown_connections 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560641167b30 0x560641168f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x5606411665a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 --2- 172.21.15.120:0/2832488136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56064115b280 0x5606411589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 -- 172.21.15.120:0/2832488136 >> 172.21.15.120:0/2832488136 conn(0x5606411590b0 msgr2=0x560641165e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 -- 172.21.15.120:0/2832488136 shutdown_connections 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.676+0000 7f2058bd6740 1 -- 172.21.15.120:0/2832488136 wait complete. 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 -- start start 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 Processor -- start 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x56064116ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 0x560641160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20501aab70 con 0x56064115b280 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20501abfe0 con 0x56064115bba0 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.677+0000 7f2058bd6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20501ab240 con 0x560641167b30 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f2058bd6740 10 client.0 _ll_register_callbacks cb 0x7f204802f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54548/0 (socket says 172.21.15.120:54548) 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 -- 172.21.15.120:0/701479491 learned_addr learned my addr 172.21.15.120:0/701479491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f2054d9e640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 0x560641160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f2047fff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x56064116ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 -- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 msgr2=0x560641160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 0x560641160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 -- 172.21.15.120:0/701479491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 msgr2=0x56064116ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x56064116ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 -- 172.21.15.120:0/701479491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560641170740 con 0x56064115b280 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f2054d9e640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 0x560641160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:13.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f204ffff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2040011510 tx=0x7f2040020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5ff358b74666b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f202f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040028040 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2040009cc0 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.678+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20380b0a90 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.679+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f204001e040 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.679+0000 7f2058bd6740 10 client.16482 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.679+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560641162030 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.679+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20380b0a90 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f204003a070 con 0x56064115b280 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 10 client.16482 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:13.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 20 client.16482 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606411625a0 0x560641175b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 4 client.16482 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606410cf850 con 0x5606411625a0 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2047fff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606411625a0 0x560641175b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f202f7fe640 10 client.16482 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f2047fff640 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606411625a0 0x560641175b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d3c87fbad3e897 server_cookie=e533b573b6cfc7e in_seq=0 out_seq=0 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f202f7fe640 1 -- 172.21.15.120:0/701479491 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x5606410cf850 con 0x5606411625a0 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.680+0000 7f202f7fe640 10 client.16482 handle_command_reply: tid=0 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 client.16482 shutdown 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 msgr2=0x56064116fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2040011510 tx=0x7f2040020000 comp rx=0 tx=0).stop 2024-09-10T22:58:13.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 shutdown_connections 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606411625a0 0x560641175b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560641167b30 0x560641160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56064115bba0 0x56064116ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 --2- 172.21.15.120:0/701479491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56064115b280 0x56064116fa70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f202f7fe640 0 client.16482 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 >> 172.21.15.120:0/701479491 conn(0x5606411590b0 msgr2=0x5606411666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 shutdown_connections 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 1 -- 172.21.15.120:0/701479491 wait complete. 2024-09-10T22:58:13.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f2058bd6740 20 client.16482 trim_cache size 0 max 16384 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:58:13.690 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:58:13.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.694 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:13.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:13.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:13.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:13.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:13.696 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 msgr2=0x7f203c0b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 0x7f203c0b7770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20400115c0 tx=0x7f2040005850 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 msgr2=0x7f2050199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.681+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f203800d6b0 tx=0x7f203800db70 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 shutdown_connections 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f203c0b52b0 0x7f203c0b7770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f205010cf60 0x7f205019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2050104420 0x7f2050199540 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 --2- 172.21.15.120:0/190878470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2050103b00 0x7f2050199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 >> 172.21.15.120:0/190878470 conn(0x7f20500fd560 msgr2=0x7f205010f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 shutdown_connections 2024-09-10T22:58:13.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:13.682+0000 7f205559f640 1 -- 172.21.15.120:0/190878470 wait complete. 2024-09-10T22:58:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:14 smithi120 ceph-mon[31247]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T22:58:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:14 smithi153 ceph-mon[34951]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T22:58:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:14 smithi176 ceph-mon[34982]: pgmap v520: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T22:58:14.617 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:58:14.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:14.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 967831 16.68 MB/sec execute 189 sec latency 49.607 ms 2024-09-10T22:58:14.882 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:15.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 -- 172.21.15.120:0/461500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c108020 msgr2=0x7f917c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 --2- 172.21.15.120:0/461500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c108020 0x7f917c10e820 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ca0 tx=0x7f917402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 -- 172.21.15.120:0/461500482 shutdown_connections 2024-09-10T22:58:15.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 --2- 172.21.15.120:0/461500482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c108020 0x7f917c10e820 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 --2- 172.21.15.120:0/461500482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 0x7f917c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 --2- 172.21.15.120:0/461500482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c101c90 0x7f917c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 -- 172.21.15.120:0/461500482 >> 172.21.15.120:0/461500482 conn(0x7f917c0fd700 msgr2=0x7f917c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 -- 172.21.15.120:0/461500482 shutdown_connections 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.407+0000 7f91819d2640 1 -- 172.21.15.120:0/461500482 wait complete. 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 Processor -- start 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 -- start start 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 0x7f917c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 0x7f917c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.408+0000 7f91819d2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f917c111ce0 con 0x7f917c101c90 2024-09-10T22:58:15.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91819d2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f917c111b60 con 0x7f917c108020 2024-09-10T22:58:15.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91819d2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f917c111e60 con 0x7f917c1076a0 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54574/0 (socket says 172.21.15.120:54574) 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 -- 172.21.15.120:0/4044919645 learned_addr learned my addr 172.21.15.120:0/4044919645 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917a7fc640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 0x7f917c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917b7fe640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 0x7f917c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 -- 172.21.15.120:0/4044919645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 msgr2=0x7f917c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 0x7f917c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 -- 172.21.15.120:0/4044919645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 msgr2=0x7f917c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 0x7f917c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 -- 172.21.15.120:0/4044919645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c19e1f0 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917b7fe640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 0x7f917c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f917affd640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c017070 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916c009c30 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917c19e4e0 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c00f620 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.409+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917c1a4ba0 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.410+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917c102070 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.412+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f916c010460 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.413+0000 7f91809d0640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 0x7f91640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.413+0000 7f917a7fc640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 0x7f91640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.413+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f916c0e5840 con 0x7f917c101c90 2024-09-10T22:58:15.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.413+0000 7f917a7fc640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 0x7f91640b7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f917c107160 tx=0x7f917000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.415+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f916c0ac6d0 con 0x7f917c101c90 2024-09-10T22:58:15.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 970993 16.68 MB/sec execute 190 sec latency 25.868 ms 2024-09-10T22:58:15.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.647+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f917c199270 con 0x7f917c101c90 2024-09-10T22:58:15.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.648+0000 7f91809d0640 1 -- 172.21.15.120:0/4044919645 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f916c009de0 con 0x7f917c101c90 2024-09-10T22:58:15.649 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:58:15.649 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:58:15.650 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 msgr2=0x7f91640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 0x7f91640b7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f917c107160 tx=0x7f917000b410 comp rx=0 tx=0).stop 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 msgr2=0x7f917c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00de10 comp rx=0 tx=0).stop 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 shutdown_connections 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91640b5360 0x7f91640b7820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f917c108020 0x7f917c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.650+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f917c1076a0 0x7f917c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.651+0000 7f91819d2640 1 --2- 172.21.15.120:0/4044919645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f917c101c90 0x7f917c198140 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.651+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 >> 172.21.15.120:0/4044919645 conn(0x7f917c0fd700 msgr2=0x7f917c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.651+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 shutdown_connections 2024-09-10T22:58:15.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:15.651+0000 7f91819d2640 1 -- 172.21.15.120:0/4044919645 wait complete. 2024-09-10T22:58:16.347 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:16 smithi120 ceph-mon[31247]: pgmap v521: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-10T22:58:16.348 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:16 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4044919645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:16.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:58:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:16 smithi153 ceph-mon[34951]: pgmap v521: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-10T22:58:16.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:16 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4044919645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:16 smithi176 ceph-mon[34982]: pgmap v521: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 3.8 MiB/s wr, 344 op/s 2024-09-10T22:58:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:16 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4044919645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:16.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 974270 16.69 MB/sec execute 191 sec latency 12.294 ms 2024-09-10T22:58:16.683 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 -- 172.21.15.120:0/668047223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63201076a0 msgr2=0x7f6320107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 --2- 172.21.15.120:0/668047223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63201076a0 0x7f6320107ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6310009900 tx=0x7f631002a410 comp rx=0 tx=0).stop 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 -- 172.21.15.120:0/668047223 shutdown_connections 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 --2- 172.21.15.120:0/668047223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6320108020 0x7f632010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 --2- 172.21.15.120:0/668047223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63201076a0 0x7f6320107ae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 --2- 172.21.15.120:0/668047223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320101c90 0x7f6320102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 -- 172.21.15.120:0/668047223 >> 172.21.15.120:0/668047223 conn(0x7f63200fd700 msgr2=0x7f63200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 -- 172.21.15.120:0/668047223 shutdown_connections 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.183+0000 7f632763c640 1 -- 172.21.15.120:0/668047223 wait complete. 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 Processor -- start 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 -- start start 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63201076a0 0x7f63201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320108020 0x7f6320198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6320111cc0 con 0x7f6320101c90 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6320111b40 con 0x7f6320108020 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f632763c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6320111e40 con 0x7f63201076a0 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54596/0 (socket says 172.21.15.120:54596) 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 -- 172.21.15.120:0/4100826724 learned_addr learned my addr 172.21.15.120:0/4100826724 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f6324bd7640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63201076a0 0x7f63201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f6325bd9640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320108020 0x7f6320198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 -- 172.21.15.120:0/4100826724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63201076a0 msgr2=0x7f63201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63201076a0 0x7f63201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 -- 172.21.15.120:0/4100826724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320108020 msgr2=0x7f6320198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320108020 0x7f6320198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.184+0000 7f63253d8640 1 -- 172.21.15.120:0/4100826724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63201a7680 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f63253d8640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e9f0 tx=0x7f631c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f63167fc640 1 -- 172.21.15.120:0/4100826724 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00cd80 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63201a7970 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f63167fc640 1 -- 172.21.15.120:0/4100826724 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c0108e0 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f63167fc640 1 -- 172.21.15.120:0/4100826724 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c018980 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.185+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63201a7ed0 con 0x7f6320101c90 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 Processor -- start 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 -- start start 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9db0 0x562d285d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f63167fc640 1 -- 172.21.15.120:0/4100826724 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f631c018b20 con 0x7f6320101c90 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d2853fea0 con 0x562d285c9db0 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d2852f2d0 con 0x562d285d5d40 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6329a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562d285330e0 con 0x562d285c9490 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6325bd9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9db0 0x562d285d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6325bd9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60194/0 (socket says 172.21.15.120:60194) 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6325bd9640 1 -- 172.21.15.120:0/3132795197 learned_addr learned my addr 172.21.15.120:0/3132795197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 -- 172.21.15.120:0/3132795197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 msgr2=0x562d285c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 -- 172.21.15.120:0/3132795197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9db0 msgr2=0x562d285d5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9db0 0x562d285d5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.188+0000 7f6324bd7640 1 -- 172.21.15.120:0/3132795197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2828d650 con 0x562d285d5d40 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d8130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f631002aa00 tx=0x7f6310005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9191cc69b32df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f63167fc640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 0x7f630c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f630b7fe640 1 -- 172.21.15.120:0/3132795197 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310035040 con 0x562d285d5d40 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f630b7fe640 1 -- 172.21.15.120:0/3132795197 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63100054f0 con 0x562d285d5d40 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f630b7fe640 1 -- 172.21.15.120:0/3132795197 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310004e40 con 0x562d285d5d40 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f6325bd9640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 0x7f630c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.189+0000 7f63167fc640 1 -- 172.21.15.120:0/4100826724 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f631c0e5ed0 con 0x7f6320101c90 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- 172.21.15.120:0/3132795197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 msgr2=0x562d285d8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6325bd9640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 0x7f630c0b77a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63180116b0 tx=0x7f6318009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d8130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f631002aa00 tx=0x7f6310005c00 comp rx=0 tx=0).stop 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- 172.21.15.120:0/3132795197 shutdown_connections 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d8130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9db0 0x562d285d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 --2- 172.21.15.120:0/3132795197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- 172.21.15.120:0/3132795197 >> 172.21.15.120:0/3132795197 conn(0x562d285c72c0 msgr2=0x562d285c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- 172.21.15.120:0/3132795197 shutdown_connections 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- 172.21.15.120:0/3132795197 wait complete. 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 -- start start 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.190+0000 7f6329a11740 1 Processor -- start 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9db0 0x562d285d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d28390700 con 0x562d285c9490 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d284ca160 con 0x562d285d5d40 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562d285d8960 con 0x562d285c9db0 2024-09-10T22:58:17.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6329a11740 10 client.0 _ll_register_callbacks cb 0x562d284e39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54604/0 (socket says 172.21.15.120:54604) 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 -- 172.21.15.120:0/381857603 learned_addr learned my addr 172.21.15.120:0/381857603 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6324bd7640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9db0 0x562d285d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6325bd9640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 -- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9db0 msgr2=0x562d285d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9db0 0x562d285d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 -- 172.21.15.120:0/381857603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 msgr2=0x562d285d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 -- 172.21.15.120:0/381857603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d284f7c90 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f63253d8640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f631c000c00 tx=0x7f631c0e95a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a016193f956b62c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.191+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c0ac0c0 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f631c010430 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63100054f0 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f631c0abee0 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 10 client.16500 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d285e4320 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63100054f0 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f631c0e9020 con 0x562d285c9490 2024-09-10T22:58:17.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 10 client.16500 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 20 client.16500 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d285e56c0 0x562d285e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 4 client.16500 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d28541df0 con 0x562d285e56c0 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.192+0000 7f6324bd7640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d285e56c0 0x562d285e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.193+0000 7f6324bd7640 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d285e56c0 0x562d285e7ab0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d22cbbb3a84869 server_cookie=d6cb91fd42905898 in_seq=0 out_seq=0 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.193+0000 7f6308ff9640 10 client.16500 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6308ff9640 1 -- 172.21.15.120:0/381857603 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d28541df0 con 0x562d285e56c0 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6308ff9640 10 client.16500 handle_command_reply: tid=0 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 client.16500 shutdown 2024-09-10T22:58:17.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 msgr2=0x562d285d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f631c000c00 tx=0x7f631c0e95a0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 shutdown_connections 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d285e56c0 0x562d285e7ab0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285d5d40 0x562d285d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9db0 0x562d285d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6308ff9640 0 client.16500 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 --2- 172.21.15.120:0/381857603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285d5650 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 >> 172.21.15.120:0/381857603 conn(0x562d285c72c0 msgr2=0x562d285cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 shutdown_connections 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 1 -- 172.21.15.120:0/381857603 wait complete. 2024-09-10T22:58:17.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.194+0000 7f6329a11740 20 client.16500 trim_cache size 0 max 16384 2024-09-10T22:58:17.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 Processor -- start 2024-09-10T22:58:17.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 -- start start 2024-09-10T22:58:17.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63201ae4f0 con 0x562d285d5d40 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63201b0d70 con 0x562d285c9db0 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6329a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63201aeb60 con 0x562d285c9490 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6325bd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6325bd9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54610/0 (socket says 172.21.15.120:54610) 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6325bd9640 1 -- 172.21.15.120:0/2988938112 learned_addr learned my addr 172.21.15.120:0/2988938112 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6324bd7640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f63253d8640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6324bd7640 1 -- 172.21.15.120:0/2988938112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 msgr2=0x562d285e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.208+0000 7f6324bd7640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f6324bd7640 1 -- 172.21.15.120:0/2988938112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 msgr2=0x562d285c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f6324bd7640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f6324bd7640 1 -- 172.21.15.120:0/2988938112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d285c7d40 con 0x562d285c9db0 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f6325bd9640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f6324bd7640 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285c8a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f631002a450 tx=0x7f6310005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec319a2da87992a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f630affd640 1 -- 172.21.15.120:0/2988938112 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310035040 con 0x562d285c9db0 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f630affd640 1 -- 172.21.15.120:0/2988938112 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6310003aa0 con 0x562d285c9db0 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.209+0000 7f630affd640 1 -- 172.21.15.120:0/2988938112 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310031970 con 0x562d285c9db0 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- 172.21.15.120:0/2988938112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 msgr2=0x562d285c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285c8a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f631002a450 tx=0x7f6310005a20 comp rx=0 tx=0).stop 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- 172.21.15.120:0/2988938112 shutdown_connections 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285d5d40 0x562d285c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285c8a20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 --2- 172.21.15.120:0/2988938112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285c9490 0x562d285e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- 172.21.15.120:0/2988938112 >> 172.21.15.120:0/2988938112 conn(0x562d285c72c0 msgr2=0x562d285c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- 172.21.15.120:0/2988938112 shutdown_connections 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- 172.21.15.120:0/2988938112 wait complete. 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 -- start start 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 Processor -- start 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.210+0000 7f6329a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285d5d40 0x562d285d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63201af230 con 0x562d285c9490 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63201b06a0 con 0x562d285c9db0 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63201af900 con 0x562d285d5d40 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 10 client.0 _ll_register_callbacks cb 0x7f631c0e96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54624/0 (socket says 172.21.15.120:54624) 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6324bd7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 -- 172.21.15.120:0/3923701244 learned_addr learned my addr 172.21.15.120:0/3923701244 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6325bd9640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285d5d40 0x562d285d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 -- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285d5d40 msgr2=0x562d285d8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285d5d40 0x562d285d8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 -- 172.21.15.120:0/3923701244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 msgr2=0x562d285de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 -- 172.21.15.120:0/3923701244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d285de700 con 0x562d285c9490 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f63253d8640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ac470 tx=0x7f631c0a8520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc5e62414222b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c0ac0c0 con 0x562d285c9490 2024-09-10T22:58:17.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f631c0ab440 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.211+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6310003aa0 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f631c010720 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 10 client.16506 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d28652940 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6310003aa0 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f631c0ab640 con 0x562d285c9490 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 10 client.16506 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 20 client.16506 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d286530c0 0x562d286554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 4 client.16506 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d2853d8a0 con 0x562d286530c0 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.212+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d286530c0 0x562d286554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:17.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6308ff9640 10 client.16506 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6324bd7640 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d286530c0 0x562d286554e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87fff7ab4be42f0 server_cookie=cabed8b3c411ded7 in_seq=0 out_seq=0 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6308ff9640 1 -- 172.21.15.120:0/3923701244 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d2853d8a0 con 0x562d286530c0 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6308ff9640 10 client.16506 handle_command_reply: tid=0 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 client.16506 shutdown 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 msgr2=0x562d285ddc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ac470 tx=0x7f631c0a8520 comp rx=0 tx=0).stop 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 shutdown_connections 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d286530c0 0x562d286554e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d285d5d40 0x562d285d8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d285c9db0 0x562d285de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 --2- 172.21.15.120:0/3923701244 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d285c9490 0x562d285ddc00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 >> 172.21.15.120:0/3923701244 conn(0x562d285c72c0 msgr2=0x562d285d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6308ff9640 0 client.16506 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:17.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 shutdown_connections 2024-09-10T22:58:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.213+0000 7f6329a11740 1 -- 172.21.15.120:0/3923701244 wait complete. 2024-09-10T22:58:17.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f6329a11740 20 client.16506 trim_cache size 0 max 16384 2024-09-10T22:58:17.228 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 msgr2=0x7f630c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 0x7f630c0b77a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63180116b0 tx=0x7f6318009500 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 msgr2=0x7f6320197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e9f0 tx=0x7f631c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 shutdown_connections 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f630c0b52e0 0x7f630c0b77a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6320108020 0x7f6320198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63201076a0 0x7f63201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 --2- 172.21.15.120:0/4100826724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6320101c90 0x7f6320197ea0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 >> 172.21.15.120:0/4100826724 conn(0x7f63200fd700 msgr2=0x7f63200ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 shutdown_connections 2024-09-10T22:58:17.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:17.214+0000 7f632763c640 1 -- 172.21.15.120:0/4100826724 wait complete. 2024-09-10T22:58:17.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 977493 16.70 MB/sec execute 192 sec latency 23.715 ms 2024-09-10T22:58:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:18 smithi120 ceph-mon[31247]: pgmap v522: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-09-10T22:58:18.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:18 smithi153 ceph-mon[34951]: pgmap v522: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-09-10T22:58:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:18 smithi176 ceph-mon[34982]: pgmap v522: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-09-10T22:58:18.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 980654 16.70 MB/sec execute 193 sec latency 76.245 ms 2024-09-10T22:58:18.911 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:58:18.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:58:19.193 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:19.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 983831 16.70 MB/sec execute 194 sec latency 17.970 ms 2024-09-10T22:58:19.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 -- 172.21.15.120:0/1606918451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc64109170 msgr2=0x7fdc6410f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 --2- 172.21.15.120:0/1606918451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc64109170 0x7fdc6410f970 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009ca0 tx=0x7fdc5c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 -- 172.21.15.120:0/1606918451 shutdown_connections 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 --2- 172.21.15.120:0/1606918451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc64109170 0x7fdc6410f970 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 --2- 172.21.15.120:0/1606918451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc641087f0 0x7fdc64108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 --2- 172.21.15.120:0/1606918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64102de0 0x7fdc641031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 -- 172.21.15.120:0/1606918451 >> 172.21.15.120:0/1606918451 conn(0x7fdc640fe830 msgr2=0x7fdc64100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 -- 172.21.15.120:0/1606918451 shutdown_connections 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.686+0000 7fdc68980640 1 -- 172.21.15.120:0/1606918451 wait complete. 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.687+0000 7fdc68980640 1 Processor -- start 2024-09-10T22:58:19.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.687+0000 7fdc68980640 1 -- start start 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 0x7fdc641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64109170 0x7fdc6419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112df0 con 0x7fdc641087f0 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112c70 con 0x7fdc64109170 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc68980640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112f70 con 0x7fdc64102de0 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 0x7fdc641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60252/0 (socket says 172.21.15.120:60252) 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 0x7fdc641997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54632/0 (socket says 172.21.15.120:54632) 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 -- 172.21.15.120:0/709795329 learned_addr learned my addr 172.21.15.120:0/709795329 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:19.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64109170 0x7fdc6419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 -- 172.21.15.120:0/709795329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64109170 msgr2=0x7fdc6419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64109170 0x7fdc6419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 -- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 msgr2=0x7fdc641997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 0x7fdc641997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 -- 172.21.15.120:0/709795329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6419f310 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.688+0000 7fdc6259c640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc54007b20 tx=0x7fdc54010050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.689+0000 7fdc5bfff640 1 -- 172.21.15.120:0/709795329 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5401b020 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.689+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6419f600 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.689+0000 7fdc5bfff640 1 -- 172.21.15.120:0/709795329 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5400e2e0 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.689+0000 7fdc5bfff640 1 -- 172.21.15.120:0/709795329 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400c8a0 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.689+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc64075550 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.691+0000 7fdc6ad55740 1 Processor -- start 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.691+0000 7fdc5bfff640 1 -- 172.21.15.120:0/709795329 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdc54016070 con 0x7fdc64102de0 2024-09-10T22:58:19.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.691+0000 7fdc6ad55740 1 -- start start 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e986d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595e97d7e60 con 0x5595e986d790 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595e97c7290 con 0x5595e9860ee0 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5595e97cb0a0 con 0x5595e9861800 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e986d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54644/0 (socket says 172.21.15.120:54644) 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e986d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60268/0 (socket says 172.21.15.120:60268) 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 -- 172.21.15.120:0/1560312098 learned_addr learned my addr 172.21.15.120:0/1560312098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 -- 172.21.15.120:0/1560312098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 msgr2=0x5595e986d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e986d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 -- 172.21.15.120:0/1560312098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 msgr2=0x5595e98612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 -- 172.21.15.120:0/1560312098 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e9525930 con 0x5595e986d790 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc5bfff640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 0x7fdc500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b770 tx=0x7fdc4c00bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cfa65ed22a206a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.692+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 0x7fdc500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc5bfff640 1 -- 172.21.15.120:0/709795329 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc540e8ff0 con 0x7fdc64102de0 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc58d99640 1 -- 172.21.15.120:0/1560312098 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003dd0 con 0x5595e986d790 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc58d99640 1 -- 172.21.15.120:0/1560312098 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c015070 con 0x5595e986d790 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc58d99640 1 -- 172.21.15.120:0/1560312098 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c004d80 con 0x5595e986d790 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 -- 172.21.15.120:0/1560312098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 msgr2=0x5595e986fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b770 tx=0x7fdc4c00bc30 comp rx=0 tx=0).stop 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 0x7fdc500b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02f500 tx=0x7fdc5c005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 -- 172.21.15.120:0/1560312098 shutdown_connections 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e986d790 0x5595e986fb80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e986d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/1560312098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 -- 172.21.15.120:0/1560312098 >> 172.21.15.120:0/1560312098 conn(0x5595e985ed10 msgr2=0x5595e985feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.693+0000 7fdc6ad55740 1 -- 172.21.15.120:0/1560312098 shutdown_connections 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 -- 172.21.15.120:0/1560312098 wait complete. 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 -- start start 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 Processor -- start 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 0x5595e986a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e986b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5595e9628890 con 0x5595e9861800 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5595e9761d40 con 0x5595e986d790 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5595e9870440 con 0x5595e9860ee0 2024-09-10T22:58:19.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6ad55740 10 client.0 _ll_register_callbacks cb 0x5595e97c5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54646/0 (socket says 172.21.15.120:54646) 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 0x5595e986a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.694+0000 7fdc59d9b640 1 -- 172.21.15.120:0/478620822 learned_addr learned my addr 172.21.15.120:0/478620822 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e986b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 -- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 msgr2=0x5595e986a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 0x5595e986a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 -- 172.21.15.120:0/478620822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 msgr2=0x5595e986b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e986b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 -- 172.21.15.120:0/478620822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e9879840 con 0x5595e9861800 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e986b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc6259c640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 0x5595e986a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:19.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005cb0 tx=0x7fdc4c004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd14cc638cd38e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003e60 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc4c011440 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4c015070 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c011640 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc6ad55740 10 client.16518 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595e987c1f0 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.695+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c015070 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdc4c024980 con 0x5595e9861800 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 10 client.16518 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 20 client.16518 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e987d650 0x5595e987fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 4 client.16518 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e97d9db0 con 0x5595e987d650 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6259c640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e987d650 0x5595e987fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc49ffb640 10 client.16518 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.696+0000 7fdc6259c640 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e987d650 0x5595e987fa40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4e181ac4e937e7 server_cookie=ca773b8b89f2f8fe in_seq=0 out_seq=0 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc49ffb640 1 -- 172.21.15.120:0/478620822 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595e97d9db0 con 0x5595e987d650 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc49ffb640 10 client.16518 handle_command_reply: tid=0 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 client.16518 shutdown 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 msgr2=0x5595e986acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005cb0 tx=0x7fdc4c004380 comp rx=0 tx=0).stop 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 shutdown_connections 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e987d650 0x5595e987fa40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e986b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc49ffb640 0 client.16518 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:19.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e986acb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/478620822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9860ee0 0x5595e986a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 >> 172.21.15.120:0/478620822 conn(0x5595e985ed10 msgr2=0x5595e985df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 shutdown_connections 2024-09-10T22:58:19.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.697+0000 7fdc6ad55740 1 -- 172.21.15.120:0/478620822 wait complete. 2024-09-10T22:58:19.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.698+0000 7fdc6ad55740 20 client.16518 trim_cache size 0 max 16384 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.719+0000 7fdc6ad55740 1 Processor -- start 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.719+0000 7fdc6ad55740 1 -- start start 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9860ee0 0x5595e9867b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e98851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641aa0c0 con 0x5595e9860ee0 2024-09-10T22:58:19.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac940 con 0x5595e986d790 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641aa730 con 0x5595e9861800 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60292/0 (socket says 172.21.15.120:60292) 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc6259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9860ee0 0x5595e9867b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 -- 172.21.15.120:0/4043560786 learned_addr learned my addr 172.21.15.120:0/4043560786 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e98851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 -- 172.21.15.120:0/4043560786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 msgr2=0x5595e98851d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e98851d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 -- 172.21.15.120:0/4043560786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9860ee0 msgr2=0x5595e9867b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9860ee0 0x5595e9867b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 -- 172.21.15.120:0/4043560786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e9861630 con 0x5595e9861800 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e98851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.720+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b770 tx=0x7fdc4c003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56fc62c44b807a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc4bfff640 1 -- 172.21.15.120:0/4043560786 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003e60 con 0x5595e9861800 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc4bfff640 1 -- 172.21.15.120:0/4043560786 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c01d070 con 0x5595e9861800 2024-09-10T22:58:19.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc4bfff640 1 -- 172.21.15.120:0/4043560786 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c02f5a0 con 0x5595e9861800 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 -- 172.21.15.120:0/4043560786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 msgr2=0x5595e98680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00b770 tx=0x7fdc4c003b70 comp rx=0 tx=0).stop 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 -- 172.21.15.120:0/4043560786 shutdown_connections 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e986d790 0x5595e98851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e9861800 0x5595e98680d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/4043560786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9860ee0 0x5595e9867b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 -- 172.21.15.120:0/4043560786 >> 172.21.15.120:0/4043560786 conn(0x5595e985ed10 msgr2=0x5595e9868ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.721+0000 7fdc6ad55740 1 -- 172.21.15.120:0/4043560786 shutdown_connections 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 -- 172.21.15.120:0/4043560786 wait complete. 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 -- start start 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 Processor -- start 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e986d790 0x5595e9870690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641aae00 con 0x5595e9861800 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac270 con 0x5595e9860ee0 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ab4d0 con 0x5595e986d790 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6ad55740 10 client.0 _ll_register_callbacks cb 0x7fdc5c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54672/0 (socket says 172.21.15.120:54672) 2024-09-10T22:58:19.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 -- 172.21.15.120:0/925665627 learned_addr learned my addr 172.21.15.120:0/925665627 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc62d9d640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e986d790 0x5595e9870690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc6259c640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 -- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e986d790 msgr2=0x5595e9870690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e986d790 0x5595e9870690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 -- 172.21.15.120:0/925665627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 msgr2=0x5595e98756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.722+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc59d9b640 1 -- 172.21.15.120:0/925665627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e9870bd0 con 0x5595e9861800 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc6259c640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc59d9b640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c024d20 tx=0x7fdc4c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafd122a8d76d35f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003e60 con 0x5595e9861800 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc4c020040 con 0x5595e9861800 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4c01d070 con 0x5595e9861800 2024-09-10T22:58:19.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c011440 con 0x5595e9861800 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc6ad55740 10 client.16524 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595e98eaaf0 con 0x5595e9861800 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c01d070 con 0x5595e9861800 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.723+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdc4c042a70 con 0x5595e9861800 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 10 client.16524 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 20 client.16524 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e98eb1a0 0x5595e98ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 4 client.16524 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5595e97d5860 con 0x5595e98eb1a0 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6259c640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e98eb1a0 0x5595e98ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc49ffb640 10 client.16524 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.724+0000 7fdc6259c640 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e98eb1a0 0x5595e98ed590 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e2b521be3a8719 server_cookie=185e0807a94ac4f6 in_seq=0 out_seq=0 2024-09-10T22:58:19.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.725+0000 7fdc49ffb640 1 -- 172.21.15.120:0/925665627 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5595e97d5860 con 0x5595e98eb1a0 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.725+0000 7fdc49ffb640 10 client.16524 handle_command_reply: tid=0 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.725+0000 7fdc6ad55740 1 client.16524 shutdown 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 msgr2=0x5595e9875c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c024d20 tx=0x7fdc4c00b4a0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 shutdown_connections 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5595e98eb1a0 0x5595e98ed590 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc49ffb640 0 client.16524 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5595e986d790 0x5595e9870690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5595e9861800 0x5595e9875c20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 --2- 172.21.15.120:0/925665627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5595e9860ee0 0x5595e98756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 >> 172.21.15.120:0/925665627 conn(0x5595e985ed10 msgr2=0x5595e9864090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 shutdown_connections 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 1 -- 172.21.15.120:0/925665627 wait complete. 2024-09-10T22:58:19.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc6ad55740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T22:58:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:58:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:58:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48", 2024-09-10T22:58:19.733 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:58:19.733 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 msgr2=0x7fdc500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 0x7fdc500b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02f500 tx=0x7fdc5c005fa0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 msgr2=0x7fdc64199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.726+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc54007b20 tx=0x7fdc54010050 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 shutdown_connections 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc500b52e0 0x7fdc500b77a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc64109170 0x7fdc6419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc641087f0 0x7fdc641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 --2- 172.21.15.120:0/709795329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc64102de0 0x7fdc64199260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 >> 172.21.15.120:0/709795329 conn(0x7fdc640fe830 msgr2=0x7fdc64100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 shutdown_connections 2024-09-10T22:58:19.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:19.727+0000 7fdc68980640 1 -- 172.21.15.120:0/709795329 wait complete. 2024-09-10T22:58:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: pgmap v523: 97 pgs: 97 active+clean; 26 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T22:58:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:58:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:58:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:20 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:58:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: pgmap v523: 97 pgs: 97 active+clean; 26 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T22:58:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:58:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:58:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:20 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:58:20.483 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: pgmap v523: 97 pgs: 97 active+clean; 26 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 4.0 MiB/s wr, 460 op/s 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:20 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:58:20.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 986002 16.66 MB/sec execute 195 sec latency 27.402 ms 2024-09-10T22:58:20.762 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:21.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.285+0000 7fde2b9e9640 1 -- 172.21.15.120:0/4006912430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde241023b0 msgr2=0x7fde24106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.285+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/4006912430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde241023b0 0x7fde24106880 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fde14009900 tx=0x7fde1402a410 comp rx=0 tx=0).stop 2024-09-10T22:58:21.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- 172.21.15.120:0/4006912430 shutdown_connections 2024-09-10T22:58:21.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/4006912430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24106ed0 0x7fde24107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/4006912430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde241023b0 0x7fde24106880 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/4006912430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde24101a90 0x7fde24101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- 172.21.15.120:0/4006912430 >> 172.21.15.120:0/4006912430 conn(0x7fde240fd700 msgr2=0x7fde240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- 172.21.15.120:0/4006912430 shutdown_connections 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- 172.21.15.120:0/4006912430 wait complete. 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 Processor -- start 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- start start 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24101a90 0x7fde2419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde241023b0 0x7fde2419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde24119350 con 0x7fde24106ed0 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde241191d0 con 0x7fde24101a90 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.286+0000 7fde2b9e9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde241194d0 con 0x7fde241023b0 2024-09-10T22:58:21.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54702/0 (socket says 172.21.15.120:54702) 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 -- 172.21.15.120:0/2932979433 learned_addr learned my addr 172.21.15.120:0/2932979433 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde28f84640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde241023b0 0x7fde2419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29785640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24101a90 0x7fde2419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 -- 172.21.15.120:0/2932979433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde241023b0 msgr2=0x7fde2419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde241023b0 0x7fde2419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 -- 172.21.15.120:0/2932979433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24101a90 msgr2=0x7fde2419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24101a90 0x7fde2419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 -- 172.21.15.120:0/2932979433 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde241a5950 con 0x7fde24106ed0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde29f86640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00cce0 tx=0x7fde1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde1a7fc640 1 -- 172.21.15.120:0/2932979433 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00f450 con 0x7fde24106ed0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde1a7fc640 1 -- 172.21.15.120:0/2932979433 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1c00faa0 con 0x7fde24106ed0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241a5c40 con 0x7fde24106ed0 2024-09-10T22:58:21.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde1a7fc640 1 -- 172.21.15.120:0/2932979433 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c018b70 con 0x7fde24106ed0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.287+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241ac300 con 0x7fde24106ed0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.290+0000 7fde2ddbe740 1 Processor -- start 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.290+0000 7fde1a7fc640 1 -- 172.21.15.120:0/2932979433 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fde1c00f5f0 con 0x7fde24106ed0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.290+0000 7fde2ddbe740 1 -- start start 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5636b1f26e50 con 0x5636b1fb0480 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5636b1f16280 con 0x5636b1fbcd30 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5636b1f1a090 con 0x5636b1fb0da0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde28f84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:60332/0 (socket says 172.21.15.120:60332) 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 -- 172.21.15.120:0/2006486829 learned_addr learned my addr 172.21.15.120:0/2006486829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:21.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 -- 172.21.15.120:0/2006486829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 msgr2=0x5636b1fbf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 -- 172.21.15.120:0/2006486829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 msgr2=0x5636b1fb0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fb0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 -- 172.21.15.120:0/2006486829 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b1c74650 con 0x5636b1fb0da0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde1a7fc640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 0x7fde100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde28f84640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 0x7fde100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde29785640 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde20009d20 tx=0x7fde20011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd8bc9e292c11ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde1a7fc640 1 -- 172.21.15.120:0/2932979433 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde1c0e6240 con 0x7fde24106ed0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.291+0000 7fde0f7fe640 1 -- 172.21.15.120:0/2006486829 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2001e070 con 0x5636b1fb0da0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde0f7fe640 1 -- 172.21.15.120:0/2006486829 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde20009e10 con 0x5636b1fb0da0 2024-09-10T22:58:21.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde0f7fe640 1 -- 172.21.15.120:0/2006486829 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde200187a0 con 0x5636b1fb0da0 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 -- 172.21.15.120:0/2006486829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 msgr2=0x5636b1fbc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde28f84640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 0x7fde100b77a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde14000c00 tx=0x7fde14009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde20009d20 tx=0x7fde20011810 comp rx=0 tx=0).stop 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 -- 172.21.15.120:0/2006486829 shutdown_connections 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fbc7f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/2006486829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fb0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 -- 172.21.15.120:0/2006486829 >> 172.21.15.120:0/2006486829 conn(0x5636b1fae2b0 msgr2=0x5636b1faf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 -- 172.21.15.120:0/2006486829 shutdown_connections 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.292+0000 7fde2ddbe740 1 -- 172.21.15.120:0/2006486829 wait complete. 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 -- start start 2024-09-10T22:58:21.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 Processor -- start 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 0x5636b1fbc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fb9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5636b1d77700 con 0x5636b1fbcd30 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5636b1eb1110 con 0x5636b1fb0480 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5636b1fbf9b0 con 0x5636b1fb0da0 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde2ddbe740 10 client.0 _ll_register_callbacks cb 0x5636b1eca990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde29f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde28f84640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fb9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde29f86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54726/0 (socket says 172.21.15.120:54726) 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.293+0000 7fde29f86640 1 -- 172.21.15.120:0/849168505 learned_addr learned my addr 172.21.15.120:0/849168505 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29785640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 0x5636b1fbc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 -- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 msgr2=0x5636b1fb9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fb9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 -- 172.21.15.120:0/849168505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 msgr2=0x5636b1fbc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 0x5636b1fbc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 -- 172.21.15.120:0/849168505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b1edec40 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29785640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 0x5636b1fbc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde29f86640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0a9000 tx=0x7fde1c0ac8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db90c27e8b6265c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0b1070 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde1c0b5b30 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20009e10 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde1c0ac540 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde2ddbe740 10 client.16536 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636b1fcb380 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.294+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde20009e10 con 0x5636b1fbcd30 2024-09-10T22:58:21.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fde1c0a8380 con 0x5636b1fbcd30 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 10 client.16536 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 20 client.16536 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b1fcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 4 client.16536 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b1f28da0 con 0x5636b1fcc840 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde29785640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b1fcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde0cff9640 10 client.16536 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.295+0000 7fde29785640 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b1fcec50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c00dfe7cb0b0c2 server_cookie=15978291459c8a5c in_seq=0 out_seq=0 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde0cff9640 1 -- 172.21.15.120:0/849168505 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636b1f28da0 con 0x5636b1fcc840 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde0cff9640 10 client.16536 handle_command_reply: tid=0 2024-09-10T22:58:21.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 client.16536 shutdown 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 msgr2=0x5636b1fba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0a9000 tx=0x7fde1c0ac8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 shutdown_connections 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b1fcec50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fbcd30 0x5636b1fba130 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fb9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde0cff9640 0 client.16536 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/849168505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0480 0x5636b1fbc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 >> 172.21.15.120:0/849168505 conn(0x5636b1fae2b0 msgr2=0x5636b1fb3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 shutdown_connections 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 1 -- 172.21.15.120:0/849168505 wait complete. 2024-09-10T22:58:21.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.296+0000 7fde2ddbe740 20 client.16536 trim_cache size 0 max 16384 2024-09-10T22:58:21.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.314+0000 7fde2ddbe740 1 Processor -- start 2024-09-10T22:58:21.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.314+0000 7fde2ddbe740 1 -- start start 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0da0 0x5636b1fad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fbcd30 0x5636b1fbe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b2850 con 0x5636b1fb0480 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b50d0 con 0x5636b1fb0da0 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b2ec0 con 0x5636b1fbcd30 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54742/0 (socket says 172.21.15.120:54742) 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 -- 172.21.15.120:0/38551173 learned_addr learned my addr 172.21.15.120:0/38551173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29f86640 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fbcd30 0x5636b1fbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 -- 172.21.15.120:0/38551173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fbcd30 msgr2=0x5636b1fbe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fbcd30 0x5636b1fbe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 -- 172.21.15.120:0/38551173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0da0 msgr2=0x5636b1fad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0da0 0x5636b1fad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 -- 172.21.15.120:0/38551173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b1fb0bd0 con 0x5636b1fb0480 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde29785640 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fde20011d10 tx=0x7fde20009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6c47b6b401cdf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde0effd640 1 -- 172.21.15.120:0/38551173 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2001e040 con 0x5636b1fb0480 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde0effd640 1 -- 172.21.15.120:0/38551173 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde20005370 con 0x5636b1fb0480 2024-09-10T22:58:21.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.315+0000 7fde0effd640 1 -- 172.21.15.120:0/38551173 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20019670 con 0x5636b1fb0480 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- 172.21.15.120:0/38551173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 msgr2=0x5636b1fc8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fde20011d10 tx=0x7fde20009a10 comp rx=0 tx=0).stop 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- 172.21.15.120:0/38551173 shutdown_connections 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fbcd30 0x5636b1fbe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fb0da0 0x5636b1fad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/38551173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc8be0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- 172.21.15.120:0/38551173 >> 172.21.15.120:0/38551173 conn(0x5636b1fae2b0 msgr2=0x5636b1facbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- 172.21.15.120:0/38551173 shutdown_connections 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- 172.21.15.120:0/38551173 wait complete. 2024-09-10T22:58:21.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 -- start start 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.316+0000 7fde2ddbe740 1 Processor -- start 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fc5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b3590 con 0x5636b1fb0480 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b4a00 con 0x5636b1fbcd30 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b3c60 con 0x5636b1fb0da0 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde2ddbe740 10 client.0 _ll_register_callbacks cb 0x7fde1c018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde28f84640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54746/0 (socket says 172.21.15.120:54746) 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 -- 172.21.15.120:0/1311116584 learned_addr learned my addr 172.21.15.120:0/1311116584 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29f86640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 -- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 msgr2=0x5636b1fc5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fc5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 -- 172.21.15.120:0/1311116584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 msgr2=0x5636b1fbfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29785640 1 -- 172.21.15.120:0/1311116584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b1fc0190 con 0x5636b1fb0480 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.317+0000 7fde29f86640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde29785640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fde20009ca0 tx=0x7fde20009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5573d196f8afd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2003a020 con 0x5636b1fb0480 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde20009e10 con 0x5636b1fb0480 2024-09-10T22:58:21.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20005370 con 0x5636b1fb0480 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde2001e040 con 0x5636b1fb0480 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 10 client.16542 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636b2039fb0 con 0x5636b1fb0480 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde20005370 con 0x5636b1fb0480 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fde20019680 con 0x5636b1fb0480 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 10 client.16542 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 20 client.16542 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b203c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 4 client.16542 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.318+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636b1f24850 con 0x5636b1fcc840 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.319+0000 7fde28f84640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b203c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.319+0000 7fde28f84640 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b203c430 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b2d1002ea67751 server_cookie=ffd6636ca8dd5d12 in_seq=0 out_seq=0 2024-09-10T22:58:21.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.319+0000 7fde0cff9640 10 client.16542 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:21.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde0cff9640 1 -- 172.21.15.120:0/1311116584 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x5636b1f24850 con 0x5636b1fcc840 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde0cff9640 10 client.16542 handle_command_reply: tid=0 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 client.16542 shutdown 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 msgr2=0x5636b1fc4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fde20009ca0 tx=0x7fde20009620 comp rx=0 tx=0).stop 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 shutdown_connections 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636b1fcc840 0x5636b203c430 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636b1fbcd30 0x5636b1fbfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636b1fb0da0 0x5636b1fc5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 --2- 172.21.15.120:0/1311116584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636b1fb0480 0x5636b1fc4c50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde0cff9640 0 client.16542 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 >> 172.21.15.120:0/1311116584 conn(0x5636b1fae2b0 msgr2=0x5636b1fad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 shutdown_connections 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 1 -- 172.21.15.120:0/1311116584 wait complete. 2024-09-10T22:58:21.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.321+0000 7fde2ddbe740 20 client.16542 trim_cache size 0 max 16384 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.329 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:21.333 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 msgr2=0x7fde100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 0x7fde100b77a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde14000c00 tx=0x7fde14009560 comp rx=0 tx=0).stop 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 msgr2=0x7fde241a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00cce0 tx=0x7fde1c00a620 comp rx=0 tx=0).stop 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 shutdown_connections 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde100b52e0 0x7fde100b77a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde24106ed0 0x7fde241a5180 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde241023b0 0x7fde2419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 --2- 172.21.15.120:0/2932979433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde24101a90 0x7fde2419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:21.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 >> 172.21.15.120:0/2932979433 conn(0x7fde240fd700 msgr2=0x7fde24104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:21.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 shutdown_connections 2024-09-10T22:58:21.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:21.322+0000 7fde2b9e9640 1 -- 172.21.15.120:0/2932979433 wait complete. 2024-09-10T22:58:21.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 989246 16.67 MB/sec execute 196 sec latency 12.212 ms 2024-09-10T22:58:22.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:58:22.164 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:22.428 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:22 smithi120 ceph-mon[31247]: pgmap v524: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2024-09-10T22:58:22.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:22 smithi153 ceph-mon[34951]: pgmap v524: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2024-09-10T22:58:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:22 smithi176 ceph-mon[34982]: pgmap v524: 97 pgs: 97 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2024-09-10T22:58:22.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 992452 16.67 MB/sec execute 197 sec latency 12.885 ms 2024-09-10T22:58:22.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.958+0000 7f3b51c61640 1 -- 172.21.15.120:0/3376900199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 msgr2=0x7f3b4c111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.958+0000 7f3b51c61640 1 --2- 172.21.15.120:0/3376900199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c111b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009ca0 tx=0x7f3b4402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 -- 172.21.15.120:0/3376900199 shutdown_connections 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 --2- 172.21.15.120:0/3376900199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c111b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 --2- 172.21.15.120:0/3376900199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c10a9d0 0x7f3b4c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 --2- 172.21.15.120:0/3376900199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c104fc0 0x7f3b4c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 -- 172.21.15.120:0/3376900199 >> 172.21.15.120:0/3376900199 conn(0x7f3b4c1009f0 msgr2=0x7f3b4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 -- 172.21.15.120:0/3376900199 shutdown_connections 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 -- 172.21.15.120:0/3376900199 wait complete. 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 Processor -- start 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.959+0000 7f3b51c61640 1 -- start start 2024-09-10T22:58:22.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c104fc0 0x7f3b4c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c10a9d0 0x7f3b4c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c114f80 con 0x7f3b4c10b350 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c114e00 con 0x7f3b4c10a9d0 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c115100 con 0x7f3b4c104fc0 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54754/0 (socket says 172.21.15.120:54754) 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 -- 172.21.15.120:0/804568020 learned_addr learned my addr 172.21.15.120:0/804568020 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4affd640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c10a9d0 0x7f3b4c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4b7fe640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c104fc0 0x7f3b4c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 -- 172.21.15.120:0/804568020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c104fc0 msgr2=0x7f3b4c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c104fc0 0x7f3b4c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 -- 172.21.15.120:0/804568020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c10a9d0 msgr2=0x7f3b4c19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c10a9d0 0x7f3b4c19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 -- 172.21.15.120:0/804568020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1a14b0 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b4bfff640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3b440098d0 tx=0x7f3b440317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44003b70 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44003d10 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4402fa20 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a1740 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.960+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a7e60 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.962+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c1053a0 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.963+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3b4402fc40 con 0x7f3b4c10b350 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.964+0000 7f3b48ff9640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 0x7f3b340b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.964+0000 7f3b4b7fe640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 0x7f3b340b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:22.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.964+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3b44106da0 con 0x7f3b4c10b350 2024-09-10T22:58:22.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.964+0000 7f3b4b7fe640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 0x7f3b340b7820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005d00 tx=0x7f3b3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:22.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:22.968+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3b440cdc90 con 0x7f3b4c10b350 2024-09-10T22:58:23.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.212+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b4c02e5c0 con 0x7f3b4c10b350 2024-09-10T22:58:23.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.212+0000 7f3b48ff9640 1 -- 172.21.15.120:0/804568020 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f3b440cd590 con 0x7f3b4c10b350 2024-09-10T22:58:23.214 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:58:23.214 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:58:23.214 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 msgr2=0x7f3b340b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 0x7f3b340b7820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005d00 tx=0x7f3b3c00b410 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 msgr2=0x7f3b4c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3b440098d0 tx=0x7f3b440317b0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 shutdown_connections 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3b340b5360 0x7f3b340b7820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b4c10b350 0x7f3b4c1a0ce0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4c10a9d0 0x7f3b4c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 --2- 172.21.15.120:0/804568020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b4c104fc0 0x7f3b4c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 >> 172.21.15.120:0/804568020 conn(0x7f3b4c1009f0 msgr2=0x7f3b4c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 shutdown_connections 2024-09-10T22:58:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:23.215+0000 7f3b51c61640 1 -- 172.21.15.120:0/804568020 wait complete. 2024-09-10T22:58:23.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 995329 16.67 MB/sec execute 198 sec latency 85.163 ms 2024-09-10T22:58:23.937 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:58:24.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:24 smithi120 ceph-mon[31247]: pgmap v525: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-09-10T22:58:24.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:24 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/804568020' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 -- 172.21.15.120:0/2664706942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c101860 msgr2=0x7ff24c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 --2- 172.21.15.120:0/2664706942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c101860 0x7ff24c101c60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c02a420 comp rx=0 tx=0).stop 2024-09-10T22:58:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 -- 172.21.15.120:0/2664706942 shutdown_connections 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 --2- 172.21.15.120:0/2664706942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 0x7ff24c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 --2- 172.21.15.120:0/2664706942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c1029b0 0x7ff24c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 --2- 172.21.15.120:0/2664706942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c101860 0x7ff24c101c60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 -- 172.21.15.120:0/2664706942 >> 172.21.15.120:0/2664706942 conn(0x7ff24c0fd140 msgr2=0x7ff24c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 -- 172.21.15.120:0/2664706942 shutdown_connections 2024-09-10T22:58:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.420+0000 7ff252575640 1 -- 172.21.15.120:0/2664706942 wait complete. 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 Processor -- start 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 -- start start 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c101860 0x7ff24c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 0x7ff24c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c10d4b0 con 0x7ff24c1029b0 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c10d330 con 0x7ff24c101860 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff252575640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c10d630 con 0x7ff24c103370 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c101860 0x7ff24c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58716/0 (socket says 172.21.15.120:58716) 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 -- 172.21.15.120:0/329838831 learned_addr learned my addr 172.21.15.120:0/329838831 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff250b12640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 0x7ff24c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 -- 172.21.15.120:0/329838831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 msgr2=0x7ff24c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 0x7ff24c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 -- 172.21.15.120:0/329838831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c101860 msgr2=0x7ff24c197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c101860 0x7ff24c197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 -- 172.21.15.120:0/329838831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c1997f0 con 0x7ff24c1029b0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.421+0000 7ff243fff640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff23400ea60 tx=0x7ff23400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.422+0000 7ff249ffb640 1 -- 172.21.15.120:0/329838831 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23400cde0 con 0x7ff24c1029b0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.422+0000 7ff249ffb640 1 -- 172.21.15.120:0/329838831 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340035f0 con 0x7ff24c1029b0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.422+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c199ae0 con 0x7ff24c1029b0 2024-09-10T22:58:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.422+0000 7ff249ffb640 1 -- 172.21.15.120:0/329838831 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234010690 con 0x7ff24c1029b0 2024-09-10T22:58:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.422+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a01b0 con 0x7ff24c1029b0 2024-09-10T22:58:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.424+0000 7ff249ffb640 1 -- 172.21.15.120:0/329838831 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff234004000 con 0x7ff24c1029b0 2024-09-10T22:58:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.424+0000 7ff255421740 1 Processor -- start 2024-09-10T22:58:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.424+0000 7ff255421740 1 -- start start 2024-09-10T22:58:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb41ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb41d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb38a6f0 con 0x55d5cb41dbd0 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb379b20 con 0x55d5cb41b190 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff255421740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb37d930 con 0x55d5cb418860 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff249ffb640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 0x7ff2380b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58728/0 (socket says 172.21.15.120:58728) 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb41d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 -- 172.21.15.120:0/4235842480 learned_addr learned my addr 172.21.15.120:0/4235842480 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff250b12640 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb41ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff250b12640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 0x7ff2380b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff249ffb640 1 -- 172.21.15.120:0/329838831 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff2340e5e60 con 0x7ff24c1029b0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 -- 172.21.15.120:0/4235842480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 msgr2=0x55d5cb41ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb41ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 -- 172.21.15.120:0/4235842480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 msgr2=0x55d5cb41d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb41d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 -- 172.21.15.120:0/4235842480 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5cb33f530 con 0x55d5cb41dbd0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff24bfff640 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb41d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.425+0000 7ff243fff640 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff2340e3990 tx=0x7ff2340a8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbd91b77a606912 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff250b12640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 0x7ff2380b77a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff244005c40 tx=0x7ff244009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff2427fc640 1 -- 172.21.15.120:0/4235842480 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340ac160 con 0x55d5cb41dbd0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff2427fc640 1 -- 172.21.15.120:0/4235842480 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340e3e50 con 0x55d5cb41dbd0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff2427fc640 1 -- 172.21.15.120:0/4235842480 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340a88c0 con 0x55d5cb41dbd0 2024-09-10T22:58:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 -- 172.21.15.120:0/4235842480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 msgr2=0x55d5cb420090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff2340e3990 tx=0x7ff2340a8620 comp rx=0 tx=0).stop 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 -- 172.21.15.120:0/4235842480 shutdown_connections 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41dbd0 0x55d5cb420090 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb41d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 --2- 172.21.15.120:0/4235842480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb41ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.426+0000 7ff255421740 1 -- 172.21.15.120:0/4235842480 >> 172.21.15.120:0/4235842480 conn(0x55d5cb4139b0 msgr2=0x55d5cb414740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- 172.21.15.120:0/4235842480 shutdown_connections 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- 172.21.15.120:0/4235842480 wait complete. 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- start start 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 Processor -- start 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb422530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb423100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb2b1ab0 con 0x55d5cb418860 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb31f730 con 0x55d5cb41b190 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d5cb420b50 con 0x55d5cb41dbd0 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.427+0000 7ff255421740 10 client.0 _ll_register_callbacks cb 0x55d5cb3736b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff243fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb422530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58732/0 (socket says 172.21.15.120:58732) 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 -- 172.21.15.120:0/3293657424 learned_addr learned my addr 172.21.15.120:0/3293657424 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff250b12640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb423100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 -- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 msgr2=0x55d5cb423100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb423100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 -- 172.21.15.120:0/3293657424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 msgr2=0x55d5cb422530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb422530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 -- 172.21.15.120:0/3293657424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5cb0db790 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff24bfff640 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c031d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49907cf533b6cfe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c04a060 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2340e3e50 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff23c004040 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff23c035070 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff255421740 10 client.16560 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5cb154620 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.428+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340e3e50 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff23c044070 con 0x55d5cb418860 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 10 client.16560 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 20 client.16560 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb430010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 4 client.16560 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5cb38c640 con 0x55d5cb42dbb0 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff243fff640 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb430010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff22b7fe640 10 client.16560 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.429+0000 7ff243fff640 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb430010 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fabdcab999c3d0c server_cookie=6fb4a29eb96fd824 in_seq=0 out_seq=0 2024-09-10T22:58:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff22b7fe640 1 -- 172.21.15.120:0/3293657424 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5cb38c640 con 0x55d5cb42dbb0 2024-09-10T22:58:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff22b7fe640 10 client.16560 handle_command_reply: tid=0 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 client.16560 shutdown 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 msgr2=0x55d5cb421e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c031d40 comp rx=0 tx=0).stop 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 shutdown_connections 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb430010 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb423100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff22b7fe640 0 client.16560 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41b190 0x55d5cb422530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 --2- 172.21.15.120:0/3293657424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb418860 0x55d5cb421e90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.432+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 >> 172.21.15.120:0/3293657424 conn(0x55d5cb4139b0 msgr2=0x55d5cb412bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.433+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 shutdown_connections 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.433+0000 7ff255421740 1 -- 172.21.15.120:0/3293657424 wait complete. 2024-09-10T22:58:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.433+0000 7ff255421740 20 client.16560 trim_cache size 0 max 16384 2024-09-10T22:58:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 Processor -- start 2024-09-10T22:58:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 -- start start 2024-09-10T22:58:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb418860 0x55d5cb421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb412910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a5c90 con 0x55d5cb41b190 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a8510 con 0x55d5cb418860 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.447+0000 7ff255421740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a6300 con 0x55d5cb41dbd0 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58738/0 (socket says 172.21.15.120:58738) 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 -- 172.21.15.120:0/1181915989 learned_addr learned my addr 172.21.15.120:0/1181915989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff250b12640 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb412910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff24bfff640 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb418860 0x55d5cb421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 -- 172.21.15.120:0/1181915989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 msgr2=0x55d5cb412910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb412910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 -- 172.21.15.120:0/1181915989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb418860 msgr2=0x55d5cb421f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb418860 0x55d5cb421f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 -- 172.21.15.120:0/1181915989 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5cb420cc0 con 0x55d5cb41b190 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff243fff640 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff2340a98c0 tx=0x7ff2340ac0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994501561da40f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff241ffb640 1 -- 172.21.15.120:0/1181915989 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234097070 con 0x55d5cb41b190 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff241ffb640 1 -- 172.21.15.120:0/1181915989 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340e3e50 con 0x55d5cb41b190 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.448+0000 7ff241ffb640 1 -- 172.21.15.120:0/1181915989 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340a88c0 con 0x55d5cb41b190 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- 172.21.15.120:0/1181915989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 msgr2=0x55d5cb4123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff2340a98c0 tx=0x7ff2340ac0a0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- 172.21.15.120:0/1181915989 shutdown_connections 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb41dbd0 0x55d5cb412910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb4123d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 --2- 172.21.15.120:0/1181915989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb418860 0x55d5cb421f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- 172.21.15.120:0/1181915989 >> 172.21.15.120:0/1181915989 conn(0x55d5cb4139b0 msgr2=0x55d5cb41e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- 172.21.15.120:0/1181915989 shutdown_connections 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- 172.21.15.120:0/1181915989 wait complete. 2024-09-10T22:58:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 -- start start 2024-09-10T22:58:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.449+0000 7ff255421740 1 Processor -- start 2024-09-10T22:58:24.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:24 smithi153 ceph-mon[34951]: pgmap v525: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-09-10T22:58:24.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:24 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/804568020' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb429560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 0x55d5cb4283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a69d0 con 0x55d5cb41b190 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a7e40 con 0x55d5cb41dbd0 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1a70a0 con 0x55d5cb418860 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 10 client.0 _ll_register_callbacks cb 0x7ff23c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb429560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58752/0 (socket says 172.21.15.120:58752) 2024-09-10T22:58:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 -- 172.21.15.120:0/353466591 learned_addr learned my addr 172.21.15.120:0/353466591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff250b12640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 0x55d5cb4283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 -- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 msgr2=0x55d5cb429560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb429560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 -- 172.21.15.120:0/353466591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 msgr2=0x55d5cb4283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 0x55d5cb4283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 -- 172.21.15.120:0/353466591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5cb428ba0 con 0x55d5cb41b190 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff24bfff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb429560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff250b12640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 0x55d5cb4283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff243fff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff2340e3d00 tx=0x7ff2340e6160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82baffcc7af5e841 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234097050 con 0x55d5cb41b190 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff2340037a0 con 0x55d5cb41b190 2024-09-10T22:58:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.450+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2340e3e50 con 0x55d5cb41b190 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2340ac650 con 0x55d5cb41b190 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 10 client.16566 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5cb4279d0 con 0x55d5cb41b190 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340e3e50 con 0x55d5cb41b190 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff2340a8030 con 0x55d5cb41b190 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 10 client.16566 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 20 client.16566 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb4a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 4 client.16566 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5cb3880f0 con 0x55d5cb42dbb0 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff24bfff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb4a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff24bfff640 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb4a11e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=122fe51144107fde server_cookie=c2e0cb84dd22fa2f in_seq=0 out_seq=0 2024-09-10T22:58:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.451+0000 7ff22b7fe640 10 client.16566 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:24 smithi176 ceph-mon[34982]: pgmap v525: 97 pgs: 97 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-09-10T22:58:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:24 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/804568020' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:24.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 998408 16.68 MB/sec execute 199 sec latency 88.598 ms 2024-09-10T22:58:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff22b7fe640 1 -- 172.21.15.120:0/353466591 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5cb3880f0 con 0x55d5cb42dbb0 2024-09-10T22:58:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff22b7fe640 10 client.16566 handle_command_reply: tid=0 2024-09-10T22:58:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 client.16566 shutdown 2024-09-10T22:58:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 msgr2=0x55d5cb429c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff2340e3d00 tx=0x7ff2340e6160 comp rx=0 tx=0).stop 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 shutdown_connections 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5cb42dbb0 0x55d5cb4a11e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff22b7fe640 0 client.16566 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.635+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5cb41dbd0 0x55d5cb4283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5cb41b190 0x55d5cb429c00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 1 --2- 172.21.15.120:0/353466591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cb418860 0x55d5cb429560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 >> 172.21.15.120:0/353466591 conn(0x55d5cb4139b0 msgr2=0x55d5cb418100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 shutdown_connections 2024-09-10T22:58:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 1 -- 172.21.15.120:0/353466591 wait complete. 2024-09-10T22:58:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff255421740 20 client.16566 trim_cache size 0 max 16384 2024-09-10T22:58:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:58:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:58:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:58:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 msgr2=0x7ff2380b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 0x7ff2380b77a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff244005c40 tx=0x7ff244009500 comp rx=0 tx=0).stop 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 msgr2=0x7ff24c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.636+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff23400ea60 tx=0x7ff23400ef20 comp rx=0 tx=0).stop 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 shutdown_connections 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2380b52e0 0x7ff2380b77a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff24c103370 0x7ff24c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff24c1029b0 0x7ff24c198300 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 --2- 172.21.15.120:0/329838831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff24c101860 0x7ff24c197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 >> 172.21.15.120:0/329838831 conn(0x7ff24c0fd140 msgr2=0x7ff24c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 shutdown_connections 2024-09-10T22:58:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:24.637+0000 7ff252575640 1 -- 172.21.15.120:0/329838831 wait complete. 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 -- 172.21.15.120:0/2108976583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c103ce0 msgr2=0x7fb08c1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 --2- 172.21.15.120:0/2108976583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c103ce0 0x7fb08c1040c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 -- 172.21.15.120:0/2108976583 shutdown_connections 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 --2- 172.21.15.120:0/2108976583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 0x7fb08c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 --2- 172.21.15.120:0/2108976583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c104600 0x7fb08c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 --2- 172.21.15.120:0/2108976583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c103ce0 0x7fb08c1040c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 -- 172.21.15.120:0/2108976583 >> 172.21.15.120:0/2108976583 conn(0x7fb08c0fd740 msgr2=0x7fb08c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 -- 172.21.15.120:0/2108976583 shutdown_connections 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.747+0000 7fb0945b7640 1 -- 172.21.15.120:0/2108976583 wait complete. 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 Processor -- start 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 -- start start 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 0x7fb08c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 0x7fb08c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c112db0 con 0x7fb08c104600 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c112c30 con 0x7fb08c103ce0 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb0945b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c112f30 con 0x7fb08c10d140 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb091b52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb091b52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58774/0 (socket says 172.21.15.120:58774) 2024-09-10T22:58:24.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb091b52640 1 -- 172.21.15.120:0/4221659309 learned_addr learned my addr 172.21.15.120:0/4221659309 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb092353640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 0x7fb08c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.748+0000 7fb092b54640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 0x7fb08c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 -- 172.21.15.120:0/4221659309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 msgr2=0x7fb08c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 0x7fb08c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 -- 172.21.15.120:0/4221659309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 msgr2=0x7fb08c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 0x7fb08c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 -- 172.21.15.120:0/4221659309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c19f370 con 0x7fb08c104600 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb092353640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 0x7fb08c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb091b52640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00e9c0 tx=0x7fb07c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb0837fe640 1 -- 172.21.15.120:0/4221659309 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00ccc0 con 0x7fb08c104600 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb0837fe640 1 -- 172.21.15.120:0/4221659309 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c0035f0 con 0x7fb08c104600 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c19f660 con 0x7fb08c104600 2024-09-10T22:58:24.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb0837fe640 1 -- 172.21.15.120:0/4221659309 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00ce60 con 0x7fb08c104600 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.749+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a5d20 con 0x7fb08c104600 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 Processor -- start 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 -- start start 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb0837fe640 1 -- 172.21.15.120:0/4221659309 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb07c004000 con 0x7fb08c104600 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a5888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a594840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 0x55ab9a597170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a4feea0 con 0x55ab9a594d80 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a4ee2d0 con 0x55ab9a588df0 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.752+0000 7fb09698c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a4f20e0 con 0x55ab9a5884d0 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092353640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a594840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a5888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092353640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a594840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41062/0 (socket says 172.21.15.120:41062) 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092353640 1 -- 172.21.15.120:0/2348707351 learned_addr learned my addr 172.21.15.120:0/2348707351 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb091b52640 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 0x55ab9a597170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 -- 172.21.15.120:0/2348707351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 msgr2=0x55ab9a594840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a594840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 -- 172.21.15.120:0/2348707351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 msgr2=0x55ab9a597170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 0x55ab9a597170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 -- 172.21.15.120:0/2348707351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9a24c650 con 0x55ab9a5884d0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb091b52640 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 0x55ab9a597170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb092b54640 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a5888b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb084009f00 tx=0x7fb084011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e680cac88b80fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb080ff9640 1 -- 172.21.15.120:0/2348707351 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004d20 con 0x55ab9a5884d0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb080ff9640 1 -- 172.21.15.120:0/2348707351 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb084004030 con 0x55ab9a5884d0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb080ff9640 1 -- 172.21.15.120:0/2348707351 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084005260 con 0x55ab9a5884d0 2024-09-10T22:58:24.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.753+0000 7fb0837fe640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 0x7fb0780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb092353640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 0x7fb0780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb0837fe640 1 -- 172.21.15.120:0/4221659309 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb07c0e5e40 con 0x7fb08c104600 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 -- 172.21.15.120:0/2348707351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 msgr2=0x55ab9a5888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb092353640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 0x7fb0780b77a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0880096c0 tx=0x7fb088005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a5888b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb084009f00 tx=0x7fb084011810 comp rx=0 tx=0).stop 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 -- 172.21.15.120:0/2348707351 shutdown_connections 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a594d80 0x55ab9a597170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a594840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 --2- 172.21.15.120:0/2348707351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a5888b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 -- 172.21.15.120:0/2348707351 >> 172.21.15.120:0/2348707351 conn(0x55ab9a586300 msgr2=0x55ab9a5874a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 -- 172.21.15.120:0/2348707351 shutdown_connections 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.754+0000 7fb09698c740 1 -- 172.21.15.120:0/2348707351 wait complete. 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 -- start start 2024-09-10T22:58:24.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 Processor -- start 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a591a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a594d80 0x55ab9a591f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a34f700 con 0x55ab9a5884d0 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a489160 con 0x55ab9a588df0 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab9a59be10 con 0x55ab9a594d80 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb09698c740 10 client.0 _ll_register_callbacks cb 0x55ab9a4a29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58792/0 (socket says 172.21.15.120:58792) 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 -- 172.21.15.120:0/2193621167 learned_addr learned my addr 172.21.15.120:0/2193621167 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb091b52640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a591a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092b54640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a594d80 0x55ab9a591f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 -- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a594d80 msgr2=0x55ab9a591f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a594d80 0x55ab9a591f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 -- 172.21.15.120:0/2193621167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 msgr2=0x55ab9a591a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a591a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb092353640 1 -- 172.21.15.120:0/2193621167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9a4b6c90 con 0x55ab9a5884d0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.755+0000 7fb091b52640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a591a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb092353640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb088033000 tx=0x7fb088009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc6679a5660c3a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08804d030 con 0x55ab9a5884d0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb084004030 con 0x55ab9a5884d0 2024-09-10T22:58:24.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0880055c0 con 0x55ab9a5884d0 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb088035040 con 0x55ab9a5884d0 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 10 client.16578 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab9a5a76d0 con 0x55ab9a5884d0 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb084004030 con 0x55ab9a5884d0 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb08803d910 con 0x55ab9a5884d0 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 10 client.16578 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 20 client.16578 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a5a8be0 0x55ab9a5ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 4 client.16578 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.756+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab9a500df0 con 0x55ab9a5a8be0 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.757+0000 7fb091b52640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a5a8be0 0x55ab9a5ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.757+0000 7fb075ffb640 10 client.16578 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.757+0000 7fb091b52640 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a5a8be0 0x55ab9a5ab010 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed181bdbdf53331 server_cookie=cacb43dfa2805d9c in_seq=0 out_seq=0 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb075ffb640 1 -- 172.21.15.120:0/2193621167 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab9a500df0 con 0x55ab9a5a8be0 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb075ffb640 10 client.16578 handle_command_reply: tid=0 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 client.16578 shutdown 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 msgr2=0x55ab9a5945d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb088033000 tx=0x7fb088009260 comp rx=0 tx=0).stop 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 shutdown_connections 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a5a8be0 0x55ab9a5ab010 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a594d80 0x55ab9a591f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a588df0 0x55ab9a591a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb075ffb640 0 client.16578 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 --2- 172.21.15.120:0/2193621167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a5945d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 >> 172.21.15.120:0/2193621167 conn(0x55ab9a586300 msgr2=0x55ab9a58be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 shutdown_connections 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 1 -- 172.21.15.120:0/2193621167 wait complete. 2024-09-10T22:58:24.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.758+0000 7fb09698c740 20 client.16578 trim_cache size 0 max 16384 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 Processor -- start 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 -- start start 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a588df0 0x55ab9a593760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a58ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1aa110 con 0x55ab9a5884d0 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ac970 con 0x55ab9a594d80 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.771+0000 7fb09698c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1aa760 con 0x55ab9a588df0 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58804/0 (socket says 172.21.15.120:58804) 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 -- 172.21.15.120:0/806686206 learned_addr learned my addr 172.21.15.120:0/806686206 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb091b52640 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a588df0 0x55ab9a593760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092b54640 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a58ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 -- 172.21.15.120:0/806686206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a588df0 msgr2=0x55ab9a593760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a588df0 0x55ab9a593760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 -- 172.21.15.120:0/806686206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 msgr2=0x55ab9a58ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a58ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 -- 172.21.15.120:0/806686206 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9a5ab690 con 0x55ab9a5884d0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb092353640 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb088031440 tx=0x7fb0880318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b171bd36feb69a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb077fff640 1 -- 172.21.15.120:0/806686206 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08804d040 con 0x55ab9a5884d0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb077fff640 1 -- 172.21.15.120:0/806686206 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08803d450 con 0x55ab9a5884d0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.772+0000 7fb077fff640 1 -- 172.21.15.120:0/806686206 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880044e0 con 0x55ab9a5884d0 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- 172.21.15.120:0/806686206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 msgr2=0x55ab9a585c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb088031440 tx=0x7fb0880318d0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- 172.21.15.120:0/806686206 shutdown_connections 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a58ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a588df0 0x55ab9a593760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 --2- 172.21.15.120:0/806686206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a5884d0 0x55ab9a585c10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- 172.21.15.120:0/806686206 >> 172.21.15.120:0/806686206 conn(0x55ab9a586300 msgr2=0x55ab9a593050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- 172.21.15.120:0/806686206 shutdown_connections 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- 172.21.15.120:0/806686206 wait complete. 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.773+0000 7fb09698c740 1 -- start start 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 Processor -- start 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 0x55ab9a59d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a614ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1aae30 con 0x55ab9a588df0 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ac2a0 con 0x55ab9a594d80 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c1ab500 con 0x55ab9a5884d0 2024-09-10T22:58:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb09698c740 10 client.0 _ll_register_callbacks cb 0x7fb088009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb091b52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 0x55ab9a59d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34838/0 (socket says 172.21.15.120:34838) 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 -- 172.21.15.120:0/2803930082 learned_addr learned my addr 172.21.15.120:0/2803930082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092b54640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a614ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 -- 172.21.15.120:0/2803930082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 msgr2=0x55ab9a614ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a614ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 -- 172.21.15.120:0/2803930082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 msgr2=0x55ab9a59d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 0x55ab9a59d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 -- 172.21.15.120:0/2803930082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9a615020 con 0x55ab9a5884d0 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb091b52640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 0x55ab9a59d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb092353640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb08802a930 tx=0x7fb08803da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c63c7616bdeb01d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.774+0000 7fb075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088069020 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb088004b80 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08803d450 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb08804d040 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 10 client.26056 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab9a616240 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08803d450 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb088049a10 con 0x55ab9a5884d0 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 10 client.26056 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 20 client.26056 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a616840 0x55ab9a618c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 4 client.26056 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab9a4fc8a0 con 0x55ab9a616840 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.775+0000 7fb091b52640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a616840 0x55ab9a618c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.776+0000 7fb075ffb640 10 client.26056 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.776+0000 7fb091b52640 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a616840 0x55ab9a618c30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2e830d92858ecd server_cookie=897d060cb83e7daf in_seq=0 out_seq=0 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.777+0000 7fb075ffb640 1 -- 172.21.15.120:0/2803930082 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab9a4fc8a0 con 0x55ab9a616840 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb075ffb640 10 client.26056 handle_command_reply: tid=0 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 client.26056 shutdown 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 msgr2=0x55ab9a59d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb08802a930 tx=0x7fb08803da30 comp rx=0 tx=0).stop 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 shutdown_connections 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab9a616840 0x55ab9a618c30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9a594d80 0x55ab9a614ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab9a588df0 0x55ab9a59d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb075ffb640 0 client.26056 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 --2- 172.21.15.120:0/2803930082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab9a5884d0 0x55ab9a59d450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 >> 172.21.15.120:0/2803930082 conn(0x55ab9a586300 msgr2=0x55ab9a593870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 shutdown_connections 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 1 -- 172.21.15.120:0/2803930082 wait complete. 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb09698c740 20 client.26056 trim_cache size 0 max 16384 2024-09-10T22:58:24.786 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 msgr2=0x7fb0780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 0x7fb0780b77a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0880096c0 tx=0x7fb088005b50 comp rx=0 tx=0).stop 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 msgr2=0x7fb08c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.778+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00e9c0 tx=0x7fb07c00ee80 comp rx=0 tx=0).stop 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 shutdown_connections 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0780b52e0 0x7fb0780b77a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb08c10d140 0x7fb08c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb08c104600 0x7fb08c199800 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 --2- 172.21.15.120:0/4221659309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb08c103ce0 0x7fb08c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 >> 172.21.15.120:0/4221659309 conn(0x7fb08c0fd740 msgr2=0x7fb08c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 shutdown_connections 2024-09-10T22:58:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:24.779+0000 7fb0945b7640 1 -- 172.21.15.120:0/4221659309 wait complete. 2024-09-10T22:58:25.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1001811 16.69 MB/sec execute 200 sec latency 17.730 ms 2024-09-10T22:58:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:26 smithi120 ceph-mon[31247]: pgmap v526: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-09-10T22:58:26.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:26 smithi153 ceph-mon[34951]: pgmap v526: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-09-10T22:58:26.558 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:58:26.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:58:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:26 smithi176 ceph-mon[34982]: pgmap v526: 97 pgs: 97 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-09-10T22:58:26.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1005228 16.69 MB/sec execute 201 sec latency 12.494 ms 2024-09-10T22:58:26.820 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:27.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- 172.21.15.120:0/1844071509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c103b30 msgr2=0x7f215c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 --2- 172.21.15.120:0/1844071509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c103b30 0x7f215c103f10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f214c009900 tx=0x7f214c02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:27.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- 172.21.15.120:0/1844071509 shutdown_connections 2024-09-10T22:58:27.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 --2- 172.21.15.120:0/1844071509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c10d0e0 0x7f215c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 --2- 172.21.15.120:0/1844071509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c104530 0x7f215c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 --2- 172.21.15.120:0/1844071509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c103b30 0x7f215c103f10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- 172.21.15.120:0/1844071509 >> 172.21.15.120:0/1844071509 conn(0x7f215c0fd700 msgr2=0x7f215c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- 172.21.15.120:0/1844071509 shutdown_connections 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- 172.21.15.120:0/1844071509 wait complete. 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 Processor -- start 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.328+0000 7f21628a4640 1 -- start start 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c103b30 0x7f215c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c104530 0x7f215c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f215c112db0 con 0x7f215c10d0e0 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f215c112c30 con 0x7f215c103b30 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f21628a4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f215c112f30 con 0x7f215c104530 2024-09-10T22:58:27.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58842/0 (socket says 172.21.15.120:58842) 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 -- 172.21.15.120:0/375797158 learned_addr learned my addr 172.21.15.120:0/375797158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f215b7fe640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c104530 0x7f215c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f215bfff640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c103b30 0x7f215c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 -- 172.21.15.120:0/375797158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c104530 msgr2=0x7f215c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c104530 0x7f215c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 -- 172.21.15.120:0/375797158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c103b30 msgr2=0x7f215c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c103b30 0x7f215c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 -- 172.21.15.120:0/375797158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1a1510 con 0x7f215c10d0e0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.329+0000 7f2160e41640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f215400eb10 tx=0x7f215400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.330+0000 7f21597fa640 1 -- 172.21.15.120:0/375797158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154004b20 con 0x7f215c10d0e0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.330+0000 7f21597fa640 1 -- 172.21.15.120:0/375797158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2154004cc0 con 0x7f215c10d0e0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.330+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1a1800 con 0x7f215c10d0e0 2024-09-10T22:58:27.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.330+0000 7f21597fa640 1 -- 172.21.15.120:0/375797158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154010670 con 0x7f215c10d0e0 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.330+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1a7ec0 con 0x7f215c10d0e0 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.332+0000 7f2164c79740 1 Processor -- start 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.332+0000 7f21597fa640 1 -- 172.21.15.120:0/375797158 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2154005020 con 0x7f215c10d0e0 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.332+0000 7f2164c79740 1 -- start start 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7449c9e60 con 0x55b744a5f790 2024-09-10T22:58:27.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7449b9290 con 0x55b744a52ee0 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f2164c79740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7449bd0a0 con 0x55b744a53800 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34864/0 (socket says 172.21.15.120:34864) 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 -- 172.21.15.120:0/1081172980 learned_addr learned my addr 172.21.15.120:0/1081172980 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 -- 172.21.15.120:0/1081172980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 msgr2=0x55b744a532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 -- 172.21.15.120:0/1081172980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 msgr2=0x55b744a61b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f21597fa640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 0x7f21440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 -- 172.21.15.120:0/1081172980 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b744717930 con 0x55b744a53800 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215b7fe640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 0x7f21440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f21597fa640 1 -- 172.21.15.120:0/375797158 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f21540e6100 con 0x7f215c10d0e0 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.333+0000 7f215bfff640 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f214c009230 tx=0x7f214c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3bb8fdb6eeec1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f21427fc640 1 -- 172.21.15.120:0/1081172980 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c040070 con 0x55b744a53800 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f21427fc640 1 -- 172.21.15.120:0/1081172980 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c005500 con 0x55b744a53800 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f21427fc640 1 -- 172.21.15.120:0/1081172980 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c035040 con 0x55b744a53800 2024-09-10T22:58:27.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f2164c79740 1 -- 172.21.15.120:0/1081172980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 msgr2=0x55b744a5f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f215b7fe640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 0x7f21440b77a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b744a52e90 tx=0x7f2150009970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.334+0000 7f2164c79740 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f214c009230 tx=0x7f214c004040 comp rx=0 tx=0).stop 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- 172.21.15.120:0/1081172980 shutdown_connections 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a61b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5f250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- 172.21.15.120:0/1081172980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- 172.21.15.120:0/1081172980 >> 172.21.15.120:0/1081172980 conn(0x55b744a50d10 msgr2=0x55b744a51eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- 172.21.15.120:0/1081172980 shutdown_connections 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- 172.21.15.120:0/1081172980 wait complete. 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- start start 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 Processor -- start 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 0x55b744a5d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b74481a890 con 0x55b744a52ee0 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b744953d40 con 0x55b744a5f790 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b744a623e0 con 0x55b744a53800 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f2164c79740 10 client.0 _ll_register_callbacks cb 0x55b7449b7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.335+0000 7f215bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58860/0 (socket says 172.21.15.120:58860) 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34878/0 (socket says 172.21.15.120:34878) 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215bfff640 1 -- 172.21.15.120:0/2889248114 learned_addr learned my addr 172.21.15.120:0/2889248114 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f2160e41640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 0x55b744a5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 -- 172.21.15.120:0/2889248114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 msgr2=0x55b744a5d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 0x55b744a5d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 -- 172.21.15.120:0/2889248114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 msgr2=0x55b744a5c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 -- 172.21.15.120:0/2889248114 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b744a6b770 con 0x55b744a53800 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215bfff640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f2160e41640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 0x55b744a5d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f215b7fe640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2150009730 tx=0x7f21500096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4fb8a50e887116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f213b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e830 con 0x55b744a53800 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c005500 con 0x55b744a53800 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f215001e9d0 con 0x55b744a53800 2024-09-10T22:58:27.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f215002d070 con 0x55b744a53800 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f2164c79740 10 client.26068 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.336+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b744a6e1b0 con 0x55b744a53800 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c005500 con 0x55b744a53800 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2150031b80 con 0x55b744a53800 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 10 client.26068 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 20 client.26068 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744a6f5c0 0x55b744a719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 4 client.26068 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7449cbdb0 con 0x55b744a6f5c0 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.337+0000 7f215bfff640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744a6f5c0 0x55b744a719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.338+0000 7f213b7fe640 10 client.26068 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.338+0000 7f215bfff640 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744a6f5c0 0x55b744a719f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf45b7cef738e039 server_cookie=885ff28004d5f06e in_seq=0 out_seq=0 2024-09-10T22:58:27.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f213b7fe640 1 -- 172.21.15.120:0/2889248114 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7449cbdb0 con 0x55b744a6f5c0 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f213b7fe640 10 client.26068 handle_command_reply: tid=0 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 client.26068 shutdown 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 msgr2=0x55b744a5cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2150009730 tx=0x7f21500096c0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 shutdown_connections 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744a6f5c0 0x55b744a719f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a5f790 0x55b744a5d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f213b7fe640 0 client.26068 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a5cc00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 --2- 172.21.15.120:0/2889248114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 >> 172.21.15.120:0/2889248114 conn(0x55b744a50d10 msgr2=0x55b744a4ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 shutdown_connections 2024-09-10T22:58:27.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 1 -- 172.21.15.120:0/2889248114 wait complete. 2024-09-10T22:58:27.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.339+0000 7f2164c79740 20 client.26068 trim_cache size 0 max 16384 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 Processor -- start 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 -- start start 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a5f790 0x55b744a59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1ae4b0 con 0x55b744a52ee0 2024-09-10T22:58:27.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b0d30 con 0x55b744a53800 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2164c79740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1aeb20 con 0x55b744a5f790 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f215bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f215b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f215b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41116/0 (socket says 172.21.15.120:41116) 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f215bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58876/0 (socket says 172.21.15.120:58876) 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f215b7fe640 1 -- 172.21.15.120:0/1698927664 learned_addr learned my addr 172.21.15.120:0/1698927664 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.359+0000 7f2160e41640 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a5f790 0x55b744a59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 -- 172.21.15.120:0/1698927664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a5f790 msgr2=0x55b744a59e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a5f790 0x55b744a59e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 -- 172.21.15.120:0/1698927664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 msgr2=0x55b744a5e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 -- 172.21.15.120:0/1698927664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b744a5b620 con 0x55b744a53800 2024-09-10T22:58:27.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f215b7fe640 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2150016040 tx=0x7f2150005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940cad9b488324db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f2141ffb640 1 -- 172.21.15.120:0/1698927664 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e830 con 0x55b744a53800 2024-09-10T22:58:27.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f2141ffb640 1 -- 172.21.15.120:0/1698927664 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f215001d5a0 con 0x55b744a53800 2024-09-10T22:58:27.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.360+0000 7f2141ffb640 1 -- 172.21.15.120:0/1698927664 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21500315d0 con 0x55b744a53800 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- 172.21.15.120:0/1698927664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 msgr2=0x55b744a598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2150016040 tx=0x7f2150005480 comp rx=0 tx=0).stop 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- 172.21.15.120:0/1698927664 shutdown_connections 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a5f790 0x55b744a59e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a53800 0x55b744a598d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 --2- 172.21.15.120:0/1698927664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a52ee0 0x55b744a5e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- 172.21.15.120:0/1698927664 >> 172.21.15.120:0/1698927664 conn(0x55b744a50d10 msgr2=0x55b744a5dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- 172.21.15.120:0/1698927664 shutdown_connections 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- 172.21.15.120:0/1698927664 wait complete. 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 -- start start 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.361+0000 7f2164c79740 1 Processor -- start 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1af1f0 con 0x55b744a5f790 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b0660 con 0x55b744a52ee0 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1af8c0 con 0x55b744a53800 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 10 client.0 _ll_register_callbacks cb 0x7f2150009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41132/0 (socket says 172.21.15.120:41132) 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2160e41640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 -- 172.21.15.120:0/3454646908 learned_addr learned my addr 172.21.15.120:0/3454646908 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215b7fe640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 -- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 msgr2=0x55b744a67f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a67f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 -- 172.21.15.120:0/3454646908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 msgr2=0x55b744a62700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a62700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 -- 172.21.15.120:0/3454646908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b744a62f30 con 0x55b744a52ee0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f215bfff640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f214c004b40 tx=0x7f214c004b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3615b2debea76c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f213b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c040040 con 0x55b744a52ee0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f215001d5a0 con 0x55b744a52ee0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.362+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f214c004f30 con 0x55b744a52ee0 2024-09-10T22:58:27.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f214c035040 con 0x55b744a52ee0 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 10 client.26009 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b744add100 con 0x55b744a52ee0 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f215001d5a0 con 0x55b744a52ee0 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f214c003b50 con 0x55b744a52ee0 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 10 client.26009 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 20 client.26009 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744add460 0x55b744adf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 4 client.26009 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7449c7860 con 0x55b744add460 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.363+0000 7f215b7fe640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744add460 0x55b744adf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.364+0000 7f213b7fe640 10 client.26009 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:27.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.364+0000 7f215b7fe640 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744add460 0x55b744adf850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5bdd7808e74a6 server_cookie=a35112aab1ff597f in_seq=0 out_seq=0 2024-09-10T22:58:27.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f213b7fe640 1 -- 172.21.15.120:0/3454646908 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7449c7860 con 0x55b744add460 2024-09-10T22:58:27.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f213b7fe640 10 client.26009 handle_command_reply: tid=0 2024-09-10T22:58:27.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 client.26009 shutdown 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 msgr2=0x55b744a679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f214c004b40 tx=0x7f214c004b70 comp rx=0 tx=0).stop 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 shutdown_connections 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b744add460 0x55b744adf850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b744a5f790 0x55b744a62700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b744a53800 0x55b744a67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f213b7fe640 0 client.26009 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:27.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 --2- 172.21.15.120:0/3454646908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b744a52ee0 0x55b744a679d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.365+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 >> 172.21.15.120:0/3454646908 conn(0x55b744a50d10 msgr2=0x55b744a5e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 shutdown_connections 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f2164c79740 1 -- 172.21.15.120:0/3454646908 wait complete. 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f2164c79740 20 client.26009 trim_cache size 0 max 16384 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0", 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:58:27.372 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 msgr2=0x7f21440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 0x7f21440b77a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b744a52e90 tx=0x7f2150009970 comp rx=0 tx=0).stop 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 msgr2=0x7f215c1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f215400eb10 tx=0x7f215400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 shutdown_connections 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f21440b52e0 0x7f21440b77a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f215c10d0e0 0x7f215c1a0d40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f215c104530 0x7f215c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 --2- 172.21.15.120:0/375797158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f215c103b30 0x7f215c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:27.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.366+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 >> 172.21.15.120:0/375797158 conn(0x7f215c0fd700 msgr2=0x7f215c10f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:27.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.367+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 shutdown_connections 2024-09-10T22:58:27.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:27.367+0000 7f21628a4640 1 -- 172.21.15.120:0/375797158 wait complete. 2024-09-10T22:58:27.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1007906 16.67 MB/sec execute 202 sec latency 31.665 ms 2024-09-10T22:58:28.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:28 smithi120 ceph-mon[31247]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-10T22:58:28.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:28 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:58:28.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:28 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:28 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:58:28.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:28 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:58:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:28 smithi153 ceph-mon[34951]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-10T22:58:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:28 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:58:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:28 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:28 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:58:28.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:28 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:28 smithi176 ceph-mon[34982]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-10T22:58:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:28 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:58:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:28 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:28 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:58:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:28 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:28.588 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:28.626 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1010140 16.65 MB/sec execute 203 sec latency 91.605 ms 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 -- 172.21.15.120:0/74747299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d81076a0 msgr2=0x7f50d8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/74747299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d81076a0 0x7f50d8107ae0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009900 tx=0x7f50cc02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 -- 172.21.15.120:0/74747299 shutdown_connections 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/74747299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d8108020 0x7f50d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/74747299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d81076a0 0x7f50d8107ae0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/74747299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 -- 172.21.15.120:0/74747299 >> 172.21.15.120:0/74747299 conn(0x7f50d80fd700 msgr2=0x7f50d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 -- 172.21.15.120:0/74747299 shutdown_connections 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.101+0000 7f50dd2ae640 1 -- 172.21.15.120:0/74747299 wait complete. 2024-09-10T22:58:29.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 Processor -- start 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 -- start start 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d81076a0 0x7f50d819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112e00 con 0x7f50d8108020 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112c80 con 0x7f50d8101c90 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50dd2ae640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112f80 con 0x7f50d81076a0 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58906/0 (socket says 172.21.15.120:58906) 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 -- 172.21.15.120:0/3077113953 learned_addr learned my addr 172.21.15.120:0/3077113953 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d67fc640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d81076a0 0x7f50d819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 -- 172.21.15.120:0/3077113953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d81076a0 msgr2=0x7f50d819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d81076a0 0x7f50d819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 -- 172.21.15.120:0/3077113953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 msgr2=0x7f50d819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.102+0000 7f50d77fe640 1 -- 172.21.15.120:0/3077113953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d81a1530 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50d77fe640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e970 tx=0x7f50d000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50bffff640 1 -- 172.21.15.120:0/3077113953 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d000ccc0 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d81a1820 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50bffff640 1 -- 172.21.15.120:0/3077113953 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d000ce60 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50bffff640 1 -- 172.21.15.120:0/3077113953 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0010430 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.103+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81a7ee0 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.105+0000 7f50bffff640 1 -- 172.21.15.120:0/3077113953 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f50d0004f70 con 0x7f50d8108020 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 Processor -- start 2024-09-10T22:58:29.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 -- start start 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecb64e50 con 0x55b6ecbee480 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecb54280 con 0x55b6ecbeeda0 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50df683740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecb58090 con 0x55b6ecbfad30 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50bffff640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 0x7f50c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58918/0 (socket says 172.21.15.120:58918) 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41158/0 (socket says 172.21.15.120:41158) 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d67fc640 1 -- 172.21.15.120:0/2788477815 learned_addr learned my addr 172.21.15.120:0/2788477815 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:29.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d77fe640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 0x7f50c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 -- 172.21.15.120:0/2788477815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 msgr2=0x55b6ecbfd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 -- 172.21.15.120:0/2788477815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 msgr2=0x55b6ecbee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50d6ffd640 1 -- 172.21.15.120:0/2788477815 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec8b2650 con 0x55b6ecbeeda0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.106+0000 7f50bffff640 1 -- 172.21.15.120:0/3077113953 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50d00e5f10 con 0x7f50d8108020 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50c8005fd0 tx=0x7f50c8011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f809c730b76793 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50d77fe640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 0x7f50c00b77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50d00e3a40 tx=0x7f50d0079bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50bd7fa640 1 -- 172.21.15.120:0/2788477815 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8003c30 con 0x55b6ecbeeda0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50bd7fa640 1 -- 172.21.15.120:0/2788477815 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c800fe50 con 0x55b6ecbeeda0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50bd7fa640 1 -- 172.21.15.120:0/2788477815 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80193e0 con 0x55b6ecbeeda0 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50df683740 1 -- 172.21.15.120:0/2788477815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 msgr2=0x55b6ecbfa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.107+0000 7f50df683740 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50c8005fd0 tx=0x7f50c8011c70 comp rx=0 tx=0).stop 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- 172.21.15.120:0/2788477815 shutdown_connections 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbfa7f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- 172.21.15.120:0/2788477815 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- 172.21.15.120:0/2788477815 >> 172.21.15.120:0/2788477815 conn(0x55b6ecbec2b0 msgr2=0x55b6ecbed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- 172.21.15.120:0/2788477815 shutdown_connections 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- 172.21.15.120:0/2788477815 wait complete. 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- start start 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 Processor -- start 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbf7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbf8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec9b5700 con 0x55b6ecbee480 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecaef110 con 0x55b6ecbeeda0 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecbfd9b0 con 0x55b6ecbfad30 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50df683740 10 client.0 _ll_register_callbacks cb 0x55b6ecb08990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58928/0 (socket says 172.21.15.120:58928) 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50d67fc640 1 -- 172.21.15.120:0/2750333934 learned_addr learned my addr 172.21.15.120:0/2750333934 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbf7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.108+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbf8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 -- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 msgr2=0x55b6ecbf8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbf8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 -- 172.21.15.120:0/2750333934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 msgr2=0x55b6ecbf7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbf7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 -- 172.21.15.120:0/2750333934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ecb1cc40 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000c00 tx=0x7f50cc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edaee175321375f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc035040 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50cc005260 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c800fe50 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50cc005460 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50df683740 10 client.16611 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6ecc09360 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c800fe50 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.109+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f50cc04d4b0 con 0x55b6ecbee480 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 10 client.16611 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 20 client.16611 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:29.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc0a7c0 0x55b6ecc0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 4 client.16611 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ecb66da0 con 0x55b6ecc0a7c0 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc0a7c0 0x55b6ecc0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc0a7c0 0x55b6ecc0cbf0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8726b32b5ee1219a server_cookie=a7ef3b31f6746be7 in_seq=0 out_seq=0 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.110+0000 7f50baff5640 10 client.16611 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50baff5640 1 -- 172.21.15.120:0/2750333934 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ecb66da0 con 0x55b6ecc0a7c0 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50baff5640 10 client.16611 handle_command_reply: tid=0 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 client.16611 shutdown 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 msgr2=0x55b6ecbfa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000c00 tx=0x7f50cc005db0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 shutdown_connections 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc0a7c0 0x55b6ecc0cbf0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbf8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbf7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 --2- 172.21.15.120:0/2750333934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecbfa690 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 >> 172.21.15.120:0/2750333934 conn(0x55b6ecbec2b0 msgr2=0x55b6ecbf1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50baff5640 0 client.16611 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 shutdown_connections 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 1 -- 172.21.15.120:0/2750333934 wait complete. 2024-09-10T22:58:29.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.112+0000 7f50df683740 20 client.16611 trim_cache size 0 max 16384 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.129+0000 7f50df683740 1 Processor -- start 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.129+0000 7f50df683740 1 -- start start 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbe72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81ae570 con 0x55b6ecbee480 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81b0df0 con 0x55b6ecbeeda0 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50df683740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81aebe0 con 0x55b6ecbfad30 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58932/0 (socket says 172.21.15.120:58932) 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 -- 172.21.15.120:0/2741776961 learned_addr learned my addr 172.21.15.120:0/2741776961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:29.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbe72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 -- 172.21.15.120:0/2741776961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 msgr2=0x55b6ecbfc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 -- 172.21.15.120:0/2741776961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 msgr2=0x55b6ecbe72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbe72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 -- 172.21.15.120:0/2741776961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ecbecf90 con 0x55b6ecbee480 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbe72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50d67fc640 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0315f0 tx=0x7f50cc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b68d99c93c7967 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50bcff9640 1 -- 172.21.15.120:0/2741776961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc035040 con 0x55b6ecbee480 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.130+0000 7f50bcff9640 1 -- 172.21.15.120:0/2741776961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50cc005260 con 0x55b6ecbee480 2024-09-10T22:58:29.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50bcff9640 1 -- 172.21.15.120:0/2741776961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc04d2a0 con 0x55b6ecbee480 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 -- 172.21.15.120:0/2741776961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 msgr2=0x55b6ecc06be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0315f0 tx=0x7f50cc005e40 comp rx=0 tx=0).stop 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 -- 172.21.15.120:0/2741776961 shutdown_connections 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbfad30 0x55b6ecbfc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecbe72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 --2- 172.21.15.120:0/2741776961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbee480 0x55b6ecc06be0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 -- 172.21.15.120:0/2741776961 >> 172.21.15.120:0/2741776961 conn(0x55b6ecbec2b0 msgr2=0x55b6ecbf2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 -- 172.21.15.120:0/2741776961 shutdown_connections 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.131+0000 7f50df683740 1 -- 172.21.15.120:0/2741776961 wait complete. 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 -- start start 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 Processor -- start 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbee480 0x55b6ecbf5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecc02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 0x55b6ecc031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81af2b0 con 0x55b6ecbfad30 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81b0720 con 0x55b6ecbeeda0 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50d81af980 con 0x55b6ecbee480 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50df683740 10 client.0 _ll_register_callbacks cb 0x7f50cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:29.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 0x55b6ecc031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 0x55b6ecc031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58942/0 (socket says 172.21.15.120:58942) 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecc02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d67fc640 1 -- 172.21.15.120:0/3632824024 learned_addr learned my addr 172.21.15.120:0/3632824024 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d67fc640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbee480 0x55b6ecbf5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 -- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbee480 msgr2=0x55b6ecbf5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbee480 0x55b6ecbf5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 -- 172.21.15.120:0/3632824024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 msgr2=0x55b6ecc031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 0x55b6ecc031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.132+0000 7f50d6ffd640 1 -- 172.21.15.120:0/3632824024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ecc121a0 con 0x55b6ecbeeda0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50d6ffd640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecc02c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a010 tx=0x7f50c8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b1abf2185c5c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c801f040 con 0x55b6ecbeeda0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50c800f9a0 con 0x55b6ecbeeda0 2024-09-10T22:58:29.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50cc005260 con 0x55b6ecbeeda0 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50c800fba0 con 0x55b6ecbeeda0 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50df683740 10 client.26027 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6ecc77810 con 0x55b6ecbeeda0 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50cc005260 con 0x55b6ecbeeda0 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.133+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f50c8018070 con 0x55b6ecbeeda0 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 10 client.26027 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 20 client.26027 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc77e30 0x55b6ecc7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 4 client.26027 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6ecb62850 con 0x55b6ecc77e30 2024-09-10T22:58:29.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50d67fc640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc77e30 0x55b6ecc7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50d67fc640 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc77e30 0x55b6ecc7a250 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1292fd281608fa44 server_cookie=280fdffb3434dbf2 in_seq=0 out_seq=0 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50baff5640 10 client.26027 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50baff5640 1 -- 172.21.15.120:0/3632824024 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55b6ecb62850 con 0x55b6ecc77e30 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50baff5640 10 client.26027 handle_command_reply: tid=0 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.134+0000 7f50df683740 1 client.26027 shutdown 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 msgr2=0x55b6ecc02c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecc02c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a010 tx=0x7f50c8009f90 comp rx=0 tx=0).stop 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 shutdown_connections 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b6ecc77e30 0x55b6ecc7a250 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b6ecbfad30 0x55b6ecc031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b6ecbeeda0 0x55b6ecc02c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50baff5640 0 client.26027 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:29.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 --2- 172.21.15.120:0/3632824024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b6ecbee480 0x55b6ecbf5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 >> 172.21.15.120:0/3632824024 conn(0x55b6ecbec2b0 msgr2=0x55b6ecbeabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 shutdown_connections 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 1 -- 172.21.15.120:0/3632824024 wait complete. 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.135+0000 7f50df683740 20 client.26027 trim_cache size 0 max 16384 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0", 2024-09-10T22:58:29.142 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:58:29.143 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.144 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:58:29.148 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:29.149 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:58:29.150 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:58:29.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 msgr2=0x7f50c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 0x7f50c00b77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50d00e3a40 tx=0x7f50d0079bd0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 msgr2=0x7f50d81a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e970 tx=0x7f50d000ed70 comp rx=0 tx=0).stop 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 shutdown_connections 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50c00b52e0 0x7f50c00b77a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50d8108020 0x7f50d81a0d60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50d81076a0 0x7f50d819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 --2- 172.21.15.120:0/3077113953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d8101c90 0x7f50d819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 >> 172.21.15.120:0/3077113953 conn(0x7f50d80fd700 msgr2=0x7f50d8103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 shutdown_connections 2024-09-10T22:58:29.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:58:29.136+0000 7f50dd2ae640 1 -- 172.21.15.120:0/3077113953 wait complete. 2024-09-10T22:58:29.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1013237 16.65 MB/sec execute 204 sec latency 42.901 ms 2024-09-10T22:58:29.853 INFO:tasks.cephfs.filesystem:scrub status for tag:1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0 - {'path': '/', 'tag': '1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0', 'options': 'recursive,force'} 2024-09-10T22:58:30.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:30 smithi120 ceph-mon[31247]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-10T22:58:30.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:30 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:58:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:30 smithi153 ceph-mon[34951]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-10T22:58:30.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:30 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:58:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:30 smithi176 ceph-mon[34982]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-10T22:58:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:30 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:58:30.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1016527 16.66 MB/sec execute 205 sec latency 15.218 ms 2024-09-10T22:58:31.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1019872 16.67 MB/sec execute 206 sec latency 15.548 ms 2024-09-10T22:58:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:32 smithi120 ceph-mon[31247]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-10T22:58:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:32 smithi153 ceph-mon[34951]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-10T22:58:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:32 smithi176 ceph-mon[34982]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-10T22:58:32.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1023153 16.68 MB/sec execute 207 sec latency 16.319 ms 2024-09-10T22:58:33.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1025580 16.64 MB/sec execute 208 sec latency 72.535 ms 2024-09-10T22:58:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:34 smithi120 ceph-mon[31247]: pgmap v530: 97 pgs: 97 active+clean; 47 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 482 op/s 2024-09-10T22:58:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:34 smithi153 ceph-mon[34951]: pgmap v530: 97 pgs: 97 active+clean; 47 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 482 op/s 2024-09-10T22:58:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:34 smithi176 ceph-mon[34982]: pgmap v530: 97 pgs: 97 active+clean; 47 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.9 MiB/s wr, 482 op/s 2024-09-10T22:58:34.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1027910 16.63 MB/sec execute 209 sec latency 37.324 ms 2024-09-10T22:58:35.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1031058 16.63 MB/sec execute 210 sec latency 15.283 ms 2024-09-10T22:58:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:36 smithi120 ceph-mon[31247]: pgmap v531: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-09-10T22:58:36.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:36 smithi153 ceph-mon[34951]: pgmap v531: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-09-10T22:58:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:36 smithi176 ceph-mon[34982]: pgmap v531: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-09-10T22:58:36.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1034348 16.64 MB/sec execute 211 sec latency 10.915 ms 2024-09-10T22:58:37.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1037724 16.65 MB/sec execute 212 sec latency 12.240 ms 2024-09-10T22:58:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 -- 172.21.15.120:0/4087833591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c103ae0 msgr2=0x7f938c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 --2- 172.21.15.120:0/4087833591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c103ae0 0x7f938c103f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f937c009930 tx=0x7f937c02a420 comp rx=0 tx=0).stop 2024-09-10T22:58:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 -- 172.21.15.120:0/4087833591 shutdown_connections 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 --2- 172.21.15.120:0/4087833591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c1044a0 0x7f938c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 --2- 172.21.15.120:0/4087833591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c103ae0 0x7f938c103f60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 --2- 172.21.15.120:0/4087833591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c102990 0x7f938c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 -- 172.21.15.120:0/4087833591 >> 172.21.15.120:0/4087833591 conn(0x7f938c0fe250 msgr2=0x7f938c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 -- 172.21.15.120:0/4087833591 shutdown_connections 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.723+0000 7f93934bd640 1 -- 172.21.15.120:0/4087833591 wait complete. 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 Processor -- start 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 -- start start 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c103ae0 0x7f938c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c1044a0 0x7f938c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10e600 con 0x7f938c102990 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10e480 con 0x7f938c1044a0 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f93934bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f938c10e780 con 0x7f938c103ae0 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41848/0 (socket says 172.21.15.120:41848) 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 -- 172.21.15.120:0/3258912799 learned_addr learned my addr 172.21.15.120:0/3258912799 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9390a58640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c103ae0 0x7f938c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391a5a640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c1044a0 0x7f938c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 -- 172.21.15.120:0/3258912799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c103ae0 msgr2=0x7f938c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c103ae0 0x7f938c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 -- 172.21.15.120:0/3258912799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c1044a0 msgr2=0x7f938c19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c1044a0 0x7f938c19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 -- 172.21.15.120:0/3258912799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c19a900 con 0x7f938c102990 2024-09-10T22:58:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.724+0000 7f9391259640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.725+0000 7f93827fc640 1 -- 172.21.15.120:0/3258912799 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388017070 con 0x7f938c102990 2024-09-10T22:58:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.725+0000 7f93827fc640 1 -- 172.21.15.120:0/3258912799 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9388009c30 con 0x7f938c102990 2024-09-10T22:58:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.725+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c19abf0 con 0x7f938c102990 2024-09-10T22:58:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.725+0000 7f93827fc640 1 -- 172.21.15.120:0/3258912799 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800f620 con 0x7f938c102990 2024-09-10T22:58:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.725+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c1a12c0 con 0x7f938c102990 2024-09-10T22:58:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 Processor -- start 2024-09-10T22:58:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 -- start start 2024-09-10T22:58:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 0x55e177cb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb2bd0 0x55e177cb5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e177c1f6f0 con 0x55e177cb0190 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e177c0eb20 con 0x55e177cb2bd0 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f939628d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e177c12930 con 0x55e177cad860 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f93827fc640 1 -- 172.21.15.120:0/3258912799 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9388010460 con 0x7f938c102990 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9390a58640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb2bd0 0x55e177cb5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57798/0 (socket says 172.21.15.120:57798) 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9391a5a640 1 -- 172.21.15.120:0/1587186914 learned_addr learned my addr 172.21.15.120:0/1587186914 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9391259640 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 0x55e177cb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.728+0000 7f9391a5a640 1 -- 172.21.15.120:0/1587186914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb2bd0 msgr2=0x55e177cb5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391a5a640 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb2bd0 0x55e177cb5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391a5a640 1 -- 172.21.15.120:0/1587186914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 msgr2=0x55e177cb2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391a5a640 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 0x55e177cb2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391a5a640 1 -- 172.21.15.120:0/1587186914 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e177bd4530 con 0x55e177cad860 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391259640 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 0x55e177cb2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391a5a640 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9384009f90 tx=0x7f93840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b2352836b7d196 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f93827fc640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 0x7f93780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f93777fe640 1 -- 172.21.15.120:0/1587186914 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384004d20 con 0x55e177cad860 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f93777fe640 1 -- 172.21.15.120:0/1587186914 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384004050 con 0x55e177cad860 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f93777fe640 1 -- 172.21.15.120:0/1587186914 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384005280 con 0x55e177cad860 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f9391259640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 0x7f93780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.729+0000 7f93827fc640 1 -- 172.21.15.120:0/3258912799 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f93880e5790 con 0x7f938c102990 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- 172.21.15.120:0/1587186914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 msgr2=0x55e177cafc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f9391259640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 0x7f93780b77a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93880a8600 tx=0x7f9388010cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9384009f90 tx=0x7f93840118d0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- 172.21.15.120:0/1587186914 shutdown_connections 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb2bd0 0x55e177cb5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb0190 0x55e177cb2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 --2- 172.21.15.120:0/1587186914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cad860 0x55e177cafc50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- 172.21.15.120:0/1587186914 >> 172.21.15.120:0/1587186914 conn(0x55e177ca89b0 msgr2=0x55e177ca9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- 172.21.15.120:0/1587186914 shutdown_connections 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- 172.21.15.120:0/1587186914 wait complete. 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 -- start start 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.730+0000 7f939628d740 1 Processor -- start 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e177b46ab0 con 0x55e177cad860 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e177bb4730 con 0x55e177cb0190 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e177cb5bb0 con 0x55e177cb2bd0 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f939628d740 10 client.0 _ll_register_callbacks cb 0x55e177c086b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41860/0 (socket says 172.21.15.120:41860) 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 -- 172.21.15.120:0/2582629685 learned_addr learned my addr 172.21.15.120:0/2582629685 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9390a58640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 -- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 msgr2=0x55e177cb80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 -- 172.21.15.120:0/2582629685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 msgr2=0x55e177cb7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 -- 172.21.15.120:0/2582629685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e177970790 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9384009250 tx=0x7f9384009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e41cf49cc4e2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938402c020 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.731+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f938400f4e0 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384004050 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9384004e10 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 10 client.16629 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1779e9620 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384004050 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f93840052a0 con 0x55e177cad860 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 10 client.16629 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 20 client.16629 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177cc24c0 0x55e177cc48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 4 client.16629 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e177c21640 con 0x55e177cc24c0 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.732+0000 7f9390a58640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177cc24c0 0x55e177cc48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.733+0000 7f9390a58640 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177cc24c0 0x55e177cc48b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8739521f6b9a31db server_cookie=5b5b9161c3de90a3 in_seq=0 out_seq=0 2024-09-10T22:58:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.733+0000 7f9374ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f9374ff9640 1 -- 172.21.15.120:0/2582629685 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e177c21640 con 0x55e177cc24c0 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f9374ff9640 10 client.16629 handle_command_reply: tid=0 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 client.16629 shutdown 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 msgr2=0x55e177cb6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9384009250 tx=0x7f9384009c40 comp rx=0 tx=0).stop 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 shutdown_connections 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177cc24c0 0x55e177cc48b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f9374ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 --2- 172.21.15.120:0/2582629685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6e60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 >> 172.21.15.120:0/2582629685 conn(0x55e177ca89b0 msgr2=0x55e177ca7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 shutdown_connections 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 1 -- 172.21.15.120:0/2582629685 wait complete. 2024-09-10T22:58:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.734+0000 7f939628d740 20 client.16629 trim_cache size 0 max 16384 2024-09-10T22:58:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 Processor -- start 2024-09-10T22:58:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 -- start start 2024-09-10T22:58:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1a7ee0 con 0x55e177cad860 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1aa760 con 0x55e177cb0190 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f939628d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1a8550 con 0x55e177cb2bd0 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41876/0 (socket says 172.21.15.120:41876) 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.750+0000 7f9391a5a640 1 -- 172.21.15.120:0/2052231841 learned_addr learned my addr 172.21.15.120:0/2052231841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9390a58640 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 -- 172.21.15.120:0/2052231841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 msgr2=0x55e177cb34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 -- 172.21.15.120:0/2052231841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 msgr2=0x55e177cb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 -- 172.21.15.120:0/2052231841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e177cb28f0 con 0x55e177cad860 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9391a5a640 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9384003b70 tx=0x7f9384014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98792ffa969775da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9376ffd640 1 -- 172.21.15.120:0/2052231841 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938402c040 con 0x55e177cad860 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9376ffd640 1 -- 172.21.15.120:0/2052231841 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384005a90 con 0x55e177cad860 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.751+0000 7f9376ffd640 1 -- 172.21.15.120:0/2052231841 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384028540 con 0x55e177cad860 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- 172.21.15.120:0/2052231841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 msgr2=0x55e177cb6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9384003b70 tx=0x7f9384014040 comp rx=0 tx=0).stop 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- 172.21.15.120:0/2052231841 shutdown_connections 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb2bd0 0x55e177cb34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cb0190 0x55e177cb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- 172.21.15.120:0/2052231841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cad860 0x55e177cb6a00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- 172.21.15.120:0/2052231841 >> 172.21.15.120:0/2052231841 conn(0x55e177ca89b0 msgr2=0x55e177ca3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- 172.21.15.120:0/2052231841 shutdown_connections 2024-09-10T22:58:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- 172.21.15.120:0/2052231841 wait complete. 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- start start 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 Processor -- start 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cad860 0x55e177cbbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb0190 0x55e177cbc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1a8c20 con 0x55e177cb2bd0 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1aa090 con 0x55e177cad860 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1a92f0 con 0x55e177cb0190 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f939628d740 10 client.0 _ll_register_callbacks cb 0x7f937c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.752+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41892/0 (socket says 172.21.15.120:41892) 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 -- 172.21.15.120:0/4040829743 learned_addr learned my addr 172.21.15.120:0/4040829743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9390a58640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb0190 0x55e177cbc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cad860 0x55e177cbbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 -- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb0190 msgr2=0x55e177cbc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb0190 0x55e177cbc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 -- 172.21.15.120:0/4040829743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cad860 msgr2=0x55e177cbbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cad860 0x55e177cbbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 -- 172.21.15.120:0/4040829743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e177cbd5f0 con 0x55e177cb2bd0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9391a5a640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9384002230 tx=0x7f9384003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae27739a87981ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384005890 con 0x55e177cb2bd0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384005a90 con 0x55e177cb2bd0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9384004fb0 con 0x55e177cb2bd0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93840051b0 con 0x55e177cb2bd0 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f939628d740 10 client.16635 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e177cbe960 con 0x55e177cb2bd0 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.753+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384005a90 con 0x55e177cb2bd0 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f938400f4e0 con 0x55e177cb2bd0 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 10 client.16635 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 20 client.16635 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177d2f3c0 0x55e177d317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 4 client.16635 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e177c1d0f0 con 0x55e177d2f3c0 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f9390a58640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177d2f3c0 0x55e177d317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f9374ff9640 10 client.16635 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.754+0000 7f9390a58640 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177d2f3c0 0x55e177d317b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45de340366d9e021 server_cookie=69d13314239db491 in_seq=0 out_seq=0 2024-09-10T22:58:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.967+0000 7f9374ff9640 1 -- 172.21.15.120:0/4040829743 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e177c1d0f0 con 0x55e177d2f3c0 2024-09-10T22:58:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.967+0000 7f9374ff9640 10 client.16635 handle_command_reply: tid=0 2024-09-10T22:58:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.967+0000 7f939628d740 1 client.16635 shutdown 2024-09-10T22:58:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 msgr2=0x55e177cbcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9384002230 tx=0x7f9384003cb0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 shutdown_connections 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e177d2f3c0 0x55e177d317b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e177cb2bd0 0x55e177cbcdc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e177cb0190 0x55e177cbc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f9374ff9640 0 client.16635 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 --2- 172.21.15.120:0/4040829743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e177cad860 0x55e177cbbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 >> 172.21.15.120:0/4040829743 conn(0x55e177ca89b0 msgr2=0x55e177cb6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 shutdown_connections 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 1 -- 172.21.15.120:0/4040829743 wait complete. 2024-09-10T22:58:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f939628d740 20 client.16635 trim_cache size 0 max 16384 2024-09-10T22:58:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:58:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:58:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:58:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 msgr2=0x7f93780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 0x7f93780b77a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93880a8600 tx=0x7f9388010cb0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 msgr2=0x7f938c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 shutdown_connections 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.968+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93780b52e0 0x7f93780b77a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f938c1044a0 0x7f938c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f938c103ae0 0x7f938c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 --2- 172.21.15.120:0/3258912799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f938c102990 0x7f938c198dc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 >> 172.21.15.120:0/3258912799 conn(0x7f938c0fe250 msgr2=0x7f938c106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 shutdown_connections 2024-09-10T22:58:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:37.969+0000 7f93934bd640 1 -- 172.21.15.120:0/3258912799 wait complete. 2024-09-10T22:58:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:38 smithi120 ceph-mon[31247]: pgmap v532: 97 pgs: 97 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-10T22:58:38.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:38 smithi153 ceph-mon[34951]: pgmap v532: 97 pgs: 97 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-10T22:58:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:38 smithi176 ceph-mon[34982]: pgmap v532: 97 pgs: 97 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 460 op/s 2024-09-10T22:58:38.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1040342 16.64 MB/sec execute 213 sec latency 62.061 ms 2024-09-10T22:58:39.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1042433 16.61 MB/sec execute 214 sec latency 68.207 ms 2024-09-10T22:58:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:40 smithi120 ceph-mon[31247]: pgmap v533: 97 pgs: 97 active+clean; 53 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-10T22:58:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:58:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:40 smithi153 ceph-mon[34951]: pgmap v533: 97 pgs: 97 active+clean; 53 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-10T22:58:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:40.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:58:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:40 smithi176 ceph-mon[34982]: pgmap v533: 97 pgs: 97 active+clean; 53 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 340 op/s 2024-09-10T22:58:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:58:40.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1045581 16.61 MB/sec execute 215 sec latency 16.861 ms 2024-09-10T22:58:41.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1048702 16.61 MB/sec execute 216 sec latency 16.856 ms 2024-09-10T22:58:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:42 smithi120 ceph-mon[31247]: pgmap v534: 97 pgs: 97 active+clean; 52 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-09-10T22:58:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:42 smithi153 ceph-mon[34951]: pgmap v534: 97 pgs: 97 active+clean; 52 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-09-10T22:58:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:42 smithi176 ceph-mon[34982]: pgmap v534: 97 pgs: 97 active+clean; 52 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 441 op/s 2024-09-10T22:58:42.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1051986 16.62 MB/sec execute 217 sec latency 18.721 ms 2024-09-10T22:58:43.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1055468 16.63 MB/sec execute 218 sec latency 12.630 ms 2024-09-10T22:58:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:44 smithi120 ceph-mon[31247]: pgmap v535: 97 pgs: 97 active+clean; 44 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-10T22:58:44.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:44 smithi153 ceph-mon[34951]: pgmap v535: 97 pgs: 97 active+clean; 44 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-10T22:58:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:44 smithi176 ceph-mon[34982]: pgmap v535: 97 pgs: 97 active+clean; 44 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-10T22:58:44.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1058078 16.62 MB/sec execute 219 sec latency 65.929 ms 2024-09-10T22:58:45.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1061023 16.60 MB/sec execute 220 sec latency 61.693 ms 2024-09-10T22:58:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:46 smithi120 ceph-mon[31247]: pgmap v536: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T22:58:46.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:46 smithi153 ceph-mon[34951]: pgmap v536: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T22:58:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:46 smithi176 ceph-mon[34982]: pgmap v536: 97 pgs: 97 active+clean; 48 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T22:58:46.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1063600 16.60 MB/sec execute 221 sec latency 19.775 ms 2024-09-10T22:58:47.627 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1066814 16.60 MB/sec execute 222 sec latency 18.811 ms 2024-09-10T22:58:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:48 smithi120 ceph-mon[31247]: pgmap v537: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-09-10T22:58:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:48 smithi153 ceph-mon[34951]: pgmap v537: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-09-10T22:58:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:48 smithi176 ceph-mon[34982]: pgmap v537: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-09-10T22:58:48.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1069971 16.61 MB/sec execute 223 sec latency 14.016 ms 2024-09-10T22:58:49.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1073027 16.61 MB/sec execute 224 sec latency 62.337 ms 2024-09-10T22:58:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:50 smithi120 ceph-mon[31247]: pgmap v538: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T22:58:50.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:50 smithi153 ceph-mon[34951]: pgmap v538: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T22:58:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:50 smithi176 ceph-mon[34982]: pgmap v538: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T22:58:50.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1076290 16.62 MB/sec execute 225 sec latency 13.861 ms 2024-09-10T22:58:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- 172.21.15.120:0/1348699093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb4806bf50 msgr2=0x7fcb48105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/1348699093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb4806bf50 0x7fcb48105560 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009900 tx=0x7fcb3c02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- 172.21.15.120:0/1348699093 shutdown_connections 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/1348699093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb481086a0 0x7fcb4810ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/1348699093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb48105af0 0x7fcb48107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/1348699093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb4806bf50 0x7fcb48105560 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- 172.21.15.120:0/1348699093 >> 172.21.15.120:0/1348699093 conn(0x7fcb480fd150 msgr2=0x7fcb480ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- 172.21.15.120:0/1348699093 shutdown_connections 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- 172.21.15.120:0/1348699093 wait complete. 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 Processor -- start 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.064+0000 7fcb4ef52640 1 -- start start 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48105af0 0x7fcb481993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb481086a0 0x7fcb4819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e5f0 con 0x7fcb481086a0 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e470 con 0x7fcb4806bf50 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ef52640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810e770 con 0x7fcb48105af0 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52732/0 (socket says 172.21.15.120:52732) 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4d4ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb481086a0 0x7fcb4819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 -- 172.21.15.120:0/900154218 learned_addr learned my addr 172.21.15.120:0/900154218 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb47fff640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48105af0 0x7fcb481993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 -- 172.21.15.120:0/900154218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48105af0 msgr2=0x7fcb481993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48105af0 0x7fcb481993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 -- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb481086a0 msgr2=0x7fcb4819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb481086a0 0x7fcb4819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 -- 172.21.15.120:0/900154218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4819a8c0 con 0x7fcb4806bf50 2024-09-10T22:58:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.065+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005940 tx=0x7fcb3c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.066+0000 7fcb45ffb640 1 -- 172.21.15.120:0/900154218 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c03f030 con 0x7fcb4806bf50 2024-09-10T22:58:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.066+0000 7fcb45ffb640 1 -- 172.21.15.120:0/900154218 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c003cb0 con 0x7fcb4806bf50 2024-09-10T22:58:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.066+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4819ab50 con 0x7fcb4806bf50 2024-09-10T22:58:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.066+0000 7fcb45ffb640 1 -- 172.21.15.120:0/900154218 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c035070 con 0x7fcb4806bf50 2024-09-10T22:58:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.066+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a1280 con 0x7fcb4806bf50 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.068+0000 7fcb45ffb640 1 -- 172.21.15.120:0/900154218 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcb3c03bbe0 con 0x7fcb4806bf50 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.068+0000 7fcb51dfe740 1 Processor -- start 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.068+0000 7fcb51dfe740 1 -- start start 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 0x559a1fa06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559a1f9736f0 con 0x559a1fa04190 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a1f962b20 con 0x559a1fa01860 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559a1f966930 con 0x559a1fa06bd0 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 0x559a1fa06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4d4ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 0x559a1fa06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52916/0 (socket says 172.21.15.120:52916) 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4ccee640 1 -- 172.21.15.120:0/3934345829 learned_addr learned my addr 172.21.15.120:0/3934345829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4d4ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52748/0 (socket says 172.21.15.120:52748) 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 -- 172.21.15.120:0/3934345829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 msgr2=0x559a1fa03c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 -- 172.21.15.120:0/3934345829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 msgr2=0x559a1fa06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 0x559a1fa06580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 -- 172.21.15.120:0/3934345829 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1f928530 con 0x559a1fa06bd0 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa09090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009c20 tx=0x7fcb34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93260864ff88c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb45ffb640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 0x7fcb380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb32ffd640 1 -- 172.21.15.120:0/3934345829 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34004050 con 0x559a1fa06bd0 2024-09-10T22:58:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb32ffd640 1 -- 172.21.15.120:0/3934345829 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb340054d0 con 0x559a1fa06bd0 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.069+0000 7fcb32ffd640 1 -- 172.21.15.120:0/3934345829 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34004ca0 con 0x559a1fa06bd0 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 0x7fcb380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb45ffb640 1 -- 172.21.15.120:0/900154218 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb3c106260 con 0x7fcb4806bf50 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3934345829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 msgr2=0x559a1fa09090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 0x7fcb380b7820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009fd0 tx=0x7fcb40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa09090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009c20 tx=0x7fcb34011810 comp rx=0 tx=0).stop 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.070+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3934345829 shutdown_connections 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa09090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa04190 0x559a1fa06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3934345829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa01860 0x559a1fa03c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3934345829 >> 172.21.15.120:0/3934345829 conn(0x559a1f9fc9b0 msgr2=0x559a1f9fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3934345829 shutdown_connections 2024-09-10T22:58:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3934345829 wait complete. 2024-09-10T22:58:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 -- start start 2024-09-10T22:58:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.071+0000 7fcb51dfe740 1 Processor -- start 2024-09-10T22:58:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa0c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559a1f89aab0 con 0x559a1fa01860 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559a1f908730 con 0x559a1fa04190 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559a1fa09bb0 con 0x559a1fa06bd0 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb51dfe740 10 client.0 _ll_register_callbacks cb 0x559a1f95c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb47fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52928/0 (socket says 172.21.15.120:52928) 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb47fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52764/0 (socket says 172.21.15.120:52764) 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 -- 172.21.15.120:0/3706146208 learned_addr learned my addr 172.21.15.120:0/3706146208 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa0c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 -- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 msgr2=0x559a1fa0c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa0c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 -- 172.21.15.120:0/3706146208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 msgr2=0x559a1fa0b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 -- 172.21.15.120:0/3706146208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1f6c4790 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006010 tx=0x7fcb3c031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77817ffecff03f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb27fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.072+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c049070 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb3c0ccf00 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb340054d0 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c0c90f0 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 10 client.16647 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a1f73d620 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb340054d0 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb3c0d1080 con 0x559a1fa01860 2024-09-10T22:58:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 10 client.16647 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 20 client.16647 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa1ab70 0x559a1fa1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 4 client.16647 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.073+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1f975640 con 0x559a1fa1ab70 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.074+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa1ab70 0x559a1fa1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.074+0000 7fcb47fff640 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa1ab70 0x559a1fa1d010 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c877554d06a8f1 server_cookie=c5b1aa590f8a2400 in_seq=0 out_seq=0 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.074+0000 7fcb27fff640 10 client.16647 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb27fff640 1 -- 172.21.15.120:0/3706146208 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a1f975640 con 0x559a1fa1ab70 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb27fff640 10 client.16647 handle_command_reply: tid=0 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 client.16647 shutdown 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 msgr2=0x559a1fa0ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006010 tx=0x7fcb3c031a10 comp rx=0 tx=0).stop 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 shutdown_connections 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa1ab70 0x559a1fa1d010 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa0c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb27fff640 0 client.16647 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa0b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/3706146208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa0ae90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 >> 172.21.15.120:0/3706146208 conn(0x559a1f9fc9b0 msgr2=0x559a1f9fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 shutdown_connections 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 1 -- 172.21.15.120:0/3706146208 wait complete. 2024-09-10T22:58:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.075+0000 7fcb51dfe740 20 client.16647 trim_cache size 0 max 16384 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.090+0000 7fcb51dfe740 1 Processor -- start 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 -- start start 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa06fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a5c00 con 0x559a1fa01860 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a8480 con 0x559a1fa04190 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a6270 con 0x559a1fa06bd0 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52934/0 (socket says 172.21.15.120:52934) 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 -- 172.21.15.120:0/1235488886 learned_addr learned my addr 172.21.15.120:0/1235488886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb47fff640 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa06fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 -- 172.21.15.120:0/1235488886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 msgr2=0x559a1fa074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 -- 172.21.15.120:0/1235488886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 msgr2=0x559a1fa06fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa06fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 -- 172.21.15.120:0/1235488886 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1fa0c470 con 0x559a1fa01860 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0d6140 tx=0x7fcb3c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e861027ad0532768 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.091+0000 7fcb327fc640 1 -- 172.21.15.120:0/1235488886 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c049030 con 0x559a1fa01860 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb327fc640 1 -- 172.21.15.120:0/1235488886 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c0cc5b0 con 0x559a1fa01860 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb327fc640 1 -- 172.21.15.120:0/1235488886 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0044e0 con 0x559a1fa01860 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 -- 172.21.15.120:0/1235488886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 msgr2=0x559a1fa09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0d6140 tx=0x7fcb3c005c90 comp rx=0 tx=0).stop 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 -- 172.21.15.120:0/1235488886 shutdown_connections 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa06bd0 0x559a1fa074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa06fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/1235488886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa01860 0x559a1fa09fb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 -- 172.21.15.120:0/1235488886 >> 172.21.15.120:0/1235488886 conn(0x559a1f9fc9b0 msgr2=0x559a1fa0aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 -- 172.21.15.120:0/1235488886 shutdown_connections 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.092+0000 7fcb51dfe740 1 -- 172.21.15.120:0/1235488886 wait complete. 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 -- start start 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 Processor -- start 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa10480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a6940 con 0x559a1fa06bd0 2024-09-10T22:58:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a7db0 con 0x559a1fa04190 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481a7010 con 0x559a1fa01860 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb51dfe740 10 client.0 _ll_register_callbacks cb 0x7fcb3c109990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52936/0 (socket says 172.21.15.120:52936) 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4ccee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:49310/0 (socket says 172.21.15.120:49310) 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 -- 172.21.15.120:0/968633347 learned_addr learned my addr 172.21.15.120:0/968633347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb47fff640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa10480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 -- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 msgr2=0x559a1fa0fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 -- 172.21.15.120:0/968633347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 msgr2=0x559a1fa10480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa10480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4d4ef640 1 -- 172.21.15.120:0/968633347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1fa11880 con 0x559a1fa06bd0 2024-09-10T22:58:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.093+0000 7fcb47fff640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb4d4ef640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcb40011ff0 tx=0x7fcb40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607fedbb5ba31448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40004b20 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb40004cc0 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c0cc5b0 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb400107c0 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 10 client.16653 fetch_fsmap learned FSMap version 26 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a1fa12cc0 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c0cc5b0 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb40005460 con 0x559a1fa06bd0 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 10 client.16653 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 20 client.16653 populate_metadata read hostname 'smithi120' 2024-09-10T22:58:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa88050 0x559a1fa8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 4 client.16653 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.094+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a1f9710f0 con 0x559a1fa88050 2024-09-10T22:58:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.095+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa88050 0x559a1fa8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.095+0000 7fcb27fff640 10 client.16653 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.095+0000 7fcb4ccee640 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa88050 0x559a1fa8a470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6bb4e9d4b875a2 server_cookie=10a3806c79faf795 in_seq=0 out_seq=0 2024-09-10T22:58:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb27fff640 1 -- 172.21.15.120:0/968633347 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a1f9710f0 con 0x559a1fa88050 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb27fff640 10 client.16653 handle_command_reply: tid=0 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb51dfe740 1 client.16653 shutdown 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 msgr2=0x559a1fa11050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcb40011ff0 tx=0x7fcb40004a40 comp rx=0 tx=0).stop 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 shutdown_connections 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559a1fa88050 0x559a1fa8a470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.291+0000 7fcb27fff640 0 client.16653 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559a1fa06bd0 0x559a1fa11050 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559a1fa04190 0x559a1fa10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 --2- 172.21.15.120:0/968633347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a1fa01860 0x559a1fa0fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 >> 172.21.15.120:0/968633347 conn(0x559a1f9fc9b0 msgr2=0x559a1fa09fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 shutdown_connections 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 1 -- 172.21.15.120:0/968633347 wait complete. 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.292+0000 7fcb51dfe740 20 client.16653 trim_cache size 0 max 16384 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:58:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:58:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 msgr2=0x7fcb380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 0x7fcb380b7820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009fd0 tx=0x7fcb40009500 comp rx=0 tx=0).stop 2024-09-10T22:58:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 msgr2=0x7fcb48198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005940 tx=0x7fcb3c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 shutdown_connections 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb380b5360 0x7fcb380b7820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb481086a0 0x7fcb4819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb48105af0 0x7fcb481993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 --2- 172.21.15.120:0/900154218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb4806bf50 0x7fcb48198d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 >> 172.21.15.120:0/900154218 conn(0x7fcb480fd150 msgr2=0x7fcb48103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 shutdown_connections 2024-09-10T22:58:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:58:51.293+0000 7fcb4ef52640 1 -- 172.21.15.120:0/900154218 wait complete. 2024-09-10T22:58:51.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1078188 16.58 MB/sec execute 226 sec latency 77.071 ms 2024-09-10T22:58:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:52 smithi120 ceph-mon[31247]: pgmap v539: 97 pgs: 97 active+clean; 55 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-09-10T22:58:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:52 smithi153 ceph-mon[34951]: pgmap v539: 97 pgs: 97 active+clean; 55 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-09-10T22:58:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:52 smithi176 ceph-mon[34982]: pgmap v539: 97 pgs: 97 active+clean; 55 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-09-10T22:58:52.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1081089 16.57 MB/sec execute 227 sec latency 12.180 ms 2024-09-10T22:58:53.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1084294 16.58 MB/sec execute 228 sec latency 12.028 ms 2024-09-10T22:58:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:54 smithi120 ceph-mon[31247]: pgmap v540: 97 pgs: 97 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-09-10T22:58:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:54 smithi153 ceph-mon[34951]: pgmap v540: 97 pgs: 97 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-09-10T22:58:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:54 smithi176 ceph-mon[34982]: pgmap v540: 97 pgs: 97 active+clean; 56 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-09-10T22:58:54.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1087206 16.59 MB/sec execute 229 sec latency 63.701 ms 2024-09-10T22:58:55.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1090515 16.59 MB/sec execute 230 sec latency 14.171 ms 2024-09-10T22:58:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:56 smithi120 ceph-mon[31247]: pgmap v541: 97 pgs: 97 active+clean; 50 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-09-10T22:58:56.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:56 smithi153 ceph-mon[34951]: pgmap v541: 97 pgs: 97 active+clean; 50 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-09-10T22:58:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:56 smithi176 ceph-mon[34982]: pgmap v541: 97 pgs: 97 active+clean; 50 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-09-10T22:58:56.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1093441 16.58 MB/sec execute 231 sec latency 15.841 ms 2024-09-10T22:58:57.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1096443 16.57 MB/sec execute 232 sec latency 12.473 ms 2024-09-10T22:58:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:58:58 smithi120 ceph-mon[31247]: pgmap v542: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-10T22:58:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:58:58 smithi153 ceph-mon[34951]: pgmap v542: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-10T22:58:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:58:58 smithi176 ceph-mon[34982]: pgmap v542: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-10T22:58:58.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1099106 16.57 MB/sec execute 233 sec latency 12.926 ms 2024-09-10T22:58:59.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1101749 16.56 MB/sec execute 234 sec latency 53.533 ms 2024-09-10T22:58:59.854 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:00.111 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:00 smithi120 ceph-mon[31247]: pgmap v543: 97 pgs: 97 active+clean; 47 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:59:00.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:00 smithi153 ceph-mon[34951]: pgmap v543: 97 pgs: 97 active+clean; 47 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:59:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:00 smithi176 ceph-mon[34982]: pgmap v543: 97 pgs: 97 active+clean; 47 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 401 op/s 2024-09-10T22:59:00.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1105065 16.57 MB/sec execute 235 sec latency 31.077 ms 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 -- 172.21.15.120:0/872864449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c108020 msgr2=0x7f1b9c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/872864449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c108020 0x7f1b9c10e820 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009ca0 tx=0x7f1b9402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 -- 172.21.15.120:0/872864449 shutdown_connections 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/872864449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c108020 0x7f1b9c10e820 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/872864449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c1076a0 0x7f1b9c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.629+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/872864449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c101c90 0x7f1b9c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- 172.21.15.120:0/872864449 >> 172.21.15.120:0/872864449 conn(0x7f1b9c0fd700 msgr2=0x7f1b9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- 172.21.15.120:0/872864449 shutdown_connections 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- 172.21.15.120:0/872864449 wait complete. 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 Processor -- start 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- start start 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c1076a0 0x7f1b9c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c108020 0x7f1b9c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c111ce0 con 0x7f1b9c101c90 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c111b60 con 0x7f1b9c108020 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba2d89640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c111e60 con 0x7f1b9c1076a0 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.630+0000 7f1ba0b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58780/0 (socket says 172.21.15.120:58780) 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3427938038 learned_addr learned my addr 172.21.15.120:0/3427938038 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba1326640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c108020 0x7f1b9c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c1076a0 0x7f1b9c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3427938038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c1076a0 msgr2=0x7f1b9c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c1076a0 0x7f1b9c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3427938038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c108020 msgr2=0x7f1b9c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c108020 0x7f1b9c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3427938038 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c19f2f0 con 0x7f1b9c101c90 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000e9f0 tx=0x7f1b9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1b99ffb640 1 -- 172.21.15.120:0/3427938038 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000cd80 con 0x7f1b9c101c90 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c19f5e0 con 0x7f1b9c101c90 2024-09-10T22:59:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1b99ffb640 1 -- 172.21.15.120:0/3427938038 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b900054d0 con 0x7f1b9c101c90 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1b99ffb640 1 -- 172.21.15.120:0/3427938038 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90010690 con 0x7f1b9c101c90 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.631+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1a5ca0 con 0x7f1b9c101c90 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1b99ffb640 1 -- 172.21.15.120:0/3427938038 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1b90004e90 con 0x7f1b9c101c90 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 Processor -- start 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 -- start start 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 0x55e577106860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e577116c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577117140 0x55e577119530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e57707ce50 con 0x55e577106da0 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e57706c280 con 0x55e577106480 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba515e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e577070090 con 0x55e577117140 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba1326640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 0x55e577106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba0b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e577116c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba1326640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 0x55e577106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44260/0 (socket says 172.21.15.120:44260) 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.634+0000 7f1ba1326640 1 -- 172.21.15.120:0/3851147671 learned_addr learned my addr 172.21.15.120:0/3851147671 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577117140 0x55e577119530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b99ffb640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 0x7f1b8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba1326640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 0x7f1b8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3851147671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577117140 msgr2=0x55e577119530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577117140 0x55e577119530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3851147671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 msgr2=0x55e577106860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 0x55e577106860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3851147671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e576dca650 con 0x55e577106da0 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b99ffb640 1 -- 172.21.15.120:0/3427938038 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1b900e5fb0 con 0x7f1b9c101c90 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e577116c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1b900e3a80 tx=0x7f1b90004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a17f7244e6787a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1ba1326640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 0x7f1b8c0b77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402f500 tx=0x7f1b9402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b86ffd640 1 -- 172.21.15.120:0/3851147671 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900ac570 con 0x55e577106da0 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b86ffd640 1 -- 172.21.15.120:0/3851147671 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b900cf070 con 0x55e577106da0 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.635+0000 7f1b86ffd640 1 -- 172.21.15.120:0/3851147671 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9001baa0 con 0x55e577106da0 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- 172.21.15.120:0/3851147671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 msgr2=0x55e577116c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e577116c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1b900e3a80 tx=0x7f1b90004a90 comp rx=0 tx=0).stop 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- 172.21.15.120:0/3851147671 shutdown_connections 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577117140 0x55e577119530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e577116c00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3851147671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106480 0x55e577106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- 172.21.15.120:0/3851147671 >> 172.21.15.120:0/3851147671 conn(0x55e5771042b0 msgr2=0x55e577105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- 172.21.15.120:0/3851147671 shutdown_connections 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- 172.21.15.120:0/3851147671 wait complete. 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 -- start start 2024-09-10T22:59:00.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.636+0000 7f1ba515e740 1 Processor -- start 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e5771143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e576ecd700 con 0x55e577106480 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e577007110 con 0x55e577117140 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e577119d90 con 0x55e577106da0 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba515e740 10 client.0 _ll_register_callbacks cb 0x55e577020990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba1326640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e5771143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40030/0 (socket says 172.21.15.120:40030) 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1b9bfff640 1 -- 172.21.15.120:0/3076328291 learned_addr learned my addr 172.21.15.120:0/3076328291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:00.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58802/0 (socket says 172.21.15.120:58802) 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 msgr2=0x55e577113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3076328291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 msgr2=0x55e5771143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e5771143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 -- 172.21.15.120:0/3076328291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e577034c40 con 0x55e577106480 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1b900e3a80 tx=0x7f1b900e6320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dea5cd55367d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.637+0000 7f1b7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000f3c0 con 0x55e577106480 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1b900aca00 con 0x55e577106480 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b900cf070 con 0x55e577106480 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b900203a0 con 0x55e577106480 2024-09-10T22:59:00.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 10 client.16665 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e577125680 con 0x55e577106480 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b900cf070 con 0x55e577106480 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1b900b58a0 con 0x55e577106480 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 10 client.16665 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 20 client.16665 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771269d0 0x55e577128dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 4 client.16665 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e57707eda0 con 0x55e5771269d0 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.638+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771269d0 0x55e577128dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.639+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771269d0 0x55e577128dc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a56fc5cc065f14 server_cookie=af5e98eb199b3601 in_seq=0 out_seq=0 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.639+0000 7f1b7bfff640 10 client.16665 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:00.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1b7bfff640 1 -- 172.21.15.120:0/3076328291 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e57707eda0 con 0x55e5771269d0 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1b7bfff640 10 client.16665 handle_command_reply: tid=0 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 client.16665 shutdown 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 msgr2=0x55e5771169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1b900e3a80 tx=0x7f1b900e6320 comp rx=0 tx=0).stop 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 shutdown_connections 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771269d0 0x55e577128dc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e5771143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1b7bfff640 0 client.16665 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106da0 0x55e577113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 --2- 172.21.15.120:0/3076328291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106480 0x55e5771169e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.640+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 >> 172.21.15.120:0/3076328291 conn(0x55e5771042b0 msgr2=0x55e577109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.641+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 shutdown_connections 2024-09-10T22:59:00.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.641+0000 7f1ba515e740 1 -- 172.21.15.120:0/3076328291 wait complete. 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.641+0000 7f1ba515e740 20 client.16665 trim_cache size 0 max 16384 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 Processor -- start 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 -- start start 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106da0 0x55e57712c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1aa080 con 0x55e577117140 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ac900 con 0x55e577106da0 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba515e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1aa6f0 con 0x55e577106480 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58806/0 (socket says 172.21.15.120:58806) 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 -- 172.21.15.120:0/401777866 learned_addr learned my addr 172.21.15.120:0/401777866 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:00.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106da0 0x55e57712c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 -- 172.21.15.120:0/401777866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 msgr2=0x55e57712bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 -- 172.21.15.120:0/401777866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106da0 msgr2=0x55e57712c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106da0 0x55e57712c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.658+0000 7f1ba1326640 1 -- 172.21.15.120:0/401777866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e577114760 con 0x55e577117140 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba1326640 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402ae60 tx=0x7f1b94045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad314b2e49d4a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1b867fc640 1 -- 172.21.15.120:0/401777866 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94055070 con 0x55e577117140 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:00.691 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:59:00.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.696 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.697 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:59:00.698 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1b867fc640 1 -- 172.21.15.120:0/401777866 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b94036050 con 0x55e577117140 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1b867fc640 1 -- 172.21.15.120:0/401777866 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9402f8c0 con 0x55e577117140 2024-09-10T22:59:00.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 -- 172.21.15.120:0/401777866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 msgr2=0x55e577102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402ae60 tx=0x7f1b94045000 comp rx=0 tx=0).stop 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 -- 172.21.15.120:0/401777866 shutdown_connections 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577117140 0x55e577102bb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577106da0 0x55e57712c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 --2- 172.21.15.120:0/401777866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 -- 172.21.15.120:0/401777866 >> 172.21.15.120:0/401777866 conn(0x55e5771042b0 msgr2=0x55e5771053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.659+0000 7f1ba515e740 1 -- 172.21.15.120:0/401777866 shutdown_connections 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 -- 172.21.15.120:0/401777866 wait complete. 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 -- start start 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 Processor -- start 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e577192540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1aadc0 con 0x55e577106da0 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ac230 con 0x55e577117140 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c1ab490 con 0x55e577106480 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba515e740 10 client.0 _ll_register_callbacks cb 0x7f1b94009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1b9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58820/0 (socket says 172.21.15.120:58820) 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1b9bfff640 1 -- 172.21.15.120:0/143653122 learned_addr learned my addr 172.21.15.120:0/143653122 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.660+0000 7f1ba1326640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e577192540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 -- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 msgr2=0x55e57712a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 -- 172.21.15.120:0/143653122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 msgr2=0x55e577192540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e577192540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 -- 172.21.15.120:0/143653122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e577192a80 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b9bfff640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55e577007170 tx=0x7f1b880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344bfcf180676b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88029060 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1b88011850 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b94036050 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b88011a50 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 10 client.16671 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e577193df0 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b94036050 con 0x55e577106da0 2024-09-10T22:59:00.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1b900051b0 con 0x55e577106da0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 10 client.16671 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 20 client.16671 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.661+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771945d0 0x55e5771969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1ba515e740 4 client.16671 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e57707a850 con 0x55e5771945d0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771945d0 0x55e5771969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1ba0b25640 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771945d0 0x55e5771969f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59da96e0d46622ce server_cookie=ad4bcf54e3052c85 in_seq=0 out_seq=0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1b7bfff640 10 client.16671 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1b7bfff640 1 -- 172.21.15.120:0/143653122 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55e57707a850 con 0x55e5771945d0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1b7bfff640 10 client.16671 handle_command_reply: tid=0 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.662+0000 7f1ba515e740 1 client.16671 shutdown 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 msgr2=0x55e57712a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55e577007170 tx=0x7f1b880039c0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 shutdown_connections 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5771945d0 0x55e5771969f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e577117140 0x55e577192540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e577106da0 0x55e57712a7a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1b7bfff640 0 client.16671 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 --2- 172.21.15.120:0/143653122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e577106480 0x55e57712a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 >> 172.21.15.120:0/143653122 conn(0x55e5771042b0 msgr2=0x55e577115460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 shutdown_connections 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 1 -- 172.21.15.120:0/143653122 wait complete. 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba515e740 20 client.16671 trim_cache size 0 max 16384 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 msgr2=0x7f1b8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 0x7f1b8c0b77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402f500 tx=0x7f1b9402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 msgr2=0x7f1b9c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:00.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000e9f0 tx=0x7f1b9000edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 shutdown_connections 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1b8c0b52e0 0x7f1b8c0b77a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b9c108020 0x7f1b9c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b9c1076a0 0x7f1b9c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.663+0000 7f1ba2d89640 1 --2- 172.21.15.120:0/3427938038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b9c101c90 0x7f1b9c199240 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.664+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 >> 172.21.15.120:0/3427938038 conn(0x7f1b9c0fd700 msgr2=0x7f1b9c06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.664+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 shutdown_connections 2024-09-10T22:59:00.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:00.664+0000 7f1ba2d89640 1 -- 172.21.15.120:0/3427938038 wait complete. 2024-09-10T22:59:01.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:59:01.560 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:01.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1108315 16.58 MB/sec execute 236 sec latency 18.206 ms 2024-09-10T22:59:01.811 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 -- 172.21.15.120:0/1393639805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 msgr2=0x7f2a74107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/1393639805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a74107ae0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c02a410 comp rx=0 tx=0).stop 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 -- 172.21.15.120:0/1393639805 shutdown_connections 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/1393639805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 0x7f2a7410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/1393639805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a74107ae0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/1393639805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 0x7f2a74102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 -- 172.21.15.120:0/1393639805 >> 172.21.15.120:0/1393639805 conn(0x7f2a740fd700 msgr2=0x7f2a740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 -- 172.21.15.120:0/1393639805 shutdown_connections 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.321+0000 7f2a78e0d640 1 -- 172.21.15.120:0/1393639805 wait complete. 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 Processor -- start 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 -- start start 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 0x7f2a74198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 0x7f2a7419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74111ca0 con 0x7f2a741076a0 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74111b20 con 0x7f2a74101c90 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a78e0d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74111e20 con 0x7f2a74108020 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a69d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a69d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58832/0 (socket says 172.21.15.120:58832) 2024-09-10T22:59:02.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a69d9b640 1 -- 172.21.15.120:0/2893125087 learned_addr learned my addr 172.21.15.120:0/2893125087 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a72d9d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 0x7f2a7419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.322+0000 7f2a7259c640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 0x7f2a74198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 -- 172.21.15.120:0/2893125087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 msgr2=0x7f2a7419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 0x7f2a7419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 -- 172.21.15.120:0/2893125087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 msgr2=0x7f2a74198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 0x7f2a74198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 -- 172.21.15.120:0/2893125087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7419e240 con 0x7f2a741076a0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a69d9b640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005940 tx=0x7f2a5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c03f030 con 0x7f2a741076a0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7419e4d0 con 0x7f2a741076a0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c0044c0 con 0x7f2a741076a0 2024-09-10T22:59:02.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c035070 con 0x7f2a741076a0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.323+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a741a4bf0 con 0x7f2a741076a0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.324+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a74102070 con 0x7f2a741076a0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.325+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2a5c03ba20 con 0x7f2a741076a0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.326+0000 7f2a6bfff640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 0x7f2a600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.326+0000 7f2a7259c640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 0x7f2a600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.326+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2a5c1063f0 con 0x7f2a741076a0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.326+0000 7f2a7259c640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 0x7f2a600b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009bd0 tx=0x7f2a64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.329+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2a5c0ccc90 con 0x7f2a741076a0 2024-09-10T22:59:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:02 smithi120 ceph-mon[31247]: pgmap v544: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-09-10T22:59:02.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:02 smithi153 ceph-mon[34951]: pgmap v544: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-09-10T22:59:02.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.558+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a74109e00 con 0x7f2a741076a0 2024-09-10T22:59:02.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.559+0000 7f2a6bfff640 1 -- 172.21.15.120:0/2893125087 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f2a5c0d16c0 con 0x7f2a741076a0 2024-09-10T22:59:02.560 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:59:02.561 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:59:02.561 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T22:59:02.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 msgr2=0x7f2a600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 0x7f2a600b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009bd0 tx=0x7f2a64005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 msgr2=0x7f2a741986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005940 tx=0x7f2a5c0040c0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 shutdown_connections 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2a600b52e0 0x7f2a600b77a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a74108020 0x7f2a7419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a741076a0 0x7f2a741986d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 --2- 172.21.15.120:0/2893125087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a74101c90 0x7f2a74198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 >> 172.21.15.120:0/2893125087 conn(0x7f2a740fd700 msgr2=0x7f2a740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 shutdown_connections 2024-09-10T22:59:02.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:02.561+0000 7f2a78e0d640 1 -- 172.21.15.120:0/2893125087 wait complete. 2024-09-10T22:59:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:02 smithi176 ceph-mon[34982]: pgmap v544: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 495 op/s 2024-09-10T22:59:02.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1111167 16.56 MB/sec execute 237 sec latency 17.487 ms 2024-09-10T22:59:03.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:03 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2893125087' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:03.257 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:59:03.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:03 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2893125087' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:03.515 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:03.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:03 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2893125087' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:03.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1114201 16.55 MB/sec execute 238 sec latency 13.651 ms 2024-09-10T22:59:04.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- 172.21.15.120:0/2989902478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8102160 msgr2=0x7f5cf810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 --2- 172.21.15.120:0/2989902478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8102160 0x7f5cf810eeb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce802a410 comp rx=0 tx=0).stop 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- 172.21.15.120:0/2989902478 shutdown_connections 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 --2- 172.21.15.120:0/2989902478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf810f3f0 0x7f5cf81117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 --2- 172.21.15.120:0/2989902478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8102160 0x7f5cf810eeb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 --2- 172.21.15.120:0/2989902478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8101840 0x7f5cf8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- 172.21.15.120:0/2989902478 >> 172.21.15.120:0/2989902478 conn(0x7f5cf80fd6f0 msgr2=0x7f5cf80ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- 172.21.15.120:0/2989902478 shutdown_connections 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- 172.21.15.120:0/2989902478 wait complete. 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 Processor -- start 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.024+0000 7f5cffc54640 1 -- start start 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102160 0x7f5cf819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810f3f0 0x7f5cf81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8114f60 con 0x7f5cf8101840 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8114de0 con 0x7f5cf8102160 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81150e0 con 0x7f5cf810f3f0 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38318/0 (socket says 172.21.15.120:38318) 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 -- 172.21.15.120:0/1904471001 learned_addr learned my addr 172.21.15.120:0/1904471001 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810f3f0 0x7f5cf81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102160 0x7f5cf819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 -- 172.21.15.120:0/1904471001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810f3f0 msgr2=0x7f5cf81a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810f3f0 0x7f5cf81a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 -- 172.21.15.120:0/1904471001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102160 msgr2=0x7f5cf819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102160 0x7f5cf819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 -- 172.21.15.120:0/1904471001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a1620 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cfd9f0640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e9f0 tx=0x7f5cf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5ceeffd640 1 -- 172.21.15.120:0/1904471001 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400cd80 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a1910 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5ceeffd640 1 -- 172.21.15.120:0/1904471001 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf40054d0 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.025+0000 7f5ceeffd640 1 -- 172.21.15.120:0/1904471001 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4010690 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.026+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a7fd0 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.028+0000 7f5d02029740 1 Processor -- start 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 -- start start 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5ceeffd640 1 -- 172.21.15.120:0/1904471001 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5cf4004e90 con 0x7f5cf8101840 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561875762df0 0x56187576e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5618756d8ea0 con 0x5618757624d0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5618756c82d0 con 0x56187576ed80 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5d02029740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5618756cc0e0 con 0x561875762df0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfd9f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561875762df0 0x56187576e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38330/0 (socket says 172.21.15.120:38330) 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/3945401177 learned_addr learned my addr 172.21.15.120:0/3945401177 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/3945401177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561875762df0 msgr2=0x56187576e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561875762df0 0x56187576e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/3945401177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 msgr2=0x561875771170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/3945401177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561875426650 con 0x5618757624d0 2024-09-10T22:59:04.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.029+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009f50 tx=0x7f5cf0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c1048e3d23b3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5ce3fff640 1 -- 172.21.15.120:0/3945401177 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003b70 con 0x5618757624d0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5ce3fff640 1 -- 172.21.15.120:0/3945401177 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0015070 con 0x5618757624d0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5ce3fff640 1 -- 172.21.15.120:0/3945401177 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00117f0 con 0x5618757624d0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5ceeffd640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 0x7f5ce40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5cfd9f0640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 0x7f5ce40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.030+0000 7f5ceeffd640 1 -- 172.21.15.120:0/1904471001 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5cf40e5fb0 con 0x7f5cf8101840 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- 172.21.15.120:0/3945401177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 msgr2=0x5618757628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5cfd9f0640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 0x7f5ce40b77a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40e3e50 tx=0x7f5cf40e9a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009f50 tx=0x7f5cf0010860 comp rx=0 tx=0).stop 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- 172.21.15.120:0/3945401177 shutdown_connections 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561875762df0 0x56187576e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 --2- 172.21.15.120:0/3945401177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x5618757628b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- 172.21.15.120:0/3945401177 >> 172.21.15.120:0/3945401177 conn(0x561875760300 msgr2=0x5618757614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- 172.21.15.120:0/3945401177 shutdown_connections 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- 172.21.15.120:0/3945401177 wait complete. 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 -- start start 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.031+0000 7f5d02029740 1 Processor -- start 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5618757624d0 0x56187576e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x56187576c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561875529700 con 0x561875762df0 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561875663160 con 0x5618757624d0 2024-09-10T22:59:04.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561875771a00 con 0x56187576ed80 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 10 client.0 _ll_register_callbacks cb 0x56187567c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38332/0 (socket says 172.21.15.120:38332) 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/3194331265 learned_addr learned my addr 172.21.15.120:0/3194331265 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x56187576c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5618757624d0 0x56187576e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 msgr2=0x56187576c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x56187576c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/3194331265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5618757624d0 msgr2=0x56187576e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5618757624d0 0x56187576e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/3194331265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561875690c90 con 0x561875762df0 2024-09-10T22:59:04.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x56187576c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce8032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236b03bf7fdabb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8035040 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5ce80316a0 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.032+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf0015070 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ce80318a0 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 10 client.16689 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56187577d360 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0015070 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5ce8004750 con 0x561875762df0 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 10 client.16689 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:04.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 20 client.16689 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56187577e820 0x561875780c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 4 client.16689 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618756dadf0 con 0x56187577e820 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.033+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56187577e820 0x561875780c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.034+0000 7f5ce17fa640 10 client.16689 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.034+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56187577e820 0x561875780c10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f321f79ca993afa server_cookie=302316595e94f70c in_seq=0 out_seq=0 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.034+0000 7f5ce17fa640 1 -- 172.21.15.120:0/3194331265 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618756dadf0 con 0x56187577e820 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.034+0000 7f5ce17fa640 10 client.16689 handle_command_reply: tid=0 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.034+0000 7f5d02029740 1 client.16689 shutdown 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 msgr2=0x56187576bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce8032e90 comp rx=0 tx=0).stop 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 shutdown_connections 2024-09-10T22:59:04.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56187577e820 0x561875780c10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x56187576c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5ce17fa640 0 client.16689 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x56187576bae0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 --2- 172.21.15.120:0/3194331265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5618757624d0 0x56187576e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 >> 172.21.15.120:0/3194331265 conn(0x561875760300 msgr2=0x561875765e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 shutdown_connections 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 1 -- 172.21.15.120:0/3194331265 wait complete. 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.035+0000 7f5d02029740 20 client.16689 trim_cache size 0 max 16384 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 Processor -- start 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 -- start start 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561875762df0 0x56187576afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x561875770170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ae580 con 0x5618757624d0 2024-09-10T22:59:04.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b0e00 con 0x561875762df0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5d02029740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81aebf0 con 0x56187576ed80 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfe1f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfe1f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38348/0 (socket says 172.21.15.120:38348) 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/1952264573 learned_addr learned my addr 172.21.15.120:0/1952264573 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561875762df0 0x56187576afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x561875770170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.048+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/1952264573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 msgr2=0x561875770170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x561875770170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/1952264573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561875762df0 msgr2=0x56187576afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561875762df0 0x56187576afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5cfe1f1640 1 -- 172.21.15.120:0/1952264573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561875781290 con 0x5618757624d0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0010d60 tx=0x7f5cf0009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492ed3fd1e050fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5ce37fe640 1 -- 172.21.15.120:0/1952264573 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003d50 con 0x5618757624d0 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5ce37fe640 1 -- 172.21.15.120:0/1952264573 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0015040 con 0x5618757624d0 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5ce37fe640 1 -- 172.21.15.120:0/1952264573 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0011a90 con 0x5618757624d0 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5d02029740 1 -- 172.21.15.120:0/1952264573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 msgr2=0x56187577ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.049+0000 7f5d02029740 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0010d60 tx=0x7f5cf0009ab0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- 172.21.15.120:0/1952264573 shutdown_connections 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56187576ed80 0x561875770170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561875762df0 0x56187576afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- 172.21.15.120:0/1952264573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5618757624d0 0x56187577ac30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- 172.21.15.120:0/1952264573 >> 172.21.15.120:0/1952264573 conn(0x561875760300 msgr2=0x56187576a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- 172.21.15.120:0/1952264573 shutdown_connections 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- 172.21.15.120:0/1952264573 wait complete. 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- start start 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 Processor -- start 2024-09-10T22:59:04.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5618757624d0 0x561875777130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81af2c0 con 0x561875762df0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b0730 con 0x56187576ed80 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81af990 con 0x5618757624d0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5d02029740 10 client.0 _ll_register_callbacks cb 0x7f5ce8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5cfd1ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.050+0000 7f5cfd1ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38356/0 (socket says 172.21.15.120:38356) 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/2010063259 learned_addr learned my addr 172.21.15.120:0/2010063259 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5618757624d0 0x561875777130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5618757624d0 msgr2=0x561875777130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5618757624d0 0x561875777130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/2010063259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 msgr2=0x561875771e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 -- 172.21.15.120:0/2010063259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561875760d80 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5cfd1ef640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005ec0 tx=0x7f5ce802aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91129bc4755579b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8035070 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5ce8004f60 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf0015040 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ce8060050 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5d02029740 10 client.16695 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5618757ec2a0 con 0x561875762df0 2024-09-10T22:59:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.051+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0015040 con 0x561875762df0 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5ce80458c0 con 0x561875762df0 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 10 client.16695 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 20 client.16695 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5618757ec9f0 0x5618757eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 4 client.16695 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618756d68a0 con 0x5618757ec9f0 2024-09-10T22:59:04.083 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5618757ec9f0 0x5618757eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5ce17fa640 10 client.16695 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.052+0000 7f5cfe1f1640 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5618757ec9f0 0x5618757eee10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91ceeda5099a5bb server_cookie=afd3a4a0206e37cb in_seq=0 out_seq=0 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5ce17fa640 1 -- 172.21.15.120:0/2010063259 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618756d68a0 con 0x5618757ec9f0 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5ce17fa640 10 client.16695 handle_command_reply: tid=0 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 client.16695 shutdown 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 msgr2=0x5618757776b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005ec0 tx=0x7f5ce802aba0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 shutdown_connections 2024-09-10T22:59:04.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5618757ec9f0 0x5618757eee10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56187576ed80 0x561875771e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561875762df0 0x5618757776b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5ce17fa640 0 client.16695 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 --2- 172.21.15.120:0/2010063259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5618757624d0 0x561875777130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 >> 172.21.15.120:0/2010063259 conn(0x561875760300 msgr2=0x56187575f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 shutdown_connections 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 1 -- 172.21.15.120:0/2010063259 wait complete. 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.054+0000 7f5d02029740 20 client.16695 trim_cache size 0 max 16384 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 msgr2=0x7f5ce40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 0x7f5ce40b77a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40e3e50 tx=0x7f5cf40e9a90 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 msgr2=0x7f5cf819b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e9f0 tx=0x7f5cf400edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 shutdown_connections 2024-09-10T22:59:04.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5ce40b52e0 0x7f5ce40b77a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810f3f0 0x7f5cf81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf8102160 0x7f5cf819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 --2- 172.21.15.120:0/1904471001 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cf8101840 0x7f5cf819b570 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 >> 172.21.15.120:0/1904471001 conn(0x7f5cf80fd6f0 msgr2=0x7f5cf80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 shutdown_connections 2024-09-10T22:59:04.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:04.055+0000 7f5cffc54640 1 -- 172.21.15.120:0/1904471001 wait complete. 2024-09-10T22:59:04.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:04 smithi120 ceph-mon[31247]: pgmap v545: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 -- 172.21.15.120:0/1073801707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d4101860 msgr2=0x7fc5d4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 --2- 172.21.15.120:0/1073801707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d4101860 0x7fc5d4101c60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c402a420 comp rx=0 tx=0).stop 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 -- 172.21.15.120:0/1073801707 shutdown_connections 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 --2- 172.21.15.120:0/1073801707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4103370 0x7fc5d4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 --2- 172.21.15.120:0/1073801707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d41029b0 0x7fc5d4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 --2- 172.21.15.120:0/1073801707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d4101860 0x7fc5d4101c60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 -- 172.21.15.120:0/1073801707 >> 172.21.15.120:0/1073801707 conn(0x7fc5d40fd140 msgr2=0x7fc5d40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 -- 172.21.15.120:0/1073801707 shutdown_connections 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 -- 172.21.15.120:0/1073801707 wait complete. 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.382+0000 7fc5da310640 1 Processor -- start 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 -- start start 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4101860 0x7fc5d4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 0x7fc5d4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d410d4d0 con 0x7fc5d41029b0 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d410d350 con 0x7fc5d4103370 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5da310640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d410d650 con 0x7fc5d4101860 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4101860 0x7fc5d4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38378/0 (socket says 172.21.15.120:38378) 2024-09-10T22:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 -- 172.21.15.120:0/3273829588 learned_addr learned my addr 172.21.15.120:0/3273829588 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 0x7fc5d4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 -- 172.21.15.120:0/3273829588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4101860 msgr2=0x7fc5d4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4101860 0x7fc5d4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 -- 172.21.15.120:0/3273829588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 msgr2=0x7fc5d4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 0x7fc5d4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d37fe640 1 -- 172.21.15.120:0/3273829588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d41997d0 con 0x7fc5d41029b0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.383+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 0x7fc5d4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5d17fa640 1 -- 172.21.15.120:0/3273829588 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800cd80 con 0x7fc5d41029b0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5d17fa640 1 -- 172.21.15.120:0/3273829588 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c80108e0 con 0x7fc5d41029b0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d4199ac0 con 0x7fc5d41029b0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5d17fa640 1 -- 172.21.15.120:0/3273829588 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8018980 con 0x7fc5d41029b0 2024-09-10T22:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.384+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a0190 con 0x7fc5d41029b0 2024-09-10T22:59:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.386+0000 7fc5d17fa640 1 -- 172.21.15.120:0/3273829588 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc5c8010430 con 0x7fc5d41029b0 2024-09-10T22:59:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.386+0000 7fc5dd1ba740 1 Processor -- start 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.386+0000 7fc5dd1ba740 1 -- start start 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7300 0x5623785e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56237855e6f0 con 0x5623785e7c40 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56237854db20 con 0x5623785e8600 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562378551930 con 0x5623785e7300 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38388/0 (socket says 172.21.15.120:38388) 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 -- 172.21.15.120:0/3273401416 learned_addr learned my addr 172.21.15.120:0/3273401416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d88ad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7300 0x5623785e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d17fa640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 0x7fc5bc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 -- 172.21.15.120:0/3273401416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7300 msgr2=0x5623785e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7300 0x5623785e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 -- 172.21.15.120:0/3273401416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 msgr2=0x5623785e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 -- 172.21.15.120:0/3273401416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562378513530 con 0x5623785e7c40 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 0x7fc5bc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d17fa640 1 -- 172.21.15.120:0/3273829588 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc5c80e5e70 con 0x7fc5d41029b0 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc5c402a940 tx=0x7fc5c4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c265932e2f5454 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5ba7fc640 1 -- 172.21.15.120:0/3273401416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4035030 con 0x5623785e7c40 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5ba7fc640 1 -- 172.21.15.120:0/3273401416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4040070 con 0x5623785e7c40 2024-09-10T22:59:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.387+0000 7fc5ba7fc640 1 -- 172.21.15.120:0/3273401416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4003850 con 0x5623785e7c40 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/3273401416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 msgr2=0x5623785e80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 0x7fc5bc0b7820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc5c402a940 tx=0x7fc5c4005de0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/3273401416 shutdown_connections 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785e80c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/3273401416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7300 0x5623785e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/3273401416 >> 172.21.15.120:0/3273401416 conn(0x5623785e9bc0 msgr2=0x5623785ea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/3273401416 shutdown_connections 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.388+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/3273401416 wait complete. 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 -- start start 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 Processor -- start 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562378485ab0 con 0x5623785e7c40 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5623784f3730 con 0x5623785e7300 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5623785f8fa0 con 0x5623785e8600 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5dd1ba740 10 client.0 _ll_register_callbacks cb 0x5623785476b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38392/0 (socket says 172.21.15.120:38392) 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5d37fe640 1 -- 172.21.15.120:0/2740786487 learned_addr learned my addr 172.21.15.120:0/2740786487 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.389+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 -- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 msgr2=0x5623785fb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785fb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 -- 172.21.15.120:0/2740786487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 msgr2=0x5623785fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 -- 172.21.15.120:0/2740786487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623782af790 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80ac6a0 tx=0x7fc5c80e9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf678c085099a57b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80e3e60 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc5c80e9020 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c4040070 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5c80ac4e0 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 10 client.16707 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562378328620 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4040070 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc5c80a8060 con 0x5623785e7c40 2024-09-10T22:59:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 10 client.16707 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 20 client.16707 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378605bd0 0x562378608000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 4 client.16707 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.390+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562378560640 con 0x562378605bd0 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.391+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378605bd0 0x562378608000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.391+0000 7fc5b37fe640 10 client.16707 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.391+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378605bd0 0x562378608000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547d1de372776b86 server_cookie=f31bc87c7da374fe in_seq=0 out_seq=0 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5b37fe640 1 -- 172.21.15.120:0/2740786487 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562378560640 con 0x562378605bd0 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5b37fe640 10 client.16707 handle_command_reply: tid=0 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 client.16707 shutdown 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 msgr2=0x5623785fa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80ac6a0 tx=0x7fc5c80e9540 comp rx=0 tx=0).stop 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 shutdown_connections 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378605bd0 0x562378608000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fa920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5b37fe640 0 client.16707 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2740786487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 >> 172.21.15.120:0/2740786487 conn(0x5623785e9bc0 msgr2=0x5623785e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 shutdown_connections 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2740786487 wait complete. 2024-09-10T22:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.392+0000 7fc5dd1ba740 20 client.16707 trim_cache size 0 max 16384 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 Processor -- start 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 -- start start 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7c40 0x5623785e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a5bb0 con 0x5623785e7300 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a8430 con 0x5623785e8600 2024-09-10T22:59:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a6220 con 0x5623785e7c40 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38404/0 (socket says 172.21.15.120:38404) 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5d3fff640 1 -- 172.21.15.120:0/2766369106 learned_addr learned my addr 172.21.15.120:0/2766369106 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.407+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7c40 0x5623785e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 -- 172.21.15.120:0/2766369106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7c40 msgr2=0x5623785e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7c40 0x5623785e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 -- 172.21.15.120:0/2766369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 msgr2=0x5623785f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 -- 172.21.15.120:0/2766369106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623785fb860 con 0x5623785e7300 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad779b4e5942de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5b9ffb640 1 -- 172.21.15.120:0/2766369106 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4035040 con 0x5623785e7300 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5b9ffb640 1 -- 172.21.15.120:0/2766369106 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4040070 con 0x5623785e7300 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.408+0000 7fc5b9ffb640 1 -- 172.21.15.120:0/2766369106 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4003b90 con 0x5623785e7300 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2766369106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 msgr2=0x5623785e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c4005b50 comp rx=0 tx=0).stop 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2766369106 shutdown_connections 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e8600 0x5623785f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e7c40 0x5623785e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/2766369106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7300 0x5623785e63d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2766369106 >> 172.21.15.120:0/2766369106 conn(0x5623785e9bc0 msgr2=0x5623785f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2766369106 shutdown_connections 2024-09-10T22:59:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/2766369106 wait complete. 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- start start 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 Processor -- start 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a68f0 con 0x5623785e7c40 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a7d60 con 0x5623785e7300 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41a6fc0 con 0x5623785e8600 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5dd1ba740 10 client.0 _ll_register_callbacks cb 0x7fc5c8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38416/0 (socket says 172.21.15.120:38416) 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.409+0000 7fc5d37fe640 1 -- 172.21.15.120:0/1592824110 learned_addr learned my addr 172.21.15.120:0/1592824110 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d88ad640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 -- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 msgr2=0x5623785ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785ff040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 -- 172.21.15.120:0/1592824110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 msgr2=0x5623785fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 -- 172.21.15.120:0/1592824110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623786023d0 con 0x5623785e7c40 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5d37fe640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80a8c90 tx=0x7fc5c80e96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891c5f4626f6b46b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80e3e60 con 0x5623785e7c40 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc5c8018d20 con 0x5623785e7c40 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c4040070 con 0x5623785e7c40 2024-09-10T22:59:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5c80abe80 con 0x5623785e7c40 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 10 client.16713 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623786731d0 con 0x5623785e7c40 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4040070 con 0x5623785e7c40 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc5c80e9a80 con 0x5623785e7c40 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 10 client.16713 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.410+0000 7fc5dd1ba740 20 client.16713 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378673b80 0x562378675fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5dd1ba740 4 client.16713 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56237855c0f0 con 0x562378673b80 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378673b80 0x562378675fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5b37fe640 10 client.16713 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.411+0000 7fc5d3fff640 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378673b80 0x562378675fa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a9d3ef968b278d server_cookie=9e3fe96872ee6cd in_seq=0 out_seq=0 2024-09-10T22:59:04.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:04 smithi153 ceph-mon[34951]: pgmap v545: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T22:59:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:04 smithi176 ceph-mon[34982]: pgmap v545: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T22:59:04.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1115884 16.53 MB/sec execute 239 sec latency 75.106 ms 2024-09-10T22:59:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5b37fe640 1 -- 172.21.15.120:0/1592824110 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56237855c0f0 con 0x562378673b80 2024-09-10T22:59:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5b37fe640 10 client.16713 handle_command_reply: tid=0 2024-09-10T22:59:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 client.16713 shutdown 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 msgr2=0x5623785fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80a8c90 tx=0x7fc5c80e96b0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 shutdown_connections 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562378673b80 0x562378675fa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623785e8600 0x5623785ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5623785e7c40 0x5623785fe330 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5b37fe640 0 client.16713 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 --2- 172.21.15.120:0/1592824110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5623785e7300 0x5623785fddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.741+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 >> 172.21.15.120:0/1592824110 conn(0x5623785e9bc0 msgr2=0x5623785e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 shutdown_connections 2024-09-10T22:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5dd1ba740 1 -- 172.21.15.120:0/1592824110 wait complete. 2024-09-10T22:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5dd1ba740 20 client.16713 trim_cache size 0 max 16384 2024-09-10T22:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:59:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:59:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 msgr2=0x7fc5bc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 0x7fc5bc0b7820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc004a40 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 msgr2=0x7fc5d41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 shutdown_connections 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc5bc0b5360 0x7fc5bc0b7820 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5d4103370 0x7fc5d4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5d41029b0 0x7fc5d41982e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 --2- 172.21.15.120:0/3273829588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5d4101860 0x7fc5d4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 >> 172.21.15.120:0/3273829588 conn(0x7fc5d40fd140 msgr2=0x7fc5d4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 shutdown_connections 2024-09-10T22:59:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:04.742+0000 7fc5da310640 1 -- 172.21.15.120:0/3273829588 wait complete. 2024-09-10T22:59:05.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1118960 16.54 MB/sec execute 240 sec latency 49.234 ms 2024-09-10T22:59:05.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:59:05.793 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:59:06.058 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:06.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:06 smithi120 ceph-mon[31247]: pgmap v546: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2024-09-10T22:59:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:06 smithi153 ceph-mon[34951]: pgmap v546: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2024-09-10T22:59:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:06 smithi176 ceph-mon[34982]: pgmap v546: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 361 op/s 2024-09-10T22:59:06.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1121896 16.53 MB/sec execute 241 sec latency 34.446 ms 2024-09-10T22:59:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- 172.21.15.120:0/1003440679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 msgr2=0x7f66fc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 --2- 172.21.15.120:0/1003440679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc108c10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002a410 comp rx=0 tx=0).stop 2024-09-10T22:59:06.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- 172.21.15.120:0/1003440679 shutdown_connections 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 --2- 172.21.15.120:0/1003440679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc109150 0x7f66fc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 --2- 172.21.15.120:0/1003440679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc108c10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 --2- 172.21.15.120:0/1003440679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc102dc0 0x7f66fc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- 172.21.15.120:0/1003440679 >> 172.21.15.120:0/1003440679 conn(0x7f66fc0fe870 msgr2=0x7f66fc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- 172.21.15.120:0/1003440679 shutdown_connections 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- 172.21.15.120:0/1003440679 wait complete. 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 Processor -- start 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.607+0000 7f670231b640 1 -- start start 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc102dc0 0x7f66fc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc109150 0x7f66fc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc112da0 con 0x7f66fc1087d0 2024-09-10T22:59:06.630 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc112c20 con 0x7f66fc102dc0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f670231b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc112f20 con 0x7f66fc109150 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38422/0 (socket says 172.21.15.120:38422) 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 -- 172.21.15.120:0/1677982102 learned_addr learned my addr 172.21.15.120:0/1677982102 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc102dc0 0x7f66fc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f67008b8640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc109150 0x7f66fc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 -- 172.21.15.120:0/1677982102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc109150 msgr2=0x7f66fc075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc109150 0x7f66fc075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 -- 172.21.15.120:0/1677982102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc102dc0 msgr2=0x7f66fc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc102dc0 0x7f66fc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 -- 172.21.15.120:0/1677982102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc1a76b0 con 0x7f66fc1087d0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.608+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66f0005940 tx=0x7f66f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.609+0000 7f66f97fa640 1 -- 172.21.15.120:0/1677982102 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f003f030 con 0x7f66fc1087d0 2024-09-10T22:59:06.631 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.609+0000 7f66f97fa640 1 -- 172.21.15.120:0/1677982102 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f0003aa0 con 0x7f66fc1087d0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.609+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc1a7940 con 0x7f66fc1087d0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.609+0000 7f66f97fa640 1 -- 172.21.15.120:0/1677982102 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0035070 con 0x7f66fc1087d0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.609+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc1a7e20 con 0x7f66fc1087d0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.611+0000 7f67046f0740 1 Processor -- start 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.611+0000 7f67046f0740 1 -- start start 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.611+0000 7f66f97fa640 1 -- 172.21.15.120:0/1677982102 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f66f003bce0 con 0x7f66fc1087d0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ebf0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebf2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56548eb5ae60 con 0x56548ebe3ee0 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56548eb4a290 con 0x56548ebe4800 2024-09-10T22:59:06.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67046f0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56548eb4e0a0 con 0x56548ebf0790 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38428/0 (socket says 172.21.15.120:38428) 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 -- 172.21.15.120:0/1931801534 learned_addr learned my addr 172.21.15.120:0/1931801534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebf2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ebf0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 -- 172.21.15.120:0/1931801534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 msgr2=0x56548ebf2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebf2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 -- 172.21.15.120:0/1931801534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 msgr2=0x56548ebf0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ebf0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 -- 172.21.15.120:0/1931801534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548e8a8930 con 0x56548ebe3ee0 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebf2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ebf0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:06.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f67008b8640 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f66f4010370 tx=0x7f66f4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30791219f7433e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66f97fa640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 0x7f66e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66e27fc640 1 -- 172.21.15.120:0/1931801534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4016070 con 0x56548ebe3ee0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66e27fc640 1 -- 172.21.15.120:0/1931801534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f4012bc0 con 0x56548ebe3ee0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.612+0000 7f66e27fc640 1 -- 172.21.15.120:0/1931801534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f401bbf0 con 0x56548ebe3ee0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 0x7f66e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f66f97fa640 1 -- 172.21.15.120:0/1677982102 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f66f01064d0 con 0x7f66fc1087d0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 -- 172.21.15.120:0/1931801534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 msgr2=0x56548ebe42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 0x7f66e40b77a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56548ebe7060 tx=0x7f66ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f66f4010370 tx=0x7f66f4010830 comp rx=0 tx=0).stop 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 -- 172.21.15.120:0/1931801534 shutdown_connections 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebf2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ebf0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 --2- 172.21.15.120:0/1931801534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe42c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 -- 172.21.15.120:0/1931801534 >> 172.21.15.120:0/1931801534 conn(0x56548ebe1d10 msgr2=0x56548ebe2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 -- 172.21.15.120:0/1931801534 shutdown_connections 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.613+0000 7f67046f0740 1 -- 172.21.15.120:0/1931801534 wait complete. 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 -- start start 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 Processor -- start 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56548e9ab890 con 0x56548ebe4800 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56548eae4d40 con 0x56548ebe3ee0 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56548ebf3440 con 0x56548ebf0790 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67046f0740 10 client.0 _ll_register_callbacks cb 0x56548eb48c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67008b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38432/0 (socket says 172.21.15.120:38432) 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55862/0 (socket says 172.21.15.120:55862) 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f66fb7fe640 1 -- 172.21.15.120:0/2107320341 learned_addr learned my addr 172.21.15.120:0/2107320341 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.614+0000 7f67008b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:51334/0 (socket says 172.21.15.120:51334) 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 -- 172.21.15.120:0/2107320341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 msgr2=0x56548ebed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 -- 172.21.15.120:0/2107320341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 msgr2=0x56548ebedd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 -- 172.21.15.120:0/2107320341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548ebfc900 con 0x56548ebf0790 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67008b8640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f4013d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e340378d8d0132a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f66db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f402a060 con 0x56548ebf0790 2024-09-10T22:59:06.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f66f4004c50 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f4012bc0 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66f4016050 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67046f0740 10 client.26200 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.615+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56548ebff2b0 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f4012bc0 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f66f400f040 con 0x56548ebf0790 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 10 client.26200 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 20 client.26200 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec00710 0x56548ec02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 4 client.26200 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56548eb5cdb0 con 0x56548ec00710 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f66fbfff640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec00710 0x56548ec02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f66db7fe640 10 client.26200 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.616+0000 7f66fbfff640 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec00710 0x56548ec02b00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca28f2596415634 server_cookie=1cf13062c97b04f5 in_seq=0 out_seq=0 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f66db7fe640 1 -- 172.21.15.120:0/2107320341 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56548eb5cdb0 con 0x56548ec00710 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f66db7fe640 10 client.26200 handle_command_reply: tid=0 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 client.26200 shutdown 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 msgr2=0x56548ebee280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f4013d80 comp rx=0 tx=0).stop 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 shutdown_connections 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec00710 0x56548ec02b00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ebee280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f66db7fe640 0 client.26200 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 --2- 172.21.15.120:0/2107320341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 >> 172.21.15.120:0/2107320341 conn(0x56548ebe1d10 msgr2=0x56548ebe0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.617+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 shutdown_connections 2024-09-10T22:59:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.618+0000 7f67046f0740 1 -- 172.21.15.120:0/2107320341 wait complete. 2024-09-10T22:59:06.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.618+0000 7f67046f0740 20 client.26200 trim_cache size 0 max 16384 2024-09-10T22:59:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.635+0000 7f67046f0740 1 Processor -- start 2024-09-10T22:59:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.635+0000 7f67046f0740 1 -- start start 2024-09-10T22:59:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ec080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ae4a0 con 0x56548ebe3ee0 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1b0d20 con 0x56548ebe4800 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67046f0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1aeb10 con 0x56548ebf0790 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38442/0 (socket says 172.21.15.120:38442) 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 -- 172.21.15.120:0/1542412189 learned_addr learned my addr 172.21.15.120:0/1542412189 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f67008b8640 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ec080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 -- 172.21.15.120:0/1542412189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 msgr2=0x56548ec08610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec08610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 -- 172.21.15.120:0/1542412189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 msgr2=0x56548ec080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ec080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 -- 172.21.15.120:0/1542412189 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548ebe1b40 con 0x56548ebe3ee0 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f66ec012cb0 tx=0x7f66ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbcb2f89c244ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66e1ffb640 1 -- 172.21.15.120:0/1542412189 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01dd70 con 0x56548ebe3ee0 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66e1ffb640 1 -- 172.21.15.120:0/1542412189 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec02a8c0 con 0x56548ebe3ee0 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.636+0000 7f66e1ffb640 1 -- 172.21.15.120:0/1542412189 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0308a0 con 0x56548ebe3ee0 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 -- 172.21.15.120:0/1542412189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 msgr2=0x56548ebe77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f66ec012cb0 tx=0x7f66ec019000 comp rx=0 tx=0).stop 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 -- 172.21.15.120:0/1542412189 shutdown_connections 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec08610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe4800 0x56548ec080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 --2- 172.21.15.120:0/1542412189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe3ee0 0x56548ebe77a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 -- 172.21.15.120:0/1542412189 >> 172.21.15.120:0/1542412189 conn(0x56548ebe1d10 msgr2=0x56548ebe7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 -- 172.21.15.120:0/1542412189 shutdown_connections 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.637+0000 7f67046f0740 1 -- 172.21.15.120:0/1542412189 wait complete. 2024-09-10T22:59:06.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 -- start start 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 Processor -- start 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebf4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec6c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1af1e0 con 0x56548ebe4800 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1b0650 con 0x56548ebe3ee0 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1af8b0 con 0x56548ebf0790 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67046f0740 10 client.0 _ll_register_callbacks cb 0x7f66f4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebf4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38458/0 (socket says 172.21.15.120:38458) 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f66fb7fe640 1 -- 172.21.15.120:0/1632249760 learned_addr learned my addr 172.21.15.120:0/1632249760 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.638+0000 7f67008b8640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec6c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 -- 172.21.15.120:0/1632249760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 msgr2=0x56548ebf4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebf4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 -- 172.21.15.120:0/1632249760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 msgr2=0x56548ebf53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 -- 172.21.15.120:0/1632249760 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548ec6c970 con 0x56548ebf0790 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66fb7fe640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebf4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67008b8640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec6c2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009f20 tx=0x7f66f4005c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe8fa7dd1894195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4035020 con 0x56548ebf0790 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f66f4021070 con 0x56548ebf0790 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec02a8c0 con 0x56548ebf0790 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66f402a040 con 0x56548ebf0790 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67046f0740 10 client.26206 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:06.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.639+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56548ec6dc90 con 0x56548ebf0790 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec02a8c0 con 0x56548ebf0790 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f66f4004e80 con 0x56548ebf0790 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 10 client.26206 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 20 client.26206 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec6e380 0x56548ec707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 4 client.26206 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56548eb58860 con 0x56548ec6e380 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.640+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec6e380 0x56548ec707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.641+0000 7f66fbfff640 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec6e380 0x56548ec707a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5451c17835d908b server_cookie=51f3ca87f11d7df9 in_seq=0 out_seq=0 2024-09-10T22:59:06.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.641+0000 7f66db7fe640 10 client.26206 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:06.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f66db7fe640 1 -- 172.21.15.120:0/1632249760 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56548eb58860 con 0x56548ec6e380 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f66db7fe640 10 client.26206 handle_command_reply: tid=0 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 client.26206 shutdown 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 msgr2=0x56548ec6c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec6c2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009f20 tx=0x7f66f4005c10 comp rx=0 tx=0).stop 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 shutdown_connections 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56548ec6e380 0x56548ec707a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56548ebf0790 0x56548ec6c2d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56548ebe4800 0x56548ebf53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 --2- 172.21.15.120:0/1632249760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56548ebe3ee0 0x56548ebf4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f66db7fe640 0 client.26206 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:06.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 >> 172.21.15.120:0/1632249760 conn(0x56548ebe1d10 msgr2=0x56548ebef650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 shutdown_connections 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 1 -- 172.21.15.120:0/1632249760 wait complete. 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.643+0000 7f67046f0740 20 client.26206 trim_cache size 0 max 16384 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "76fd4309-53dc-472d-bac3-a65eb1ac4495", 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:59:06.651 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 msgr2=0x7f66e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 0x7f66e40b77a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56548ebe7060 tx=0x7f66ec005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 msgr2=0x7f66fc0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66f0005940 tx=0x7f66f0004270 comp rx=0 tx=0).stop 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 shutdown_connections 2024-09-10T22:59:06.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f66e40b52e0 0x7f66e40b77a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66fc109150 0x7f66fc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66fc1087d0 0x7f66fc0759e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 --2- 172.21.15.120:0/1677982102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66fc102dc0 0x7f66fc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 >> 172.21.15.120:0/1677982102 conn(0x7f66fc0fe870 msgr2=0x7f66fc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 shutdown_connections 2024-09-10T22:59:06.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:06.644+0000 7f670231b640 1 -- 172.21.15.120:0/1677982102 wait complete. 2024-09-10T22:59:07.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:07 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:59:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:07 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:07 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:59:07.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:07 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:07 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:59:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:07 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:07 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:59:07.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:07 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:07 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:59:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:07 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:07 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:59:07.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:07 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1124567 16.51 MB/sec execute 242 sec latency 46.015 ms 2024-09-10T22:59:07.803 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:08.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 -- 172.21.15.120:0/2412349292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 msgr2=0x7f836810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 --2- 172.21.15.120:0/2412349292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836810e820 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ca0 tx=0x7f836002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 -- 172.21.15.120:0/2412349292 shutdown_connections 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 --2- 172.21.15.120:0/2412349292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836810e820 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 --2- 172.21.15.120:0/2412349292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83681076a0 0x7f8368107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 --2- 172.21.15.120:0/2412349292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8368101c90 0x7f8368102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 -- 172.21.15.120:0/2412349292 >> 172.21.15.120:0/2412349292 conn(0x7f83680fd700 msgr2=0x7f83680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.347+0000 7f836efdb640 1 -- 172.21.15.120:0/2412349292 shutdown_connections 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 -- 172.21.15.120:0/2412349292 wait complete. 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 Processor -- start 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 -- start start 2024-09-10T22:59:08.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8368101c90 0x7f8368199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83681076a0 0x7f8368199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111ca0 con 0x7f8368108020 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111b20 con 0x7f8368101c90 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836efdb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8368111e20 con 0x7f83681076a0 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.348+0000 7f836d578640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38468/0 (socket says 172.21.15.120:38468) 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f8367fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83681076a0 0x7f8368199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 -- 172.21.15.120:0/3258612405 learned_addr learned my addr 172.21.15.120:0/3258612405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:08.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836cd77640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8368101c90 0x7f8368199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 -- 172.21.15.120:0/3258612405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83681076a0 msgr2=0x7f8368199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83681076a0 0x7f8368199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 -- 172.21.15.120:0/3258612405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8368101c90 msgr2=0x7f8368199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8368101c90 0x7f8368199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 -- 172.21.15.120:0/3258612405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836819f2d0 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836d578640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8360005cc0 tx=0x7f836002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f8365ffb640 1 -- 172.21.15.120:0/3258612405 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360003bf0 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f8365ffb640 1 -- 172.21.15.120:0/3258612405 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8360003d90 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836819f560 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.349+0000 7f8365ffb640 1 -- 172.21.15.120:0/3258612405 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836002fbc0 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.350+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83680fdd70 con 0x7f8368108020 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.352+0000 7f83713b0740 1 Processor -- start 2024-09-10T22:59:08.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.352+0000 7f8365ffb640 1 -- 172.21.15.120:0/3258612405 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8360004610 con 0x7f8368108020 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.352+0000 7f83713b0740 1 -- start start 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 0x565416e767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416e79120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565416de0e50 con 0x565416e6a480 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565416dd0280 con 0x565416e76d30 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f83713b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565416dd4090 con 0x565416e6ada0 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8367fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 0x565416e767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8367fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38478/0 (socket says 172.21.15.120:38478) 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8367fff640 1 -- 172.21.15.120:0/2591013878 learned_addr learned my addr 172.21.15.120:0/2591013878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8367fff640 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416e79120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 -- 172.21.15.120:0/2591013878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 msgr2=0x565416e79120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8365ffb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 0x7f83580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416e79120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 -- 172.21.15.120:0/2591013878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 msgr2=0x565416e6a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 -- 172.21.15.120:0/2591013878 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565416b2e650 con 0x565416e6ada0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8367fff640 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836d578640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 0x7f83580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f8365ffb640 1 -- 172.21.15.120:0/3258612405 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8360106f70 con 0x7f8368108020 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.353+0000 7f836cd77640 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 0x565416e767f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f835c005fd0 tx=0x7f835c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab76053ef4d20ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f8352ffd640 1 -- 172.21.15.120:0/2591013878 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c01c680 con 0x565416e6ada0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f8352ffd640 1 -- 172.21.15.120:0/2591013878 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c00be20 con 0x565416e6ada0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f8352ffd640 1 -- 172.21.15.120:0/2591013878 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c011c00 con 0x565416e6ada0 2024-09-10T22:59:08.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 -- 172.21.15.120:0/2591013878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 msgr2=0x565416e767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f836d578640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 0x7f83580b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83600c9dd0 tx=0x7f836002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 0x565416e767f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f835c005fd0 tx=0x7f835c0107a0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 -- 172.21.15.120:0/2591013878 shutdown_connections 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416e79120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6ada0 0x565416e767f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 --2- 172.21.15.120:0/2591013878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6a480 0x565416e6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 -- 172.21.15.120:0/2591013878 >> 172.21.15.120:0/2591013878 conn(0x565416e682b0 msgr2=0x565416e69450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 -- 172.21.15.120:0/2591013878 shutdown_connections 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.354+0000 7f83713b0740 1 -- 172.21.15.120:0/2591013878 wait complete. 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 -- start start 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 Processor -- start 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e76610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x565416c31700 con 0x565416e6ada0 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x565416d6b110 con 0x565416e6a480 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565416e799b0 con 0x565416e76d30 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f83713b0740 10 client.0 _ll_register_callbacks cb 0x565416d84990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38484/0 (socket says 172.21.15.120:38484) 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f836cd77640 1 -- 172.21.15.120:0/3263140068 learned_addr learned my addr 172.21.15.120:0/3263140068 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.355+0000 7f8367fff640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8367fff640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e76610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 -- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 msgr2=0x565416e74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 -- 172.21.15.120:0/3263140068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 msgr2=0x565416e76610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e76610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 -- 172.21.15.120:0/3263140068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565416d98c40 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8347fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f836cd77640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f835c000c00 tx=0x7f835c011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99ee6f99823cfec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c02a060 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f835c011d50 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c00be20 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f835c005040 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f83713b0740 10 client.16743 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565416e852d0 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.356+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835c00be20 con 0x565416e6ada0 2024-09-10T22:59:08.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f835c01c480 con 0x565416e6ada0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 10 client.16743 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 20 client.16743 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416e866d0 0x565416e88ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 4 client.16743 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565416de2da0 con 0x565416e866d0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f8367fff640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416e866d0 0x565416e88ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f8347fff640 10 client.16743 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.357+0000 7f8367fff640 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416e866d0 0x565416e88ac0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1ff8ab69800db6 server_cookie=8a4dd813d014c98b in_seq=0 out_seq=0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f8347fff640 1 -- 172.21.15.120:0/3263140068 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565416de2da0 con 0x565416e866d0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f8347fff640 10 client.16743 handle_command_reply: tid=0 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 client.16743 shutdown 2024-09-10T22:59:08.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 msgr2=0x565416e73a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f835c000c00 tx=0x7f835c011670 comp rx=0 tx=0).stop 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 shutdown_connections 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416e866d0 0x565416e88ac0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e73a60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 --2- 172.21.15.120:0/3263140068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e76610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 >> 172.21.15.120:0/3263140068 conn(0x565416e682b0 msgr2=0x565416e6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f8347fff640 0 client.16743 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 shutdown_connections 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.358+0000 7f83713b0740 1 -- 172.21.15.120:0/3263140068 wait complete. 2024-09-10T22:59:08.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.359+0000 7f83713b0740 20 client.16743 trim_cache size 0 max 16384 2024-09-10T22:59:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 Processor -- start 2024-09-10T22:59:08.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 -- start start 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e68f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83681aa090 con 0x565416e6ada0 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ac8f0 con 0x565416e6a480 2024-09-10T22:59:08.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.379+0000 7f83713b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83681aa6e0 con 0x565416e76d30 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e68f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e68f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38496/0 (socket says 172.21.15.120:38496) 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f836cd77640 1 -- 172.21.15.120:0/3062829977 learned_addr learned my addr 172.21.15.120:0/3062829977 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e67bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 -- 172.21.15.120:0/3062829977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 msgr2=0x565416e67bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e67bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 -- 172.21.15.120:0/3062829977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 msgr2=0x565416e68f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e68f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 -- 172.21.15.120:0/3062829977 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565416e69670 con 0x565416e76d30 2024-09-10T22:59:08.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f8367fff640 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e78120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x565416c31810 tx=0x7f8354004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c902543c4d4a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.380+0000 7f83527fc640 1 -- 172.21.15.120:0/3062829977 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354004220 con 0x565416e76d30 2024-09-10T22:59:08.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83527fc640 1 -- 172.21.15.120:0/3062829977 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8354026e30 con 0x565416e76d30 2024-09-10T22:59:08.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83527fc640 1 -- 172.21.15.120:0/3062829977 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540047a0 con 0x565416e76d30 2024-09-10T22:59:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 -- 172.21.15.120:0/3062829977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 msgr2=0x565416e78120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e78120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x565416c31810 tx=0x7f8354004730 comp rx=0 tx=0).stop 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 -- 172.21.15.120:0/3062829977 shutdown_connections 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e76d30 0x565416e78120 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e68f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 --2- 172.21.15.120:0/3062829977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e6a480 0x565416e67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 -- 172.21.15.120:0/3062829977 >> 172.21.15.120:0/3062829977 conn(0x565416e682b0 msgr2=0x565416e722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 -- 172.21.15.120:0/3062829977 shutdown_connections 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.381+0000 7f83713b0740 1 -- 172.21.15.120:0/3062829977 wait complete. 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 -- start start 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 Processor -- start 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6a480 0x565416e87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416eec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83681aadb0 con 0x565416e6ada0 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ac220 con 0x565416e76d30 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ab480 con 0x565416e6a480 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f83713b0740 10 client.0 _ll_register_callbacks cb 0x7f835400ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f836cd77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38502/0 (socket says 172.21.15.120:38502) 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.382+0000 7f836cd77640 1 -- 172.21.15.120:0/2766029091 learned_addr learned my addr 172.21.15.120:0/2766029091 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8367fff640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6a480 0x565416e87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8367fff640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416eec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 -- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6a480 msgr2=0x565416e87fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6a480 0x565416e87fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 -- 172.21.15.120:0/2766029091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 msgr2=0x565416eec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416eec200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 -- 172.21.15.120:0/2766029091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565416eec910 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8367fff640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416eec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f836cd77640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f835c01ca10 tx=0x7f835c00b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad31af307d05345d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8347fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c03e070 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f835c011b50 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8354026e30 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f835c035050 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.383+0000 7f83713b0740 10 client.16749 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565416ef3ce0 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8354026e30 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f835c041020 con 0x565416e6ada0 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 10 client.16749 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:08.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 20 client.16749 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416ef43c0 0x565416ef67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 4 client.16749 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565416dde850 con 0x565416ef43c0 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f8367fff640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416ef43c0 0x565416ef67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.384+0000 7f8347fff640 10 client.16749 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f8367fff640 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416ef43c0 0x565416ef67e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98472889ee52bda2 server_cookie=63163f8c06f4b05a in_seq=0 out_seq=0 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f8347fff640 1 -- 172.21.15.120:0/2766029091 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x565416dde850 con 0x565416ef43c0 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f8347fff640 10 client.16749 handle_command_reply: tid=0 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 client.16749 shutdown 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 msgr2=0x565416e88520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f835c01ca10 tx=0x7f835c00b490 comp rx=0 tx=0).stop 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 shutdown_connections 2024-09-10T22:59:08.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x565416ef43c0 0x565416ef67e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565416e76d30 0x565416eec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x565416e6ada0 0x565416e88520 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f8347fff640 0 client.16749 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 --2- 172.21.15.120:0/2766029091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565416e6a480 0x565416e87fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 >> 172.21.15.120:0/2766029091 conn(0x565416e682b0 msgr2=0x565416e75090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 shutdown_connections 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 1 -- 172.21.15.120:0/2766029091 wait complete. 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.385+0000 7f83713b0740 20 client.16749 trim_cache size 0 max 16384 2024-09-10T22:59:08.391 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "76fd4309-53dc-472d-bac3-a65eb1ac4495", 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:59:08.393 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.394 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:59:08.396 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:59:08.400 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:08.401 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 msgr2=0x7f83580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 0x7f83580b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83600c9dd0 tx=0x7f836002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 msgr2=0x7f836819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8360005cc0 tx=0x7f836002f380 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 shutdown_connections 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f83580b52e0 0x7f83580b77a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8368108020 0x7f836819eb00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83681076a0 0x7f8368199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 --2- 172.21.15.120:0/3258612405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8368101c90 0x7f8368199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 >> 172.21.15.120:0/3258612405 conn(0x7f83680fd700 msgr2=0x7f836806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 shutdown_connections 2024-09-10T22:59:08.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:08.386+0000 7f836efdb640 1 -- 172.21.15.120:0/3258612405 wait complete. 2024-09-10T22:59:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:08 smithi120 ceph-mon[31247]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 566 op/s 2024-09-10T22:59:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:08 smithi153 ceph-mon[34951]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 566 op/s 2024-09-10T22:59:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:08 smithi176 ceph-mon[34982]: pgmap v547: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 566 op/s 2024-09-10T22:59:08.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1127489 16.52 MB/sec execute 243 sec latency 15.033 ms 2024-09-10T22:59:09.218 INFO:tasks.cephfs.filesystem:scrub status for tag:76fd4309-53dc-472d-bac3-a65eb1ac4495 - {'path': '/', 'tag': '76fd4309-53dc-472d-bac3-a65eb1ac4495', 'options': 'recursive,force'} 2024-09-10T22:59:09.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1130106 16.51 MB/sec execute 244 sec latency 54.136 ms 2024-09-10T22:59:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:10 smithi153 ceph-mon[34951]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T22:59:10.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:10 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:59:10.490 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:10 smithi120 ceph-mon[31247]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T22:59:10.490 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:10 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:59:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:10 smithi176 ceph-mon[34982]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T22:59:10.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:10 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:59:10.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1133359 16.52 MB/sec execute 245 sec latency 41.905 ms 2024-09-10T22:59:11.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1136727 16.52 MB/sec execute 246 sec latency 16.660 ms 2024-09-10T22:59:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:12 smithi176 ceph-mon[34982]: pgmap v549: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T22:59:12.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1139849 16.52 MB/sec execute 247 sec latency 17.775 ms 2024-09-10T22:59:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:12 smithi120 ceph-mon[31247]: pgmap v549: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T22:59:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:12 smithi153 ceph-mon[34951]: pgmap v549: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 501 op/s 2024-09-10T22:59:13.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1142803 16.51 MB/sec execute 248 sec latency 14.054 ms 2024-09-10T22:59:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:14 smithi176 ceph-mon[34982]: pgmap v550: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-09-10T22:59:14.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1145237 16.50 MB/sec execute 249 sec latency 79.618 ms 2024-09-10T22:59:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:14 smithi120 ceph-mon[31247]: pgmap v550: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-09-10T22:59:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:14 smithi153 ceph-mon[34951]: pgmap v550: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-09-10T22:59:15.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1148073 16.50 MB/sec execute 250 sec latency 30.797 ms 2024-09-10T22:59:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:16 smithi176 ceph-mon[34982]: pgmap v551: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-10T22:59:16.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1151310 16.50 MB/sec execute 251 sec latency 12.089 ms 2024-09-10T22:59:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:16 smithi120 ceph-mon[31247]: pgmap v551: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-10T22:59:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:16 smithi153 ceph-mon[34951]: pgmap v551: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-10T22:59:17.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1154685 16.51 MB/sec execute 252 sec latency 19.098 ms 2024-09-10T22:59:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 -- 172.21.15.120:0/2936726084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4101860 msgr2=0x7f6de4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2936726084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4101860 0x7f6de4101c60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009930 tx=0x7f6dd402a420 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 -- 172.21.15.120:0/2936726084 shutdown_connections 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2936726084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4103370 0x7f6de4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2936726084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 0x7f6de4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2936726084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4101860 0x7f6de4101c60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 -- 172.21.15.120:0/2936726084 >> 172.21.15.120:0/2936726084 conn(0x7f6de40fd140 msgr2=0x7f6de40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 -- 172.21.15.120:0/2936726084 shutdown_connections 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 -- 172.21.15.120:0/2936726084 wait complete. 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.831+0000 7f6de96b8640 1 Processor -- start 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 -- start start 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 0x7f6de4198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 0x7f6de4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d4b0 con 0x7f6de4103370 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d330 con 0x7f6de4101860 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de96b8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6de410d630 con 0x7f6de41029b0 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37960/0 (socket says 172.21.15.120:37960) 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 -- 172.21.15.120:0/2599821658 learned_addr learned my addr 172.21.15.120:0/2599821658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 0x7f6de4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 0x7f6de4198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 -- 172.21.15.120:0/2599821658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 msgr2=0x7f6de4199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 0x7f6de4199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 -- 172.21.15.120:0/2599821658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 msgr2=0x7f6de4198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 0x7f6de4198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 -- 172.21.15.120:0/2599821658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de419a910 con 0x7f6de4103370 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 0x7f6de4198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.832+0000 7f6de37fe640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.833+0000 7f6de0ff9640 1 -- 172.21.15.120:0/2599821658 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc017070 con 0x7f6de4103370 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.833+0000 7f6de0ff9640 1 -- 172.21.15.120:0/2599821658 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc003e20 con 0x7f6de4103370 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.833+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de419ac00 con 0x7f6de4103370 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.833+0000 7f6de0ff9640 1 -- 172.21.15.120:0/2599821658 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc010460 con 0x7f6de4103370 2024-09-10T22:59:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.833+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41a12d0 con 0x7f6de4103370 2024-09-10T22:59:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.836+0000 7f6dec481740 1 Processor -- start 2024-09-10T22:59:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.836+0000 7f6de0ff9640 1 -- 172.21.15.120:0/2599821658 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6ddc003970 con 0x7f6de4103370 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.836+0000 7f6dec481740 1 -- start start 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeaf190 0x5605ffeb1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5605ffe1e6f0 con 0x5605ffeac860 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605ffe0db20 con 0x5605ffeb1bd0 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dec481740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5605ffe11930 con 0x5605ffeaf190 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37968/0 (socket says 172.21.15.120:37968) 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeaf190 0x5605ffeb1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 -- 172.21.15.120:0/4068837344 learned_addr learned my addr 172.21.15.120:0/4068837344 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 -- 172.21.15.120:0/4068837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeaf190 msgr2=0x5605ffeb1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeaf190 0x5605ffeb1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 -- 172.21.15.120:0/4068837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeb1bd0 msgr2=0x5605ffeb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 -- 172.21.15.120:0/4068837344 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ffdd3530 con 0x5605ffeac860 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6de0ff9640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 0x7f6dd00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009da0 tx=0x7f6dcc013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43f1b038513f34d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 0x7f6dd00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.837+0000 7f6dd97fa640 1 -- 172.21.15.120:0/4068837344 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc010430 con 0x5605ffeac860 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6de0ff9640 1 -- 172.21.15.120:0/2599821658 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6ddc0e5940 con 0x7f6de4103370 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dd97fa640 1 -- 172.21.15.120:0/4068837344 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc010a80 con 0x5605ffeac860 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dd97fa640 1 -- 172.21.15.120:0/4068837344 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc01b940 con 0x5605ffeac860 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 -- 172.21.15.120:0/4068837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 msgr2=0x5605ffeaec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 0x7f6dd00b77a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000c00 tx=0x7f6dd40058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009da0 tx=0x7f6dcc013830 comp rx=0 tx=0).stop 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 -- 172.21.15.120:0/4068837344 shutdown_connections 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeaf190 0x5605ffeb1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 --2- 172.21.15.120:0/4068837344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeac860 0x5605ffeaec50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 -- 172.21.15.120:0/4068837344 >> 172.21.15.120:0/4068837344 conn(0x5605ffea79b0 msgr2=0x5605ffea8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 -- 172.21.15.120:0/4068837344 shutdown_connections 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.838+0000 7f6dec481740 1 -- 172.21.15.120:0/4068837344 wait complete. 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 -- start start 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 Processor -- start 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5605ffd45ab0 con 0x5605ffeaf190 2024-09-10T22:59:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605ffdb3730 con 0x5605ffeac860 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5605ffeb4b50 con 0x5605ffeb1bd0 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dec481740 10 client.0 _ll_register_callbacks cb 0x5605ffe076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37970/0 (socket says 172.21.15.120:37970) 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2684101066 learned_addr learned my addr 172.21.15.120:0/2684101066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6de37fe640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 msgr2=0x5605ffeb7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2684101066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 msgr2=0x5605ffeb5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.839+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2684101066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ffb6f790 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009f90 tx=0x7f6dcc0104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897dabed0d505215 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0052c0 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc010a80 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dcc005460 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc02d070 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 10 client.16761 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605ffbe8620 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc010a80 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dcc0035f0 con 0x5605ffeaf190 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 10 client.16761 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 20 client.16761 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605ffec14d0 0x5605ffec38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 4 client.16761 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.840+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ffe20640 con 0x5605ffec14d0 2024-09-10T22:59:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.841+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605ffec14d0 0x5605ffec38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.841+0000 7f6dca7fc640 10 client.16761 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.841+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605ffec14d0 0x5605ffec38c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e14f60a8bf837b2 server_cookie=da4bfd0bfc77b9e0 in_seq=0 out_seq=0 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2684101066 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605ffe20640 con 0x5605ffec14d0 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dca7fc640 10 client.16761 handle_command_reply: tid=0 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 client.16761 shutdown 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 msgr2=0x5605ffeb6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009f90 tx=0x7f6dcc0104b0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 shutdown_connections 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605ffec14d0 0x5605ffec38c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6560 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 --2- 172.21.15.120:0/2684101066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dca7fc640 0 client.16761 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 >> 172.21.15.120:0/2684101066 conn(0x5605ffea79b0 msgr2=0x5605ffea6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 shutdown_connections 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 1 -- 172.21.15.120:0/2684101066 wait complete. 2024-09-10T22:59:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.842+0000 7f6dec481740 20 client.16761 trim_cache size 0 max 16384 2024-09-10T22:59:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.857+0000 7f6dec481740 1 Processor -- start 2024-09-10T22:59:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.857+0000 7f6dec481740 1 -- start start 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a7e90 con 0x5605ffeaf190 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41aa710 con 0x5605ffeac860 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dec481740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a8500 con 0x5605ffeb1bd0 2024-09-10T22:59:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37980/0 (socket says 172.21.15.120:37980) 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45462/0 (socket says 172.21.15.120:45462) 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2722475626 learned_addr learned my addr 172.21.15.120:0/2722475626 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2722475626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 msgr2=0x5605ffeb4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2722475626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 msgr2=0x5605ffeb5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.858+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2722475626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ffeb18f0 con 0x5605ffeaf190 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.859+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc003a40 tx=0x7f6dcc009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3917ccd7aee94ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.859+0000 7f6dd8ff9640 1 -- 172.21.15.120:0/2722475626 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc036070 con 0x5605ffeaf190 2024-09-10T22:59:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.859+0000 7f6dd8ff9640 1 -- 172.21.15.120:0/2722475626 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc021070 con 0x5605ffeaf190 2024-09-10T22:59:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.859+0000 7f6dd8ff9640 1 -- 172.21.15.120:0/2722475626 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc005600 con 0x5605ffeaf190 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- 172.21.15.120:0/2722475626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 msgr2=0x5605ffeb6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc003a40 tx=0x7f6dcc009b70 comp rx=0 tx=0).stop 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- 172.21.15.120:0/2722475626 shutdown_connections 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeb1bd0 0x5605ffeb4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeaf190 0x5605ffeb6400 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- 172.21.15.120:0/2722475626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeac860 0x5605ffeb5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- 172.21.15.120:0/2722475626 >> 172.21.15.120:0/2722475626 conn(0x5605ffea79b0 msgr2=0x5605ffeaaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- 172.21.15.120:0/2722475626 shutdown_connections 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- 172.21.15.120:0/2722475626 wait complete. 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- start start 2024-09-10T22:59:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 Processor -- start 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeac860 0x5605ffebaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 0x5605ffebbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a8bd0 con 0x5605ffeb1bd0 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41aa040 con 0x5605ffeaf190 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41a92a0 con 0x5605ffeac860 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.860+0000 7f6dec481740 10 client.0 _ll_register_callbacks cb 0x7f6dcc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6de2ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeac860 0x5605ffebaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:57002/0 (socket says 172.21.15.120:57002) 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6de37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 0x5605ffebbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2138467571 learned_addr learned my addr 172.21.15.120:0/2138467571 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeac860 msgr2=0x5605ffebaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeac860 0x5605ffebaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2138467571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 msgr2=0x5605ffebbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 0x5605ffebbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 -- 172.21.15.120:0/2138467571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ffebc4f0 con 0x5605ffeaf190 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6de37fe640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 0x5605ffebbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dda7fc640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c00 tx=0x7f6dcc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb611d90dc38164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc03f070 con 0x5605ffeaf190 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc021070 con 0x5605ffeaf190 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dcc003c30 con 0x5605ffeaf190 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc04a050 con 0x5605ffeaf190 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dec481740 10 client.26168 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.861+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605ffebd860 con 0x5605ffeaf190 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc021070 con 0x5605ffeaf190 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dcc032480 con 0x5605ffeaf190 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 10 client.26168 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 20 client.26168 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605fff2e390 0x5605fff30780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 4 client.26168 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605ffe1c0f0 con 0x5605fff2e390 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.862+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605fff2e390 0x5605fff30780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.863+0000 7f6dca7fc640 10 client.26168 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:17.863+0000 7f6de2ffd640 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605fff2e390 0x5605fff30780 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1adf44be84b7b7c8 server_cookie=c48a2c9c84c854d4 in_seq=0 out_seq=0 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dca7fc640 1 -- 172.21.15.120:0/2138467571 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605ffe1c0f0 con 0x5605fff2e390 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dca7fc640 10 client.26168 handle_command_reply: tid=0 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 client.26168 shutdown 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 msgr2=0x5605ffebb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c00 tx=0x7f6dcc009f70 comp rx=0 tx=0).stop 2024-09-10T22:59:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 shutdown_connections 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605fff2e390 0x5605fff30780 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605ffeb1bd0 0x5605ffebbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dca7fc640 0 client.26168 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605ffeaf190 0x5605ffebb140 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 --2- 172.21.15.120:0/2138467571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605ffeac860 0x5605ffebaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 >> 172.21.15.120:0/2138467571 conn(0x5605ffea79b0 msgr2=0x5605ffeb5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 shutdown_connections 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.046+0000 7f6dec481740 1 -- 172.21.15.120:0/2138467571 wait complete. 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6dec481740 20 client.26168 trim_cache size 0 max 16384 2024-09-10T22:59:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:59:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:59:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:59:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 msgr2=0x7f6dd00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 0x7f6dd00b77a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000c00 tx=0x7f6dd40058b0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 msgr2=0x7f6de419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009fd0 tx=0x7f6ddc00b260 comp rx=0 tx=0).stop 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 shutdown_connections 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6dd00b52e0 0x7f6dd00b77a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6de4103370 0x7f6de419a050 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6de41029b0 0x7f6de4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 --2- 172.21.15.120:0/2599821658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6de4101860 0x7f6de4198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 >> 172.21.15.120:0/2599821658 conn(0x7f6de40fd140 msgr2=0x7f6de4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 shutdown_connections 2024-09-10T22:59:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:18.047+0000 7f6de96b8640 1 -- 172.21.15.120:0/2599821658 wait complete. 2024-09-10T22:59:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:18 smithi176 ceph-mon[34982]: pgmap v552: 97 pgs: 97 active+clean; 59 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-09-10T22:59:18.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1157315 16.50 MB/sec execute 253 sec latency 89.647 ms 2024-09-10T22:59:18.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:18 smithi120 ceph-mon[31247]: pgmap v552: 97 pgs: 97 active+clean; 59 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-09-10T22:59:18.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:18 smithi153 ceph-mon[34951]: pgmap v552: 97 pgs: 97 active+clean; 59 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-09-10T22:59:19.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1160480 16.49 MB/sec execute 254 sec latency 15.976 ms 2024-09-10T22:59:20.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:20 smithi120 ceph-mon[31247]: pgmap v553: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-10T22:59:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:20 smithi176 ceph-mon[34982]: pgmap v553: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-10T22:59:20.628 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1163083 16.49 MB/sec execute 255 sec latency 52.467 ms 2024-09-10T22:59:20.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:20 smithi153 ceph-mon[34951]: pgmap v553: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-10T22:59:21.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1166059 16.49 MB/sec execute 256 sec latency 26.630 ms 2024-09-10T22:59:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:22 smithi176 ceph-mon[34982]: pgmap v554: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-09-10T22:59:22.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1169394 16.49 MB/sec execute 257 sec latency 12.687 ms 2024-09-10T22:59:22.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:22 smithi120 ceph-mon[31247]: pgmap v554: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-09-10T22:59:22.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:22 smithi153 ceph-mon[34951]: pgmap v554: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-09-10T22:59:23.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1172806 16.50 MB/sec execute 258 sec latency 22.761 ms 2024-09-10T22:59:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:24 smithi176 ceph-mon[34982]: pgmap v555: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:59:24.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1176324 16.51 MB/sec execute 259 sec latency 13.054 ms 2024-09-10T22:59:24.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:24 smithi120 ceph-mon[31247]: pgmap v555: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:59:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:24 smithi153 ceph-mon[34951]: pgmap v555: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-09-10T22:59:25.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1179440 16.52 MB/sec execute 260 sec latency 65.697 ms 2024-09-10T22:59:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:26 smithi176 ceph-mon[34982]: pgmap v556: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T22:59:26.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1182690 16.52 MB/sec execute 261 sec latency 15.573 ms 2024-09-10T22:59:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:26 smithi120 ceph-mon[31247]: pgmap v556: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T22:59:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:26 smithi153 ceph-mon[34951]: pgmap v556: 97 pgs: 97 active+clean; 54 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T22:59:27.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1185915 16.52 MB/sec execute 262 sec latency 21.891 ms 2024-09-10T22:59:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:28 smithi176 ceph-mon[34982]: pgmap v557: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T22:59:28.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1188720 16.51 MB/sec execute 263 sec latency 12.237 ms 2024-09-10T22:59:28.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:28 smithi120 ceph-mon[31247]: pgmap v557: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T22:59:28.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:28 smithi153 ceph-mon[34951]: pgmap v557: 97 pgs: 97 active+clean; 55 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T22:59:29.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1191685 16.51 MB/sec execute 264 sec latency 15.960 ms 2024-09-10T22:59:30.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:30 smithi153 ceph-mon[34951]: pgmap v558: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-09-10T22:59:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:30 smithi176 ceph-mon[34982]: pgmap v558: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-09-10T22:59:30.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1194103 16.50 MB/sec execute 265 sec latency 49.858 ms 2024-09-10T22:59:30.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:30 smithi120 ceph-mon[31247]: pgmap v558: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.136+0000 7f50bcd94640 1 -- 172.21.15.120:0/1203059062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8105ad0 msgr2=0x7f50b8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.136+0000 7f50bcd94640 1 --2- 172.21.15.120:0/1203059062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8105ad0 0x7f50b8107f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a002a410 comp rx=0 tx=0).stop 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 -- 172.21.15.120:0/1203059062 shutdown_connections 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 --2- 172.21.15.120:0/1203059062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b8108680 0x7f50b810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 --2- 172.21.15.120:0/1203059062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8105ad0 0x7f50b8107f40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 --2- 172.21.15.120:0/1203059062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b806b410 0x7f50b8105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 -- 172.21.15.120:0/1203059062 >> 172.21.15.120:0/1203059062 conn(0x7f50b80fd140 msgr2=0x7f50b80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 -- 172.21.15.120:0/1203059062 shutdown_connections 2024-09-10T22:59:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 -- 172.21.15.120:0/1203059062 wait complete. 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 Processor -- start 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.137+0000 7f50bcd94640 1 -- start start 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b8105ad0 0x7f50b81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e5d0 con 0x7f50b8108680 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e450 con 0x7f50b8105ad0 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810e750 con 0x7f50b806b410 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45202/0 (socket says 172.21.15.120:45202) 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38928/0 (socket says 172.21.15.120:38928) 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b659c640 1 -- 172.21.15.120:0/4110084103 learned_addr learned my addr 172.21.15.120:0/4110084103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50affff640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b8105ad0 0x7f50b81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 -- 172.21.15.120:0/4110084103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 msgr2=0x7f50b8198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 -- 172.21.15.120:0/4110084103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b8105ad0 msgr2=0x7f50b81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b8105ad0 0x7f50b81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 -- 172.21.15.120:0/4110084103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b819a8d0 con 0x7f50b8108680 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b659c640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50b6d9d640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009fd0 tx=0x7f50b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50af7fe640 1 -- 172.21.15.120:0/4110084103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000ce20 con 0x7f50b8108680 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b819abc0 con 0x7f50b8108680 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50af7fe640 1 -- 172.21.15.120:0/4110084103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b0005020 con 0x7f50b8108680 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50af7fe640 1 -- 172.21.15.120:0/4110084103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0010690 con 0x7f50b8108680 2024-09-10T22:59:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.138+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b81a1290 con 0x7f50b8108680 2024-09-10T22:59:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 Processor -- start 2024-09-10T22:59:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50af7fe640 1 -- 172.21.15.120:0/4110084103 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f50b00054d0 con 0x7f50b8108680 2024-09-10T22:59:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 -- start start 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c5c70 0x55c5405c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c5405336f0 con 0x55c5405cafe0 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c540522b20 con 0x55c5405c85a0 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c540526930 con 0x55c5405c5c70 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38938/0 (socket says 172.21.15.120:38938) 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.141+0000 7f50affff640 1 -- 172.21.15.120:0/1995831195 learned_addr learned my addr 172.21.15.120:0/1995831195 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c5c70 0x55c5405c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50b659c640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50af7fe640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 0x7f50a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 -- 172.21.15.120:0/1995831195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c5c70 msgr2=0x55c5405c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c5c70 0x55c5405c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 -- 172.21.15.120:0/1995831195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 msgr2=0x55c5405ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 -- 172.21.15.120:0/1995831195 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5404e8530 con 0x55c5405cafe0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50b659c640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50b6d9d640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 0x7f50a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50affff640 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f50b806b910 tx=0x7f50a00046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe707c941c47d78e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50af7fe640 1 -- 172.21.15.120:0/4110084103 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50b00e61c0 con 0x7f50b8108680 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50acff9640 1 -- 172.21.15.120:0/1995831195 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035040 con 0x55c5405cafe0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50acff9640 1 -- 172.21.15.120:0/1995831195 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0031650 con 0x55c5405cafe0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.142+0000 7f50acff9640 1 -- 172.21.15.120:0/1995831195 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003ae0 con 0x55c5405cafe0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50b6d9d640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 0x7f50a40b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50b00a9c20 tx=0x7f50b0076a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1995831195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 msgr2=0x55c5405cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f50b806b910 tx=0x7f50a00046e0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1995831195 shutdown_connections 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405cafe0 0x55c5405cd4a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1995831195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c5c70 0x55c5405c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1995831195 >> 172.21.15.120:0/1995831195 conn(0x55c5405bc9b0 msgr2=0x55c5405bd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1995831195 shutdown_connections 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1995831195 wait complete. 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.143+0000 7f50bfc3f740 1 -- start start 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 Processor -- start 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c54045aab0 con 0x55c5405c5c70 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5404c8730 con 0x55c5405cafe0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5405cdf60 con 0x55c5405c85a0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50bfc3f740 10 client.0 _ll_register_callbacks cb 0x55c54051c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38942/0 (socket says 172.21.15.120:38942) 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50affff640 1 -- 172.21.15.120:0/3808575206 learned_addr learned my addr 172.21.15.120:0/3808575206 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50affff640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 -- 172.21.15.120:0/3808575206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 msgr2=0x55c5405d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 -- 172.21.15.120:0/3808575206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 msgr2=0x55c5405cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.144+0000 7f50b659c640 1 -- 172.21.15.120:0/3808575206 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c540284790 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50b659c640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405cf9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f50a80127c0 tx=0x7f50a8012c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f553311a7c46197 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8010e00 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50a8004030 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0031650 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50a8004230 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 10 client.26254 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c5402fd620 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0031650 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f50a8019070 con 0x55c5405c85a0 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 10 client.26254 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 20 client.26254 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5405dac40 0x55c5405dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 4 client.26254 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.145+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c540535640 con 0x55c5405dac40 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.146+0000 7f50affff640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5405dac40 0x55c5405dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.146+0000 7f50affff640 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5405dac40 0x55c5405dd080 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d55dd848b2d879a server_cookie=7b107f4f03ec0050 in_seq=0 out_seq=0 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.146+0000 7f509dffb640 10 client.26254 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.146+0000 7f509dffb640 1 -- 172.21.15.120:0/3808575206 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c540535640 con 0x55c5405dac40 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.146+0000 7f509dffb640 10 client.26254 handle_command_reply: tid=0 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 client.26254 shutdown 2024-09-10T22:59:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 msgr2=0x55c5405cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405cf9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f50a80127c0 tx=0x7f50a8012c80 comp rx=0 tx=0).stop 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 shutdown_connections 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5405dac40 0x55c5405dd080 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405cf9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f509dffb640 0 client.26254 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/3808575206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 >> 172.21.15.120:0/3808575206 conn(0x55c5405bc9b0 msgr2=0x55c5405bbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 shutdown_connections 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 1 -- 172.21.15.120:0/3808575206 wait complete. 2024-09-10T22:59:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.147+0000 7f50bfc3f740 20 client.26254 trim_cache size 0 max 16384 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 Processor -- start 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 -- start start 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a5c80 con 0x55c5405c5c70 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a8500 con 0x55c5405cafe0 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a62f0 con 0x55c5405c85a0 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38952/0 (socket says 172.21.15.120:38952) 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 -- 172.21.15.120:0/691460755 learned_addr learned my addr 172.21.15.120:0/691460755 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50b659c640 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 -- 172.21.15.120:0/691460755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 msgr2=0x55c5405bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405bd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 -- 172.21.15.120:0/691460755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 msgr2=0x55c5405cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 -- 172.21.15.120:0/691460755 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5405cf0d0 con 0x55c5405c5c70 2024-09-10T22:59:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f50affff640 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec6f53fb50512e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f509ffff640 1 -- 172.21.15.120:0/691460755 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035040 con 0x55c5405c5c70 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.163+0000 7f509ffff640 1 -- 172.21.15.120:0/691460755 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0031c80 con 0x55c5405c5c70 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f509ffff640 1 -- 172.21.15.120:0/691460755 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a004a810 con 0x55c5405c5c70 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- 172.21.15.120:0/691460755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 msgr2=0x55c5405bd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a00035f0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- 172.21.15.120:0/691460755 shutdown_connections 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405cafe0 0x55c5405cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405c85a0 0x55c5405bd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/691460755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405bd210 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- 172.21.15.120:0/691460755 >> 172.21.15.120:0/691460755 conn(0x55c5405bc9b0 msgr2=0x55c5405cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- 172.21.15.120:0/691460755 shutdown_connections 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- 172.21.15.120:0/691460755 wait complete. 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.164+0000 7f50bfc3f740 1 -- start start 2024-09-10T22:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 Processor -- start 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405cafe0 0x55c540645b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a69c0 con 0x55c5405c5c70 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a7e30 con 0x55c5405c85a0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f50b81a7090 con 0x55c5405cafe0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 10 client.0 _ll_register_callbacks cb 0x7f50a800bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38954/0 (socket says 172.21.15.120:38954) 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 -- 172.21.15.120:0/1646149379 learned_addr learned my addr 172.21.15.120:0/1646149379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405cafe0 0x55c540645b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50b659c640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 -- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405cafe0 msgr2=0x55c540645b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405cafe0 0x55c540645b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 -- 172.21.15.120:0/1646149379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 msgr2=0x55c5405d6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405d6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 -- 172.21.15.120:0/1646149379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c540646130 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50affff640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f50a0004720 tx=0x7f50a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca41e1602d7bc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f509dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035030 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f50a004bb70 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.165+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0031c80 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50a0069050 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 10 client.16785 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c540647440 con 0x55c5405c5c70 2024-09-10T22:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0031c80 con 0x55c5405c5c70 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f50a0060070 con 0x55c5405c5c70 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 10 client.16785 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 20 client.16785 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c540647950 0x55c540649d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 4 client.16785 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5405310f0 con 0x55c540647950 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.166+0000 7f50b659c640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c540647950 0x55c540649d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.167+0000 7f509dffb640 10 client.16785 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.167+0000 7f50b659c640 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c540647950 0x55c540649d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf66adb75e95ef10 server_cookie=d6c625517a62eada in_seq=0 out_seq=0 2024-09-10T22:59:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.385+0000 7f509dffb640 1 -- 172.21.15.120:0/1646149379 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5405310f0 con 0x55c540647950 2024-09-10T22:59:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.385+0000 7f509dffb640 10 client.16785 handle_command_reply: tid=0 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.385+0000 7f50bfc3f740 1 client.16785 shutdown 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 msgr2=0x55c5405d6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f50a0004720 tx=0x7f50a002a410 comp rx=0 tx=0).stop 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 shutdown_connections 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c540647950 0x55c540649d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5405cafe0 0x55c540645b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5405c85a0 0x55c5405d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 --2- 172.21.15.120:0/1646149379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5405c5c70 0x55c5405d6840 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 >> 172.21.15.120:0/1646149379 conn(0x55c5405bc9b0 msgr2=0x55c5405bd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f509dffb640 0 client.16785 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 shutdown_connections 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 1 -- 172.21.15.120:0/1646149379 wait complete. 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.386+0000 7f50bfc3f740 20 client.16785 trim_cache size 0 max 16384 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:59:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 msgr2=0x7f50a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 0x7f50a40b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50b00a9c20 tx=0x7f50b0076a80 comp rx=0 tx=0).stop 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 msgr2=0x7f50b819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009fd0 tx=0x7f50b000ef10 comp rx=0 tx=0).stop 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 shutdown_connections 2024-09-10T22:59:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f50a40b52e0 0x7f50a40b77a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f50b8108680 0x7f50b819a010 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50b8105ad0 0x7f50b81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 --2- 172.21.15.120:0/4110084103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50b806b410 0x7f50b8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 >> 172.21.15.120:0/4110084103 conn(0x7f50b80fd140 msgr2=0x7f50b8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 shutdown_connections 2024-09-10T22:59:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:31.387+0000 7f50bcd94640 1 -- 172.21.15.120:0/4110084103 wait complete. 2024-09-10T22:59:31.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1196939 16.50 MB/sec execute 266 sec latency 90.628 ms 2024-09-10T22:59:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:32 smithi176 ceph-mon[34982]: pgmap v559: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T22:59:32.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1199696 16.49 MB/sec execute 267 sec latency 16.537 ms 2024-09-10T22:59:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:32 smithi120 ceph-mon[31247]: pgmap v559: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T22:59:32.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:32 smithi153 ceph-mon[34951]: pgmap v559: 97 pgs: 97 active+clean; 59 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T22:59:33.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1202615 16.48 MB/sec execute 268 sec latency 17.001 ms 2024-09-10T22:59:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:34 smithi176 ceph-mon[34982]: pgmap v560: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-10T22:59:34.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1205058 16.47 MB/sec execute 269 sec latency 13.717 ms 2024-09-10T22:59:34.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:34 smithi120 ceph-mon[31247]: pgmap v560: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-10T22:59:34.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:34 smithi153 ceph-mon[34951]: pgmap v560: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-10T22:59:35.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1207959 16.48 MB/sec execute 270 sec latency 64.195 ms 2024-09-10T22:59:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:36 smithi176 ceph-mon[34982]: pgmap v561: 97 pgs: 97 active+clean; 60 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-10T22:59:36.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1210937 16.47 MB/sec execute 271 sec latency 25.623 ms 2024-09-10T22:59:36.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:36 smithi120 ceph-mon[31247]: pgmap v561: 97 pgs: 97 active+clean; 60 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-10T22:59:36.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:36 smithi153 ceph-mon[34951]: pgmap v561: 97 pgs: 97 active+clean; 60 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-10T22:59:37.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1213611 16.45 MB/sec execute 272 sec latency 12.897 ms 2024-09-10T22:59:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:38 smithi176 ceph-mon[34982]: pgmap v562: 97 pgs: 97 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-09-10T22:59:38.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1216086 16.45 MB/sec execute 273 sec latency 68.868 ms 2024-09-10T22:59:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:38 smithi120 ceph-mon[31247]: pgmap v562: 97 pgs: 97 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-09-10T22:59:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:38 smithi153 ceph-mon[34951]: pgmap v562: 97 pgs: 97 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-09-10T22:59:39.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:39.479 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:39.629 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1219000 16.45 MB/sec execute 274 sec latency 12.161 ms 2024-09-10T22:59:40.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/2201832032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 msgr2=0x7f0ce411e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/2201832032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 0x7f0ce411e490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009ca0 tx=0x7f0cdc02a960 comp rx=0 tx=0).stop 2024-09-10T22:59:40.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/2201832032 shutdown_connections 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/2201832032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 0x7f0ce411e490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/2201832032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ce410a530 0x7f0ce410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/2201832032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a150 0x7f0ce41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/2201832032 >> 172.21.15.120:0/2201832032 conn(0x7f0ce406dfc0 msgr2=0x7f0ce406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.092+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/2201832032 shutdown_connections 2024-09-10T22:59:40.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.093+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/2201832032 wait complete. 2024-09-10T22:59:40.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.094+0000 7f0ce8e6c640 1 Processor -- start 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.094+0000 7f0ce8e6c640 1 -- start start 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ce410a150 0x7f0ce41a8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 0x7f0ce41ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4121b60 con 0x7f0ce410a150 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41219e0 con 0x7f0ce411c0a0 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce8e6c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4121ce0 con 0x7f0ce410a530 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce1d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce1d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50880/0 (socket says 172.21.15.120:50880) 2024-09-10T22:59:40.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.095+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/960986201 learned_addr learned my addr 172.21.15.120:0/960986201 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/960986201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 msgr2=0x7f0ce41ad910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 0x7f0ce41ad910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ce410a150 msgr2=0x7f0ce41a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ce410a150 0x7f0ce41a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/960986201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce41ae0e0 con 0x7f0ce410a530 2024-09-10T22:59:40.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.096+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00d950 tx=0x7f0ccc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.098+0000 7f0cdb7fe640 1 -- 172.21.15.120:0/960986201 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc017070 con 0x7f0ce410a530 2024-09-10T22:59:40.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.098+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce41ae3d0 con 0x7f0ce410a530 2024-09-10T22:59:40.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.098+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41b4a90 con 0x7f0ce410a530 2024-09-10T22:59:40.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.098+0000 7f0cdb7fe640 1 -- 172.21.15.120:0/960986201 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc009bb0 con 0x7f0ce410a530 2024-09-10T22:59:40.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.099+0000 7f0cdb7fe640 1 -- 172.21.15.120:0/960986201 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00f620 con 0x7f0ce410a530 2024-09-10T22:59:40.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.103+0000 7f0ceb241740 1 Processor -- start 2024-09-10T22:59:40.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 -- start start 2024-09-10T22:59:40.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e70710480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7071c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e70686e50 con 0x564e707100a0 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e70676280 con 0x564e707109c0 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ceb241740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564e7067a090 con 0x564e7071c950 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0cdb7fe640 1 -- 172.21.15.120:0/960986201 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0ccc023060 con 0x7f0ce410a530 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce1d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e70710480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e70710480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42084/0 (socket says 172.21.15.120:42084) 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7071c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50888/0 (socket says 172.21.15.120:50888) 2024-09-10T22:59:40.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.104+0000 7f0ce2d9d640 1 -- 172.21.15.120:0/1410352579 learned_addr learned my addr 172.21.15.120:0/1410352579 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:40.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/1410352579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 msgr2=0x564e7071c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7071c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/1410352579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 msgr2=0x564e70710480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e70710480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/1410352579 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e703d4650 con 0x564e7071c950 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0e2a90 tx=0x7f0ccc0ed6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ce2679774b17d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0cdb7fe640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 0x7f0cd00b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0cd8ff9640 1 -- 172.21.15.120:0/1410352579 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0f8070 con 0x564e7071c950 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0cd8ff9640 1 -- 172.21.15.120:0/1410352579 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0ab290 con 0x564e7071c950 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.105+0000 7f0cd8ff9640 1 -- 172.21.15.120:0/1410352579 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0a7620 con 0x564e7071c950 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.106+0000 7f0cdb7fe640 1 -- 172.21.15.120:0/960986201 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0ccc0e4f90 con 0x7f0ce410a530 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.106+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 0x7f0cd00b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.106+0000 7f0ceb241740 1 -- 172.21.15.120:0/1410352579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 msgr2=0x564e7071ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.106+0000 7f0ceb241740 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0e2a90 tx=0x7f0ccc0ed6f0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 -- 172.21.15.120:0/1410352579 shutdown_connections 2024-09-10T22:59:40.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e7071ed40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7071c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 --2- 172.21.15.120:0/1410352579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e70710480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 -- 172.21.15.120:0/1410352579 >> 172.21.15.120:0/1410352579 conn(0x564e7070ded0 msgr2=0x564e7070f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 -- 172.21.15.120:0/1410352579 shutdown_connections 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 -- 172.21.15.120:0/1410352579 wait complete. 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 0x7f0cd00b7930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004af0 tx=0x7f0cdc005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 -- start start 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.107+0000 7f0ceb241740 1 Processor -- start 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e707109c0 0x564e70719610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 0x564e70719b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564e704d7700 con 0x564e7071c950 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564e70611110 con 0x564e707100a0 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564e7071f5d0 con 0x564e707109c0 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 10 client.0 _ll_register_callbacks cb 0x564e7062a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:40.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42804/0 (socket says 172.21.15.120:42804) 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 -- 172.21.15.120:0/612927370 learned_addr learned my addr 172.21.15.120:0/612927370 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e707109c0 0x564e70719610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 0x564e70719b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 -- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e707109c0 msgr2=0x564e70719610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e707109c0 0x564e70719610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 -- 172.21.15.120:0/612927370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 msgr2=0x564e70719b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 0x564e70719b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 -- 172.21.15.120:0/612927370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7063ec40 con 0x564e707100a0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 0x564e70719b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ce259c640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40116f0 tx=0x7f0cd4011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c9c78b0f711a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0cc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4004460 con 0x564e707100a0 2024-09-10T22:59:40.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ccc0ab290 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.108+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0cd400f920 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cd400fb20 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 10 client.26189 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e7072afc0 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0ab290 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0cd4018070 con 0x564e707100a0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 10 client.26189 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 20 client.26189 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e7072c490 0x564e7072e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 4 client.26189 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e70688da0 con 0x564e7072c490 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.109+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e7072c490 0x564e7072e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.110+0000 7f0cc9ffb640 10 client.26189 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:40.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.110+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e7072c490 0x564e7072e8e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48678c62eac68e7c server_cookie=722a11a1073038e3 in_seq=0 out_seq=0 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/612927370 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e70688da0 con 0x564e7072c490 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0cc9ffb640 10 client.26189 handle_command_reply: tid=0 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 client.26189 shutdown 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 msgr2=0x564e7071c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40116f0 tx=0x7f0cd4011bb0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 shutdown_connections 2024-09-10T22:59:40.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e7072c490 0x564e7072e8e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e7071c950 0x564e70719b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e707109c0 0x564e70719610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 --2- 172.21.15.120:0/612927370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e7071c1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 >> 172.21.15.120:0/612927370 conn(0x564e7070ded0 msgr2=0x564e70713a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.111+0000 7f0cc9ffb640 0 client.26189 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.112+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 shutdown_connections 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.112+0000 7f0ceb241740 1 -- 172.21.15.120:0/612927370 wait complete. 2024-09-10T22:59:40.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.112+0000 7f0ceb241740 20 client.26189 trim_cache size 0 max 16384 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.139+0000 7f0ceb241740 1 Processor -- start 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.139+0000 7f0ceb241740 1 -- start start 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.139+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e7072e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7070d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ceb241740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400c360 con 0x564e707100a0 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ceb241740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400ebe0 con 0x564e707109c0 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ceb241740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400c9d0 con 0x564e7071c950 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50914/0 (socket says 172.21.15.120:50914) 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce2d9d640 1 -- 172.21.15.120:0/2293802783 learned_addr learned my addr 172.21.15.120:0/2293802783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce259c640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e7072e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7070d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2293802783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 msgr2=0x564e707185a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2293802783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 msgr2=0x564e7072e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e7072e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2293802783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7070dc50 con 0x564e707109c0 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.140+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7070d710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc082df0 tx=0x7f0ccc0a7ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ac38353a8e146c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ccbfff640 1 -- 172.21.15.120:0/2293802783 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc020040 con 0x564e707109c0 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ccbfff640 1 -- 172.21.15.120:0/2293802783 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0e8dc0 con 0x564e707109c0 2024-09-10T22:59:40.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ccbfff640 1 -- 172.21.15.120:0/2293802783 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc084510 con 0x564e707109c0 2024-09-10T22:59:40.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ceb241740 1 -- 172.21.15.120:0/2293802783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 msgr2=0x564e7070d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7070d710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc082df0 tx=0x7f0ccc0a7ca0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ceb241740 1 -- 172.21.15.120:0/2293802783 shutdown_connections 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.141+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707109c0 0x564e7070d710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2293802783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707100a0 0x564e7072e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 -- 172.21.15.120:0/2293802783 >> 172.21.15.120:0/2293802783 conn(0x564e7070ded0 msgr2=0x564e70717e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 -- 172.21.15.120:0/2293802783 shutdown_connections 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 -- 172.21.15.120:0/2293802783 wait complete. 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 -- start start 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.142+0000 7f0ceb241740 1 Processor -- start 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400d0a0 con 0x564e707109c0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400e510 con 0x564e707100a0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc400d770 con 0x564e7071c950 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ceb241740 10 client.0 _ll_register_callbacks cb 0x7f0cd4009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42830/0 (socket says 172.21.15.120:42830) 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce259c640 1 -- 172.21.15.120:0/2935496233 learned_addr learned my addr 172.21.15.120:0/2935496233 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42114/0 (socket says 172.21.15.120:42114) 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce2d9d640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 msgr2=0x564e707983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2935496233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 msgr2=0x564e70720be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 -- 172.21.15.120:0/2935496233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e70798930 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce259c640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0ce1d9b640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0846b0 tx=0x7f0ccc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a128461034ddc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.143+0000 7f0cc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc020070 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0ccc0e8350 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ccc0e8dc0 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc0846e0 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 10 client.16797 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e70799ae0 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ccc0ab290 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0ccc085370 con 0x564e707109c0 2024-09-10T22:59:40.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 10 client.16797 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 20 client.16797 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e70799dd0 0x564e7079c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 4 client.16797 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.144+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0ccc0e8dc0 con 0x564e70799dd0 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.145+0000 7f0ce259c640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e70799dd0 0x564e7079c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.145+0000 7f0ce259c640 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e70799dd0 0x564e7079c1c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e1711a1e0d4f42 server_cookie=ba6ad21a8c67dc9 in_seq=0 out_seq=0 2024-09-10T22:59:40.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.145+0000 7f0cc9ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.146+0000 7f0cc9ffb640 1 -- 172.21.15.120:0/2935496233 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x7f0ccc0e8dc0 con 0x564e70799dd0 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.146+0000 7f0cc9ffb640 10 client.16797 handle_command_reply: tid=0 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.146+0000 7f0ceb241740 1 client.16797 shutdown 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.146+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 msgr2=0x564e70721120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.146+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0846b0 tx=0x7f0ccc005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 shutdown_connections 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564e70799dd0 0x564e7079c1c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564e7071c950 0x564e707983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564e707109c0 0x564e70721120 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 --2- 172.21.15.120:0/2935496233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564e707100a0 0x564e70720be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 >> 172.21.15.120:0/2935496233 conn(0x564e7070ded0 msgr2=0x564e707186b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0cc9ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 shutdown_connections 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 1 -- 172.21.15.120:0/2935496233 wait complete. 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ceb241740 20 client.16797 trim_cache size 0 max 16384 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T22:59:40.154 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:59:40.160 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:40.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 msgr2=0x7f0cd00b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 0x7f0cd00b7930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004af0 tx=0x7f0cdc005d80 comp rx=0 tx=0).stop 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 msgr2=0x7f0ce41a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.147+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00d950 tx=0x7f0ccc00de10 comp rx=0 tx=0).stop 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 shutdown_connections 2024-09-10T22:59:40.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0cd00b5470 0x7f0cd00b7930 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ce411c0a0 0x7f0ce41ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ce410a530 0x7f0ce41a8570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 --2- 172.21.15.120:0/960986201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ce410a150 0x7f0ce41a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 >> 172.21.15.120:0/960986201 conn(0x7f0ce406dfc0 msgr2=0x7f0ce411c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 shutdown_connections 2024-09-10T22:59:40.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:40.148+0000 7f0ce8e6c640 1 -- 172.21.15.120:0/960986201 wait complete. 2024-09-10T22:59:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: pgmap v563: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-10T22:59:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:40 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: pgmap v563: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-10T22:59:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:40 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1221969 16.45 MB/sec execute 275 sec latency 45.675 ms 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: pgmap v563: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:40.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:40 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.048 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:59:41.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:41.335 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:41.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:59:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T22:59:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T22:59:41.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:59:41.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1224667 16.44 MB/sec execute 276 sec latency 34.065 ms 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:59:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T22:59:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- 172.21.15.120:0/1562686393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 msgr2=0x7fc058101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 --2- 172.21.15.120:0/1562686393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc058101ab0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc048009900 tx=0x7fc04802a410 comp rx=0 tx=0).stop 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- 172.21.15.120:0/1562686393 shutdown_connections 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 --2- 172.21.15.120:0/1562686393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 --2- 172.21.15.120:0/1562686393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 --2- 172.21.15.120:0/1562686393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc058101ab0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- 172.21.15.120:0/1562686393 >> 172.21.15.120:0/1562686393 conn(0x7fc058077fb0 msgr2=0x7fc0580783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- 172.21.15.120:0/1562686393 shutdown_connections 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- 172.21.15.120:0/1562686393 wait complete. 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 Processor -- start 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.875+0000 7fc05d672640 1 -- start start 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc058112c30 con 0x7fc0581016d0 2024-09-10T22:59:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc058112ab0 con 0x7fc0581075b0 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05d672640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc058112db0 con 0x7fc058106c30 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42130/0 (socket says 172.21.15.120:42130) 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 -- 172.21.15.120:0/4115665513 learned_addr learned my addr 172.21.15.120:0/4115665513 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc0577fe640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05ce71640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 -- 172.21.15.120:0/4115665513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 msgr2=0x7fc058199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 -- 172.21.15.120:0/4115665513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 msgr2=0x7fc05819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 -- 172.21.15.120:0/4115665513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05819f190 con 0x7fc0581016d0 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc0577fe640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc05ce71640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.876+0000 7fc057fff640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc048005940 tx=0x7fc048004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.877+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04803f030 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.877+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc048003aa0 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.877+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048035070 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.877+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05819f420 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.877+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581a5b40 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.878+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc058101ab0 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.879+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc04803bab0 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.880+0000 7fc0557fa640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 0x7fc0400b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.881+0000 7fc0577fe640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 0x7fc0400b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.881+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc048106800 con 0x7fc0581016d0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.881+0000 7fc0577fe640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 0x7fc0400b7850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0581040c0 tx=0x7fc04c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:41.884+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc0480cd6f0 con 0x7fc0581016d0 2024-09-10T22:59:42.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.115+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0581a5dd0 con 0x7fc0581016d0 2024-09-10T22:59:42.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.115+0000 7fc0557fa640 1 -- 172.21.15.120:0/4115665513 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fc0480ccff0 con 0x7fc0581016d0 2024-09-10T22:59:42.117 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T22:59:42.117 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T22:59:42.117 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.117+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 msgr2=0x7fc0400b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.117+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 0x7fc0400b7850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0581040c0 tx=0x7fc04c005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.117+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 msgr2=0x7fc0581990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.117+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc048005940 tx=0x7fc048004270 comp rx=0 tx=0).stop 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 shutdown_connections 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0400b5390 0x7fc0400b7850 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0581075b0 0x7fc05819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc058106c30 0x7fc058199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 --2- 172.21.15.120:0/4115665513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0581016d0 0x7fc0581990e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 >> 172.21.15.120:0/4115665513 conn(0x7fc058077fb0 msgr2=0x7fc058104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 shutdown_connections 2024-09-10T22:59:42.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:42.118+0000 7fc05d672640 1 -- 172.21.15.120:0/4115665513 wait complete. 2024-09-10T22:59:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T22:59:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T22:59:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: pgmap v564: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-09-10T22:59:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T22:59:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T22:59:42.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:42 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4115665513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: pgmap v564: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T22:59:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:42 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4115665513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1227648 16.43 MB/sec execute 277 sec latency 15.583 ms 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: pgmap v564: 97 pgs: 97 active+clean; 55 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T22:59:42.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:42 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4115665513' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.835 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:59:43.096 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:43.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1230022 16.42 MB/sec execute 278 sec latency 47.762 ms 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 -- 172.21.15.120:0/1935113012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 msgr2=0x7fc1a0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/1935113012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0107ae0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc194009900 tx=0x7fc19402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 -- 172.21.15.120:0/1935113012 shutdown_connections 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/1935113012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/1935113012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0107ae0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/1935113012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 0x7fc1a0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 -- 172.21.15.120:0/1935113012 >> 172.21.15.120:0/1935113012 conn(0x7fc1a00fd700 msgr2=0x7fc1a00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 -- 172.21.15.120:0/1935113012 shutdown_connections 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.632+0000 7fc1a5fab640 1 -- 172.21.15.120:0/1935113012 wait complete. 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 Processor -- start 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 -- start start 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 0x7fc1a01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0111ca0 con 0x7fc1a01076a0 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0111b20 con 0x7fc1a0101c90 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc1a5fab640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0111e20 con 0x7fc1a0108020 2024-09-10T22:59:43.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc19effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc19effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44112/0 (socket says 172.21.15.120:44112) 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc19effd640 1 -- 172.21.15.120:0/3541727180 learned_addr learned my addr 172.21.15.120:0/3541727180 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.633+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 0x7fc1a01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 -- 172.21.15.120:0/3541727180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 msgr2=0x7fc1a019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 -- 172.21.15.120:0/3541727180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 msgr2=0x7fc1a01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 0x7fc1a01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 -- 172.21.15.120:0/3541727180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a019f370 con 0x7fc1a01076a0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19effd640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc194005940 tx=0x7fc194004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19cff9640 1 -- 172.21.15.120:0/3541727180 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19403f030 con 0x7fc1a01076a0 2024-09-10T22:59:43.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19cff9640 1 -- 172.21.15.120:0/3541727180 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194003aa0 con 0x7fc1a01076a0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a019f600 con 0x7fc1a01076a0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc19cff9640 1 -- 172.21.15.120:0/3541727180 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194035070 con 0x7fc1a01076a0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.634+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a01a5d20 con 0x7fc1a01076a0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19cff9640 1 -- 172.21.15.120:0/3541727180 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc194003c50 con 0x7fc1a01076a0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 Processor -- start 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 -- start start 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b51e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b531580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d1b494ea0 con 0x557d1b51edf0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d1b4842d0 con 0x557d1b51e4d0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc1a8380740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557d1b4880e0 con 0x557d1b52f190 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44118/0 (socket says 172.21.15.120:44118) 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3730644889 learned_addr learned my addr 172.21.15.120:0/3730644889 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b51e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19effd640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b531580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3730644889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 msgr2=0x557d1b531580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b531580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3730644889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 msgr2=0x557d1b51e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b51e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3730644889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1b1e2650 con 0x557d1b51edf0 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.637+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b51e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc19000ac20 tx=0x7fc19000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c7a6600bda84e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc19cff9640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 0x7fc1880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 0x7fc1880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc185ffb640 1 -- 172.21.15.120:0/3730644889 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19001d070 con 0x557d1b51edf0 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc185ffb640 1 -- 172.21.15.120:0/3730644889 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc190018af0 con 0x557d1b51edf0 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc185ffb640 1 -- 172.21.15.120:0/3730644889 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190010aa0 con 0x557d1b51edf0 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.638+0000 7fc19cff9640 1 -- 172.21.15.120:0/3541727180 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc194106500 con 0x7fc1a01076a0 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- 172.21.15.120:0/3730644889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 msgr2=0x557d1b52ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc19000ac20 tx=0x7fc19000f790 comp rx=0 tx=0).stop 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 0x7fc1880b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc198009fd0 tx=0x7fc198009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- 172.21.15.120:0/3730644889 shutdown_connections 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b531580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52ec50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3730644889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b51e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- 172.21.15.120:0/3730644889 >> 172.21.15.120:0/3730644889 conn(0x557d1b51c300 msgr2=0x557d1b51d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- 172.21.15.120:0/3730644889 shutdown_connections 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- 172.21.15.120:0/3730644889 wait complete. 2024-09-10T22:59:43.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 -- start start 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.639+0000 7fc1a8380740 1 Processor -- start 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b52ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d1b2e5700 con 0x557d1b51edf0 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557d1b41f160 con 0x557d1b51e4d0 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557d1b531e10 con 0x557d1b52f190 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc1a8380740 10 client.0 _ll_register_callbacks cb 0x557d1b4389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44120/0 (socket says 172.21.15.120:44120) 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 -- 172.21.15.120:0/699387498 learned_addr learned my addr 172.21.15.120:0/699387498 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:43.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36990/0 (socket says 172.21.15.120:36990) 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b52ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 -- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 msgr2=0x557d1b52c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 -- 172.21.15.120:0/699387498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 msgr2=0x557d1b52ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b52ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 -- 172.21.15.120:0/699387498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1b44cc90 con 0x557d1b51edf0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19ffff640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc19effd640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c9310 tx=0x7fc194031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26265b58a7d8b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.640+0000 7fc17effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194049070 con 0x557d1b51edf0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc1940c8fb0 con 0x557d1b51edf0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc190018af0 con 0x557d1b51edf0 2024-09-10T22:59:43.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1940c8be0 con 0x557d1b51edf0 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 10 client.16815 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d1b53d700 con 0x557d1b51edf0 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc190018af0 con 0x557d1b51edf0 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc194106110 con 0x557d1b51edf0 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 10 client.16815 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 20 client.16815 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b53eb60 0x557d1b540f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 4 client.16815 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:43.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1b496df0 con 0x557d1b53eb60 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.641+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b53eb60 0x557d1b540f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.642+0000 7fc17effd640 10 client.16815 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.642+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b53eb60 0x557d1b540f70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734bbd9115e7bd6c server_cookie=68b753af94d2b2e3 in_seq=0 out_seq=0 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.642+0000 7fc17effd640 1 -- 172.21.15.120:0/699387498 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d1b496df0 con 0x557d1b53eb60 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.642+0000 7fc17effd640 10 client.16815 handle_command_reply: tid=0 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.642+0000 7fc1a8380740 1 client.16815 shutdown 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 msgr2=0x557d1b52bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c9310 tx=0x7fc194031a10 comp rx=0 tx=0).stop 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 shutdown_connections 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b53eb60 0x557d1b540f70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b52c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51edf0 0x557d1b52bee0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 --2- 172.21.15.120:0/699387498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51e4d0 0x557d1b52ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 >> 172.21.15.120:0/699387498 conn(0x557d1b51c300 msgr2=0x557d1b521ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc17effd640 0 client.16815 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:43.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 shutdown_connections 2024-09-10T22:59:43.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 1 -- 172.21.15.120:0/699387498 wait complete. 2024-09-10T22:59:43.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.643+0000 7fc1a8380740 20 client.16815 trim_cache size 0 max 16384 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 Processor -- start 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 -- start start 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b51d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b540b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01aa080 con 0x557d1b51e4d0 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01ac900 con 0x557d1b51edf0 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc1a8380740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01aa6f0 con 0x557d1b52f190 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44128/0 (socket says 172.21.15.120:44128) 2024-09-10T22:59:43.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3418294290 learned_addr learned my addr 172.21.15.120:0/3418294290 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19effd640 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b51d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19ffff640 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b540b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3418294290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 msgr2=0x557d1b540b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b540b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3418294290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 msgr2=0x557d1b51d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b51d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 -- 172.21.15.120:0/3418294290 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1b51ec20 con 0x557d1b51e4d0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.657+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc1900106b0 tx=0x7fc19000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b807a8391d70f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1857fa640 1 -- 172.21.15.120:0/3418294290 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190030070 con 0x557d1b51e4d0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1857fa640 1 -- 172.21.15.120:0/3418294290 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1900054f0 con 0x557d1b51e4d0 2024-09-10T22:59:43.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1857fa640 1 -- 172.21.15.120:0/3418294290 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19001d040 con 0x557d1b51e4d0 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 -- 172.21.15.120:0/3418294290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 msgr2=0x557d1b53b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc1900106b0 tx=0x7fc19000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 -- 172.21.15.120:0/3418294290 shutdown_connections 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b52f190 0x557d1b540b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b51d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 --2- 172.21.15.120:0/3418294290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b51e4d0 0x557d1b53b040 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.658+0000 7fc1a8380740 1 -- 172.21.15.120:0/3418294290 >> 172.21.15.120:0/3418294290 conn(0x557d1b51c300 msgr2=0x557d1b51d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 -- 172.21.15.120:0/3418294290 shutdown_connections 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 -- 172.21.15.120:0/3418294290 wait complete. 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 -- start start 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 Processor -- start 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 0x557d1b5370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.659+0000 7fc1a8380740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc1a8380740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01aadc0 con 0x557d1b52f190 2024-09-10T22:59:43.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc1a8380740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01ac230 con 0x557d1b51edf0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc1a8380740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a01ab490 con 0x557d1b51e4d0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc1a8380740 10 client.0 _ll_register_callbacks cb 0x7fc194005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 0x557d1b5370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51804/0 (socket says 172.21.15.120:51804) 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 0x557d1b5370a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:36998/0 (socket says 172.21.15.120:36998) 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44138/0 (socket says 172.21.15.120:44138) 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 -- 172.21.15.120:0/1663049654 learned_addr learned my addr 172.21.15.120:0/1663049654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 -- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 msgr2=0x557d1b5370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 0x557d1b5370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 -- 172.21.15.120:0/1663049654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 msgr2=0x557d1b534070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:43.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 -- 172.21.15.120:0/1663049654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1b5345b0 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19ffff640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc19effd640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c9fb0 tx=0x7fc194005cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5e8abf6a54c9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc17effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194049070 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1900054f0 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.660+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc1940d5d90 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1940cc4b0 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 10 client.26216 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d1b5ac130 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1900054f0 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc1940b2b60 con 0x557d1b51edf0 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 10 client.26216 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:43.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 20 client.26216 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b5ac700 0x557d1b5aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 4 client.26216 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.661+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d1b4928a0 con 0x557d1b5ac700 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b5ac700 0x557d1b5aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc19f7fe640 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b5ac700 0x557d1b5aeb20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266c42bb0171a5c4 server_cookie=12ff4c2921c553fd in_seq=0 out_seq=0 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc17effd640 10 client.26216 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc17effd640 1 -- 172.21.15.120:0/1663049654 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d1b4928a0 con 0x557d1b5ac700 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc17effd640 10 client.26216 handle_command_reply: tid=0 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc1a8380740 1 client.26216 shutdown 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 msgr2=0x557d1b5375e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.662+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c9fb0 tx=0x7fc194005cb0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 shutdown_connections 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557d1b5ac700 0x557d1b5aeb20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d1b52f190 0x557d1b534070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557d1b51edf0 0x557d1b5375e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 --2- 172.21.15.120:0/1663049654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d1b51e4d0 0x557d1b5370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc17effd640 0 client.26216 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 >> 172.21.15.120:0/1663049654 conn(0x557d1b51c300 msgr2=0x557d1b52d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 shutdown_connections 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 1 -- 172.21.15.120:0/1663049654 wait complete. 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a8380740 20 client.26216 trim_cache size 0 max 16384 2024-09-10T22:59:43.669 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 msgr2=0x7fc1880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 0x7fc1880b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc198009fd0 tx=0x7fc198009500 comp rx=0 tx=0).stop 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 msgr2=0x7fc1a0199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc194005940 tx=0x7fc194004270 comp rx=0 tx=0).stop 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 shutdown_connections 2024-09-10T22:59:43.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1880b52e0 0x7fc1880b77a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1a0108020 0x7fc1a019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a01076a0 0x7fc1a0199800 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 --2- 172.21.15.120:0/3541727180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a0101c90 0x7fc1a01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.663+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 >> 172.21.15.120:0/3541727180 conn(0x7fc1a00fd700 msgr2=0x7fc1a01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.664+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 shutdown_connections 2024-09-10T22:59:43.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:43.664+0000 7fc1a5fab640 1 -- 172.21.15.120:0/3541727180 wait complete. 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 -- 172.21.15.120:0/634837806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5490103370 msgr2=0x7f5490109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 --2- 172.21.15.120:0/634837806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5490103370 0x7f5490109cf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5488009cd0 tx=0x7f548802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 -- 172.21.15.120:0/634837806 shutdown_connections 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 --2- 172.21.15.120:0/634837806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5490103370 0x7f5490109cf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 --2- 172.21.15.120:0/634837806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54901029b0 0x7f5490102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 --2- 172.21.15.120:0/634837806 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490101860 0x7f5490101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 -- 172.21.15.120:0/634837806 >> 172.21.15.120:0/634837806 conn(0x7f54900fd140 msgr2=0x7f54900ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 -- 172.21.15.120:0/634837806 shutdown_connections 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 -- 172.21.15.120:0/634837806 wait complete. 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.480+0000 7f5495416640 1 Processor -- start 2024-09-10T22:59:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 -- start start 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490101860 0x7f5490198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490103370 0x7f5490199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f549010d4d0 con 0x7f54901029b0 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f549010d350 con 0x7f5490101860 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f5495416640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f549010d650 con 0x7f5490103370 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44160/0 (socket says 172.21.15.120:44160) 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490103370 0x7f5490199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 -- 172.21.15.120:0/3260556331 learned_addr learned my addr 172.21.15.120:0/3260556331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548effd640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490101860 0x7f5490198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 -- 172.21.15.120:0/3260556331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490103370 msgr2=0x7f5490199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490103370 0x7f5490199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 -- 172.21.15.120:0/3260556331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490101860 msgr2=0x7f5490198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490101860 0x7f5490198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 -- 172.21.15.120:0/3260556331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549019a870 con 0x7f54901029b0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.481+0000 7f548e7fc640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f548400e9f0 tx=0x7f548400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.482+0000 7f5477fff640 1 -- 172.21.15.120:0/3260556331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548400cd80 con 0x7f54901029b0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.482+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549019ab60 con 0x7f54901029b0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.482+0000 7f5477fff640 1 -- 172.21.15.120:0/3260556331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840108e0 con 0x7f54901029b0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.482+0000 7f5477fff640 1 -- 172.21.15.120:0/3260556331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484018980 con 0x7f54901029b0 2024-09-10T22:59:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.482+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a1230 con 0x7f54901029b0 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.484+0000 7f5477fff640 1 -- 172.21.15.120:0/3260556331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5484010430 con 0x7f54901029b0 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.484+0000 7f54981e9740 1 Processor -- start 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 -- start start 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5cfe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562eb5c696f0 con 0x562eb5cfbc70 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562eb5c58b20 con 0x562eb5cfe5a0 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f54981e9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562eb5c5c930 con 0x562eb5d00fe0 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5cfe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5cfe060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44172/0 (socket says 172.21.15.120:44172) 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548f7fe640 1 -- 172.21.15.120:0/189979900 learned_addr learned my addr 172.21.15.120:0/189979900 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548e7fc640 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f5477fff640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 0x7f54780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548f7fe640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 0x7f54780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 -- 172.21.15.120:0/189979900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 msgr2=0x562eb5d034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 -- 172.21.15.120:0/189979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 msgr2=0x562eb5cfe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5cfe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 -- 172.21.15.120:0/189979900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb5c1e530 con 0x562eb5cfe5a0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f5477fff640 1 -- 172.21.15.120:0/3260556331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f54840e5ef0 con 0x7f54901029b0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.485+0000 7f548effd640 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d00990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5480009c00 tx=0x7f5480011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6b84539bcdd99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54757fa640 1 -- 172.21.15.120:0/189979900 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480004230 con 0x562eb5cfe5a0 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f548f7fe640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 0x7f54780b77a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f548802ae00 tx=0x7f548802a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54757fa640 1 -- 172.21.15.120:0/189979900 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54800043d0 con 0x562eb5cfe5a0 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54757fa640 1 -- 172.21.15.120:0/189979900 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480004c30 con 0x562eb5cfe5a0 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 -- 172.21.15.120:0/189979900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 msgr2=0x562eb5d00990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d00990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5480009c00 tx=0x7f5480011810 comp rx=0 tx=0).stop 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 -- 172.21.15.120:0/189979900 shutdown_connections 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d00990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 --2- 172.21.15.120:0/189979900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5cfe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 -- 172.21.15.120:0/189979900 >> 172.21.15.120:0/189979900 conn(0x562eb5cf29b0 msgr2=0x562eb5cf3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 -- 172.21.15.120:0/189979900 shutdown_connections 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.486+0000 7f54981e9740 1 -- 172.21.15.120:0/189979900 wait complete. 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 -- start start 2024-09-10T22:59:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 Processor -- start 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562eb5b90ab0 con 0x562eb5cfbc70 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562eb5bfe730 con 0x562eb5cfe5a0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562eb5d03fc0 con 0x562eb5d00fe0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f54981e9740 10 client.0 _ll_register_callbacks cb 0x562eb5c526b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44186/0 (socket says 172.21.15.120:44186) 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 -- 172.21.15.120:0/4282140316 learned_addr learned my addr 172.21.15.120:0/4282140316 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548f7fe640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548e7fc640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 -- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 msgr2=0x562eb5d064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 -- 172.21.15.120:0/4282140316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 msgr2=0x562eb5d05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.487+0000 7f548effd640 1 -- 172.21.15.120:0/4282140316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb59ba790 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f548effd640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5480019af0 tx=0x7f5480019b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b815ebb26e1bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480028020 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54800043d0 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5480024620 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5480009e70 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 10 client.16833 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562eb5a33620 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54800043d0 con 0x562eb5cfbc70 2024-09-10T22:59:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5480018070 con 0x562eb5cfbc70 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 10 client.16833 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 20 client.16833 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d109f0 0x562eb5d12e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 4 client.16833 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb5c6b640 con 0x562eb5d109f0 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.488+0000 7f548e7fc640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d109f0 0x562eb5d12e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.489+0000 7f5472ff5640 10 client.16833 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.489+0000 7f548e7fc640 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d109f0 0x562eb5d12e00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bdc3943bf1e777 server_cookie=8926d8fc06460512 in_seq=0 out_seq=0 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.489+0000 7f5472ff5640 1 -- 172.21.15.120:0/4282140316 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562eb5c6b640 con 0x562eb5d109f0 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.489+0000 7f5472ff5640 10 client.16833 handle_command_reply: tid=0 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 client.16833 shutdown 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 msgr2=0x562eb5d05270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5480019af0 tx=0x7f5480019b20 comp rx=0 tx=0).stop 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 shutdown_connections 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d109f0 0x562eb5d12e00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5d00fe0 0x562eb5d064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f5472ff5640 0 client.16833 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 --2- 172.21.15.120:0/4282140316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfbc70 0x562eb5d05270 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 >> 172.21.15.120:0/4282140316 conn(0x562eb5cf29b0 msgr2=0x562eb5cf1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 shutdown_connections 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 1 -- 172.21.15.120:0/4282140316 wait complete. 2024-09-10T22:59:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.490+0000 7f54981e9740 20 client.16833 trim_cache size 0 max 16384 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 Processor -- start 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 -- start start 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5cf6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 0x562eb5d01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a5c10 con 0x562eb5cfe5a0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a8490 con 0x562eb5d00fe0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.505+0000 7f54981e9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a6280 con 0x562eb5cfbc70 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44202/0 (socket says 172.21.15.120:44202) 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 -- 172.21.15.120:0/2433313956 learned_addr learned my addr 172.21.15.120:0/2433313956 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548effd640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5cf6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548f7fe640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 0x562eb5d01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 -- 172.21.15.120:0/2433313956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 msgr2=0x562eb5cf6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5cf6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 -- 172.21.15.120:0/2433313956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 msgr2=0x562eb5d01f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 0x562eb5d01f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 -- 172.21.15.120:0/2433313956 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb5cfe3d0 con 0x562eb5cfe5a0 2024-09-10T22:59:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548effd640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5cf6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548f7fe640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 0x562eb5d01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f548e7fc640 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54840ac490 tx=0x7f54840e38f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a498f891db6e4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f5474ff9640 1 -- 172.21.15.120:0/2433313956 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840ac2a0 con 0x562eb5cfe5a0 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f5474ff9640 1 -- 172.21.15.120:0/2433313956 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840b5c70 con 0x562eb5cfe5a0 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.506+0000 7f5474ff9640 1 -- 172.21.15.120:0/2433313956 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840b57e0 con 0x562eb5cfe5a0 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- 172.21.15.120:0/2433313956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 msgr2=0x562eb5cf71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54840ac490 tx=0x7f54840e38f0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- 172.21.15.120:0/2433313956 shutdown_connections 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5d00fe0 0x562eb5d01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5cfe5a0 0x562eb5cf71d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- 172.21.15.120:0/2433313956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5cf6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- 172.21.15.120:0/2433313956 >> 172.21.15.120:0/2433313956 conn(0x562eb5cf29b0 msgr2=0x562eb5cf3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- 172.21.15.120:0/2433313956 shutdown_connections 2024-09-10T22:59:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- 172.21.15.120:0/2433313956 wait complete. 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- start start 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 Processor -- start 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5d09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a6950 con 0x562eb5d00fe0 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a7dc0 con 0x562eb5cfe5a0 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54901a7020 con 0x562eb5cfbc70 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.507+0000 7f54981e9740 10 client.0 _ll_register_callbacks cb 0x7f5480005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5d09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44214/0 (socket says 172.21.15.120:44214) 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 -- 172.21.15.120:0/984052040 learned_addr learned my addr 172.21.15.120:0/984052040 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51868/0 (socket says 172.21.15.120:51868) 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 -- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 msgr2=0x562eb5d09fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5d09fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 -- 172.21.15.120:0/984052040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 msgr2=0x562eb5d0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 -- 172.21.15.120:0/984052040 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb5d0ba70 con 0x562eb5d00fe0 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548e7fc640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f548f7fe640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5488006010 tx=0x7f54880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12d9175289e4176 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f5472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488004590 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5488004730 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54840b5c70 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.508+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54880305a0 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 10 client.16839 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562eb5d0cdf0 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54840b5c70 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5488030070 con 0x562eb5d00fe0 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 10 client.16839 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 20 client.16839 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d7db40 0x562eb5d7ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 4 client.16839 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562eb5c670f0 con 0x562eb5d7db40 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f548effd640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d7db40 0x562eb5d7ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f548effd640 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d7db40 0x562eb5d7ff60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6474b750c6639e server_cookie=b2000c0d7a31282d in_seq=0 out_seq=0 2024-09-10T22:59:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.509+0000 7f5472ff5640 10 client.16839 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:44 smithi176 ceph-mon[34982]: pgmap v565: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-09-10T22:59:44.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1232933 16.42 MB/sec execute 279 sec latency 42.368 ms 2024-09-10T22:59:44.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:44 smithi120 ceph-mon[31247]: pgmap v565: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-09-10T22:59:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:44 smithi153 ceph-mon[34951]: pgmap v565: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-09-10T22:59:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f5472ff5640 1 -- 172.21.15.120:0/984052040 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562eb5c670f0 con 0x562eb5d7db40 2024-09-10T22:59:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f5472ff5640 10 client.16839 handle_command_reply: tid=0 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f54981e9740 1 client.16839 shutdown 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 msgr2=0x562eb5d0b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5488006010 tx=0x7f54880095c0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 shutdown_connections 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.763+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562eb5d7db40 0x562eb5d7ff60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5472ff5640 0 client.16839 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562eb5d00fe0 0x562eb5d0b240 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562eb5cfe5a0 0x562eb5d0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 --2- 172.21.15.120:0/984052040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562eb5cfbc70 0x562eb5d09fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 >> 172.21.15.120:0/984052040 conn(0x562eb5cf29b0 msgr2=0x562eb5cf13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 shutdown_connections 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 1 -- 172.21.15.120:0/984052040 wait complete. 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f54981e9740 20 client.16839 trim_cache size 0 max 16384 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:59:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:59:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 msgr2=0x7f54780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 0x7f54780b77a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f548802ae00 tx=0x7f548802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 msgr2=0x7f5490199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f548400e9f0 tx=0x7f548400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 shutdown_connections 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54780b52e0 0x7f54780b77a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5490103370 0x7f5490199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54901029b0 0x7f5490199380 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 --2- 172.21.15.120:0/3260556331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490101860 0x7f5490198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.764+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 >> 172.21.15.120:0/3260556331 conn(0x7f54900fd140 msgr2=0x7f54900feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.765+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 shutdown_connections 2024-09-10T22:59:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:44.765+0000 7f5495416640 1 -- 172.21.15.120:0/3260556331 wait complete. 2024-09-10T22:59:45.465 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:59:45.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:59:45.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1235389 16.41 MB/sec execute 280 sec latency 70.128 ms 2024-09-10T22:59:45.757 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:46.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 -- 172.21.15.120:0/3651737256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 msgr2=0x7fe644104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 --2- 172.21.15.120:0/3651737256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644104080 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe634009900 tx=0x7fe63402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:46.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 -- 172.21.15.120:0/3651737256 shutdown_connections 2024-09-10T22:59:46.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 --2- 172.21.15.120:0/3651737256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe64410d100 0x7fe64410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 --2- 172.21.15.120:0/3651737256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6441045c0 0x7fe64410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 --2- 172.21.15.120:0/3651737256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644104080 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 -- 172.21.15.120:0/3651737256 >> 172.21.15.120:0/3651737256 conn(0x7fe6440fd700 msgr2=0x7fe6440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 -- 172.21.15.120:0/3651737256 shutdown_connections 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.274+0000 7fe648e8a640 1 -- 172.21.15.120:0/3651737256 wait complete. 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 Processor -- start 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 -- start start 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 0x7fe6441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe64410d100 0x7fe64419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112d70 con 0x7fe644103ca0 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112bf0 con 0x7fe6441045c0 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe648e8a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112ef0 con 0x7fe64410d100 2024-09-10T22:59:46.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44236/0 (socket says 172.21.15.120:44236) 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe641d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 0x7fe6441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 -- 172.21.15.120:0/1721855356 learned_addr learned my addr 172.21.15.120:0/1721855356 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe642d9d640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe64410d100 0x7fe64419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 -- 172.21.15.120:0/1721855356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe64410d100 msgr2=0x7fe64419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe64410d100 0x7fe64419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.275+0000 7fe64259c640 1 -- 172.21.15.120:0/1721855356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 msgr2=0x7fe6441997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe64259c640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 0x7fe6441997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe64259c640 1 -- 172.21.15.120:0/1721855356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64419f330 con 0x7fe644103ca0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe641d9b640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 0x7fe6441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe64259c640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe634005940 tx=0x7fe634004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe6337fe640 1 -- 172.21.15.120:0/1721855356 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63403f030 con 0x7fe644103ca0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe6337fe640 1 -- 172.21.15.120:0/1721855356 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe634003aa0 con 0x7fe644103ca0 2024-09-10T22:59:46.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64419f5c0 con 0x7fe644103ca0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe6337fe640 1 -- 172.21.15.120:0/1721855356 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634035070 con 0x7fe644103ca0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.276+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a5ce0 con 0x7fe644103ca0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe6337fe640 1 -- 172.21.15.120:0/1721855356 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe63404c060 con 0x7fe644103ca0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 Processor -- start 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 -- start start 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f5800 0x5617ef801250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef801790 0x5617ef803b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617ef76be60 con 0x5617ef801790 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617ef75b290 con 0x5617ef7f5800 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.279+0000 7fe64b25f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5617ef75f0a0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe64259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f5800 0x5617ef801250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37100/0 (socket says 172.21.15.120:37100) 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 -- 172.21.15.120:0/2214765084 learned_addr learned my addr 172.21.15.120:0/2214765084 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe641d9b640 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef801790 0x5617ef803b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe6337fe640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 0x7fe62c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 -- 172.21.15.120:0/2214765084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f5800 msgr2=0x5617ef801250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f5800 0x5617ef801250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 -- 172.21.15.120:0/2214765084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef801790 msgr2=0x5617ef803b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef801790 0x5617ef803b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 -- 172.21.15.120:0/2214765084 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ef4b9930 con 0x5617ef7f4ee0 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 0x7fe62c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe6337fe640 1 -- 172.21.15.120:0/1721855356 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe634105c60 con 0x7fe644103ca0 2024-09-10T22:59:46.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010540 tx=0x7fe63c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afc1fd9312e933d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe630ff9640 1 -- 172.21.15.120:0/2214765084 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0134c0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.280+0000 7fe642d9d640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 0x7fe62c0b77a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe63c021000 tx=0x7fe63c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe630ff9640 1 -- 172.21.15.120:0/2214765084 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c01d8b0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe630ff9640 1 -- 172.21.15.120:0/2214765084 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0117e0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 -- 172.21.15.120:0/2214765084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 msgr2=0x5617ef7f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010540 tx=0x7fe63c009f90 comp rx=0 tx=0).stop 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 -- 172.21.15.120:0/2214765084 shutdown_connections 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef801790 0x5617ef803b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f5800 0x5617ef801250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2214765084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f52c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.281+0000 7fe64b25f740 1 -- 172.21.15.120:0/2214765084 >> 172.21.15.120:0/2214765084 conn(0x5617ef7f2d10 msgr2=0x5617ef7f3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- 172.21.15.120:0/2214765084 shutdown_connections 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- 172.21.15.120:0/2214765084 wait complete. 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- start start 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 Processor -- start 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5617ef5bc890 con 0x5617ef7f5800 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5617ef6f5d40 con 0x5617ef7f4ee0 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5617ef804440 con 0x5617ef801790 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.282+0000 7fe64b25f740 10 client.0 _ll_register_callbacks cb 0x5617ef759c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe64259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe641d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe64259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51906/0 (socket says 172.21.15.120:51906) 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe641d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44258/0 (socket says 172.21.15.120:44258) 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe64259c640 1 -- 172.21.15.120:0/3372561419 learned_addr learned my addr 172.21.15.120:0/3372561419 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37114/0 (socket says 172.21.15.120:37114) 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 -- 172.21.15.120:0/3372561419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 msgr2=0x5617ef7fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 -- 172.21.15.120:0/3372561419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 msgr2=0x5617ef7fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 -- 172.21.15.120:0/3372561419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ef80d830 con 0x5617ef801790 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe64259c640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe641d9b640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe642d9d640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010510 tx=0x7fe63c01dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8714e098baf94baf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.304 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c034060 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe63c011770 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.283+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63c01d8b0 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe63c0134c0 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 10 client.26332 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5617ef8101e0 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c01d8b0 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe63c00f040 con 0x5617ef801790 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 10 client.26332 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 20 client.26332 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef811640 0x5617ef813a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 4 client.26332 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617ef76ddb0 con 0x5617ef811640 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.284+0000 7fe64259c640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef811640 0x5617ef813a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.285+0000 7fe629ffb640 10 client.26332 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.285+0000 7fe64259c640 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef811640 0x5617ef813a30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafc2819b59e07 server_cookie=a127f24d868c6566 in_seq=0 out_seq=0 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.286+0000 7fe629ffb640 1 -- 172.21.15.120:0/3372561419 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617ef76ddb0 con 0x5617ef811640 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.286+0000 7fe629ffb640 10 client.26332 handle_command_reply: tid=0 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.286+0000 7fe64b25f740 1 client.26332 shutdown 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 msgr2=0x5617ef7ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe63c010510 tx=0x7fe63c01dc90 comp rx=0 tx=0).stop 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 shutdown_connections 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef811640 0x5617ef813a30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7ff220 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 --2- 172.21.15.120:0/3372561419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe629ffb640 0 client.26332 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:46.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 >> 172.21.15.120:0/3372561419 conn(0x5617ef7f2d10 msgr2=0x5617ef7f1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 shutdown_connections 2024-09-10T22:59:46.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 1 -- 172.21.15.120:0/3372561419 wait complete. 2024-09-10T22:59:46.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.287+0000 7fe64b25f740 20 client.26332 trim_cache size 0 max 16384 2024-09-10T22:59:46.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 Processor -- start 2024-09-10T22:59:46.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 -- start start 2024-09-10T22:59:46.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ac350 con 0x5617ef7f5800 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441aebd0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64b25f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ac9c0 con 0x5617ef801790 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe641d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe64259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe641d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44260/0 (socket says 172.21.15.120:44260) 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe641d9b640 1 -- 172.21.15.120:0/2774449293 learned_addr learned my addr 172.21.15.120:0/2774449293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.306+0000 7fe642d9d640 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 -- 172.21.15.120:0/2774449293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 msgr2=0x5617ef7fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 -- 172.21.15.120:0/2774449293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 msgr2=0x5617ef7fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 -- 172.21.15.120:0/2774449293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ef7ee640 con 0x5617ef7f4ee0 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe641d9b640 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe64259c640 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f4530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe634103b00 tx=0x7fe634108380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106eeeab0432d0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe62bfff640 1 -- 172.21.15.120:0/2774449293 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634049040 con 0x5617ef7f4ee0 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe62bfff640 1 -- 172.21.15.120:0/2774449293 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6340d5320 con 0x5617ef7f4ee0 2024-09-10T22:59:46.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.307+0000 7fe62bfff640 1 -- 172.21.15.120:0/2774449293 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6340cc500 con 0x5617ef7f4ee0 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 -- 172.21.15.120:0/2774449293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 msgr2=0x5617ef7f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f4530 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe634103b00 tx=0x7fe634108380 comp rx=0 tx=0).stop 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 -- 172.21.15.120:0/2774449293 shutdown_connections 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef801790 0x5617ef7fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef7fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 --2- 172.21.15.120:0/2774449293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef7f4ee0 0x5617ef7f4530 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 -- 172.21.15.120:0/2774449293 >> 172.21.15.120:0/2774449293 conn(0x5617ef7f2d10 msgr2=0x5617ef7f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.308+0000 7fe64b25f740 1 -- 172.21.15.120:0/2774449293 shutdown_connections 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 -- 172.21.15.120:0/2774449293 wait complete. 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 -- start start 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 Processor -- start 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef809be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.309+0000 7fe64b25f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 0x5617ef804650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 0x5617ef804650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64b25f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ad090 con 0x5617ef7f5800 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64b25f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ae500 con 0x5617ef801790 2024-09-10T22:59:46.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37138/0 (socket says 172.21.15.120:37138) 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe642d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 0x5617ef804650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51926/0 (socket says 172.21.15.120:51926) 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64259c640 1 -- 172.21.15.120:0/1849760790 learned_addr learned my addr 172.21.15.120:0/1849760790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6441ad760 con 0x5617ef7f4ee0 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe64b25f740 10 client.0 _ll_register_callbacks cb 0x7fe63c020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.310+0000 7fe641d9b640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef809be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 -- 172.21.15.120:0/1849760790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 msgr2=0x5617ef804650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 0x5617ef804650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 -- 172.21.15.120:0/1849760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 msgr2=0x5617ef809be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef809be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 -- 172.21.15.120:0/1849760790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ef804be0 con 0x5617ef7f4ee0 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe641d9b640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef809be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64259c640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6341037e0 tx=0x7fe63402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb86a673fc1d3d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634049070 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617ef769860 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.311+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe6340cc760 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6340cbe50 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 10 client.26338 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5617ef87ec40 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5617ef769860 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe6340f3190 con 0x5617ef7f4ee0 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 10 client.26338 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:46.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 20 client.26338 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef87f220 0x5617ef881640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 4 client.26338 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5617ef766c90 con 0x5617ef87f220 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.312+0000 7fe641d9b640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef87f220 0x5617ef881640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.313+0000 7fe629ffb640 10 client.26338 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:46.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.313+0000 7fe641d9b640 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef87f220 0x5617ef881640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae15782f4eba1ae server_cookie=33b2df8d7a1abfbb in_seq=0 out_seq=0 2024-09-10T22:59:46.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe629ffb640 1 -- 172.21.15.120:0/1849760790 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5617ef766c90 con 0x5617ef87f220 2024-09-10T22:59:46.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe629ffb640 10 client.26338 handle_command_reply: tid=0 2024-09-10T22:59:46.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 client.26338 shutdown 2024-09-10T22:59:46.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 msgr2=0x5617ef8096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6341037e0 tx=0x7fe63402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 shutdown_connections 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5617ef87f220 0x5617ef881640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5617ef801790 0x5617ef804650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5617ef7f5800 0x5617ef809be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe629ffb640 0 client.26338 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 --2- 172.21.15.120:0/1849760790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5617ef7f4ee0 0x5617ef8096a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 >> 172.21.15.120:0/1849760790 conn(0x5617ef7f2d10 msgr2=0x5617ef80ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 shutdown_connections 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.315+0000 7fe64b25f740 1 -- 172.21.15.120:0/1849760790 wait complete. 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe64b25f740 20 client.26338 trim_cache size 0 max 16384 2024-09-10T22:59:46.320 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "2b3a1bd2-bd10-4048-b05f-00fd520d6700", 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 msgr2=0x7fe62c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 0x7fe62c0b77a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe63c021000 tx=0x7fe63c01e000 comp rx=0 tx=0).stop 2024-09-10T22:59:46.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 msgr2=0x7fe644199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe634005940 tx=0x7fe634004270 comp rx=0 tx=0).stop 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 shutdown_connections 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe62c0b52e0 0x7fe62c0b77a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe64410d100 0x7fe64419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6441045c0 0x7fe6441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 --2- 172.21.15.120:0/1721855356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe644103ca0 0x7fe644199280 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 >> 172.21.15.120:0/1721855356 conn(0x7fe6440fd700 msgr2=0x7fe6440fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 shutdown_connections 2024-09-10T22:59:46.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:46.316+0000 7fe648e8a640 1 -- 172.21.15.120:0/1721855356 wait complete. 2024-09-10T22:59:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:46 smithi120 ceph-mon[31247]: pgmap v566: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-09-10T22:59:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:46 smithi176 ceph-mon[34982]: pgmap v566: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-09-10T22:59:46.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1237556 16.39 MB/sec execute 281 sec latency 43.154 ms 2024-09-10T22:59:46.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:46 smithi153 ceph-mon[34951]: pgmap v566: 97 pgs: 97 active+clean; 54 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-09-10T22:59:47.132 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:47.515 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:47 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T22:59:47.515 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:47 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.515 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:47 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T22:59:47.515 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:47 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:47 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T22:59:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:47 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:47 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T22:59:47.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:47 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1240520 16.39 MB/sec execute 282 sec latency 14.644 ms 2024-09-10T22:59:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:47 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T22:59:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:47 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:47 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T22:59:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:47 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 -- 172.21.15.120:0/963974945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 msgr2=0x7f8a5010cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 --2- 172.21.15.120:0/963974945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a5010cb10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009900 tx=0x7f8a4002a410 comp rx=0 tx=0).stop 2024-09-10T22:59:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 -- 172.21.15.120:0/963974945 shutdown_connections 2024-09-10T22:59:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 --2- 172.21.15.120:0/963974945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 0x7f8a5010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 --2- 172.21.15.120:0/963974945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a5010cb10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 --2- 172.21.15.120:0/963974945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a50103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 -- 172.21.15.120:0/963974945 >> 172.21.15.120:0/963974945 conn(0x7f8a500fd700 msgr2=0x7f8a500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 -- 172.21.15.120:0/963974945 shutdown_connections 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.894+0000 7f8a578ee640 1 -- 172.21.15.120:0/963974945 wait complete. 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 Processor -- start 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 -- start start 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 0x7f8a5019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112d60 con 0x7f8a50104530 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112be0 con 0x7f8a50103b30 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a578ee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112ee0 con 0x7f8a5010d0e0 2024-09-10T22:59:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44272/0 (socket says 172.21.15.120:44272) 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51948/0 (socket says 172.21.15.120:51948) 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 -- 172.21.15.120:0/748636888 learned_addr learned my addr 172.21.15.120:0/748636888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 0x7f8a5019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 -- 172.21.15.120:0/748636888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 msgr2=0x7f8a5019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 0x7f8a5019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 -- 172.21.15.120:0/748636888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 msgr2=0x7f8a501992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a54e89640 1 -- 172.21.15.120:0/748636888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019f3a0 con 0x7f8a50104530 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.895+0000 7f8a5568a640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a54e89640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005940 tx=0x7f8a400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a467fc640 1 -- 172.21.15.120:0/748636888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4003f030 con 0x7f8a50104530 2024-09-10T22:59:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a467fc640 1 -- 172.21.15.120:0/748636888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a400044c0 con 0x7f8a50104530 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a467fc640 1 -- 172.21.15.120:0/748636888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40035070 con 0x7f8a50104530 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019f630 con 0x7f8a50104530 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.896+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a5d50 con 0x7f8a50104530 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a467fc640 1 -- 172.21.15.120:0/748636888 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8a4003ba20 con 0x7f8a50104530 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 Processor -- start 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 -- start start 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e50ce50 con 0x55dc2e5a7140 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e4fc280 con 0x55dc2e596480 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e500090 con 0x55dc2e596da0 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a55e8b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a55e8b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51950/0 (socket says 172.21.15.120:51950) 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a54e89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a55e8b640 1 -- 172.21.15.120:0/3649963744 learned_addr learned my addr 172.21.15.120:0/3649963744 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.899+0000 7f8a5568a640 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 -- 172.21.15.120:0/3649963744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 msgr2=0x55dc2e5a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 -- 172.21.15.120:0/3649963744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 msgr2=0x55dc2e5a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 -- 172.21.15.120:0/3649963744 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2e25a650 con 0x55dc2e596480 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a467fc640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 0x7f8a3c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009f50 tx=0x7f8a480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488378273c7e0140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a5568a640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 0x7f8a3c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a467fc640 1 -- 172.21.15.120:0/748636888 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8a40106160 con 0x7f8a50104530 2024-09-10T22:59:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a3b7fe640 1 -- 172.21.15.120:0/3649963744 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48004d20 con 0x55dc2e596480 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a3b7fe640 1 -- 172.21.15.120:0/3649963744 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48004050 con 0x55dc2e596480 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.900+0000 7f8a3b7fe640 1 -- 172.21.15.120:0/3649963744 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48005280 con 0x55dc2e596480 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- 172.21.15.120:0/3649963744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 msgr2=0x55dc2e596860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009f50 tx=0x7f8a480118d0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a5568a640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 0x7f8a3c0b7820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55dc2e599600 tx=0x7f8a4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- 172.21.15.120:0/3649963744 shutdown_connections 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/3649963744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596480 0x55dc2e596860 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- 172.21.15.120:0/3649963744 >> 172.21.15.120:0/3649963744 conn(0x55dc2e5942b0 msgr2=0x55dc2e595450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- 172.21.15.120:0/3649963744 shutdown_connections 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- 172.21.15.120:0/3649963744 wait complete. 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 -- start start 2024-09-10T22:59:47.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.901+0000 7f8a59cc3740 1 Processor -- start 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596480 0x55dc2e5a6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e59fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e35d700 con 0x55dc2e5a7140 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e497110 con 0x55dc2e596da0 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dc2e5a9df0 con 0x55dc2e596480 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a59cc3740 10 client.0 _ll_register_callbacks cb 0x55dc2e4b0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a55e8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e59fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596480 0x55dc2e5a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51956/0 (socket says 172.21.15.120:51956) 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 -- 172.21.15.120:0/1590141214 learned_addr learned my addr 172.21.15.120:0/1590141214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:47.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a55e8b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e59fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44294/0 (socket says 172.21.15.120:44294) 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 -- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596480 msgr2=0x55dc2e5a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596480 0x55dc2e5a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 -- 172.21.15.120:0/1590141214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 msgr2=0x55dc2e59fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e59fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 -- 172.21.15.120:0/1590141214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2e4c4c40 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a54e89640 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a40032af0 tx=0x7f8a40009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d43792ffc8ffe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.902+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40049040 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48004050 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8a400c8c70 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a400cc7d0 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 10 client.26261 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc2e5b9b70 con 0x55dc2e596da0 2024-09-10T22:59:47.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48004050 con 0x55dc2e596da0 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8a400ccc60 con 0x55dc2e596da0 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 10 client.26261 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 20 client.26261 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e5baf80 0x55dc2e5bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 4 client.26261 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.903+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc2e50eda0 con 0x55dc2e5baf80 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.904+0000 7f8a5568a640 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e5baf80 0x55dc2e5bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.904+0000 7f8a38ff9640 10 client.26261 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.904+0000 7f8a5568a640 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e5baf80 0x55dc2e5bd3a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09e867d8ca8d151 server_cookie=26c11500018fbd0d in_seq=0 out_seq=0 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.905+0000 7f8a38ff9640 1 -- 172.21.15.120:0/1590141214 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc2e50eda0 con 0x55dc2e5baf80 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.905+0000 7f8a38ff9640 10 client.26261 handle_command_reply: tid=0 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.905+0000 7f8a59cc3740 1 client.26261 shutdown 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.905+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 msgr2=0x55dc2e5a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.905+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a40032af0 tx=0x7f8a40009620 comp rx=0 tx=0).stop 2024-09-10T22:59:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 shutdown_connections 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e5baf80 0x55dc2e5bd3a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e5a7140 0x55dc2e59fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a6be0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1590141214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596480 0x55dc2e5a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a38ff9640 0 client.26261 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 >> 172.21.15.120:0/1590141214 conn(0x55dc2e5942b0 msgr2=0x55dc2e599e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 shutdown_connections 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1590141214 wait complete. 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.906+0000 7f8a59cc3740 20 client.26261 trim_cache size 0 max 16384 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 Processor -- start 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 -- start start 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501aa0c0 con 0x55dc2e596480 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501ac940 con 0x55dc2e596da0 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501aa730 con 0x55dc2e5a7140 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44300/0 (socket says 172.21.15.120:44300) 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a5568a640 1 -- 172.21.15.120:0/1635516618 learned_addr learned my addr 172.21.15.120:0/1635516618 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a54e89640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.923+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 -- 172.21.15.120:0/1635516618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e5a7140 msgr2=0x55dc2e5a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 -- 172.21.15.120:0/1635516618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 msgr2=0x55dc2e5a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 -- 172.21.15.120:0/1635516618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2e59e9d0 con 0x55dc2e596480 2024-09-10T22:59:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a54e89640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a5568a640 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55dc2e5bdba0 tx=0x7f8a4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a5591b48674aee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a3affd640 1 -- 172.21.15.120:0/1635516618 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c00fe00 con 0x55dc2e596480 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a3affd640 1 -- 172.21.15.120:0/1635516618 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c005020 con 0x55dc2e596480 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.924+0000 7f8a3affd640 1 -- 172.21.15.120:0/1635516618 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01da20 con 0x55dc2e596480 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1635516618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 msgr2=0x55dc2e5a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55dc2e5bdba0 tx=0x7f8a4c017000 comp rx=0 tx=0).stop 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1635516618 shutdown_connections 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e5a7140 0x55dc2e5a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e596da0 0x55dc2e5a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/1635516618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5a6870 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1635516618 >> 172.21.15.120:0/1635516618 conn(0x55dc2e5942b0 msgr2=0x55dc2e5a10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1635516618 shutdown_connections 2024-09-10T22:59:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- 172.21.15.120:0/1635516618 wait complete. 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 -- start start 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.925+0000 7f8a59cc3740 1 Processor -- start 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e5a7140 0x55dc2e5b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501aae00 con 0x55dc2e596480 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501ac270 con 0x55dc2e5a7140 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501ab4d0 con 0x55dc2e596da0 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a59cc3740 10 client.0 _ll_register_callbacks cb 0x7f8a40109930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a5568a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44312/0 (socket says 172.21.15.120:44312) 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a5568a640 1 -- 172.21.15.120:0/641610521 learned_addr learned my addr 172.21.15.120:0/641610521 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a55e8b640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e5a7140 0x55dc2e5b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 -- 172.21.15.120:0/641610521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e5a7140 msgr2=0x55dc2e5b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e5a7140 0x55dc2e5b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 -- 172.21.15.120:0/641610521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 msgr2=0x55dc2e5b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 -- 172.21.15.120:0/641610521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2e5a6f50 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a5568a640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a54e89640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5b2900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a400cc700 tx=0x7f8a40009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d4e90358ca37a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.926+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400f3070 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8a400d5a50 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c005020 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a4010d020 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 10 client.26356 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc2e5bd710 con 0x55dc2e596da0 2024-09-10T22:59:47.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c005020 con 0x55dc2e596da0 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8a400c8350 con 0x55dc2e596da0 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 10 client.26356 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 20 client.26356 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e628f10 0x55dc2e62b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 4 client.26356 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc2e50a850 con 0x55dc2e628f10 2024-09-10T22:59:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.927+0000 7f8a5568a640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e628f10 0x55dc2e62b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "2b3a1bd2-bd10-4048-b05f-00fd520d6700", 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T22:59:47.948 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T22:59:47.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T22:59:47.953 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T22:59:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.955 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T22:59:47.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.928+0000 7f8a5568a640 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e628f10 0x55dc2e62b330 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0ccc73e8543d05 server_cookie=94c6e5f6492e00e8 in_seq=0 out_seq=0 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.928+0000 7f8a38ff9640 10 client.26356 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a38ff9640 1 -- 172.21.15.120:0/641610521 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55dc2e50a850 con 0x55dc2e628f10 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a38ff9640 10 client.26356 handle_command_reply: tid=0 2024-09-10T22:59:47.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 client.26356 shutdown 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 msgr2=0x55dc2e5b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5b2900 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a400cc700 tx=0x7f8a40009300 comp rx=0 tx=0).stop 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 shutdown_connections 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc2e628f10 0x55dc2e62b330 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc2e5a7140 0x55dc2e5b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc2e596da0 0x55dc2e5b2900 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 --2- 172.21.15.120:0/641610521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc2e596480 0x55dc2e5b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a38ff9640 0 client.26356 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.929+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 >> 172.21.15.120:0/641610521 conn(0x55dc2e5942b0 msgr2=0x55dc2e5a18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 shutdown_connections 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a59cc3740 1 -- 172.21.15.120:0/641610521 wait complete. 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a59cc3740 20 client.26356 trim_cache size 0 max 16384 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 msgr2=0x7f8a3c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 0x7f8a3c0b7820 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55dc2e599600 tx=0x7f8a4c005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 msgr2=0x7f8a50199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8a40005940 tx=0x7f8a400040c0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 shutdown_connections 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8a3c0b5360 0x7f8a3c0b7820 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a5010d0e0 0x7f8a5019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8a50104530 0x7f8a50199830 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 --2- 172.21.15.120:0/748636888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a50103b30 0x7f8a501992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 >> 172.21.15.120:0/748636888 conn(0x7f8a500fd700 msgr2=0x7f8a500fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 shutdown_connections 2024-09-10T22:59:47.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T22:59:47.930+0000 7f8a578ee640 1 -- 172.21.15.120:0/748636888 wait complete. 2024-09-10T22:59:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:48 smithi176 ceph-mon[34982]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 525 op/s 2024-09-10T22:59:48.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1243595 16.39 MB/sec execute 283 sec latency 16.128 ms 2024-09-10T22:59:48.689 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:48 smithi120 ceph-mon[31247]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 525 op/s 2024-09-10T22:59:48.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:48 smithi153 ceph-mon[34951]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 525 op/s 2024-09-10T22:59:48.721 INFO:tasks.cephfs.filesystem:scrub status for tag:2b3a1bd2-bd10-4048-b05f-00fd520d6700 - {'path': '/', 'tag': '2b3a1bd2-bd10-4048-b05f-00fd520d6700', 'options': 'recursive,force'} 2024-09-10T22:59:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:49 smithi176 ceph-mon[34982]: pgmap v568: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-10T22:59:49.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1246993 16.40 MB/sec execute 284 sec latency 13.914 ms 2024-09-10T22:59:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:49 smithi120 ceph-mon[31247]: pgmap v568: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-10T22:59:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:49 smithi153 ceph-mon[34951]: pgmap v568: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-10T22:59:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:50 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T22:59:50.597 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:50 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T22:59:50.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1250081 16.41 MB/sec execute 285 sec latency 54.461 ms 2024-09-10T22:59:50.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:50 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T22:59:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:51 smithi176 ceph-mon[34982]: pgmap v569: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-09-10T22:59:51.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1252865 16.40 MB/sec execute 286 sec latency 19.481 ms 2024-09-10T22:59:51.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:51 smithi120 ceph-mon[31247]: pgmap v569: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-09-10T22:59:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:51 smithi153 ceph-mon[34951]: pgmap v569: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-09-10T22:59:52.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1256055 16.39 MB/sec execute 287 sec latency 41.380 ms 2024-09-10T22:59:53.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1259347 16.40 MB/sec execute 288 sec latency 16.710 ms 2024-09-10T22:59:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:53 smithi120 ceph-mon[31247]: pgmap v570: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-09-10T22:59:53.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:53 smithi153 ceph-mon[34951]: pgmap v570: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-09-10T22:59:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:53 smithi176 ceph-mon[34982]: pgmap v570: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 416 op/s 2024-09-10T22:59:54.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1262450 16.40 MB/sec execute 289 sec latency 12.476 ms 2024-09-10T22:59:55.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1265185 16.40 MB/sec execute 290 sec latency 98.993 ms 2024-09-10T22:59:55.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:55 smithi120 ceph-mon[31247]: pgmap v571: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-09-10T22:59:55.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:55 smithi153 ceph-mon[34951]: pgmap v571: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-09-10T22:59:55.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:55 smithi176 ceph-mon[34982]: pgmap v571: 97 pgs: 97 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-09-10T22:59:56.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1268269 16.40 MB/sec execute 291 sec latency 30.790 ms 2024-09-10T22:59:57.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1271684 16.41 MB/sec execute 292 sec latency 13.358 ms 2024-09-10T22:59:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:57 smithi120 ceph-mon[31247]: pgmap v572: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-10T22:59:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:57 smithi153 ceph-mon[34951]: pgmap v572: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-10T22:59:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:57 smithi176 ceph-mon[34982]: pgmap v572: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 466 op/s 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 -- 172.21.15.120:0/4033848048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c1018b0 msgr2=0x7fec7c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 --2- 172.21.15.120:0/4033848048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c1018b0 0x7fec7c101cb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fec78009cd0 tx=0x7fec7802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 -- 172.21.15.120:0/4033848048 shutdown_connections 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 --2- 172.21.15.120:0/4033848048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 0x7fec7c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 --2- 172.21.15.120:0/4033848048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c102a00 0x7fec7c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 --2- 172.21.15.120:0/4033848048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c1018b0 0x7fec7c101cb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 -- 172.21.15.120:0/4033848048 >> 172.21.15.120:0/4033848048 conn(0x7fec7c0fd150 msgr2=0x7fec7c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 -- 172.21.15.120:0/4033848048 shutdown_connections 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.854+0000 7fec839cd640 1 -- 172.21.15.120:0/4033848048 wait complete. 2024-09-10T22:59:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 Processor -- start 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 -- start start 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 0x7fec7c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 0x7fec7c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c10e650 con 0x7fec7c102a00 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c10e4d0 con 0x7fec7c1018b0 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec839cd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c10e7d0 con 0x7fec7c1033c0 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43228/0 (socket says 172.21.15.120:43228) 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 -- 172.21.15.120:0/1991967175 learned_addr learned my addr 172.21.15.120:0/1991967175 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec81f6a640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 0x7fec7c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec81769640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 0x7fec7c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 -- 172.21.15.120:0/1991967175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 msgr2=0x7fec7c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 0x7fec7c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 -- 172.21.15.120:0/1991967175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 msgr2=0x7fec7c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.855+0000 7fec80f68640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 0x7fec7c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec80f68640 1 -- 172.21.15.120:0/1991967175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c19a890 con 0x7fec7c102a00 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec81769640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 0x7fec7c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec80f68640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c920 tx=0x7fec6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec727fc640 1 -- 172.21.15.120:0/1991967175 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c007c70 con 0x7fec7c102a00 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec727fc640 1 -- 172.21.15.120:0/1991967175 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c007e10 con 0x7fec7c102a00 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec727fc640 1 -- 172.21.15.120:0/1991967175 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c018820 con 0x7fec7c102a00 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c19ab80 con 0x7fec7c102a00 2024-09-10T22:59:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.856+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1a1250 con 0x7fec7c102a00 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.858+0000 7fec86878740 1 Processor -- start 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 -- start start 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec727fc640 1 -- 172.21.15.120:0/1991967175 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fec6c00f450 con 0x7fec7c102a00 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b173c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b179090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a57b0e36f0 con 0x55a57b171860 2024-09-10T22:59:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a57b0d2b20 con 0x55a57b176bd0 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec86878740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a57b0d6930 con 0x55a57b174190 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81f6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b173c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57874/0 (socket says 172.21.15.120:57874) 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 -- 172.21.15.120:0/766731810 learned_addr learned my addr 172.21.15.120:0/766731810 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec80f68640 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b179090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 -- 172.21.15.120:0/766731810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 msgr2=0x55a57b179090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b179090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 -- 172.21.15.120:0/766731810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 msgr2=0x55a57b173c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b173c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81769640 1 -- 172.21.15.120:0/766731810 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57b098530 con 0x55a57b174190 2024-09-10T22:59:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.859+0000 7fec81f6a640 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b173c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec727fc640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 0x7fec680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec81769640 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec78005cc0 tx=0x7fec780314b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168ddcdb1d064117 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec81f6a640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 0x7fec680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec727fc640 1 -- 172.21.15.120:0/1991967175 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec6c0e5d20 con 0x7fec7c102a00 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec677fe640 1 -- 172.21.15.120:0/766731810 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78031b50 con 0x55a57b174190 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec677fe640 1 -- 172.21.15.120:0/766731810 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec78036040 con 0x55a57b174190 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.860+0000 7fec677fe640 1 -- 172.21.15.120:0/766731810 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78004710 con 0x55a57b174190 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- 172.21.15.120:0/766731810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 msgr2=0x55a57b176580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec81f6a640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 0x7fec680b77a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec7c199ad0 tx=0x7fec74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec78005cc0 tx=0x7fec780314b0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- 172.21.15.120:0/766731810 shutdown_connections 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b179090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b176580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 --2- 172.21.15.120:0/766731810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b173c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- 172.21.15.120:0/766731810 >> 172.21.15.120:0/766731810 conn(0x55a57b16c9b0 msgr2=0x55a57b16d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- 172.21.15.120:0/766731810 shutdown_connections 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- 172.21.15.120:0/766731810 wait complete. 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 -- start start 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.861+0000 7fec86878740 1 Processor -- start 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b17ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b17b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a57b00aab0 con 0x55a57b171860 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a57b078730 con 0x55a57b176bd0 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a57b179b50 con 0x55a57b174190 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec86878740 10 client.0 _ll_register_callbacks cb 0x55a57b0cc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81769640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b17ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:41518/0 (socket says 172.21.15.120:41518) 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 -- 172.21.15.120:0/44179017 learned_addr learned my addr 172.21.15.120:0/44179017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec80f68640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b17b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 -- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 msgr2=0x55a57b17b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b17b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 -- 172.21.15.120:0/44179017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 msgr2=0x55a57b17ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b17ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81f6a640 1 -- 172.21.15.120:0/44179017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57ae34790 con 0x55a57b176bd0 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec80f68640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b17b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.862+0000 7fec81769640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b17ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec81f6a640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec7401ea00 tx=0x7fec74004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc6ed5642f3ff19f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74004020 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec74018030 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec78036040 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec740041c0 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec86878740 10 client.26276 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.863+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a57aead620 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec78036040 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec74033480 con 0x55a57b176bd0 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 10 client.26276 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 20 client.26276 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b188b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 4 client.26276 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a57b0e5640 con 0x55a57b186770 2024-09-10T22:59:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec81769640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b188b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec64ff9640 10 client.26276 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.864+0000 7fec81769640 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b188b60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2e0534ddfb7314 server_cookie=553d92a825d9ba98 in_seq=0 out_seq=0 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.865+0000 7fec64ff9640 1 -- 172.21.15.120:0/44179017 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a57b0e5640 con 0x55a57b186770 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.865+0000 7fec64ff9640 10 client.26276 handle_command_reply: tid=0 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.865+0000 7fec86878740 1 client.26276 shutdown 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 msgr2=0x55a57b17c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec7401ea00 tx=0x7fec74004730 comp rx=0 tx=0).stop 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 shutdown_connections 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b188b60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b176bd0 0x55a57b17c0d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec64ff9640 0 client.26276 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b174190 0x55a57b17b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 --2- 172.21.15.120:0/44179017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b17ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 >> 172.21.15.120:0/44179017 conn(0x55a57b16c9b0 msgr2=0x55a57b16bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 shutdown_connections 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 1 -- 172.21.15.120:0/44179017 wait complete. 2024-09-10T22:59:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.866+0000 7fec86878740 20 client.26276 trim_cache size 0 max 16384 2024-09-10T22:59:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.882+0000 7fec86878740 1 Processor -- start 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.882+0000 7fec86878740 1 -- start start 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.882+0000 7fec86878740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b171860 0x55a57b17c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.882+0000 7fec86878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b176fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.882+0000 7fec86878740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec86878740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1a7d90 con 0x55a57b176bd0 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec86878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1aa610 con 0x55a57b174190 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec86878740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1a8400 con 0x55a57b171860 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43254/0 (socket says 172.21.15.120:43254) 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 -- 172.21.15.120:0/3953330396 learned_addr learned my addr 172.21.15.120:0/3953330396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81769640 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b171860 0x55a57b17c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec80f68640 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b176fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 -- 172.21.15.120:0/3953330396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b171860 msgr2=0x55a57b17c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b171860 0x55a57b17c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 -- 172.21.15.120:0/3953330396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 msgr2=0x55a57b176fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b176fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 -- 172.21.15.120:0/3953330396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57b17b870 con 0x55a57b176bd0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec81f6a640 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fec74019040 tx=0x7fec74005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e814c221b55b0260 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec66ffd640 1 -- 172.21.15.120:0/3953330396 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7401e7c0 con 0x55a57b176bd0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec66ffd640 1 -- 172.21.15.120:0/3953330396 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec74038070 con 0x55a57b176bd0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.883+0000 7fec66ffd640 1 -- 172.21.15.120:0/3953330396 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74033480 con 0x55a57b176bd0 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- 172.21.15.120:0/3953330396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 msgr2=0x55a57b1774f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fec74019040 tx=0x7fec74005b90 comp rx=0 tx=0).stop 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- 172.21.15.120:0/3953330396 shutdown_connections 2024-09-10T22:59:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b176bd0 0x55a57b1774f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b176fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 --2- 172.21.15.120:0/3953330396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b171860 0x55a57b17c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- 172.21.15.120:0/3953330396 >> 172.21.15.120:0/3953330396 conn(0x55a57b16c9b0 msgr2=0x55a57b167af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- 172.21.15.120:0/3953330396 shutdown_connections 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- 172.21.15.120:0/3953330396 wait complete. 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 -- start start 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.884+0000 7fec86878740 1 Processor -- start 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b1812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b176bd0 0x55a57b181830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1a8ad0 con 0x55a57b171860 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1a9f40 con 0x55a57b174190 2024-09-10T22:59:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1a91a0 con 0x55a57b176bd0 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec86878740 10 client.0 _ll_register_callbacks cb 0x7fec74012f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43264/0 (socket says 172.21.15.120:43264) 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 -- 172.21.15.120:0/647122202 learned_addr learned my addr 172.21.15.120:0/647122202 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81f6a640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b176bd0 0x55a57b181830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec80f68640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b1812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 -- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b176bd0 msgr2=0x55a57b181830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b176bd0 0x55a57b181830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 -- 172.21.15.120:0/647122202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 msgr2=0x55a57b1812f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b1812f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 -- 172.21.15.120:0/647122202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57b181d70 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec80f68640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b1812f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec81769640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fec78000f80 tx=0x7fec78009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747ad70e8790556a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.885+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78003c40 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec74038070 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec78003de0 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec78004040 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 10 client.16893 fetch_fsmap learned FSMap version 26 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a57b180c50 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec74038070 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec78030070 con 0x55a57b171860 2024-09-10T22:59:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 10 client.16893 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 20 client.16893 populate_metadata read hostname 'smithi120' 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 4 client.16893 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57b0e10f0 con 0x55a57b186770 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.886+0000 7fec80f68640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.887+0000 7fec80f68640 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b1f59c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4003e20c7f7c9c server_cookie=b0a2a1d247d440b9 in_seq=0 out_seq=0 2024-09-10T22:59:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:57.887+0000 7fec64ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.116+0000 7fec64ff9640 1 -- 172.21.15.120:0/647122202 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a57b0e10f0 con 0x55a57b186770 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.116+0000 7fec64ff9640 10 client.16893 handle_command_reply: tid=0 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.116+0000 7fec86878740 1 client.16893 shutdown 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.116+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 msgr2=0x55a57b1825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.116+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fec78000f80 tx=0x7fec78009360 comp rx=0 tx=0).stop 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 shutdown_connections 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a57b186770 0x55a57b1f59c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec64ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a57b176bd0 0x55a57b181830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a57b174190 0x55a57b1812f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 --2- 172.21.15.120:0/647122202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a57b171860 0x55a57b1825f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 >> 172.21.15.120:0/647122202 conn(0x55a57b16c9b0 msgr2=0x55a57b170cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 shutdown_connections 2024-09-10T22:59:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 1 -- 172.21.15.120:0/647122202 wait complete. 2024-09-10T22:59:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.117+0000 7fec86878740 20 client.16893 trim_cache size 0 max 16384 2024-09-10T22:59:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T22:59:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T22:59:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T22:59:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 msgr2=0x7fec680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 0x7fec680b77a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec7c199ad0 tx=0x7fec74005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 msgr2=0x7fec7c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c920 tx=0x7fec6c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 shutdown_connections 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec680b52e0 0x7fec680b77a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec7c1033c0 0x7fec7c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec7c102a00 0x7fec7c1993a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 --2- 172.21.15.120:0/1991967175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1018b0 0x7fec7c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 >> 172.21.15.120:0/1991967175 conn(0x7fec7c0fd150 msgr2=0x7fec7c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 shutdown_connections 2024-09-10T22:59:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T22:59:58.118+0000 7fec839cd640 1 -- 172.21.15.120:0/1991967175 wait complete. 2024-09-10T22:59:58.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1273874 16.38 MB/sec execute 293 sec latency 75.038 ms 2024-09-10T22:59:59.625 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 22:59:59 smithi176 ceph-mon[34982]: pgmap v573: 97 pgs: 97 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 345 op/s 2024-09-10T22:59:59.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1276916 16.39 MB/sec execute 294 sec latency 11.900 ms 2024-09-10T22:59:59.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 22:59:59 smithi120 ceph-mon[31247]: pgmap v573: 97 pgs: 97 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 345 op/s 2024-09-10T22:59:59.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 22:59:59 smithi153 ceph-mon[34951]: pgmap v573: 97 pgs: 97 active+clean; 49 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 345 op/s 2024-09-10T23:00:00.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1279458 16.38 MB/sec execute 295 sec latency 97.504 ms 2024-09-10T23:00:00.637 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:00 smithi120 ceph-mon[31247]: overall HEALTH_OK 2024-09-10T23:00:00.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:00 smithi153 ceph-mon[34951]: overall HEALTH_OK 2024-09-10T23:00:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:00 smithi176 ceph-mon[34982]: overall HEALTH_OK 2024-09-10T23:00:01.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1282435 16.39 MB/sec execute 296 sec latency 107.953 ms 2024-09-10T23:00:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:01 smithi120 ceph-mon[31247]: pgmap v574: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-09-10T23:00:01.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:01 smithi153 ceph-mon[34951]: pgmap v574: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-09-10T23:00:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:01 smithi176 ceph-mon[34982]: pgmap v574: 97 pgs: 97 active+clean; 44 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-09-10T23:00:02.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1285843 16.39 MB/sec execute 297 sec latency 13.293 ms 2024-09-10T23:00:03.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1289099 16.40 MB/sec execute 298 sec latency 20.770 ms 2024-09-10T23:00:03.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:03 smithi120 ceph-mon[31247]: pgmap v575: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T23:00:03.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:03 smithi153 ceph-mon[34951]: pgmap v575: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T23:00:03.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:03 smithi176 ceph-mon[34982]: pgmap v575: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-09-10T23:00:04.630 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1292012 16.39 MB/sec execute 299 sec latency 19.688 ms 2024-09-10T23:00:05.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1295097 16.38 MB/sec execute 300 sec latency 14.854 ms 2024-09-10T23:00:05.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:05 smithi120 ceph-mon[31247]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-10T23:00:05.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:05 smithi153 ceph-mon[34951]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-10T23:00:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:05 smithi176 ceph-mon[34982]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-10T23:00:06.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1297406 16.37 MB/sec execute 301 sec latency 63.137 ms 2024-09-10T23:00:07.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1300593 16.38 MB/sec execute 302 sec latency 22.101 ms 2024-09-10T23:00:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:07 smithi120 ceph-mon[31247]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 510 op/s 2024-09-10T23:00:07.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:07 smithi153 ceph-mon[34951]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 510 op/s 2024-09-10T23:00:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:07 smithi176 ceph-mon[34982]: pgmap v577: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 510 op/s 2024-09-10T23:00:08.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1303956 16.39 MB/sec execute 303 sec latency 64.243 ms 2024-09-10T23:00:09.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1307324 16.39 MB/sec execute 304 sec latency 16.350 ms 2024-09-10T23:00:09.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:09 smithi120 ceph-mon[31247]: pgmap v578: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-10T23:00:09.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:09 smithi153 ceph-mon[34951]: pgmap v578: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-10T23:00:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:09 smithi176 ceph-mon[34982]: pgmap v578: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-10T23:00:10.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1310715 16.40 MB/sec execute 305 sec latency 34.748 ms 2024-09-10T23:00:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 -- 172.21.15.120:0/2148033126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 msgr2=0x7f11e0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/2148033126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0101c60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009930 tx=0x7f11d002a420 comp rx=0 tx=0).stop 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 -- 172.21.15.120:0/2148033126 shutdown_connections 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/2148033126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/2148033126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 0x7f11e0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/2148033126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0101c60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 -- 172.21.15.120:0/2148033126 >> 172.21.15.120:0/2148033126 conn(0x7f11e00fd140 msgr2=0x7f11e00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 -- 172.21.15.120:0/2148033126 shutdown_connections 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.214+0000 7f11e5cf6640 1 -- 172.21.15.120:0/2148033126 wait complete. 2024-09-10T23:00:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 Processor -- start 2024-09-10T23:00:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 -- start start 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 0x7f11e01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e660 con 0x7f11e0101860 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e4e0 con 0x7f11e0103370 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11e5cf6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e7e0 con 0x7f11e01029b0 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50756/0 (socket says 172.21.15.120:50756) 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11df7fe640 1 -- 172.21.15.120:0/1981600359 learned_addr learned my addr 172.21.15.120:0/1981600359 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11deffd640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 0x7f11e01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.215+0000 7f11dffff640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 -- 172.21.15.120:0/1981600359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 msgr2=0x7f11e01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 0x7f11e01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 -- 172.21.15.120:0/1981600359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 msgr2=0x7f11e019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 -- 172.21.15.120:0/1981600359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e019a8e0 con 0x7f11e0101860 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11dffff640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11df7fe640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031500 tx=0x7f11d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11dcff9640 1 -- 172.21.15.120:0/1981600359 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d003f070 con 0x7f11e0101860 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e019ab70 con 0x7f11e0101860 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11dcff9640 1 -- 172.21.15.120:0/1981600359 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0031c50 con 0x7f11e0101860 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11dcff9640 1 -- 172.21.15.120:0/1981600359 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0035030 con 0x7f11e0101860 2024-09-10T23:00:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.216+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a12a0 con 0x7f11e0101860 2024-09-10T23:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 Processor -- start 2024-09-10T23:00:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11dcff9640 1 -- 172.21.15.120:0/1981600359 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f11d003b460 con 0x7f11e0101860 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 -- start start 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e1456436f0 con 0x55e1456d1860 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e145632b20 con 0x55e1456d6bd0 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e145636930 con 0x55e1456d4190 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52726/0 (socket says 172.21.15.120:52726) 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11df7fe640 1 -- 172.21.15.120:0/3844094242 learned_addr learned my addr 172.21.15.120:0/3844094242 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11dffff640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11deffd640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.219+0000 7f11dffff640 1 -- 172.21.15.120:0/3844094242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 msgr2=0x55e1456d6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dcff9640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 0x7f11c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 -- 172.21.15.120:0/3844094242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 msgr2=0x55e1456d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 -- 172.21.15.120:0/3844094242 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1455f8530 con 0x55e1456d1860 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11df7fe640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dcff9640 1 -- 172.21.15.120:0/1981600359 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11d0106600 con 0x7f11e0101860 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456d3c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005fd0 tx=0x7f11d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a394b2ec635b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11dffff640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 0x7f11c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11c5ffb640 1 -- 172.21.15.120:0/3844094242 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80116a0 con 0x55e1456d1860 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11c5ffb640 1 -- 172.21.15.120:0/3844094242 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801b490 con 0x55e1456d1860 2024-09-10T23:00:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.220+0000 7f11c5ffb640 1 -- 172.21.15.120:0/3844094242 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80106f0 con 0x55e1456d1860 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- 172.21.15.120:0/3844094242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 msgr2=0x55e1456d3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456d3c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005fd0 tx=0x7f11d800f730 comp rx=0 tx=0).stop 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11dffff640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 0x7f11c80b77a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11d8019000 tx=0x7f11d8010e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- 172.21.15.120:0/3844094242 shutdown_connections 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d4190 0x55e1456d6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/3844094242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456d3c50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- 172.21.15.120:0/3844094242 >> 172.21.15.120:0/3844094242 conn(0x55e1456cc9b0 msgr2=0x55e1456cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- 172.21.15.120:0/3844094242 shutdown_connections 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- 172.21.15.120:0/3844094242 wait complete. 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 -- start start 2024-09-10T23:00:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.221+0000 7f11e8ac9740 1 Processor -- start 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e14556aab0 con 0x55e1456d1860 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e1455d8730 con 0x55e1456d4190 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e1456d9bb0 con 0x55e1456d6bd0 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11e8ac9740 10 client.0 _ll_register_callbacks cb 0x55e14562c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50778/0 (socket says 172.21.15.120:50778) 2024-09-10T23:00:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 -- 172.21.15.120:0/2019498350 learned_addr learned my addr 172.21.15.120:0/2019498350 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11dffff640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11deffd640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 -- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 msgr2=0x55e1456dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 -- 172.21.15.120:0/2019498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 msgr2=0x55e1456db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 -- 172.21.15.120:0/2019498350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e145394790 con 0x55e1456d1860 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11dffff640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.222+0000 7f11df7fe640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11d00ccd80 tx=0x7f11d0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef345e3ef16414e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0049030 con 0x55e1456d1860 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11d00d5ef0 con 0x55e1456d1860 2024-09-10T23:00:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d801b490 con 0x55e1456d1860 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d00c8c90 con 0x55e1456d1860 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 10 client.16905 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e14540d620 con 0x55e1456d1860 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801b490 con 0x55e1456d1860 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f11d0108b80 con 0x55e1456d1860 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 10 client.16905 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 20 client.16905 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e1456e66b0 0x55e1456e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 4 client.16905 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e145645640 con 0x55e1456e66b0 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.223+0000 7f11deffd640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e1456e66b0 0x55e1456e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.224+0000 7f11beffd640 10 client.16905 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.224+0000 7f11deffd640 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e1456e66b0 0x55e1456e8b00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0ed19e858f8375 server_cookie=e4643d36d05051a9 in_seq=0 out_seq=0 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11beffd640 1 -- 172.21.15.120:0/2019498350 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e145645640 con 0x55e1456e66b0 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11beffd640 10 client.16905 handle_command_reply: tid=0 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 client.16905 shutdown 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 msgr2=0x55e1456dae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11d00ccd80 tx=0x7f11d0005e10 comp rx=0 tx=0).stop 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 shutdown_connections 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e1456e66b0 0x55e1456e8b00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11beffd640 0 client.16905 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/2019498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456dae30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 >> 172.21.15.120:0/2019498350 conn(0x55e1456cc9b0 msgr2=0x55e1456cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 shutdown_connections 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 1 -- 172.21.15.120:0/2019498350 wait complete. 2024-09-10T23:00:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.225+0000 7f11e8ac9740 20 client.16905 trim_cache size 0 max 16384 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.240+0000 7f11e8ac9740 1 Processor -- start 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.240+0000 7f11e8ac9740 1 -- start start 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.240+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d1860 0x55e1456d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01a7df0 con 0x55e1456d4190 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01aa670 con 0x55e1456d6bd0 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01a8460 con 0x55e1456d1860 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50792/0 (socket says 172.21.15.120:50792) 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 -- 172.21.15.120:0/241440386 learned_addr learned my addr 172.21.15.120:0/241440386 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11df7fe640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d1860 0x55e1456d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11dffff640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 -- 172.21.15.120:0/241440386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d1860 msgr2=0x55e1456d6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d1860 0x55e1456d6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 -- 172.21.15.120:0/241440386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 msgr2=0x55e1456d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 -- 172.21.15.120:0/241440386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1456d3fc0 con 0x55e1456d4190 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11dffff640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11deffd640 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3384935a3bba9fce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.241+0000 7f11c57fa640 1 -- 172.21.15.120:0/241440386 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4010430 con 0x55e1456d4190 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11c57fa640 1 -- 172.21.15.120:0/241440386 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d4010a80 con 0x55e1456d4190 2024-09-10T23:00:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11c57fa640 1 -- 172.21.15.120:0/241440386 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d401ea00 con 0x55e1456d4190 2024-09-10T23:00:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 -- 172.21.15.120:0/241440386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 msgr2=0x55e1456d74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4005f90 comp rx=0 tx=0).stop 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 -- 172.21.15.120:0/241440386 shutdown_connections 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d6bd0 0x55e1456d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d4190 0x55e1456d74f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/241440386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d1860 0x55e1456d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 -- 172.21.15.120:0/241440386 >> 172.21.15.120:0/241440386 conn(0x55e1456cc9b0 msgr2=0x55e1456da9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 -- 172.21.15.120:0/241440386 shutdown_connections 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.242+0000 7f11e8ac9740 1 -- 172.21.15.120:0/241440386 wait complete. 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 -- start start 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 Processor -- start 2024-09-10T23:00:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01a8b30 con 0x55e1456d1860 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01a9fa0 con 0x55e1456d4190 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01a9200 con 0x55e1456d6bd0 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11e8ac9740 10 client.0 _ll_register_callbacks cb 0x7f11d0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50798/0 (socket says 172.21.15.120:50798) 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 -- 172.21.15.120:0/211247898 learned_addr learned my addr 172.21.15.120:0/211247898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11dffff640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11deffd640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 -- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 msgr2=0x55e1456e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 -- 172.21.15.120:0/211247898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 msgr2=0x55e1456e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456e2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.243+0000 7f11df7fe640 1 -- 172.21.15.120:0/211247898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1456e1c00 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11df7fe640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f11d01044a0 tx=0x7f11d003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc4db264d0fe20b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0049070 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11d00d6120 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d4010a80 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d00cc560 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 10 client.16911 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1456e0a30 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d4010a80 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f11d0106110 con 0x55e1456d1860 2024-09-10T23:00:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 10 client.16911 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 20 client.16911 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e145753920 0x55e145755d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 4 client.16911 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1456410f0 con 0x55e145753920 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.244+0000 7f11deffd640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e145753920 0x55e145755d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.245+0000 7f11beffd640 10 client.16911 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.245+0000 7f11deffd640 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e145753920 0x55e145755d10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9f2dd59a7ae298 server_cookie=24932182dcae2009 in_seq=0 out_seq=0 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11beffd640 1 -- 172.21.15.120:0/211247898 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1456410f0 con 0x55e145753920 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11beffd640 10 client.16911 handle_command_reply: tid=0 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 client.16911 shutdown 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 msgr2=0x55e1456e2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f11d01044a0 tx=0x7f11d003b800 comp rx=0 tx=0).stop 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 shutdown_connections 2024-09-10T23:00:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e145753920 0x55e145755d10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e1456d6bd0 0x55e1456e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e1456d4190 0x55e1456e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 --2- 172.21.15.120:0/211247898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1456d1860 0x55e1456e2540 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 >> 172.21.15.120:0/211247898 conn(0x55e1456cc9b0 msgr2=0x55e1456d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11beffd640 0 client.16911 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 shutdown_connections 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.446+0000 7f11e8ac9740 1 -- 172.21.15.120:0/211247898 wait complete. 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e8ac9740 20 client.16911 trim_cache size 0 max 16384 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:00:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 msgr2=0x7f11c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 0x7f11c80b77a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11d8019000 tx=0x7f11d8010e60 comp rx=0 tx=0).stop 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 msgr2=0x7f11e0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f11d0031500 tx=0x7f11d0003c10 comp rx=0 tx=0).stop 2024-09-10T23:00:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 shutdown_connections 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11c80b52e0 0x7f11c80b77a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0103370 0x7f11e019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11e01029b0 0x7f11e01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 --2- 172.21.15.120:0/1981600359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11e0101860 0x7f11e0198da0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.447+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 >> 172.21.15.120:0/1981600359 conn(0x7f11e00fd140 msgr2=0x7f11e0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.448+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 shutdown_connections 2024-09-10T23:00:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:11.448+0000 7f11e5cf6640 1 -- 172.21.15.120:0/1981600359 wait complete. 2024-09-10T23:00:11.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1312791 16.37 MB/sec execute 306 sec latency 119.790 ms 2024-09-10T23:00:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:11 smithi120 ceph-mon[31247]: pgmap v579: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-09-10T23:00:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:11 smithi153 ceph-mon[34951]: pgmap v579: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-09-10T23:00:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:11 smithi176 ceph-mon[34982]: pgmap v579: 97 pgs: 97 active+clean; 50 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 425 op/s 2024-09-10T23:00:12.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1315107 16.37 MB/sec execute 307 sec latency 25.327 ms 2024-09-10T23:00:13.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1318223 16.37 MB/sec execute 308 sec latency 15.291 ms 2024-09-10T23:00:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:13 smithi120 ceph-mon[31247]: pgmap v580: 97 pgs: 97 active+clean; 57 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2024-09-10T23:00:13.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:13 smithi153 ceph-mon[34951]: pgmap v580: 97 pgs: 97 active+clean; 57 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2024-09-10T23:00:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:13 smithi176 ceph-mon[34982]: pgmap v580: 97 pgs: 97 active+clean; 57 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2024-09-10T23:00:14.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1321312 16.38 MB/sec execute 309 sec latency 160.403 ms 2024-09-10T23:00:15.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1324616 16.38 MB/sec execute 310 sec latency 12.039 ms 2024-09-10T23:00:15.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:15 smithi120 ceph-mon[31247]: pgmap v581: 97 pgs: 97 active+clean; 57 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 300 op/s 2024-09-10T23:00:15.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:15 smithi153 ceph-mon[34951]: pgmap v581: 97 pgs: 97 active+clean; 57 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 300 op/s 2024-09-10T23:00:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:15 smithi176 ceph-mon[34982]: pgmap v581: 97 pgs: 97 active+clean; 57 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 300 op/s 2024-09-10T23:00:16.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1327026 16.36 MB/sec execute 311 sec latency 84.660 ms 2024-09-10T23:00:17.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1329363 16.35 MB/sec execute 312 sec latency 71.699 ms 2024-09-10T23:00:17.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:17 smithi120 ceph-mon[31247]: pgmap v582: 97 pgs: 97 active+clean; 49 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 440 op/s 2024-09-10T23:00:17.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:17 smithi153 ceph-mon[34951]: pgmap v582: 97 pgs: 97 active+clean; 49 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 440 op/s 2024-09-10T23:00:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:17 smithi176 ceph-mon[34982]: pgmap v582: 97 pgs: 97 active+clean; 49 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 440 op/s 2024-09-10T23:00:18.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1332487 16.35 MB/sec execute 313 sec latency 18.111 ms 2024-09-10T23:00:18.722 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:00:18.992 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:19.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- 172.21.15.120:0/4054348450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2010d100 msgr2=0x7f2d2010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4054348450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2010d100 0x7f2d2010f550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009ca0 tx=0x7f2d1802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- 172.21.15.120:0/4054348450 shutdown_connections 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4054348450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2010d100 0x7f2d2010f550 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4054348450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d20104550 0x7f2d2010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4054348450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- 172.21.15.120:0/4054348450 >> 172.21.15.120:0/4054348450 conn(0x7f2d200fd700 msgr2=0x7f2d200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- 172.21.15.120:0/4054348450 shutdown_connections 2024-09-10T23:00:19.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- 172.21.15.120:0/4054348450 wait complete. 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 Processor -- start 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.502+0000 7f2d2688a640 1 -- start start 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2010d100 0x7f2d2019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112db0 con 0x7f2d20104550 2024-09-10T23:00:19.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112c30 con 0x7f2d20103b50 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d2688a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112f30 con 0x7f2d2010d100 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37492/0 (socket says 172.21.15.120:37492) 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33950/0 (socket says 172.21.15.120:33950) 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 -- 172.21.15.120:0/4189037073 learned_addr learned my addr 172.21.15.120:0/4189037073 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d24e27640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2010d100 0x7f2d2019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 -- 172.21.15.120:0/4189037073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2010d100 msgr2=0x7f2d2019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2010d100 0x7f2d2019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 -- 172.21.15.120:0/4189037073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 msgr2=0x7f2d20199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1f7fe640 1 -- 172.21.15.120:0/4189037073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2019f2f0 con 0x7f2d20104550 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.503+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d1d7fa640 1 -- 172.21.15.120:0/4189037073 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400cd80 con 0x7f2d20104550 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d1d7fa640 1 -- 172.21.15.120:0/4189037073 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d140108e0 con 0x7f2d20104550 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2019f5e0 con 0x7f2d20104550 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d1d7fa640 1 -- 172.21.15.120:0/4189037073 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14018980 con 0x7f2d20104550 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.504+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a5e70 con 0x7f2d20104550 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 Processor -- start 2024-09-10T23:00:19.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 -- start start 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d962540d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d9624a8e50 con 0x55d96253e950 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d962498280 con 0x55d9625320a0 2024-09-10T23:00:19.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d96249c090 con 0x55d9625329c0 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d24e27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.507+0000 7f2d24e27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33956/0 (socket says 172.21.15.120:33956) 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d24e27640 1 -- 172.21.15.120:0/579102193 learned_addr learned my addr 172.21.15.120:0/579102193 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 -- 172.21.15.120:0/579102193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 msgr2=0x55d962532480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 -- 172.21.15.120:0/579102193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 msgr2=0x55d962540d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d962540d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 -- 172.21.15.120:0/579102193 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9621f6650 con 0x55d9625329c0 2024-09-10T23:00:19.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d24e27640 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253e410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d100103d0 tx=0x7f2d10010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda5d0d48bcee0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d1d7fa640 1 -- 172.21.15.120:0/4189037073 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2d14010430 con 0x7f2d20104550 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d067fc640 1 -- 172.21.15.120:0/579102193 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10004220 con 0x55d9625329c0 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d067fc640 1 -- 172.21.15.120:0/579102193 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d100043c0 con 0x55d9625329c0 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.508+0000 7f2d067fc640 1 -- 172.21.15.120:0/579102193 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d100119a0 con 0x55d9625329c0 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d1d7fa640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 -- 172.21.15.120:0/579102193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 msgr2=0x55d96253e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253e410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d100103d0 tx=0x7f2d10010890 comp rx=0 tx=0).stop 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d1d7fa640 1 -- 172.21.15.120:0/4189037073 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2d140e5e00 con 0x7f2d20104550 2024-09-10T23:00:19.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d24e27640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 -- 172.21.15.120:0/579102193 shutdown_connections 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d962540d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253e410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/579102193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962532480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 -- 172.21.15.120:0/579102193 >> 172.21.15.120:0/579102193 conn(0x55d96252fed0 msgr2=0x55d962531070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 -- 172.21.15.120:0/579102193 shutdown_connections 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.509+0000 7f2d28c5f740 1 -- 172.21.15.120:0/579102193 wait complete. 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d24e27640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 0x7f2d080b7820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802aeb0 tx=0x7f2d18005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 -- start start 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 Processor -- start 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625329c0 0x55d96253b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d9622f9700 con 0x55d9625329c0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d962433110 con 0x55d96253e950 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d9625415d0 con 0x55d9625320a0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.510+0000 7f2d28c5f740 10 client.0 _ll_register_callbacks cb 0x55d96244c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55624/0 (socket says 172.21.15.120:55624) 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2893924713 learned_addr learned my addr 172.21.15.120:0/2893924713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d24e27640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625329c0 0x55d96253b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2893924713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 msgr2=0x55d96253bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2893924713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625329c0 msgr2=0x55d96253b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625329c0 0x55d96253b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2893924713 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d962460c40 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d100041a0 tx=0x7f2d100041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e89cdf5383b3efa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2cff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1002a060 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2d10005250 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d100043c0 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.511+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d10005450 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 10 client.26404 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d96254cf80 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d100043c0 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2d10004930 con 0x55d9625320a0 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 10 client.26404 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:19.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 20 client.26404 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d96254e380 0x55d962550770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 4 client.26404 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9624aada0 con 0x55d96254e380 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.512+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d96254e380 0x55d962550770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.513+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d96254e380 0x55d962550770 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c89ff1dd57035f29 server_cookie=3bdf1756776ec337 in_seq=0 out_seq=0 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.513+0000 7f2cff7fe640 10 client.26404 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.513+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2893924713 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9624aada0 con 0x55d96254e380 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.513+0000 7f2cff7fe640 10 client.26404 handle_command_reply: tid=0 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.513+0000 7f2d28c5f740 1 client.26404 shutdown 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 msgr2=0x55d96253e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d100041a0 tx=0x7f2d100041d0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 shutdown_connections 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d96254e380 0x55d962550770 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2cff7fe640 0 client.26404 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625329c0 0x55d96253b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2893924713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625320a0 0x55d96253e290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 >> 172.21.15.120:0/2893924713 conn(0x55d96252fed0 msgr2=0x55d962535a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 shutdown_connections 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2893924713 wait complete. 2024-09-10T23:00:19.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.514+0000 7f2d28c5f740 20 client.26404 trim_cache size 0 max 16384 2024-09-10T23:00:19.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 Processor -- start 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 -- start start 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ac360 con 0x55d9625320a0 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201aebe0 con 0x55d96253e950 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ac9d0 con 0x55d9625329c0 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37516/0 (socket says 172.21.15.120:37516) 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1ffff640 1 -- 172.21.15.120:0/4068287548 learned_addr learned my addr 172.21.15.120:0/4068287548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d24e27640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1ffff640 1 -- 172.21.15.120:0/4068287548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 msgr2=0x55d96253d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.541+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d1ffff640 1 -- 172.21.15.120:0/4068287548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 msgr2=0x55d96253fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d1ffff640 1 -- 172.21.15.120:0/4068287548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d962530950 con 0x55d9625320a0 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b3d0 tx=0x7f2d10010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0998b617ecef50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d05ffb640 1 -- 172.21.15.120:0/4068287548 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10035030 con 0x55d9625320a0 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d05ffb640 1 -- 172.21.15.120:0/4068287548 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d10005250 con 0x55d9625320a0 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.542+0000 7f2d05ffb640 1 -- 172.21.15.120:0/4068287548 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1002a040 con 0x55d9625320a0 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 -- 172.21.15.120:0/4068287548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 msgr2=0x55d96252f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b3d0 tx=0x7f2d10010890 comp rx=0 tx=0).stop 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 -- 172.21.15.120:0/4068287548 shutdown_connections 2024-09-10T23:00:19.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d96253e950 0x55d96253fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d96253d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/4068287548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d9625320a0 0x55d96252f710 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 -- 172.21.15.120:0/4068287548 >> 172.21.15.120:0/4068287548 conn(0x55d96252fed0 msgr2=0x55d96253cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 -- 172.21.15.120:0/4068287548 shutdown_connections 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.543+0000 7f2d28c5f740 1 -- 172.21.15.120:0/4068287548 wait complete. 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 -- start start 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 Processor -- start 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d9625505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d9625b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ad0a0 con 0x55d96253e950 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ae510 con 0x55d9625320a0 2024-09-10T23:00:19.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ad770 con 0x55d9625329c0 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.544+0000 7f2d28c5f740 10 client.0 _ll_register_callbacks cb 0x7f2d18009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d24e27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d9625b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33986/0 (socket says 172.21.15.120:33986) 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2331841199 learned_addr learned my addr 172.21.15.120:0/2331841199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d9625505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 msgr2=0x55d9625505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d9625505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2331841199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 msgr2=0x55d9625b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:19.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d9625b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 -- 172.21.15.120:0/2331841199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9625ba620 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d1ffff640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000bff0 tx=0x7f2d10005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a768fb991d23763 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2cff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1003f060 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2d100043c0 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d10005250 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d1002a030 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d28c5f740 10 client.26312 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.545+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9625bb930 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d10005250 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2d10011890 con 0x55d9625320a0 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 10 client.26312 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:19.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 20 client.26312 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d9625bbf80 0x55d9625be370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 4 client.26312 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9624a6850 con 0x55d9625bbf80 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d9625bbf80 0x55d9625be370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2cff7fe640 10 client.26312 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.546+0000 7f2d1f7fe640 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d9625bbf80 0x55d9625be370 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef2736fa32926c7 server_cookie=d0934e73abbd2e52 in_seq=0 out_seq=0 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2cff7fe640 1 -- 172.21.15.120:0/2331841199 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55d9624a6850 con 0x55d9625bbf80 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2cff7fe640 10 client.26312 handle_command_reply: tid=0 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 client.26312 shutdown 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 msgr2=0x55d962550030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000bff0 tx=0x7f2d10005e50 comp rx=0 tx=0).stop 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 shutdown_connections 2024-09-10T23:00:19.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d9625bbf80 0x55d9625be370 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d96253e950 0x55d9625b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9625329c0 0x55d9625505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2d28c5f740 1 --2- 172.21.15.120:0/2331841199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d9625320a0 0x55d962550030 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.547+0000 7f2cff7fe640 0 client.26312 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 >> 172.21.15.120:0/2331841199 conn(0x55d96252fed0 msgr2=0x55d96253a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 shutdown_connections 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d28c5f740 1 -- 172.21.15.120:0/2331841199 wait complete. 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d28c5f740 20 client.26312 trim_cache size 0 max 16384 2024-09-10T23:00:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:00:19.561 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.562 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:00:19.563 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.564 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.565 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:00:19.565 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:00:19.565 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.565 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:00:19.569 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:00:19.570 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:00:19.571 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:00:19.572 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.573 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.574 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:19.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:19.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:19.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:19.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:19.576 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:00:19.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 msgr2=0x7f2d080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 0x7f2d080b7820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802aeb0 tx=0x7f2d18005d80 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 msgr2=0x7f2d20199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 shutdown_connections 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2010d100 0x7f2d2019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d20104550 0x7f2d20199780 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 --2- 172.21.15.120:0/4189037073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d20103b50 0x7f2d20199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.548+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 >> 172.21.15.120:0/4189037073 conn(0x7f2d200fd700 msgr2=0x7f2d200fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:19.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.549+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 shutdown_connections 2024-09-10T23:00:19.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:19.549+0000 7f2d2688a640 1 -- 172.21.15.120:0/4189037073 wait complete. 2024-09-10T23:00:19.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1335774 16.36 MB/sec execute 314 sec latency 21.423 ms 2024-09-10T23:00:19.682 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:19 smithi120 ceph-mon[31247]: pgmap v583: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T23:00:19.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:19 smithi153 ceph-mon[34951]: pgmap v583: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T23:00:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:19 smithi176 ceph-mon[34982]: pgmap v583: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T23:00:20.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:00:20.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:00:20.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1339156 16.37 MB/sec execute 315 sec latency 15.286 ms 2024-09-10T23:00:20.867 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:21.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 -- 172.21.15.120:0/1144555336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 msgr2=0x7fdc6c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 --2- 172.21.15.120:0/1144555336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c10cab0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009900 tx=0x7fdc6002a410 comp rx=0 tx=0).stop 2024-09-10T23:00:21.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 -- 172.21.15.120:0/1144555336 shutdown_connections 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 --2- 172.21.15.120:0/1144555336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 0x7fdc6c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 --2- 172.21.15.120:0/1144555336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c10cab0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 --2- 172.21.15.120:0/1144555336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 -- 172.21.15.120:0/1144555336 >> 172.21.15.120:0/1144555336 conn(0x7fdc6c0fd700 msgr2=0x7fdc6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 -- 172.21.15.120:0/1144555336 shutdown_connections 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 -- 172.21.15.120:0/1144555336 wait complete. 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.407+0000 7fdc7134b640 1 Processor -- start 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 -- start start 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 0x7fdc6c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c112d70 con 0x7fdc6c1045c0 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c112bf0 con 0x7fdc6c103ca0 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc7134b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c112ef0 con 0x7fdc6c10d100 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37546/0 (socket says 172.21.15.120:37546) 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34002/0 (socket says 172.21.15.120:34002) 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6a7fc640 1 -- 172.21.15.120:0/3495838409 learned_addr learned my addr 172.21.15.120:0/3495838409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6b7fe640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 0x7fdc6c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 -- 172.21.15.120:0/3495838409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 msgr2=0x7fdc6c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 0x7fdc6c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 -- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 msgr2=0x7fdc6c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.408+0000 7fdc6affd640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc6affd640 1 -- 172.21.15.120:0/3495838409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19f350 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc6a7fc640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc6affd640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c009c30 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c009dd0 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19f640 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003e20 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.409+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c1a5d00 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.410+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c104080 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.411+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdc5c004000 con 0x7fdc6c103ca0 2024-09-10T23:00:21.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.412+0000 7fdc53fff640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 0x7fdc540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.412+0000 7fdc6a7fc640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 0x7fdc540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.412+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc5c0e5760 con 0x7fdc6c103ca0 2024-09-10T23:00:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.413+0000 7fdc6a7fc640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 0x7fdc540b77a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc600096c0 tx=0x7fdc60032f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.416+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdc5c017020 con 0x7fdc6c103ca0 2024-09-10T23:00:21.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1341994 16.36 MB/sec execute 316 sec latency 46.134 ms 2024-09-10T23:00:21.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.650+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc6c02e5c0 con 0x7fdc6c103ca0 2024-09-10T23:00:21.653 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:21 smithi120 ceph-mon[31247]: pgmap v584: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-09-10T23:00:21.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.651+0000 7fdc53fff640 1 -- 172.21.15.120:0/3495838409 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fdc5c0abfa0 con 0x7fdc6c103ca0 2024-09-10T23:00:21.653 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:00:21.654 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:00:21.654 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 msgr2=0x7fdc540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 0x7fdc540b77a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc600096c0 tx=0x7fdc60032f70 comp rx=0 tx=0).stop 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 msgr2=0x7fdc6c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c00dde0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 shutdown_connections 2024-09-10T23:00:21.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdc540b52e0 0x7fdc540b77a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc6c10d100 0x7fdc6c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc6c1045c0 0x7fdc6c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 --2- 172.21.15.120:0/3495838409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc6c103ca0 0x7fdc6c1992a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 >> 172.21.15.120:0/3495838409 conn(0x7fdc6c0fd700 msgr2=0x7fdc6c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 shutdown_connections 2024-09-10T23:00:21.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:21.654+0000 7fdc7134b640 1 -- 172.21.15.120:0/3495838409 wait complete. 2024-09-10T23:00:21.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:21 smithi153 ceph-mon[34951]: pgmap v584: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-09-10T23:00:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:21 smithi176 ceph-mon[34982]: pgmap v584: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 470 op/s 2024-09-10T23:00:22.413 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3495838409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:22.415 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:00:22.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1344685 16.35 MB/sec execute 317 sec latency 30.831 ms 2024-09-10T23:00:22.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3495838409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:22.708 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:22.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3495838409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3042506206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be41087a0 msgr2=0x7f0be4108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3042506206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be41087a0 0x7f0be4108be0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009900 tx=0x7f0bcc02a410 comp rx=0 tx=0).stop 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3042506206 shutdown_connections 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3042506206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be4109120 0x7f0be410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3042506206 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be41087a0 0x7f0be4108be0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.216 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3042506206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 0x7f0be4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3042506206 >> 172.21.15.120:0/3042506206 conn(0x7f0be40fe820 msgr2=0x7f0be4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3042506206 shutdown_connections 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.212+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3042506206 wait complete. 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 Processor -- start 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 -- start start 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 0x7f0be40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be41087a0 0x7f0be40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4112de0 con 0x7f0be4109120 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4112c60 con 0x7f0be4102d90 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be9b5d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4112f60 con 0x7f0be41087a0 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33510/0 (socket says 172.21.15.120:33510) 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be3fff640 1 -- 172.21.15.120:0/3240477480 learned_addr learned my addr 172.21.15.120:0/3240477480 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0be37fe640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 0x7f0be40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.213+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be41087a0 0x7f0be40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 -- 172.21.15.120:0/3240477480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be41087a0 msgr2=0x7f0be40759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be41087a0 0x7f0be40759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 -- 172.21.15.120:0/3240477480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 msgr2=0x7f0be40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 0x7f0be40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 -- 172.21.15.120:0/3240477480 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be41a7690 con 0x7f0be4109120 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be3fff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00eb10 tx=0x7f0bdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be17fa640 1 -- 172.21.15.120:0/3240477480 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc004b20 con 0x7f0be4109120 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be17fa640 1 -- 172.21.15.120:0/3240477480 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc004cc0 con 0x7f0be4109120 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be41a7980 con 0x7f0be4109120 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be17fa640 1 -- 172.21.15.120:0/3240477480 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc010670 con 0x7f0be4109120 2024-09-10T23:00:23.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.214+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be41a7e90 con 0x7f0be4109120 2024-09-10T23:00:23.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.217+0000 7f0bebf32740 1 Processor -- start 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.217+0000 7f0be17fa640 1 -- 172.21.15.120:0/3240477480 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0bdc005020 con 0x7f0be4109120 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.217+0000 7f0bebf32740 1 -- start start 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.217+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.217+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bebf32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559cbb852ea0 con 0x559cbb8dca10 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bebf32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559cbb8422d0 con 0x559cbb8e89a0 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bebf32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559cbb8460e0 con 0x559cbb8dc0f0 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8dc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59864/0 (socket says 172.21.15.120:59864) 2024-09-10T23:00:23.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bdbfff640 1 -- 172.21.15.120:0/4170962693 learned_addr learned my addr 172.21.15.120:0/4170962693 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:23.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 -- 172.21.15.120:0/4170962693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 msgr2=0x559cbb8dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 -- 172.21.15.120:0/4170962693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 msgr2=0x559cbb8ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 -- 172.21.15.120:0/4170962693 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbb5a0650 con 0x559cbb8dca10 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be17fa640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 0x7f0bd00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 0x7f0bd00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be37fe640 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e8460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c20 tx=0x7f0bd4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325022a80ce6b378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0be17fa640 1 -- 172.21.15.120:0/3240477480 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0bdc0e6100 con 0x7f0be4109120 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.218+0000 7f0bd9ffb640 1 -- 172.21.15.120:0/4170962693 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004050 con 0x559cbb8dca10 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bd9ffb640 1 -- 172.21.15.120:0/4170962693 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd40054d0 con 0x559cbb8dca10 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bd9ffb640 1 -- 172.21.15.120:0/4170962693 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004d30 con 0x559cbb8dca10 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 -- 172.21.15.120:0/4170962693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 msgr2=0x559cbb8e8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 0x7f0bd00b77a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005c40 tx=0x7f0bcc005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e8460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c20 tx=0x7f0bd4011810 comp rx=0 tx=0).stop 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 -- 172.21.15.120:0/4170962693 shutdown_connections 2024-09-10T23:00:23.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e8460 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 --2- 172.21.15.120:0/4170962693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 -- 172.21.15.120:0/4170962693 >> 172.21.15.120:0/4170962693 conn(0x559cbb8d9f20 msgr2=0x559cbb8db0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 -- 172.21.15.120:0/4170962693 shutdown_connections 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.219+0000 7f0bebf32740 1 -- 172.21.15.120:0/4170962693 wait complete. 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 -- start start 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 Processor -- start 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dca10 0x559cbb8e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559cbb6a3700 con 0x559cbb8dc0f0 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559cbb7dd160 con 0x559cbb8e89a0 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559cbb8eb670 con 0x559cbb8dca10 2024-09-10T23:00:23.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bebf32740 10 client.0 _ll_register_callbacks cb 0x559cbb7f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33532/0 (socket says 172.21.15.120:33532) 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0be37fe640 1 -- 172.21.15.120:0/994039865 learned_addr learned my addr 172.21.15.120:0/994039865 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dca10 0x559cbb8e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.220+0000 7f0be3fff640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 -- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dca10 msgr2=0x559cbb8e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dca10 0x559cbb8e5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 -- 172.21.15.120:0/994039865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 msgr2=0x559cbb8e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 -- 172.21.15.120:0/994039865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbb80ac90 con 0x559cbb8dc0f0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0be37fe640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260f334247a0416a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40046c0 con 0x559cbb8dc0f0 2024-09-10T23:00:23.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0bd4005140 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd40054d0 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bd4005c50 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bebf32740 10 client.16944 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559cbb8f6f00 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.221+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd40054d0 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0bd4018070 con 0x559cbb8dc0f0 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 10 client.16944 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 20 client.16944 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb8f8300 0x559cbb8fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 4 client.16944 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:23.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cbb854df0 con 0x559cbb8f8300 2024-09-10T23:00:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb8f8300 0x559cbb8fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bc2ffd640 10 client.16944 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.222+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb8f8300 0x559cbb8fa6f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc048e4f4e61697 server_cookie=2e15c1caf4759840 in_seq=0 out_seq=0 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/994039865 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cbb854df0 con 0x559cbb8f8300 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bc2ffd640 10 client.16944 handle_command_reply: tid=0 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 client.16944 shutdown 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 msgr2=0x559cbb8e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4004d40 comp rx=0 tx=0).stop 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 shutdown_connections 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb8f8300 0x559cbb8fa6f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dca10 0x559cbb8e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 --2- 172.21.15.120:0/994039865 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dc0f0 0x559cbb8e81f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 >> 172.21.15.120:0/994039865 conn(0x559cbb8d9f20 msgr2=0x559cbb8dfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bc2ffd640 0 client.16944 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 shutdown_connections 2024-09-10T23:00:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 1 -- 172.21.15.120:0/994039865 wait complete. 2024-09-10T23:00:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.223+0000 7f0bebf32740 20 client.16944 trim_cache size 0 max 16384 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 Processor -- start 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 -- start start 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41ae550 con 0x559cbb8dca10 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41b0dd0 con 0x559cbb8e89a0 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bebf32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41aebc0 con 0x559cbb8dc0f0 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33542/0 (socket says 172.21.15.120:33542) 2024-09-10T23:00:23.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 -- 172.21.15.120:0/2621751997 learned_addr learned my addr 172.21.15.120:0/2621751997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0be37fe640 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0be3fff640 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 -- 172.21.15.120:0/2621751997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 msgr2=0x559cbb8d9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8d9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 -- 172.21.15.120:0/2621751997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 msgr2=0x559cbb8e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.238+0000 7f0bdbfff640 1 -- 172.21.15.120:0/2621751997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbb8da9a0 con 0x559cbb8dca10 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc02a930 tx=0x7f0bcc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b28be1e333dce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bd97fa640 1 -- 172.21.15.120:0/2621751997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc05a070 con 0x559cbb8dca10 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bd97fa640 1 -- 172.21.15.120:0/2621751997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc0035f0 con 0x559cbb8dca10 2024-09-10T23:00:23.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bd97fa640 1 -- 172.21.15.120:0/2621751997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc035040 con 0x559cbb8dca10 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 -- 172.21.15.120:0/2621751997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 msgr2=0x559cbb8e7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc02a930 tx=0x7f0bcc02a410 comp rx=0 tx=0).stop 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 -- 172.21.15.120:0/2621751997 shutdown_connections 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8e89a0 0x559cbb8e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8e7380 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 --2- 172.21.15.120:0/2621751997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8dc0f0 0x559cbb8d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 -- 172.21.15.120:0/2621751997 >> 172.21.15.120:0/2621751997 conn(0x559cbb8d9f20 msgr2=0x559cbb8e6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.239+0000 7f0bebf32740 1 -- 172.21.15.120:0/2621751997 shutdown_connections 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 -- 172.21.15.120:0/2621751997 wait complete. 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 -- start start 2024-09-10T23:00:23.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 Processor -- start 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8dc0f0 0x559cbb8ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 0x559cbb8ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41af290 con 0x559cbb8dca10 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41b0700 con 0x559cbb8dc0f0 2024-09-10T23:00:23.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41af960 con 0x559cbb8e89a0 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bebf32740 10 client.0 _ll_register_callbacks cb 0x7f0bcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33550/0 (socket says 172.21.15.120:33550) 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0bdbfff640 1 -- 172.21.15.120:0/1782802868 learned_addr learned my addr 172.21.15.120:0/1782802868 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.240+0000 7f0be37fe640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8dc0f0 0x559cbb8ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0be3fff640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 0x559cbb8ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 -- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 msgr2=0x559cbb8ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 0x559cbb8ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 -- 172.21.15.120:0/1782802868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8dc0f0 msgr2=0x559cbb8ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8dc0f0 0x559cbb8ec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 -- 172.21.15.120:0/1782802868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbb9642b0 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0be3fff640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 0x559cbb8ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bdbfff640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0096c0 tx=0x7f0bcc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cbf1322963fbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc063060 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0bcc004280 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc0035f0 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc035050 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bebf32740 10 client.16950 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559cbb9655b0 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc0035f0 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.241+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0bcc03d810 con 0x559cbb8dca10 2024-09-10T23:00:23.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 10 client.16950 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 20 client.16950 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb965c60 0x559cbb968080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 4 client.16950 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cbb8508a0 con 0x559cbb965c60 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0be37fe640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb965c60 0x559cbb968080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bc2ffd640 10 client.16950 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0be37fe640 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb965c60 0x559cbb968080 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d73ce28a5fc797 server_cookie=5a4cc898e4d9aa73 in_seq=0 out_seq=0 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bc2ffd640 1 -- 172.21.15.120:0/1782802868 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cbb8508a0 con 0x559cbb965c60 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bc2ffd640 10 client.16950 handle_command_reply: tid=0 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 1 client.16950 shutdown 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.242+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 msgr2=0x559cbb8ecf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0096c0 tx=0x7f0bcc009520 comp rx=0 tx=0).stop 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 shutdown_connections 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559cbb965c60 0x559cbb968080 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559cbb8e89a0 0x559cbb8ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559cbb8dca10 0x559cbb8ecf00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 --2- 172.21.15.120:0/1782802868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559cbb8dc0f0 0x559cbb8ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 >> 172.21.15.120:0/1782802868 conn(0x559cbb8d9f20 msgr2=0x559cbb8e7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bc2ffd640 0 client.16950 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 shutdown_connections 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 1 -- 172.21.15.120:0/1782802868 wait complete. 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0bebf32740 20 client.16950 trim_cache size 0 max 16384 2024-09-10T23:00:23.251 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:00:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 msgr2=0x7f0bd00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 0x7f0bd00b77a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005c40 tx=0x7f0bcc005780 comp rx=0 tx=0).stop 2024-09-10T23:00:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 msgr2=0x7f0be4075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00eb10 tx=0x7f0bdc00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 shutdown_connections 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0bd00b52e0 0x7f0bd00b77a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0be4109120 0x7f0be4075f20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be41087a0 0x7f0be40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 --2- 172.21.15.120:0/3240477480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0be4102d90 0x7f0be40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.243+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 >> 172.21.15.120:0/3240477480 conn(0x7f0be40fe820 msgr2=0x7f0be4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.244+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 shutdown_connections 2024-09-10T23:00:23.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:23.244+0000 7f0be9b5d640 1 -- 172.21.15.120:0/3240477480 wait complete. 2024-09-10T23:00:23.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1347295 16.35 MB/sec execute 318 sec latency 20.138 ms 2024-09-10T23:00:23.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:23 smithi120 ceph-mon[31247]: pgmap v585: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-09-10T23:00:23.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:23 smithi153 ceph-mon[34951]: pgmap v585: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-09-10T23:00:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:23 smithi176 ceph-mon[34982]: pgmap v585: 97 pgs: 97 active+clean; 52 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 -- 172.21.15.120:0/2976789133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 msgr2=0x7f0e4c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 --2- 172.21.15.120:0/2976789133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c101c60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009930 tx=0x7f0e3c02a420 comp rx=0 tx=0).stop 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 -- 172.21.15.120:0/2976789133 shutdown_connections 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 --2- 172.21.15.120:0/2976789133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 0x7f0e4c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 --2- 172.21.15.120:0/2976789133 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 0x7f0e4c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 --2- 172.21.15.120:0/2976789133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c101c60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 -- 172.21.15.120:0/2976789133 >> 172.21.15.120:0/2976789133 conn(0x7f0e4c0fd140 msgr2=0x7f0e4c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 -- 172.21.15.120:0/2976789133 shutdown_connections 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.537+0000 7f0e52616640 1 -- 172.21.15.120:0/2976789133 wait complete. 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 Processor -- start 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 -- start start 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 0x7f0e4c10a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 0x7f0e4c10ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10e600 con 0x7f0e4c101860 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10e480 con 0x7f0e4c103370 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e52616640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c10e780 con 0x7f0e4c1029b0 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33578/0 (socket says 172.21.15.120:33578) 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 -- 172.21.15.120:0/3512166715 learned_addr learned my addr 172.21.15.120:0/3512166715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e43fff640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 0x7f0e4c10a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 0x7f0e4c10ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 -- 172.21.15.120:0/3512166715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 msgr2=0x7f0e4c10a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 0x7f0e4c10a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 -- 172.21.15.120:0/3512166715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 msgr2=0x7f0e4c10ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 0x7f0e4c10ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 -- 172.21.15.120:0/3512166715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c10b6f0 con 0x7f0e4c101860 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009a50 tx=0x7f0e3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.538+0000 7f0e49ffb640 1 -- 172.21.15.120:0/3512166715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c03f030 con 0x7f0e4c101860 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.539+0000 7f0e49ffb640 1 -- 172.21.15.120:0/3512166715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c031910 con 0x7f0e4c101860 2024-09-10T23:00:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.539+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c1a2e70 con 0x7f0e4c101860 2024-09-10T23:00:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.539+0000 7f0e49ffb640 1 -- 172.21.15.120:0/3512166715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c035070 con 0x7f0e4c101860 2024-09-10T23:00:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.539+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1a3280 con 0x7f0e4c101860 2024-09-10T23:00:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.541+0000 7f0e553e3740 1 Processor -- start 2024-09-10T23:00:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.541+0000 7f0e553e3740 1 -- start start 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 0x55d79d561c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 0x55d79d564580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d567090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d79d4d16f0 con 0x55d79d562190 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d79d4c0b20 con 0x55d79d564bd0 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e553e3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d79d4c4930 con 0x55d79d55f860 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e49ffb640 1 -- 172.21.15.120:0/3512166715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0e3c03ba60 con 0x7f0e4c101860 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e50bb3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 0x55d79d561c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e50bb3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 0x55d79d561c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59912/0 (socket says 172.21.15.120:59912) 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e50bb3640 1 -- 172.21.15.120:0/4121770638 learned_addr learned my addr 172.21.15.120:0/4121770638 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d567090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 0x55d79d564580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 -- 172.21.15.120:0/4121770638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 msgr2=0x55d79d561c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 0x55d79d561c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 -- 172.21.15.120:0/4121770638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 msgr2=0x55d79d564580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 0x55d79d564580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 -- 172.21.15.120:0/4121770638 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79d486530 con 0x55d79d564bd0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 0x55d79d564580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e43fff640 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d567090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009c00 tx=0x7f0e34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463d831547dcf1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.542+0000 7f0e427fc640 1 -- 172.21.15.120:0/4121770638 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004230 con 0x55d79d564bd0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e427fc640 1 -- 172.21.15.120:0/4121770638 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e340043d0 con 0x55d79d564bd0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e427fc640 1 -- 172.21.15.120:0/4121770638 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004c60 con 0x55d79d564bd0 2024-09-10T23:00:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e49ffb640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 0x7f0e380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 0x7f0e380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e49ffb640 1 -- 172.21.15.120:0/3512166715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0e3c106400 con 0x7f0e4c101860 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- 172.21.15.120:0/4121770638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 msgr2=0x55d79d567090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 0x7f0e380b7820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e44003bc0 tx=0x7f0e44009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d567090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009c00 tx=0x7f0e34011810 comp rx=0 tx=0).stop 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- 172.21.15.120:0/4121770638 shutdown_connections 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d567090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d562190 0x55d79d564580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 --2- 172.21.15.120:0/4121770638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d55f860 0x55d79d561c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- 172.21.15.120:0/4121770638 >> 172.21.15.120:0/4121770638 conn(0x55d79d55a9b0 msgr2=0x55d79d55b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- 172.21.15.120:0/4121770638 shutdown_connections 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- 172.21.15.120:0/4121770638 wait complete. 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.543+0000 7f0e553e3740 1 -- start start 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 Processor -- start 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 0x55d79d5695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d564bd0 0x55d79d56a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d79d3f8ab0 con 0x55d79d55f860 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d79d466730 con 0x55d79d562190 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d79d567bb0 con 0x55d79d564bd0 2024-09-10T23:00:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e553e3740 10 client.0 _ll_register_callbacks cb 0x55d79d4ba6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 0x55d79d5695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33596/0 (socket says 172.21.15.120:33596) 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1397540302 learned_addr learned my addr 172.21.15.120:0/1397540302 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e43fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 0x55d79d5695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35938/0 (socket says 172.21.15.120:35938) 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d564bd0 0x55d79d56a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d564bd0 msgr2=0x55d79d56a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d564bd0 0x55d79d56a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1397540302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 msgr2=0x55d79d5695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 0x55d79d5695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.544+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1397540302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79d222790 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0c91c0 tx=0x7f0e3c0c9300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e1fc520d744025 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c049040 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0e3c0c8e60 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e340043d0 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e3c0cc540 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 10 client.16962 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d79d29b620 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e340043d0 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0e3c0c8050 con 0x55d79d55f860 2024-09-10T23:00:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 10 client.16962 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 20 client.16962 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5747b0 0x55d79d576c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 4 client.16962 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d79d4d3640 con 0x55d79d5747b0 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.545+0000 7f0e43fff640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5747b0 0x55d79d576c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.546+0000 7f0e2b7fe640 10 client.16962 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.546+0000 7f0e43fff640 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5747b0 0x55d79d576c30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3effaf670caf472 server_cookie=6ce6598b5a69fcc6 in_seq=0 out_seq=0 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/1397540302 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d79d4d3640 con 0x55d79d5747b0 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e2b7fe640 10 client.16962 handle_command_reply: tid=0 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 client.16962 shutdown 2024-09-10T23:00:24.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 msgr2=0x55d79d568f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0c91c0 tx=0x7f0e3c0c9300 comp rx=0 tx=0).stop 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 shutdown_connections 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5747b0 0x55d79d576c30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d564bd0 0x55d79d56a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e2b7fe640 0 client.16962 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d562190 0x55d79d5695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1397540302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568f50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 >> 172.21.15.120:0/1397540302 conn(0x55d79d55a9b0 msgr2=0x55d79d559bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 shutdown_connections 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 1 -- 172.21.15.120:0/1397540302 wait complete. 2024-09-10T23:00:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.547+0000 7f0e553e3740 20 client.16962 trim_cache size 0 max 16384 2024-09-10T23:00:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.561+0000 7f0e553e3740 1 Processor -- start 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.561+0000 7f0e553e3740 1 -- start start 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d55ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d55f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1a9fc0 con 0x55d79d55f860 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1ac840 con 0x55d79d564bd0 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e553e3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1aa630 con 0x55d79d562190 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33600/0 (socket says 172.21.15.120:33600) 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1277279326 learned_addr learned my addr 172.21.15.120:0/1277279326 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e43fff640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d55ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d55f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1277279326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 msgr2=0x55d79d55ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d55ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1277279326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 msgr2=0x55d79d55f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d55f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 -- 172.21.15.120:0/1277279326 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79d5648f0 con 0x55d79d55f860 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d55f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0c9e60 tx=0x7f0e3c0c8430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a95344508e1ce75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e41ffb640 1 -- 172.21.15.120:0/1277279326 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c049070 con 0x55d79d55f860 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e41ffb640 1 -- 172.21.15.120:0/1277279326 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0f3020 con 0x55d79d55f860 2024-09-10T23:00:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.562+0000 7f0e41ffb640 1 -- 172.21.15.120:0/1277279326 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0cc360 con 0x55d79d55f860 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- 172.21.15.120:0/1277279326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 msgr2=0x55d79d568af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0c9e60 tx=0x7f0e3c0c8430 comp rx=0 tx=0).stop 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- 172.21.15.120:0/1277279326 shutdown_connections 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d55f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d55ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 --2- 172.21.15.120:0/1277279326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d568af0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- 172.21.15.120:0/1277279326 >> 172.21.15.120:0/1277279326 conn(0x55d79d55a9b0 msgr2=0x55d79d555af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- 172.21.15.120:0/1277279326 shutdown_connections 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- 172.21.15.120:0/1277279326 wait complete. 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 -- start start 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.563+0000 7f0e553e3740 1 Processor -- start 2024-09-10T23:00:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d570c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d56f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1aad00 con 0x55d79d55f860 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1ac170 con 0x55d79d564bd0 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1ab3d0 con 0x55d79d562190 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 10 client.0 _ll_register_callbacks cb 0x7f0e3c0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33602/0 (socket says 172.21.15.120:33602) 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 -- 172.21.15.120:0/2837266243 learned_addr learned my addr 172.21.15.120:0/2837266243 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e43fff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d570c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d56f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 -- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 msgr2=0x55d79d570c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d570c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 -- 172.21.15.120:0/2837266243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 msgr2=0x55d79d56f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d56f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 -- 172.21.15.120:0/2837266243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79d56fcb0 con 0x55d79d55f860 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e50bb3640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d56f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e4bfff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cccf0 tx=0x7f0e3c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e1b28e399f548d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c049070 con 0x55d79d55f860 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0e3c0cce50 con 0x55d79d55f860 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.564+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3c0f3020 con 0x55d79d55f860 2024-09-10T23:00:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e3c0cc360 con 0x55d79d55f860 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 10 client.16968 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d79d56fcb0 con 0x55d79d55f860 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e3c0f3020 con 0x55d79d55f860 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0e3c0d11d0 con 0x55d79d55f860 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 10 client.16968 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 20 client.16968 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5e1980 0x55d79d5e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 4 client.16968 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d79d4cf0f0 con 0x55d79d5e1980 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e43fff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5e1980 0x55d79d5e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e2b7fe640 10 client.16968 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.565+0000 7f0e43fff640 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5e1980 0x55d79d5e3da0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3c3aa9e56ea4d7 server_cookie=2a3858ea09ca203f in_seq=0 out_seq=0 2024-09-10T23:00:24.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1350164 16.34 MB/sec execute 319 sec latency 29.326 ms 2024-09-10T23:00:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e2b7fe640 1 -- 172.21.15.120:0/2837266243 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d79d4cf0f0 con 0x55d79d5e1980 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e2b7fe640 10 client.16968 handle_command_reply: tid=0 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 client.16968 shutdown 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 msgr2=0x55d79d5705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cccf0 tx=0x7f0e3c02a420 comp rx=0 tx=0).stop 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 shutdown_connections 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d79d5e1980 0x55d79d5e3da0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d79d564bd0 0x55d79d56f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e2b7fe640 0 client.16968 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d79d562190 0x55d79d570c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 --2- 172.21.15.120:0/2837266243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d79d55f860 0x55d79d5705f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 >> 172.21.15.120:0/2837266243 conn(0x55d79d55a9b0 msgr2=0x55d79d568af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 shutdown_connections 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 1 -- 172.21.15.120:0/2837266243 wait complete. 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.790+0000 7f0e553e3740 20 client.16968 trim_cache size 0 max 16384 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 msgr2=0x7f0e380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 0x7f0e380b7820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e44003bc0 tx=0x7f0e44009500 comp rx=0 tx=0).stop 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 msgr2=0x7f0e4c10c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009a50 tx=0x7f0e3c0042f0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 shutdown_connections 2024-09-10T23:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0e380b5360 0x7f0e380b7820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e4c103370 0x7f0e4c10ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e4c1029b0 0x7f0e4c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 --2- 172.21.15.120:0/3512166715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e4c101860 0x7f0e4c10c0e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 >> 172.21.15.120:0/3512166715 conn(0x7f0e4c0fd140 msgr2=0x7f0e4c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 shutdown_connections 2024-09-10T23:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:24.792+0000 7f0e52616640 1 -- 172.21.15.120:0/3512166715 wait complete. 2024-09-10T23:00:24.941 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:00:24.941 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:00:25.194 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:25.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1353060 16.35 MB/sec execute 320 sec latency 118.260 ms 2024-09-10T23:00:25.632 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:25 smithi120 ceph-mon[31247]: pgmap v586: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-09-10T23:00:25.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:25 smithi153 ceph-mon[34951]: pgmap v586: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 -- 172.21.15.120:0/3941842939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 msgr2=0x7f93cc107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 --2- 172.21.15.120:0/3941842939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc107320 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009ca0 tx=0x7f93c402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 -- 172.21.15.120:0/3941842939 shutdown_connections 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 --2- 172.21.15.120:0/3941842939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc107320 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 --2- 172.21.15.120:0/3941842939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 --2- 172.21.15.120:0/3941842939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 0x7f93cc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.719+0000 7f93d29a5640 1 -- 172.21.15.120:0/3941842939 >> 172.21.15.120:0/3941842939 conn(0x7f93cc0fd700 msgr2=0x7f93cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- 172.21.15.120:0/3941842939 shutdown_connections 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- 172.21.15.120:0/3941842939 wait complete. 2024-09-10T23:00:25.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 Processor -- start 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- start start 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 0x7f93cc19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc119350 con 0x7f93cc106ed0 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1191d0 con 0x7f93cc1023b0 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.720+0000 7f93d29a5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1194d0 con 0x7f93cc101a90 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35990/0 (socket says 172.21.15.120:35990) 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 0x7f93cc19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93cb7fe640 1 -- 172.21.15.120:0/1880393704 learned_addr learned my addr 172.21.15.120:0/1880393704 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33628/0 (socket says 172.21.15.120:33628) 2024-09-10T23:00:25.735 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 -- 172.21.15.120:0/1880393704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 msgr2=0x7f93cc19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 0x7f93cc19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 -- 172.21.15.120:0/1880393704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 msgr2=0x7f93cc19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 -- 172.21.15.120:0/1880393704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc1a5990 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d0f42640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f93c402ade0 tx=0x7f93c40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93c97fa640 1 -- 172.21.15.120:0/1880393704 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4003bf0 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc1a5c20 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93c97fa640 1 -- 172.21.15.120:0/1880393704 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c4003d90 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93c97fa640 1 -- 172.21.15.120:0/1880393704 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c402fbd0 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.721+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc1ac340 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 Processor -- start 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93c97fa640 1 -- 172.21.15.120:0/1880393704 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f93c4004610 con 0x7f93cc106ed0 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 -- start start 2024-09-10T23:00:25.736 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a043462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04352790 0x555a04354b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a042bce60 con 0x555a04346800 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555a042ac290 con 0x555a04352790 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555a042b00a0 con 0x555a04345ee0 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33638/0 (socket says 172.21.15.120:33638) 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 -- 172.21.15.120:0/3085360784 learned_addr learned my addr 172.21.15.120:0/3085360784 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a043462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04352790 0x555a04354b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 -- 172.21.15.120:0/3085360784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 msgr2=0x555a043462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a043462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 -- 172.21.15.120:0/3085360784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04352790 msgr2=0x555a04354b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.737 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04352790 0x555a04354b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.724+0000 7f93cbfff640 1 -- 172.21.15.120:0/3085360784 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a0400a930 con 0x555a04346800 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93c97fa640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 0x7f93b40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00bd40 tx=0x7f93bc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848d309a90e3b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 0x7f93b40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93c97fa640 1 -- 172.21.15.120:0/1880393704 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f93c4106e70 con 0x7f93cc106ed0 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93b27fc640 1 -- 172.21.15.120:0/3085360784 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc004230 con 0x555a04346800 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93b27fc640 1 -- 172.21.15.120:0/3085360784 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00be30 con 0x555a04346800 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.725+0000 7f93b27fc640 1 -- 172.21.15.120:0/3085360784 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc01a760 con 0x555a04346800 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3085360784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 msgr2=0x555a04352250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 0x7f93b40b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x555a04345e90 tx=0x7f93c000b3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00bd40 tx=0x7f93bc0107a0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3085360784 shutdown_connections 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04352790 0x555a04354b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.738 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a04352250 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3085360784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a043462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3085360784 >> 172.21.15.120:0/3085360784 conn(0x555a04343d10 msgr2=0x555a04344eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3085360784 shutdown_connections 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3085360784 wait complete. 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 -- start start 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.726+0000 7f93d4d7a740 1 Processor -- start 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a043501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a0410d890 con 0x555a04352790 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555a04246d40 con 0x555a04345ee0 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555a04355440 con 0x555a04346800 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 10 client.0 _ll_register_callbacks cb 0x555a042aac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:25.739 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d0f42640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a043501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59966/0 (socket says 172.21.15.120:59966) 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36006/0 (socket says 172.21.15.120:36006) 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 -- 172.21.15.120:0/3803257271 learned_addr learned my addr 172.21.15.120:0/3803257271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 -- 172.21.15.120:0/3803257271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 msgr2=0x555a0434f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 -- 172.21.15.120:0/3803257271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 msgr2=0x555a043501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a043501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 -- 172.21.15.120:0/3803257271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a0435e7f0 con 0x555a04346800 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d0f42640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a043501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:25.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93c0012930 tx=0x7f93c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6319d640e1bd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c002e060 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc00be30 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.727+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f93c001b050 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93c0010930 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 10 client.26461 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a04361250 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc00be30 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f93c001d070 con 0x555a04346800 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 10 client.26461 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 20 client.26461 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a04362700 0x555a04364b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 4 client.26461 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a042bedb0 con 0x555a04362700 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.728+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a04362700 0x555a04364b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.729+0000 7f93ab7fe640 10 client.26461 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.729+0000 7f93cbfff640 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a04362700 0x555a04364b60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9cba9824ccc631 server_cookie=75651b722c474743 in_seq=0 out_seq=0 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.730+0000 7f93ab7fe640 1 -- 172.21.15.120:0/3803257271 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a042bedb0 con 0x555a04362700 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.730+0000 7f93ab7fe640 10 client.26461 handle_command_reply: tid=0 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.730+0000 7f93d4d7a740 1 client.26461 shutdown 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 msgr2=0x555a0434fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f93c0012930 tx=0x7f93c00046c0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 shutdown_connections 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a04362700 0x555a04364b60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a043501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04346800 0x555a0434fc60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93ab7fe640 0 client.26461 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:25.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/3803257271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a0434f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 >> 172.21.15.120:0/3803257271 conn(0x555a04343d10 msgr2=0x555a04342f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 shutdown_connections 2024-09-10T23:00:25.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 1 -- 172.21.15.120:0/3803257271 wait complete. 2024-09-10T23:00:25.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.731+0000 7f93d4d7a740 20 client.26461 trim_cache size 0 max 16384 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 Processor -- start 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 -- start start 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a0434c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04352790 0x555a0434ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b29d0 con 0x555a04346800 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b5250 con 0x555a04345ee0 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b3040 con 0x555a04352790 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a0434c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93d0f42640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04352790 0x555a0434ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36012/0 (socket says 172.21.15.120:36012) 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cbfff640 1 -- 172.21.15.120:0/763045071 learned_addr learned my addr 172.21.15.120:0/763045071 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 -- 172.21.15.120:0/763045071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04352790 msgr2=0x555a0434ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04352790 0x555a0434ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 -- 172.21.15.120:0/763045071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 msgr2=0x555a04345100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 -- 172.21.15.120:0/763045071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a04344790 con 0x555a04346800 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cbfff640 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a0434c950 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f93c0000c00 tx=0x7f93c000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20671922224383 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93b1ffb640 1 -- 172.21.15.120:0/763045071 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0039020 con 0x555a04346800 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93b1ffb640 1 -- 172.21.15.120:0/763045071 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c000f3c0 con 0x555a04346800 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.749+0000 7f93b1ffb640 1 -- 172.21.15.120:0/763045071 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c000fcd0 con 0x555a04346800 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- 172.21.15.120:0/763045071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 msgr2=0x555a0434c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a0434c950 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f93c0000c00 tx=0x7f93c000bf70 comp rx=0 tx=0).stop 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- 172.21.15.120:0/763045071 shutdown_connections 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04352790 0x555a0434ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04346800 0x555a0434c950 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/763045071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04345ee0 0x555a04345100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- 172.21.15.120:0/763045071 >> 172.21.15.120:0/763045071 conn(0x555a04343d10 msgr2=0x555a043449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- 172.21.15.120:0/763045071 shutdown_connections 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- 172.21.15.120:0/763045071 wait complete. 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.750+0000 7f93d4d7a740 1 -- start start 2024-09-10T23:00:25.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 Processor -- start 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a0435b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a04358bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b3710 con 0x555a04352790 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b4b80 con 0x555a04346800 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b3de0 con 0x555a04345ee0 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d4d7a740 10 client.0 _ll_register_callbacks cb 0x7f93c0012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:25.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a0435b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36026/0 (socket says 172.21.15.120:36026) 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 -- 172.21.15.120:0/4127118062 learned_addr learned my addr 172.21.15.120:0/4127118062 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93d0f42640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a04358bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 -- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 msgr2=0x555a0435b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a0435b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 -- 172.21.15.120:0/4127118062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 msgr2=0x555a04358bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a04358bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cb7fe640 1 -- 172.21.15.120:0/4127118062 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a043ce640 con 0x555a04346800 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.751+0000 7f93cbfff640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a0435b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93cb7fe640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93c000def0 tx=0x7f93c00108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f98c982414823884 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:00:25.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0039030 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f93c000f570 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c000f3c0 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93c000f7f0 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 10 client.26372 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a043cf880 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c000f3c0 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f93c001d040 con 0x555a04346800 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 10 client.26372 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 20 client.26372 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a043cff70 0x555a043d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 4 client.26372 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:00:25.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.752+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a042ba860 con 0x555a043cff70 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.753+0000 7f93cbfff640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a043cff70 0x555a043d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.753+0000 7f93ab7fe640 10 client.26372 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.753+0000 7f93cbfff640 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a043cff70 0x555a043d2390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c10a07f1f04c76 server_cookie=b7e8e3fd55ff699d in_seq=0 out_seq=0 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.754+0000 7f93ab7fe640 1 -- 172.21.15.120:0/4127118062 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a042ba860 con 0x555a043cff70 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.754+0000 7f93ab7fe640 10 client.26372 handle_command_reply: tid=0 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.754+0000 7f93d4d7a740 1 client.26372 shutdown 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 msgr2=0x555a043584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93c000def0 tx=0x7f93c00108c0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 shutdown_connections 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555a043cff70 0x555a043d2390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a04352790 0x555a04358bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555a04346800 0x555a043584f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93ab7fe640 0 client.26372 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 --2- 172.21.15.120:0/4127118062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a04345ee0 0x555a0435b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 >> 172.21.15.120:0/4127118062 conn(0x555a04343d10 msgr2=0x555a04345210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 shutdown_connections 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 1 -- 172.21.15.120:0/4127118062 wait complete. 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d4d7a740 20 client.26372 trim_cache size 0 max 16384 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1108fa36-9551-4546-98e9-d41c113b5b06", 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:00:25.763 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:00:25.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 msgr2=0x7f93b40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 0x7f93b40b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x555a04345e90 tx=0x7f93c000b3f0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 msgr2=0x7f93cc1a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:25.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.755+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f93c402ade0 tx=0x7f93c40319f0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 shutdown_connections 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f93b40b52e0 0x7f93b40b77a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93cc106ed0 0x7f93cc1a51c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93cc1023b0 0x7f93cc19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 --2- 172.21.15.120:0/1880393704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93cc101a90 0x7f93cc19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 >> 172.21.15.120:0/1880393704 conn(0x7f93cc0fd700 msgr2=0x7f93cc104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 shutdown_connections 2024-09-10T23:00:25.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:25.756+0000 7f93d29a5640 1 -- 172.21.15.120:0/1880393704 wait complete. 2024-09-10T23:00:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:25 smithi176 ceph-mon[34982]: pgmap v586: 97 pgs: 97 active+clean; 51 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-09-10T23:00:26.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:00:26.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1354660 16.32 MB/sec execute 321 sec latency 66.321 ms 2024-09-10T23:00:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:26 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:00:26.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:26 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:26.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:26 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:00:26.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:26 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:26 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:00:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:26 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:26.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:26 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:00:26.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:26 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:26.718 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:26.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:26 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:00:26.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:26 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:26.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:26 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:00:26.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:26 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:27.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 -- 172.21.15.120:0/4218087764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d001076a0 msgr2=0x7f4d00107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 --2- 172.21.15.120:0/4218087764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d001076a0 0x7f4d00107ae0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce802a410 comp rx=0 tx=0).stop 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 -- 172.21.15.120:0/4218087764 shutdown_connections 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 --2- 172.21.15.120:0/4218087764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00108020 0x7f4d0010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 --2- 172.21.15.120:0/4218087764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d001076a0 0x7f4d00107ae0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 --2- 172.21.15.120:0/4218087764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d00101c90 0x7f4d00102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 -- 172.21.15.120:0/4218087764 >> 172.21.15.120:0/4218087764 conn(0x7f4d000fd700 msgr2=0x7f4d000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 -- 172.21.15.120:0/4218087764 shutdown_connections 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.219+0000 7f4d0563c640 1 -- 172.21.15.120:0/4218087764 wait complete. 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 Processor -- start 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 -- start start 2024-09-10T23:00:27.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 0x7f4d00199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d001076a0 0x7f4d001997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00111ce0 con 0x7f4d00108020 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00111b60 con 0x7f4d00101c90 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4d0563c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00111e60 con 0x7f4d001076a0 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4cff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4cff7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33690/0 (socket says 172.21.15.120:33690) 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d001076a0 0x7f4d001997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4cff7fe640 1 -- 172.21.15.120:0/3596841889 learned_addr learned my addr 172.21.15.120:0/3596841889 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.220+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 0x7f4d00199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 -- 172.21.15.120:0/3596841889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d001076a0 msgr2=0x7f4d001997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d001076a0 0x7f4d001997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 -- 172.21.15.120:0/3596841889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 msgr2=0x7f4d00199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 0x7f4d00199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 -- 172.21.15.120:0/3596841889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0019f320 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 0x7f4d00199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cff7fe640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800eb10 tx=0x7f4cf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cf7fff640 1 -- 172.21.15.120:0/3596841889 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8004b20 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0019f610 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cf7fff640 1 -- 172.21.15.120:0/3596841889 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf8004cc0 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4cf7fff640 1 -- 172.21.15.120:0/3596841889 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8010670 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.221+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a5cd0 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.223+0000 7f4cf7fff640 1 -- 172.21.15.120:0/3596841889 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4cf8005020 con 0x7f4d00108020 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 Processor -- start 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 -- start start 2024-09-10T23:00:27.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587dd32ee50 con 0x5587dd3b8480 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5587dd31e280 con 0x5587dd3c9140 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4d07a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5587dd322090 con 0x5587dd3b8da0 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33706/0 (socket says 172.21.15.120:33706) 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/1145813366 learned_addr learned my addr 172.21.15.120:0/1145813366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.224+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cf7fff640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 0x7f4cec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/1145813366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 msgr2=0x5587dd3c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/1145813366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 msgr2=0x5587dd3cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/1145813366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587dd07c650 con 0x5587dd3b8480 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cff7fe640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 0x7f4cec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257dcae8353208d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cf7fff640 1 -- 172.21.15.120:0/3596841889 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4cf80e6190 con 0x7f4d00108020 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cf57fa640 1 -- 172.21.15.120:0/1145813366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8035040 con 0x5587dd3b8480 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cf57fa640 1 -- 172.21.15.120:0/1145813366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce8045070 con 0x5587dd3b8480 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.225+0000 7f4cf57fa640 1 -- 172.21.15.120:0/1145813366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003ab0 con 0x5587dd3b8480 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- 172.21.15.120:0/1145813366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 msgr2=0x5587dd3b8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4cff7fe640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 0x7f4cec0b77a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80a9bf0 tx=0x7f4cf80dcdf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce8005b50 comp rx=0 tx=0).stop 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- 172.21.15.120:0/1145813366 shutdown_connections 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1145813366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b8860 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- 172.21.15.120:0/1145813366 >> 172.21.15.120:0/1145813366 conn(0x5587dd3b62b0 msgr2=0x5587dd3b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- 172.21.15.120:0/1145813366 shutdown_connections 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- 172.21.15.120:0/1145813366 wait complete. 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 -- start start 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.226+0000 7f4d07a11740 1 Processor -- start 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8480 0x5587dd3c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587dd17f700 con 0x5587dd3b8da0 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5587dd2b9110 con 0x5587dd3c9140 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5587dd3cbd90 con 0x5587dd3b8480 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4d07a11740 10 client.0 _ll_register_callbacks cb 0x5587dd2d2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33722/0 (socket says 172.21.15.120:33722) 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 -- 172.21.15.120:0/3943209924 learned_addr learned my addr 172.21.15.120:0/3943209924 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8480 0x5587dd3c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 -- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8480 msgr2=0x5587dd3c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8480 0x5587dd3c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 -- 172.21.15.120:0/3943209924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 msgr2=0x5587dd3c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 -- 172.21.15.120:0/3943209924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587dd2e6c40 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012d20 tx=0x7f4cf000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20646a223f56b85a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4ce67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00120d0 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.227+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4cf0010430 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce8045070 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4cf0012270 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 10 client.16998 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587dd3d7740 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce8045070 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4cf0027700 con 0x5587dd3b8da0 2024-09-10T23:00:27.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 10 client.16998 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 20 client.16998 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd3d8cf0 0x5587dd3db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 4 client.16998 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587dd330da0 con 0x5587dd3d8cf0 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.228+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd3d8cf0 0x5587dd3db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.229+0000 7f4ce67fc640 10 client.16998 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.229+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd3d8cf0 0x5587dd3db110 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca91984f8f0a15c1 server_cookie=f5a3598f180fde6f in_seq=0 out_seq=0 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4ce67fc640 1 -- 172.21.15.120:0/3943209924 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587dd330da0 con 0x5587dd3d8cf0 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4ce67fc640 10 client.16998 handle_command_reply: tid=0 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 client.16998 shutdown 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 msgr2=0x5587dd3c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012d20 tx=0x7f4cf000f790 comp rx=0 tx=0).stop 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 shutdown_connections 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd3d8cf0 0x5587dd3db110 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3c5e70 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4ce67fc640 0 client.16998 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 --2- 172.21.15.120:0/3943209924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8480 0x5587dd3c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 >> 172.21.15.120:0/3943209924 conn(0x5587dd3b62b0 msgr2=0x5587dd3b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 shutdown_connections 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 1 -- 172.21.15.120:0/3943209924 wait complete. 2024-09-10T23:00:27.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.230+0000 7f4d07a11740 20 client.16998 trim_cache size 0 max 16384 2024-09-10T23:00:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.249+0000 7f4d07a11740 1 Processor -- start 2024-09-10T23:00:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.249+0000 7f4d07a11740 1 -- start start 2024-09-10T23:00:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a9fd0 con 0x5587dd3b8480 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001ac830 con 0x5587dd3c9140 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4d07a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001aa620 con 0x5587dd3b8da0 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33732/0 (socket says 172.21.15.120:33732) 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/180928067 learned_addr learned my addr 172.21.15.120:0/180928067 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/180928067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 msgr2=0x5587dd3d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3d4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/180928067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 msgr2=0x5587dd3b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 -- 172.21.15.120:0/180928067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587dd3c8650 con 0x5587dd3b8480 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009850 tx=0x7f4ce802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a571cfe1105ae46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cf4ff9640 1 -- 172.21.15.120:0/180928067 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8035040 con 0x5587dd3b8480 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cf4ff9640 1 -- 172.21.15.120:0/180928067 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce8045040 con 0x5587dd3b8480 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.250+0000 7f4cf4ff9640 1 -- 172.21.15.120:0/180928067 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004c20 con 0x5587dd3b8480 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 -- 172.21.15.120:0/180928067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 msgr2=0x5587dd3b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009850 tx=0x7f4ce802a410 comp rx=0 tx=0).stop 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 -- 172.21.15.120:0/180928067 shutdown_connections 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3c9140 0x5587dd3b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3b8da0 0x5587dd3d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 --2- 172.21.15.120:0/180928067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8480 0x5587dd3b5bc0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 -- 172.21.15.120:0/180928067 >> 172.21.15.120:0/180928067 conn(0x5587dd3b62b0 msgr2=0x5587dd3b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 -- 172.21.15.120:0/180928067 shutdown_connections 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.251+0000 7f4d07a11740 1 -- 172.21.15.120:0/180928067 wait complete. 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 -- start start 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 Processor -- start 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3b8480 0x5587dd3d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3c9140 0x5587dd3ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001aacf0 con 0x5587dd3b8da0 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001ac160 con 0x5587dd3b8480 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001ab3c0 con 0x5587dd3c9140 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4d07a11740 10 client.0 _ll_register_callbacks cb 0x7f4cf0009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33746/0 (socket says 172.21.15.120:33746) 2024-09-10T23:00:27.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfe7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3c9140 0x5587dd3ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 -- 172.21.15.120:0/1297196608 learned_addr learned my addr 172.21.15.120:0/1297196608 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3b8480 0x5587dd3d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 -- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3c9140 msgr2=0x5587dd3ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3c9140 0x5587dd3ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 -- 172.21.15.120:0/1297196608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3b8480 msgr2=0x5587dd3d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3b8480 0x5587dd3d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.252+0000 7f4cfeffd640 1 -- 172.21.15.120:0/1297196608 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587dd3ce7b0 con 0x5587dd3b8da0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4cfeffd640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012cf0 tx=0x7f4cf000fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f911f541afb57cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00120d0 con 0x5587dd3b8da0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4cf0027920 con 0x5587dd3b8da0 2024-09-10T23:00:27.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce8045040 con 0x5587dd3b8da0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4cf0036070 con 0x5587dd3b8da0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 10 client.17004 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587dd446680 con 0x5587dd3b8da0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce8045040 con 0x5587dd3b8da0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4cf002e030 con 0x5587dd3b8da0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 10 client.17004 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 20 client.17004 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd446dd0 0x5587dd4491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 4 client.17004 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.253+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587dd32c850 con 0x5587dd446dd0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd446dd0 0x5587dd4491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4ce67fc640 10 client.17004 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:27.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4cfe7fc640 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd446dd0 0x5587dd4491f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7773b0af3adca401 server_cookie=2a69334e17aa6d59 in_seq=0 out_seq=0 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4ce67fc640 1 -- 172.21.15.120:0/1297196608 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x5587dd32c850 con 0x5587dd446dd0 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4ce67fc640 10 client.17004 handle_command_reply: tid=0 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 client.17004 shutdown 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 msgr2=0x5587dd3d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012cf0 tx=0x7f4cf000fd30 comp rx=0 tx=0).stop 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 shutdown_connections 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5587dd446dd0 0x5587dd4491f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5587dd3c9140 0x5587dd3ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4ce67fc640 0 client.17004 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587dd3b8da0 0x5587dd3d1790 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 --2- 172.21.15.120:0/1297196608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5587dd3b8480 0x5587dd3d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.254+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 >> 172.21.15.120:0/1297196608 conn(0x5587dd3b62b0 msgr2=0x5587dd3c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 shutdown_connections 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d07a11740 1 -- 172.21.15.120:0/1297196608 wait complete. 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d07a11740 20 client.17004 trim_cache size 0 max 16384 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "1108fa36-9551-4546-98e9-d41c113b5b06", 2024-09-10T23:00:27.266 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.270 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:00:27.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:00:27.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:27.276 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 msgr2=0x7f4cec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 0x7f4cec0b77a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80a9bf0 tx=0x7f4cf80dcdf0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 msgr2=0x7f4d0019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800eb10 tx=0x7f4cf800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 shutdown_connections 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4cec0b52e0 0x7f4cec0b77a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d00108020 0x7f4d0019eb50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d001076a0 0x7f4d001997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 --2- 172.21.15.120:0/3596841889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d00101c90 0x7f4d00199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:27.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.255+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 >> 172.21.15.120:0/3596841889 conn(0x7f4d000fd700 msgr2=0x7f4d0006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:27.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.256+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 shutdown_connections 2024-09-10T23:00:27.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:27.256+0000 7f4d0563c640 1 -- 172.21.15.120:0/3596841889 wait complete. 2024-09-10T23:00:27.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1357456 16.31 MB/sec execute 322 sec latency 35.512 ms 2024-09-10T23:00:27.665 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:27 smithi120 ceph-mon[31247]: pgmap v587: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 536 op/s 2024-09-10T23:00:27.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:27 smithi153 ceph-mon[34951]: pgmap v587: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 536 op/s 2024-09-10T23:00:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:27 smithi176 ceph-mon[34982]: pgmap v587: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 536 op/s 2024-09-10T23:00:28.004 INFO:tasks.cephfs.filesystem:scrub status for tag:1108fa36-9551-4546-98e9-d41c113b5b06 - {'path': '/', 'tag': '1108fa36-9551-4546-98e9-d41c113b5b06', 'options': 'recursive,force'} 2024-09-10T23:00:28.631 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1360774 16.32 MB/sec execute 323 sec latency 21.831 ms 2024-09-10T23:00:29.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1364190 16.33 MB/sec execute 324 sec latency 15.702 ms 2024-09-10T23:00:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:29 smithi120 ceph-mon[31247]: pgmap v588: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 432 op/s 2024-09-10T23:00:29.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:29 smithi153 ceph-mon[34951]: pgmap v588: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 432 op/s 2024-09-10T23:00:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:29 smithi176 ceph-mon[34982]: pgmap v588: 97 pgs: 97 active+clean; 48 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 432 op/s 2024-09-10T23:00:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:30 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:00:30.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1367562 16.34 MB/sec execute 325 sec latency 23.805 ms 2024-09-10T23:00:30.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:30 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:00:30.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:30 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:00:31.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1370195 16.32 MB/sec execute 326 sec latency 134.553 ms 2024-09-10T23:00:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:31 smithi120 ceph-mon[31247]: pgmap v589: 97 pgs: 97 active+clean; 45 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T23:00:31.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:31 smithi153 ceph-mon[34951]: pgmap v589: 97 pgs: 97 active+clean; 45 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T23:00:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:31 smithi176 ceph-mon[34982]: pgmap v589: 97 pgs: 97 active+clean; 45 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-09-10T23:00:32.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1372868 16.31 MB/sec execute 327 sec latency 24.397 ms 2024-09-10T23:00:33.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1375263 16.33 MB/sec execute 328 sec latency 44.728 ms 2024-09-10T23:00:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:33 smithi120 ceph-mon[31247]: pgmap v590: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T23:00:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:33 smithi153 ceph-mon[34951]: pgmap v590: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T23:00:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:33 smithi176 ceph-mon[34982]: pgmap v590: 97 pgs: 97 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-10T23:00:34.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1378682 16.37 MB/sec execute 329 sec latency 18.606 ms 2024-09-10T23:00:35.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1381784 16.37 MB/sec execute 330 sec latency 14.869 ms 2024-09-10T23:00:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:35 smithi120 ceph-mon[31247]: pgmap v591: 97 pgs: 97 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-10T23:00:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:35 smithi153 ceph-mon[34951]: pgmap v591: 97 pgs: 97 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-10T23:00:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:35 smithi176 ceph-mon[34982]: pgmap v591: 97 pgs: 97 active+clean; 51 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 448 op/s 2024-09-10T23:00:36.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1384102 16.35 MB/sec execute 331 sec latency 89.313 ms 2024-09-10T23:00:37.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1386594 16.35 MB/sec execute 332 sec latency 52.960 ms 2024-09-10T23:00:37.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:37 smithi120 ceph-mon[31247]: pgmap v592: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-09-10T23:00:37.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:37 smithi153 ceph-mon[34951]: pgmap v592: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-09-10T23:00:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:37 smithi176 ceph-mon[34982]: pgmap v592: 97 pgs: 97 active+clean; 52 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-09-10T23:00:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 -- 172.21.15.120:0/1976986446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac06b470 msgr2=0x7f28ac105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/1976986446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac06b470 0x7f28ac105530 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f289c009900 tx=0x7f289c02a410 comp rx=0 tx=0).stop 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 -- 172.21.15.120:0/1976986446 shutdown_connections 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/1976986446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/1976986446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac105ac0 0x7f28ac107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/1976986446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac06b470 0x7f28ac105530 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 -- 172.21.15.120:0/1976986446 >> 172.21.15.120:0/1976986446 conn(0x7f28ac0fd140 msgr2=0x7f28ac0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 -- 172.21.15.120:0/1976986446 shutdown_connections 2024-09-10T23:00:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.891+0000 7f28b0a8e640 1 -- 172.21.15.120:0/1976986446 wait complete. 2024-09-10T23:00:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 Processor -- start 2024-09-10T23:00:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 -- start start 2024-09-10T23:00:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 0x7f28ac198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10e5c0 con 0x7f28ac105ac0 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10e440 con 0x7f28ac108670 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28b0a8e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10e740 con 0x7f28ac06b470 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41692/0 (socket says 172.21.15.120:41692) 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 0x7f28ac198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28a3fff640 1 -- 172.21.15.120:0/4137275751 learned_addr learned my addr 172.21.15.120:0/4137275751 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.892+0000 7f28aad9d640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 -- 172.21.15.120:0/4137275751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 msgr2=0x7f28ac198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 0x7f28ac198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 -- 172.21.15.120:0/4137275751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 msgr2=0x7f28ac199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 -- 172.21.15.120:0/4137275751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac19a8b0 con 0x7f28ac105ac0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28aad9d640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28aa59c640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 0x7f28ac198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a3fff640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f289400e9f0 tx=0x7f289400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a37fe640 1 -- 172.21.15.120:0/4137275751 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289400cd80 con 0x7f28ac105ac0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a37fe640 1 -- 172.21.15.120:0/4137275751 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28940054d0 con 0x7f28ac105ac0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28a37fe640 1 -- 172.21.15.120:0/4137275751 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894010690 con 0x7f28ac105ac0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac19aba0 con 0x7f28ac105ac0 2024-09-10T23:00:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.893+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac1a1270 con 0x7f28ac105ac0 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.896+0000 7f28a37fe640 1 -- 172.21.15.120:0/4137275751 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2894004e90 con 0x7f28ac105ac0 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 Processor -- start 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 -- start start 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e3b52c6f0 con 0x563e3b5bd190 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e3b51bb20 con 0x563e3b5ba860 2024-09-10T23:00:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28b385d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563e3b51f930 con 0x563e3b5bfbd0 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41694/0 (socket says 172.21.15.120:41694) 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 -- 172.21.15.120:0/603967187 learned_addr learned my addr 172.21.15.120:0/603967187 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28a3fff640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aad9d640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28a37fe640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 0x7f28980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 -- 172.21.15.120:0/603967187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 msgr2=0x563e3b5c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.897+0000 7f28aa59c640 1 -- 172.21.15.120:0/603967187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 msgr2=0x563e3b5bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aa59c640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5bcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aa59c640 1 -- 172.21.15.120:0/603967187 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3b4e1530 con 0x563e3b5bd190 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aad9d640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aad9d640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 0x7f28980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28a37fe640 1 -- 172.21.15.120:0/4137275751 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f28940e5f90 con 0x7f28ac105ac0 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aa59c640 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f289c009900 tx=0x7f289c0315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f0b6b019134645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28aad9d640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 0x7f28980b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28ac199af0 tx=0x7f28a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28a0ff9640 1 -- 172.21.15.120:0/603967187 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c040070 con 0x563e3b5bd190 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28a0ff9640 1 -- 172.21.15.120:0/603967187 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c005040 con 0x563e3b5bd190 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.898+0000 7f28a0ff9640 1 -- 172.21.15.120:0/603967187 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c035040 con 0x563e3b5bd190 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- 172.21.15.120:0/603967187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 msgr2=0x563e3b5bf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f289c009900 tx=0x7f289c0315a0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- 172.21.15.120:0/603967187 shutdown_connections 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bd190 0x563e3b5bf580 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 --2- 172.21.15.120:0/603967187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- 172.21.15.120:0/603967187 >> 172.21.15.120:0/603967187 conn(0x563e3b5b59b0 msgr2=0x563e3b5b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- 172.21.15.120:0/603967187 shutdown_connections 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- 172.21.15.120:0/603967187 wait complete. 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.899+0000 7f28b385d740 1 -- start start 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 Processor -- start 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e3b453ab0 con 0x563e3b5bfbd0 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563e3b4c1730 con 0x563e3b5ba860 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563e3b5c2b50 con 0x563e3b5bd190 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28b385d740 10 client.0 _ll_register_callbacks cb 0x563e3b5156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41702/0 (socket says 172.21.15.120:41702) 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 -- 172.21.15.120:0/1276213139 learned_addr learned my addr 172.21.15.120:0/1276213139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aa59c640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 -- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 msgr2=0x563e3b5c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 -- 172.21.15.120:0/1276213139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 msgr2=0x563e3b5c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 -- 172.21.15.120:0/1276213139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3b27d790 con 0x563e3b5bfbd0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aa59c640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f28aad9d640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f28a4013900 tx=0x7f28a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd75118febd73746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f2891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.900+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a401ed70 con 0x563e3b5bfbd0 2024-09-10T23:00:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f28a4028470 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c005040 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28a4028670 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 10 client.17016 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e3b2f6620 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c005040 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f28a400f040 con 0x563e3b5bfbd0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 10 client.17016 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 20 client.17016 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b5cf7f0 0x563e3b5d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 4 client.17016 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3b52e640 con 0x563e3b5cf7f0 2024-09-10T23:00:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.901+0000 7f28aa59c640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b5cf7f0 0x563e3b5d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.902+0000 7f28aa59c640 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b5cf7f0 0x563e3b5d1c80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ea887e1ba59c21 server_cookie=8f2a541a8b483d8e in_seq=0 out_seq=0 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.902+0000 7f2891ffb640 10 client.17016 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.902+0000 7f2891ffb640 1 -- 172.21.15.120:0/1276213139 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e3b52e640 con 0x563e3b5cf7f0 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.902+0000 7f2891ffb640 10 client.17016 handle_command_reply: tid=0 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.902+0000 7f28b385d740 1 client.17016 shutdown 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 msgr2=0x563e3b5c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f28a4013900 tx=0x7f28a4004ab0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 shutdown_connections 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b5cf7f0 0x563e3b5d1c80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c51c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f2891ffb640 0 client.17016 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:37.912 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 --2- 172.21.15.120:0/1276213139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5ba860 0x563e3b5c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 >> 172.21.15.120:0/1276213139 conn(0x563e3b5b59b0 msgr2=0x563e3b5b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 shutdown_connections 2024-09-10T23:00:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 1 -- 172.21.15.120:0/1276213139 wait complete. 2024-09-10T23:00:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.903+0000 7f28b385d740 20 client.17016 trim_cache size 0 max 16384 2024-09-10T23:00:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 Processor -- start 2024-09-10T23:00:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 -- start start 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a7e50 con 0x563e3b5ba860 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1aa6d0 con 0x563e3b5bfbd0 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28b385d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a84c0 con 0x563e3b5bd190 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41706/0 (socket says 172.21.15.120:41706) 2024-09-10T23:00:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 -- 172.21.15.120:0/2254582830 learned_addr learned my addr 172.21.15.120:0/2254582830 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aad9d640 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 -- 172.21.15.120:0/2254582830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 msgr2=0x563e3b5c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 -- 172.21.15.120:0/2254582830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bfbd0 msgr2=0x563e3b5c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.918+0000 7f28aa59c640 1 -- 172.21.15.120:0/2254582830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3b5bf8f0 con 0x563e3b5ba860 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.919+0000 7f28aa59c640 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f289c004620 tx=0x7f289c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9623ab4c26f9cd84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.919+0000 7f2893fff640 1 -- 172.21.15.120:0/2254582830 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c035040 con 0x563e3b5ba860 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.919+0000 7f2893fff640 1 -- 172.21.15.120:0/2254582830 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c0052f0 con 0x563e3b5ba860 2024-09-10T23:00:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.919+0000 7f2893fff640 1 -- 172.21.15.120:0/2254582830 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c040040 con 0x563e3b5ba860 2024-09-10T23:00:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- 172.21.15.120:0/2254582830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 msgr2=0x563e3b5c2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f289c004620 tx=0x7f289c02a410 comp rx=0 tx=0).stop 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- 172.21.15.120:0/2254582830 shutdown_connections 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bfbd0 0x563e3b5c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bd190 0x563e3b5c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- 172.21.15.120:0/2254582830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5c2f50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- 172.21.15.120:0/2254582830 >> 172.21.15.120:0/2254582830 conn(0x563e3b5b59b0 msgr2=0x563e3b5b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- 172.21.15.120:0/2254582830 shutdown_connections 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- 172.21.15.120:0/2254582830 wait complete. 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- start start 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 Processor -- start 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 0x563e3b5cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a8b90 con 0x563e3b5ba860 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1aa000 con 0x563e3b5bd190 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a9260 con 0x563e3b5bfbd0 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28b385d740 10 client.0 _ll_register_callbacks cb 0x7f28a4012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41720/0 (socket says 172.21.15.120:41720) 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.920+0000 7f28aa59c640 1 -- 172.21.15.120:0/260890508 learned_addr learned my addr 172.21.15.120:0/260890508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28a3fff640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 0x563e3b5cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 -- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 msgr2=0x563e3b5ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 -- 172.21.15.120:0/260890508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 msgr2=0x563e3b5cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 0x563e3b5cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 -- 172.21.15.120:0/260890508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3b5cabe0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28a3fff640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 0x563e3b5cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28aa59c640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f289c009800 tx=0x7f289c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772b50ea1ceee41f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f2891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c035070 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f289c0048c0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c0052f0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f289c040040 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28b385d740 10 client.17022 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e3b5c99d0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.921+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c0052f0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f289c06bba0 con 0x563e3b5ba860 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 10 client.17022 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 20 client.17022 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b63c850 0x563e3b63ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 4 client.17022 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e3b52a0f0 con 0x563e3b63c850 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28a3fff640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b63c850 0x563e3b63ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f28a3fff640 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b63c850 0x563e3b63ec40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0db15bfdb9b3cc server_cookie=9904a4589b733395 in_seq=0 out_seq=0 2024-09-10T23:00:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:37.922+0000 7f2891ffb640 10 client.17022 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f2891ffb640 1 -- 172.21.15.120:0/260890508 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e3b52a0f0 con 0x563e3b63c850 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f2891ffb640 10 client.17022 handle_command_reply: tid=0 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f28b385d740 1 client.17022 shutdown 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 msgr2=0x563e3b5cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f289c009800 tx=0x7f289c0095c0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.121+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 shutdown_connections 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563e3b63c850 0x563e3b63ec40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e3b5bfbd0 0x563e3b5ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f2891ffb640 0 client.17022 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563e3b5bd190 0x563e3b5cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 --2- 172.21.15.120:0/260890508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e3b5ba860 0x563e3b5cb5c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 >> 172.21.15.120:0/260890508 conn(0x563e3b5b59b0 msgr2=0x563e3b5c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 shutdown_connections 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 1 -- 172.21.15.120:0/260890508 wait complete. 2024-09-10T23:00:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b385d740 20 client.17022 trim_cache size 0 max 16384 2024-09-10T23:00:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:00:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:00:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:00:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 msgr2=0x7f28980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 0x7f28980b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28ac199af0 tx=0x7f28a4004a40 comp rx=0 tx=0).stop 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 msgr2=0x7f28ac1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.122+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f289400e9f0 tx=0x7f289400edf0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 shutdown_connections 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f28980b5360 0x7f28980b7820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28ac108670 0x7f28ac199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28ac105ac0 0x7f28ac1993c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 --2- 172.21.15.120:0/4137275751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28ac06b470 0x7f28ac198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 >> 172.21.15.120:0/4137275751 conn(0x7f28ac0fd140 msgr2=0x7f28ac103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 shutdown_connections 2024-09-10T23:00:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:38.123+0000 7f28b0a8e640 1 -- 172.21.15.120:0/4137275751 wait complete. 2024-09-10T23:00:38.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1389257 16.35 MB/sec execute 333 sec latency 63.944 ms 2024-09-10T23:00:39.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1392155 16.34 MB/sec execute 334 sec latency 11.973 ms 2024-09-10T23:00:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:39 smithi120 ceph-mon[31247]: pgmap v593: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-10T23:00:39.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:39 smithi153 ceph-mon[34951]: pgmap v593: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-10T23:00:39.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:39 smithi176 ceph-mon[34982]: pgmap v593: 97 pgs: 97 active+clean; 50 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-10T23:00:40.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1395245 16.33 MB/sec execute 335 sec latency 13.363 ms 2024-09-10T23:00:41.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1397240 16.32 MB/sec execute 336 sec latency 47.945 ms 2024-09-10T23:00:41.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:41 smithi120 ceph-mon[31247]: pgmap v594: 97 pgs: 97 active+clean; 38 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-09-10T23:00:41.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:41.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:41.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:41.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:00:41.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:41 smithi153 ceph-mon[34951]: pgmap v594: 97 pgs: 97 active+clean; 38 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-09-10T23:00:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:41.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:00:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:41 smithi176 ceph-mon[34982]: pgmap v594: 97 pgs: 97 active+clean; 38 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-09-10T23:00:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:00:42.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1400275 16.33 MB/sec execute 337 sec latency 57.414 ms 2024-09-10T23:00:43.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1403527 16.33 MB/sec execute 338 sec latency 12.820 ms 2024-09-10T23:00:43.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:43 smithi120 ceph-mon[31247]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T23:00:43.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:43 smithi153 ceph-mon[34951]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T23:00:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:43 smithi176 ceph-mon[34982]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-09-10T23:00:44.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1406440 16.32 MB/sec execute 339 sec latency 19.779 ms 2024-09-10T23:00:45.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1409426 16.32 MB/sec execute 340 sec latency 52.672 ms 2024-09-10T23:00:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:45 smithi120 ceph-mon[31247]: pgmap v596: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:00:45.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:45 smithi153 ceph-mon[34951]: pgmap v596: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:00:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:45 smithi176 ceph-mon[34982]: pgmap v596: 97 pgs: 97 active+clean; 28 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:00:46.632 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1412316 16.32 MB/sec execute 341 sec latency 49.275 ms 2024-09-10T23:00:47.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1414764 16.31 MB/sec execute 342 sec latency 71.602 ms 2024-09-10T23:00:47.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:47 smithi120 ceph-mon[31247]: pgmap v597: 97 pgs: 97 active+clean; 32 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-09-10T23:00:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:47 smithi153 ceph-mon[34951]: pgmap v597: 97 pgs: 97 active+clean; 32 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-09-10T23:00:47.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:47 smithi176 ceph-mon[34982]: pgmap v597: 97 pgs: 97 active+clean; 32 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-09-10T23:00:48.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1418174 16.32 MB/sec execute 343 sec latency 7.273 ms 2024-09-10T23:00:49.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1421480 16.33 MB/sec execute 344 sec latency 14.775 ms 2024-09-10T23:00:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:49 smithi120 ceph-mon[31247]: pgmap v598: 97 pgs: 97 active+clean; 33 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-10T23:00:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:49 smithi153 ceph-mon[34951]: pgmap v598: 97 pgs: 97 active+clean; 33 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-10T23:00:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:49 smithi176 ceph-mon[34982]: pgmap v598: 97 pgs: 97 active+clean; 33 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 3.6 MiB/s wr, 405 op/s 2024-09-10T23:00:50.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1424546 16.32 MB/sec execute 345 sec latency 27.816 ms 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 -- 172.21.15.120:0/1080892178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2708103b00 msgr2=0x7f2708103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 --2- 172.21.15.120:0/1080892178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2708103b00 0x7f2708103f80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009930 tx=0x7f26f402a420 comp rx=0 tx=0).stop 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 -- 172.21.15.120:0/1080892178 shutdown_connections 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 --2- 172.21.15.120:0/1080892178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081044c0 0x7f270810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 --2- 172.21.15.120:0/1080892178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2708103b00 0x7f2708103f80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 --2- 172.21.15.120:0/1080892178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27081029b0 0x7f2708102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 -- 172.21.15.120:0/1080892178 >> 172.21.15.120:0/1080892178 conn(0x7f27080fe270 msgr2=0x7f2708100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 -- 172.21.15.120:0/1080892178 shutdown_connections 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.213+0000 7f270ec85640 1 -- 172.21.15.120:0/1080892178 wait complete. 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 Processor -- start 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 -- start start 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081029b0 0x7f2708198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2708103b00 0x7f2708199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f270810e600 con 0x7f27081044c0 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f270810e480 con 0x7f27081029b0 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ec85640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f270810e780 con 0x7f2708103b00 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56186/0 (socket says 172.21.15.120:56186) 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 -- 172.21.15.120:0/1273268864 learned_addr learned my addr 172.21.15.120:0/1273268864 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f2707fff640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2708103b00 0x7f2708199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270ca21640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081029b0 0x7f2708198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 -- 172.21.15.120:0/1273268864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2708103b00 msgr2=0x7f2708199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2708103b00 0x7f2708199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 -- 172.21.15.120:0/1273268864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081029b0 msgr2=0x7f2708198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081029b0 0x7f2708198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 -- 172.21.15.120:0/1273268864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270819a8d0 con 0x7f27081044c0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f270d222640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f270000eb10 tx=0x7f270000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.214+0000 7f2705ffb640 1 -- 172.21.15.120:0/1273268864 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700004b20 con 0x7f27081044c0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.215+0000 7f2705ffb640 1 -- 172.21.15.120:0/1273268864 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2700004cc0 con 0x7f27081044c0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.215+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270819abc0 con 0x7f27081044c0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.215+0000 7f2705ffb640 1 -- 172.21.15.120:0/1273268864 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700010670 con 0x7f27081044c0 2024-09-10T23:00:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.215+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27081a1290 con 0x7f27081044c0 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.217+0000 7f2705ffb640 1 -- 172.21.15.120:0/1273268864 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2700005020 con 0x7f27081044c0 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.217+0000 7f2711a58740 1 Processor -- start 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.217+0000 7f2711a58740 1 -- start start 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.217+0000 7f2711a58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.217+0000 7f2711a58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2711a58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2711a58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5615062116f0 con 0x5615062a4bd0 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2711a58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561506200b20 con 0x56150629f860 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2711a58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561506204930 con 0x5615062a2190 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59384/0 (socket says 172.21.15.120:59384) 2024-09-10T23:00:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 -- 172.21.15.120:0/3727003315 learned_addr learned my addr 172.21.15.120:0/3727003315 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2707fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2707fff640 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2705ffb640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 0x7f26f80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 -- 172.21.15.120:0/3727003315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 msgr2=0x5615062a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 -- 172.21.15.120:0/3727003315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 msgr2=0x5615062a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 -- 172.21.15.120:0/3727003315 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615061c6530 con 0x5615062a2190 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270d222640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 0x7f26f80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f270ca21640 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00bd10 tx=0x7f26fc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef1f0b0601a97f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f2705ffb640 1 -- 172.21.15.120:0/1273268864 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f27000e6180 con 0x7f27081044c0 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f26f2ffd640 1 -- 172.21.15.120:0/3727003315 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc015070 con 0x5615062a2190 2024-09-10T23:00:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f26f2ffd640 1 -- 172.21.15.120:0/3727003315 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc01a910 con 0x5615062a2190 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.218+0000 7f26f2ffd640 1 -- 172.21.15.120:0/3727003315 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0119a0 con 0x5615062a2190 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- 172.21.15.120:0/3727003315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 msgr2=0x5615062a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f270d222640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 0x7f26f80b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f270008aea0 tx=0x7f27000a90a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00bd10 tx=0x7f26fc00f730 comp rx=0 tx=0).stop 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- 172.21.15.120:0/3727003315 shutdown_connections 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062a4580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 --2- 172.21.15.120:0/3727003315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- 172.21.15.120:0/3727003315 >> 172.21.15.120:0/3727003315 conn(0x56150629a9b0 msgr2=0x56150629b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- 172.21.15.120:0/3727003315 shutdown_connections 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- 172.21.15.120:0/3727003315 wait complete. 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.219+0000 7f2711a58740 1 -- start start 2024-09-10T23:00:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 Processor -- start 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a2190 0x5615062a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a4bd0 0x5615062aa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561506138ab0 con 0x56150629f860 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5615061a6730 con 0x5615062a2190 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5615062a7bb0 con 0x5615062a4bd0 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2711a58740 10 client.0 _ll_register_callbacks cb 0x5615061fa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56196/0 (socket says 172.21.15.120:56196) 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 -- 172.21.15.120:0/1353152265 learned_addr learned my addr 172.21.15.120:0/1353152265 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a4bd0 0x5615062aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f270ca21640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a2190 0x5615062a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 -- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a4bd0 msgr2=0x5615062aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a4bd0 0x5615062aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 -- 172.21.15.120:0/1353152265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a2190 msgr2=0x5615062a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a2190 0x5615062a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.220+0000 7f2707fff640 1 -- 172.21.15.120:0/1353152265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561505f62790 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2707fff640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f26f40096c0 tx=0x7f26f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23578b2e8387ae7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4035040 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f26f4004310 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc01a910 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26f40035f0 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 10 client.17034 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561505fdb620 con 0x56150629f860 2024-09-10T23:00:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc01a910 con 0x56150629f860 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f26f403d4e0 con 0x56150629f860 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 10 client.17034 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 20 client.17034 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x5615062b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 4 client.17034 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.221+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561506213640 con 0x5615062b4730 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.222+0000 7f270ca21640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x5615062b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.222+0000 7f270ca21640 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x5615062b6b40 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6ba1e32af65032 server_cookie=44c030843862df46 in_seq=0 out_seq=0 2024-09-10T23:00:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.222+0000 7f26e7fff640 10 client.17034 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f26e7fff640 1 -- 172.21.15.120:0/1353152265 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561506213640 con 0x5615062b4730 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f26e7fff640 10 client.17034 handle_command_reply: tid=0 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 client.17034 shutdown 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 msgr2=0x5615062a8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f26f40096c0 tx=0x7f26f4005dd0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 shutdown_connections 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x5615062b6b40 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a4bd0 0x5615062aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f26e7fff640 0 client.17034 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a2190 0x5615062a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 --2- 172.21.15.120:0/1353152265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56150629f860 0x5615062a8ef0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 >> 172.21.15.120:0/1353152265 conn(0x56150629a9b0 msgr2=0x561506299bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 shutdown_connections 2024-09-10T23:00:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 1 -- 172.21.15.120:0/1353152265 wait complete. 2024-09-10T23:00:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.223+0000 7f2711a58740 20 client.17034 trim_cache size 0 max 16384 2024-09-10T23:00:51.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 Processor -- start 2024-09-10T23:00:51.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 -- start start 2024-09-10T23:00:51.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56150629f860 0x56150629b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 0x5615062bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27081a7e00 con 0x5615062a2190 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27081aa680 con 0x5615062a4bd0 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2711a58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27081a8470 con 0x56150629f860 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56206/0 (socket says 172.21.15.120:56206) 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 -- 172.21.15.120:0/3046546287 learned_addr learned my addr 172.21.15.120:0/3046546287 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2707fff640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 0x5615062bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2707fff640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56150629f860 0x56150629b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 -- 172.21.15.120:0/3046546287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56150629f860 msgr2=0x56150629b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56150629f860 0x56150629b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 -- 172.21.15.120:0/3046546287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 msgr2=0x5615062bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 0x5615062bb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 -- 172.21.15.120:0/3046546287 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615062a1fc0 con 0x5615062a2190 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f2707fff640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 0x5615062bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.238+0000 7f270ca21640 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f26fc019000 tx=0x7f26fc00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1ad58b1b08ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f26f27fc640 1 -- 172.21.15.120:0/3046546287 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc015040 con 0x5615062a2190 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f26f27fc640 1 -- 172.21.15.120:0/3046546287 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc005500 con 0x5615062a2190 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f26f27fc640 1 -- 172.21.15.120:0/3046546287 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004bf0 con 0x5615062a2190 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 -- 172.21.15.120:0/3046546287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 msgr2=0x5615062bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f26fc019000 tx=0x7f26fc00ff70 comp rx=0 tx=0).stop 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 -- 172.21.15.120:0/3046546287 shutdown_connections 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5615062a4bd0 0x5615062bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a2190 0x5615062bb440 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 --2- 172.21.15.120:0/3046546287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56150629f860 0x56150629b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 -- 172.21.15.120:0/3046546287 >> 172.21.15.120:0/3046546287 conn(0x56150629a9b0 msgr2=0x56150629da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 -- 172.21.15.120:0/3046546287 shutdown_connections 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.239+0000 7f2711a58740 1 -- 172.21.15.120:0/3046546287 wait complete. 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 -- start start 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 Processor -- start 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27081a8b40 con 0x5615062a4bd0 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f27081a9fb0 con 0x56150629f860 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f27081a9210 con 0x5615062a2190 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2711a58740 10 client.0 _ll_register_callbacks cb 0x7f26f40316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:51.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f270ca21640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2707fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f270ca21640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59416/0 (socket says 172.21.15.120:59416) 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f270ca21640 1 -- 172.21.15.120:0/1510654465 learned_addr learned my addr 172.21.15.120:0/1510654465 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.240+0000 7f2707fff640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 -- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 msgr2=0x5615062b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 -- 172.21.15.120:0/1510654465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 msgr2=0x5615062b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 -- 172.21.15.120:0/1510654465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615062af1e0 con 0x5615062a4bd0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f270ca21640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2707fff640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062aeb70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f26f4031a90 tx=0x7f26f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c53b0c583cecb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:51.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4035040 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f26f403d660 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26fc005500 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26f403dad0 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 10 client.17040 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561506321fa0 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc005500 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f26f4064430 con 0x5615062a4bd0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 10 client.17040 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 20 client.17040 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x561506324990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 4 client.17040 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.241+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56150620f0f0 con 0x5615062b4730 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.242+0000 7f270ca21640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x561506324990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:51.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.242+0000 7f26e7fff640 10 client.17040 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:51.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.242+0000 7f270ca21640 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x561506324990 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769de61196613779 server_cookie=528f3bf429168bd0 in_seq=0 out_seq=0 2024-09-10T23:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.553+0000 7f26e7fff640 1 -- 172.21.15.120:0/1510654465 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56150620f0f0 con 0x5615062b4730 2024-09-10T23:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.553+0000 7f26e7fff640 10 client.17040 handle_command_reply: tid=0 2024-09-10T23:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.553+0000 7f2711a58740 1 client.17040 shutdown 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.553+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 msgr2=0x5615062aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.553+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062aeb70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f26f4031a90 tx=0x7f26f4005dd0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 shutdown_connections 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5615062b4730 0x561506324990 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5615062a4bd0 0x5615062aeb70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5615062a2190 0x5615062b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f26e7fff640 0 client.17040 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 --2- 172.21.15.120:0/1510654465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56150629f860 0x5615062b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 >> 172.21.15.120:0/1510654465 conn(0x56150629a9b0 msgr2=0x56150629f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 shutdown_connections 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 1 -- 172.21.15.120:0/1510654465 wait complete. 2024-09-10T23:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.554+0000 7f2711a58740 20 client.17040 trim_cache size 0 max 16384 2024-09-10T23:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 msgr2=0x7f26f80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 0x7f26f80b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f270008aea0 tx=0x7f27000a90a0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 msgr2=0x7f270819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f270000eb10 tx=0x7f270000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 shutdown_connections 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f26f80b52e0 0x7f26f80b77a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27081044c0 0x7f270819a010 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2708103b00 0x7f2708199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 --2- 172.21.15.120:0/1273268864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27081029b0 0x7f2708198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 >> 172.21.15.120:0/1273268864 conn(0x7f27080fe270 msgr2=0x7f2708106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 shutdown_connections 2024-09-10T23:00:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:00:51.555+0000 7f270ec85640 1 -- 172.21.15.120:0/1273268864 wait complete. 2024-09-10T23:00:51.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1426132 16.30 MB/sec execute 346 sec latency 150.871 ms 2024-09-10T23:00:51.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:51 smithi120 ceph-mon[31247]: pgmap v599: 97 pgs: 97 active+clean; 32 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-09-10T23:00:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:51 smithi153 ceph-mon[34951]: pgmap v599: 97 pgs: 97 active+clean; 32 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-09-10T23:00:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:51 smithi176 ceph-mon[34982]: pgmap v599: 97 pgs: 97 active+clean; 32 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-09-10T23:00:52.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1428741 16.30 MB/sec execute 347 sec latency 84.913 ms 2024-09-10T23:00:53.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1432041 16.30 MB/sec execute 348 sec latency 39.981 ms 2024-09-10T23:00:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:53 smithi120 ceph-mon[31247]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 3.7 MiB/s wr, 333 op/s 2024-09-10T23:00:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:53 smithi153 ceph-mon[34951]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 3.7 MiB/s wr, 333 op/s 2024-09-10T23:00:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:53 smithi176 ceph-mon[34982]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 3.7 MiB/s wr, 333 op/s 2024-09-10T23:00:54.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1435185 16.30 MB/sec execute 349 sec latency 21.568 ms 2024-09-10T23:00:55.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1438289 16.30 MB/sec execute 350 sec latency 18.075 ms 2024-09-10T23:00:55.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:55 smithi120 ceph-mon[31247]: pgmap v601: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-10T23:00:55.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:55 smithi153 ceph-mon[34951]: pgmap v601: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-10T23:00:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:55 smithi176 ceph-mon[34982]: pgmap v601: 97 pgs: 97 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-10T23:00:56.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1441461 16.30 MB/sec execute 351 sec latency 12.307 ms 2024-09-10T23:00:57.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1443904 16.30 MB/sec execute 352 sec latency 73.256 ms 2024-09-10T23:00:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:57 smithi120 ceph-mon[31247]: pgmap v602: 97 pgs: 97 active+clean; 24 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 468 op/s 2024-09-10T23:00:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:57 smithi153 ceph-mon[34951]: pgmap v602: 97 pgs: 97 active+clean; 24 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 468 op/s 2024-09-10T23:00:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:57 smithi176 ceph-mon[34982]: pgmap v602: 97 pgs: 97 active+clean; 24 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 468 op/s 2024-09-10T23:00:58.005 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:00:58.280 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:58.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1446705 16.29 MB/sec execute 353 sec latency 49.638 ms 2024-09-10T23:00:58.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 -- 172.21.15.120:0/456155085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101c90 msgr2=0x7f73d4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 --2- 172.21.15.120:0/456155085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101c90 0x7f73d4102070 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009900 tx=0x7f73c402a410 comp rx=0 tx=0).stop 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 -- 172.21.15.120:0/456155085 shutdown_connections 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 --2- 172.21.15.120:0/456155085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4108020 0x7f73d410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 --2- 172.21.15.120:0/456155085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d41076a0 0x7f73d4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 --2- 172.21.15.120:0/456155085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d4101c90 0x7f73d4102070 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 -- 172.21.15.120:0/456155085 >> 172.21.15.120:0/456155085 conn(0x7f73d40fd700 msgr2=0x7f73d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 -- 172.21.15.120:0/456155085 shutdown_connections 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.813+0000 7f73d8a71640 1 -- 172.21.15.120:0/456155085 wait complete. 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 Processor -- start 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 -- start start 2024-09-10T23:00:58.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4101c90 0x7f73d4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d4108020 0x7f73d419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111ce0 con 0x7f73d41076a0 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111b60 con 0x7f73d4101c90 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73d8a71640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4111e60 con 0x7f73d4108020 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51098/0 (socket says 172.21.15.120:51098) 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.814+0000 7f73cbfff640 1 -- 172.21.15.120:0/845793436 learned_addr learned my addr 172.21.15.120:0/845793436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d4108020 0x7f73d419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73d259c640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4101c90 0x7f73d4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 -- 172.21.15.120:0/845793436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d4108020 msgr2=0x7f73d419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d4108020 0x7f73d419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 -- 172.21.15.120:0/845793436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4101c90 msgr2=0x7f73d4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4101c90 0x7f73d4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 -- 172.21.15.120:0/845793436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d419f3b0 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cbfff640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cb7fe640 1 -- 172.21.15.120:0/845793436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00cd80 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d419f6a0 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cb7fe640 1 -- 172.21.15.120:0/845793436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc0054d0 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73cb7fe640 1 -- 172.21.15.120:0/845793436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc010690 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.815+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a5d60 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.818+0000 7f73cb7fe640 1 -- 172.21.15.120:0/845793436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f73bc004e90 con 0x7f73d41076a0 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.818+0000 7f73dae46740 1 Processor -- start 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.818+0000 7f73dae46740 1 -- start start 2024-09-10T23:00:58.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dd5950 0x560180dd7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560180d3fe50 con 0x560180dc99c0 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560180d2f280 con 0x560180dd5950 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73dae46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560180d33090 con 0x560180dc90a0 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38334/0 (socket says 172.21.15.120:38334) 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51100/0 (socket says 172.21.15.120:51100) 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 -- 172.21.15.120:0/1975232757 learned_addr learned my addr 172.21.15.120:0/1975232757 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73cbfff640 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dd5950 0x560180dd7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73cb7fe640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 0x7f73c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 -- 172.21.15.120:0/1975232757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dd5950 msgr2=0x560180dd7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dd5950 0x560180dd7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 -- 172.21.15.120:0/1975232757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 msgr2=0x560180dd5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 -- 172.21.15.120:0/1975232757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560180a8d650 con 0x560180dc90a0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d259c640 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73cb7fe640 1 -- 172.21.15.120:0/845793436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f73bc0e5fc0 con 0x7f73d41076a0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.819+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 0x7f73c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.820+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007ec0 tx=0x7f73cc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a478e7e0e99d9e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.820+0000 7f73c8ff9640 1 -- 172.21.15.120:0/1975232757 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc010ac0 con 0x560180dc90a0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.820+0000 7f73c8ff9640 1 -- 172.21.15.120:0/1975232757 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc01add0 con 0x560180dc90a0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.820+0000 7f73c8ff9640 1 -- 172.21.15.120:0/1975232757 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc019b50 con 0x560180dc90a0 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- 172.21.15.120:0/1975232757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 msgr2=0x560180dc9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 0x7f73c00b77a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f73cc01e000 tx=0x7f73cc01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007ec0 tx=0x7f73cc00f790 comp rx=0 tx=0).stop 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- 172.21.15.120:0/1975232757 shutdown_connections 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dd5950 0x560180dd7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 --2- 172.21.15.120:0/1975232757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc90a0 0x560180dc9480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- 172.21.15.120:0/1975232757 >> 172.21.15.120:0/1975232757 conn(0x560180dc6ed0 msgr2=0x560180dc8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- 172.21.15.120:0/1975232757 shutdown_connections 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- 172.21.15.120:0/1975232757 wait complete. 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 -- start start 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.821+0000 7f73dae46740 1 Processor -- start 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180dd5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dd5950 0x560180dd2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.844 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560180b90700 con 0x560180dc99c0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560180cca110 con 0x560180dc90a0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560180dd85d0 con 0x560180dd5950 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 10 client.0 _ll_register_callbacks cb 0x560180ce3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51116/0 (socket says 172.21.15.120:51116) 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73d259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180dd5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 -- 172.21.15.120:0/2592260909 learned_addr learned my addr 172.21.15.120:0/2592260909 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dd5950 0x560180dd2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 -- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dd5950 msgr2=0x560180dd2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dd5950 0x560180dd2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 -- 172.21.15.120:0/2592260909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 msgr2=0x560180dd5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.845 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180dd5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 -- 172.21.15.120:0/2592260909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560180cf7c40 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0a9ac0 tx=0x7f73bc0e9660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8fb25be7ed265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0a86f0 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73cc01add0 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f73bc0e80d0 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73bc0abfc0 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 10 client.17052 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.822+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560180de3f90 con 0x560180dc99c0 2024-09-10T23:00:58.846 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc01add0 con 0x560180dc99c0 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f73bc0b5950 con 0x560180dc99c0 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 10 client.17052 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 20 client.17052 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180de7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 4 client.17052 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560180d41da0 con 0x560180de53a0 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73d259c640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180de7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73b9ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.823+0000 7f73d259c640 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180de7790 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44142434fc5ae2c0 server_cookie=eab75f7faee5263 in_seq=0 out_seq=0 2024-09-10T23:00:58.847 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73b9ffb640 1 -- 172.21.15.120:0/2592260909 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560180d41da0 con 0x560180de53a0 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73b9ffb640 10 client.17052 handle_command_reply: tid=0 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 client.17052 shutdown 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 msgr2=0x560180dd2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0a9ac0 tx=0x7f73bc0e9660 comp rx=0 tx=0).stop 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 shutdown_connections 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180de7790 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73b9ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dd5950 0x560180dd2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dc99c0 0x560180dd2670 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 --2- 172.21.15.120:0/2592260909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180dd5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 >> 172.21.15.120:0/2592260909 conn(0x560180dc6ed0 msgr2=0x560180dcca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.848 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 shutdown_connections 2024-09-10T23:00:58.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.824+0000 7f73dae46740 1 -- 172.21.15.120:0/2592260909 wait complete. 2024-09-10T23:00:58.849 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.825+0000 7f73dae46740 20 client.17052 trim_cache size 0 max 16384 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.844+0000 7f73dae46740 1 Processor -- start 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.844+0000 7f73dae46740 1 -- start start 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180de1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dc7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dc8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41ac300 con 0x560180dd5950 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41aeb80 con 0x560180dc90a0 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73dae46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41ac970 con 0x560180dc99c0 2024-09-10T23:00:58.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73d2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dc8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dc7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73d2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dc8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51132/0 (socket says 172.21.15.120:51132) 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73d2d9d640 1 -- 172.21.15.120:0/2344448158 learned_addr learned my addr 172.21.15.120:0/2344448158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73d259c640 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180de1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 -- 172.21.15.120:0/2344448158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 msgr2=0x560180de1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180de1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 -- 172.21.15.120:0/2344448158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 msgr2=0x560180dc8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dc8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.845+0000 7f73cbfff640 1 -- 172.21.15.120:0/2344448158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560180dd0b70 con 0x560180dc99c0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73cbfff640 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dc7f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9c0 tx=0x7f73bc0e63c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ccf25fa14a172 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73bbfff640 1 -- 172.21.15.120:0/2344448158 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0ac1a0 con 0x560180dc99c0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73bbfff640 1 -- 172.21.15.120:0/2344448158 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc0ac360 con 0x560180dc99c0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73bbfff640 1 -- 172.21.15.120:0/2344448158 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0ac620 con 0x560180dc99c0 2024-09-10T23:00:58.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 -- 172.21.15.120:0/2344448158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 msgr2=0x560180dc7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dc7f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9c0 tx=0x7f73bc0e63c0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 -- 172.21.15.120:0/2344448158 shutdown_connections 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dc8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dc7f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 --2- 172.21.15.120:0/2344448158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180de1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.846+0000 7f73dae46740 1 -- 172.21.15.120:0/2344448158 >> 172.21.15.120:0/2344448158 conn(0x560180dc6ed0 msgr2=0x560180dcd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- 172.21.15.120:0/2344448158 shutdown_connections 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- 172.21.15.120:0/2344448158 wait complete. 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- start start 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 Processor -- start 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dd8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41ad040 con 0x560180dd5950 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41ae4b0 con 0x560180dc90a0 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41ad710 con 0x560180dc99c0 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73dae46740 10 client.0 _ll_register_callbacks cb 0x7f73bc0a6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73d259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73d259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37970/0 (socket says 172.21.15.120:37970) 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38358/0 (socket says 172.21.15.120:38358) 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73d259c640 1 -- 172.21.15.120:0/3710590409 learned_addr learned my addr 172.21.15.120:0/3710590409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:00:58.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.847+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dd8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 -- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 msgr2=0x560180dd8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 -- 172.21.15.120:0/3710590409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 msgr2=0x560180dd8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dd8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 -- 172.21.15.120:0/3710590409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560180dd9450 con 0x560180dc90a0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73cbfff640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d2d9d640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dd8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73d259c640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005ab0 tx=0x7f73c402aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5f73176b7da03b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4035040 con 0x560180dc90a0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f73c4005510 con 0x560180dc90a0 2024-09-10T23:00:58.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73bc0ac360 con 0x560180dc90a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73c40046c0 con 0x560180dc90a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73dae46740 10 client.26441 fetch_fsmap learned FSMap version 26 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560180e531a0 con 0x560180dc90a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.848+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc0ac360 con 0x560180dc90a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f73bc0051b0 con 0x560180dc90a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 10 client.26441 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 20 client.26441 populate_metadata read hostname 'smithi120' 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180e55990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 4 client.26441 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560180d3d850 con 0x560180de53a0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73cbfff640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180e55990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73b9ffb640 10 client.26441 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:58.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.849+0000 7f73cbfff640 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180e55990 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9cf17e80f1e704 server_cookie=72531b0d92df77ea in_seq=0 out_seq=0 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73b9ffb640 1 -- 172.21.15.120:0/3710590409 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x560180d3d850 con 0x560180de53a0 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73b9ffb640 10 client.26441 handle_command_reply: tid=0 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 client.26441 shutdown 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 msgr2=0x560180ddded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005ab0 tx=0x7f73c402aa90 comp rx=0 tx=0).stop 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 shutdown_connections 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560180de53a0 0x560180e55990 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560180dd5950 0x560180dd8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73b9ffb640 0 client.26441 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560180dc99c0 0x560180dd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 --2- 172.21.15.120:0/3710590409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560180dc90a0 0x560180ddded0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 >> 172.21.15.120:0/3710590409 conn(0x560180dc6ed0 msgr2=0x560180dc7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 shutdown_connections 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 1 -- 172.21.15.120:0/3710590409 wait complete. 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.851+0000 7f73dae46740 20 client.26441 trim_cache size 0 max 16384 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:00:58.861 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.862 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.863 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.864 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:00:58.865 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.867 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.868 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.869 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:00:58.870 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 msgr2=0x7f73c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 0x7f73c00b77a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f73cc01e000 tx=0x7f73cc01f000 comp rx=0 tx=0).stop 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 msgr2=0x7f73d4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00edf0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 shutdown_connections 2024-09-10T23:00:58.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73c00b52e0 0x7f73c00b77a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d4108020 0x7f73d419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73d41076a0 0x7f73d4199840 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 --2- 172.21.15.120:0/845793436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73d4101c90 0x7f73d4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 >> 172.21.15.120:0/845793436 conn(0x7f73d40fd700 msgr2=0x7f73d406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 shutdown_connections 2024-09-10T23:00:58.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:00:58.852+0000 7f73d8a71640 1 -- 172.21.15.120:0/845793436 wait complete. 2024-09-10T23:00:59.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1450132 16.30 MB/sec execute 354 sec latency 12.451 ms 2024-09-10T23:00:59.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:00:59 smithi120 ceph-mon[31247]: pgmap v603: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-10T23:00:59.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:00:59 smithi153 ceph-mon[34951]: pgmap v603: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-10T23:00:59.820 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:00:59.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:00:59.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:00:59 smithi176 ceph-mon[34982]: pgmap v603: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-10T23:01:00.110 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:00.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1453359 16.31 MB/sec execute 355 sec latency 26.154 ms 2024-09-10T23:01:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- 172.21.15.120:0/1364664705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 msgr2=0x7f3778102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 --2- 172.21.15.120:0/1364664705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f3778102070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f376c009900 tx=0x7f376c02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- 172.21.15.120:0/1364664705 shutdown_connections 2024-09-10T23:01:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 --2- 172.21.15.120:0/1364664705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f377810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 --2- 172.21.15.120:0/1364664705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 0x7f3778107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 --2- 172.21.15.120:0/1364664705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f3778102070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- 172.21.15.120:0/1364664705 >> 172.21.15.120:0/1364664705 conn(0x7f37780fd700 msgr2=0x7f37780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- 172.21.15.120:0/1364664705 shutdown_connections 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- 172.21.15.120:0/1364664705 wait complete. 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 Processor -- start 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.627+0000 7f377f1da640 1 -- start start 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f377819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 0x7f377819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f37781a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3778111ce0 con 0x7f3778108020 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3778111b60 con 0x7f37781076a0 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3778111e60 con 0x7f3778101c90 2024-09-10T23:01:00.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f37781a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377cf76640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f377819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377cf76640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f377819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38376/0 (socket says 172.21.15.120:38376) 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377cf76640 1 -- 172.21.15.120:0/1188979555 learned_addr learned my addr 172.21.15.120:0/1188979555 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f3777fff640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 0x7f377819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 -- 172.21.15.120:0/1188979555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 msgr2=0x7f377819b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f377819b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 -- 172.21.15.120:0/1188979555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 msgr2=0x7f377819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 0x7f377819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 -- 172.21.15.120:0/1188979555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37781a1670 con 0x7f3778108020 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377d777640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f37781a0ea0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f377000da10 tx=0x7f377000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377000bd70 con 0x7f3778108020 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37781a1960 con 0x7f3778108020 2024-09-10T23:01:00.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3770003e20 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.628+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700106c0 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.629+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781a8020 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.630+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778102070 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.632+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3770024060 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.632+0000 7f3775ffb640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 0x7f37680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.632+0000 7f377cf76640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 0x7f37680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.632+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f37700e5ba0 con 0x7f3778108020 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.633+0000 7f377cf76640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 0x7f37680b77a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f376c031440 tx=0x7f376c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:00.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.636+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f37700aca90 con 0x7f3778108020 2024-09-10T23:01:00.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.867+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37781a8340 con 0x7f3778108020 2024-09-10T23:01:00.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.868+0000 7f3775ffb640 1 -- 172.21.15.120:0/1188979555 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f37700ac390 con 0x7f3778108020 2024-09-10T23:01:00.870 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:01:00.870 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:01:00.871 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:01:00.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 msgr2=0x7f37680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:00.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 0x7f37680b77a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f376c031440 tx=0x7f376c0058d0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 msgr2=0x7f37781a0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f37781a0ea0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f377000da10 tx=0x7f377000ded0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 shutdown_connections 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f37680b52e0 0x7f37680b77a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3778108020 0x7f37781a0ea0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37781076a0 0x7f377819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 --2- 172.21.15.120:0/1188979555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3778101c90 0x7f377819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 >> 172.21.15.120:0/1188979555 conn(0x7f37780fd700 msgr2=0x7f377806a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 shutdown_connections 2024-09-10T23:01:00.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:00.870+0000 7f377f1da640 1 -- 172.21.15.120:0/1188979555 wait complete. 2024-09-10T23:01:01.501 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:01 smithi153 ceph-mon[34951]: pgmap v604: 97 pgs: 97 active+clean; 30 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 473 op/s 2024-09-10T23:01:01.501 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:01 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1188979555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:01.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:01 smithi120 ceph-mon[31247]: pgmap v604: 97 pgs: 97 active+clean; 30 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 473 op/s 2024-09-10T23:01:01.540 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:01 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1188979555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:01.621 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:01:01.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1456725 16.31 MB/sec execute 356 sec latency 13.303 ms 2024-09-10T23:01:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:01 smithi176 ceph-mon[34982]: pgmap v604: 97 pgs: 97 active+clean; 30 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 473 op/s 2024-09-10T23:01:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:01 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1188979555' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:01.889 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 -- 172.21.15.120:0/981133405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c10108020 msgr2=0x7f1c1010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 --2- 172.21.15.120:0/981133405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c10108020 0x7f1c1010e820 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009ca0 tx=0x7f1c0802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 -- 172.21.15.120:0/981133405 shutdown_connections 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 --2- 172.21.15.120:0/981133405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c10108020 0x7f1c1010e820 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 --2- 172.21.15.120:0/981133405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c101076a0 0x7f1c10107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 --2- 172.21.15.120:0/981133405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c10102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.403+0000 7f1c17292640 1 -- 172.21.15.120:0/981133405 >> 172.21.15.120:0/981133405 conn(0x7f1c100fd700 msgr2=0x7f1c100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- 172.21.15.120:0/981133405 shutdown_connections 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- 172.21.15.120:0/981133405 wait complete. 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 Processor -- start 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- start start 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c1019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 0x7f1c101a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10112d60 con 0x7f1c101076a0 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10112be0 con 0x7f1c10108020 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.404+0000 7f1c17292640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10112ee0 con 0x7f1c10101c90 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c1019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51184/0 (socket says 172.21.15.120:51184) 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c1019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38394/0 (socket says 172.21.15.120:38394) 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 -- 172.21.15.120:0/2244725580 learned_addr learned my addr 172.21.15.120:0/2244725580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1582f640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 0x7f1c101a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 -- 172.21.15.120:0/2244725580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 msgr2=0x7f1c1019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c1019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 -- 172.21.15.120:0/2244725580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 msgr2=0x7f1c101a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 0x7f1c101a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 -- 172.21.15.120:0/2244725580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c101a14e0 con 0x7f1c101076a0 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1582f640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 0x7f1c101a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c067fc640 1 -- 172.21.15.120:0/2244725580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000cd80 con 0x7f1c101076a0 2024-09-10T23:01:02.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c067fc640 1 -- 172.21.15.120:0/2244725580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000108e0 con 0x7f1c101076a0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a17d0 con 0x7f1c101076a0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c067fc640 1 -- 172.21.15.120:0/2244725580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00018980 con 0x7f1c101076a0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.405+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a7e90 con 0x7f1c101076a0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.407+0000 7f1c067fc640 1 -- 172.21.15.120:0/2244725580 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1c00010430 con 0x7f1c101076a0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 Processor -- start 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 -- start start 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c90e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c911170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56164c878ea0 con 0x56164c9024d0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56164c8682d0 con 0x56164c90ed80 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c19667740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56164c86c0e0 con 0x56164c902df0 2024-09-10T23:01:02.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c1582f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c90e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c911170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c1582f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51188/0 (socket says 172.21.15.120:51188) 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.408+0000 7f1c1582f640 1 -- 172.21.15.120:0/406927808 learned_addr learned my addr 172.21.15.120:0/406927808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c067fc640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 0x7f1bfc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 -- 172.21.15.120:0/406927808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 msgr2=0x56164c90e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c90e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 -- 172.21.15.120:0/406927808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 msgr2=0x56164c911170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c911170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 -- 172.21.15.120:0/406927808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c5c6650 con 0x56164c9024d0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c067fc640 1 -- 172.21.15.120:0/2244725580 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c000e5f40 con 0x7f1c101076a0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009390 tx=0x7f1c080093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb632f38ebd9a579 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1c1582f640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 0x7f1bfc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1bfb7fe640 1 -- 172.21.15.120:0/406927808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0804a040 con 0x56164c9024d0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1bfb7fe640 1 -- 172.21.15.120:0/406927808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08036040 con 0x56164c9024d0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.409+0000 7f1bfb7fe640 1 -- 172.21.15.120:0/406927808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004650 con 0x56164c9024d0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- 172.21.15.120:0/406927808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 msgr2=0x56164c9028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c1582f640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 0x7f1bfc0b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c08045000 tx=0x7f1c080048d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009390 tx=0x7f1c080093c0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- 172.21.15.120:0/406927808 shutdown_connections 2024-09-10T23:01:02.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c911170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c90e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 --2- 172.21.15.120:0/406927808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c9028b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- 172.21.15.120:0/406927808 >> 172.21.15.120:0/406927808 conn(0x56164c900300 msgr2=0x56164c9014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- 172.21.15.120:0/406927808 shutdown_connections 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- 172.21.15.120:0/406927808 wait complete. 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.410+0000 7f1c19667740 1 -- start start 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 Processor -- start 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c902df0 0x56164c90ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c90ed80 0x56164c90c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56164c6c9700 con 0x56164c9024d0 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56164c803160 con 0x56164c902df0 2024-09-10T23:01:02.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56164c9119a0 con 0x56164c90ed80 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c19667740 10 client.0 _ll_register_callbacks cb 0x56164c81c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51194/0 (socket says 172.21.15.120:51194) 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 -- 172.21.15.120:0/300387408 learned_addr learned my addr 172.21.15.120:0/300387408 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1582f640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c90ed80 0x56164c90c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1482d640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c902df0 0x56164c90ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 -- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c90ed80 msgr2=0x56164c90c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c90ed80 0x56164c90c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 -- 172.21.15.120:0/300387408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c902df0 msgr2=0x56164c90ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c902df0 0x56164c90ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.411+0000 7f1c1502e640 1 -- 172.21.15.120:0/300387408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c830c90 con 0x56164c9024d0 2024-09-10T23:01:02.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c1502e640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012700 tx=0x7f1c0c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e879600b51fd88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00f8a0 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c0c00fa40 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c08036040 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c00fc40 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 10 client.17076 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56164c91d330 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c08036040 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1c0c027420 con 0x56164c9024d0 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 10 client.17076 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 20 client.17076 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:02.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c91e840 0x56164c920cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 4 client.17076 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.412+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164c87adf0 con 0x56164c91e840 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.413+0000 7f1c1482d640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c91e840 0x56164c920cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.413+0000 7f1c1482d640 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c91e840 0x56164c920cb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7376124bcb7feca6 server_cookie=ad7312098580d25a in_seq=0 out_seq=0 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.413+0000 7f1bf8ff9640 10 client.17076 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.413+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/300387408 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56164c87adf0 con 0x56164c91e840 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.413+0000 7f1bf8ff9640 10 client.17076 handle_command_reply: tid=0 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 client.17076 shutdown 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 msgr2=0x56164c90e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012700 tx=0x7f1c0c012bc0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 shutdown_connections 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c91e840 0x56164c920cb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c90ed80 0x56164c90c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c902df0 0x56164c90ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 --2- 172.21.15.120:0/300387408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c9024d0 0x56164c90e640 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 >> 172.21.15.120:0/300387408 conn(0x56164c900300 msgr2=0x56164c905e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1bf8ff9640 0 client.17076 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 shutdown_connections 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 1 -- 172.21.15.120:0/300387408 wait complete. 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.414+0000 7f1c19667740 20 client.17076 trim_cache size 0 max 16384 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.427+0000 7f1c19667740 1 Processor -- start 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.427+0000 7f1c19667740 1 -- start start 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c9024d0 0x56164c91ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c9106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101ae4e0 con 0x56164c902df0 2024-09-10T23:01:02.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101b0d60 con 0x56164c90ed80 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c19667740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101aeb50 con 0x56164c9024d0 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51208/0 (socket says 172.21.15.120:51208) 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 -- 172.21.15.120:0/965434736 learned_addr learned my addr 172.21.15.120:0/965434736 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1502e640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c9024d0 0x56164c91ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1582f640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c9106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 -- 172.21.15.120:0/965434736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c9024d0 msgr2=0x56164c91ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c9024d0 0x56164c91ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 -- 172.21.15.120:0/965434736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 msgr2=0x56164c9106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c9106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 -- 172.21.15.120:0/965434736 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164c909f90 con 0x56164c902df0 2024-09-10T23:01:02.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1582f640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c9106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1c1482d640 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c6a0 tx=0x7f1c000e9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83655aec58d55e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1bfaffd640 1 -- 172.21.15.120:0/965434736 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000ac080 con 0x56164c902df0 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1bfaffd640 1 -- 172.21.15.120:0/965434736 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000ac240 con 0x56164c902df0 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.428+0000 7f1bfaffd640 1 -- 172.21.15.120:0/965434736 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000abea0 con 0x56164c902df0 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- 172.21.15.120:0/965434736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 msgr2=0x56164c910170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c6a0 tx=0x7f1c000e9a20 comp rx=0 tx=0).stop 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- 172.21.15.120:0/965434736 shutdown_connections 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c90ed80 0x56164c9106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c902df0 0x56164c910170 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 --2- 172.21.15.120:0/965434736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c9024d0 0x56164c91ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- 172.21.15.120:0/965434736 >> 172.21.15.120:0/965434736 conn(0x56164c900300 msgr2=0x56164c906cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- 172.21.15.120:0/965434736 shutdown_connections 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- 172.21.15.120:0/965434736 wait complete. 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 -- start start 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.429+0000 7f1c19667740 1 Processor -- start 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 0x56164c916c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c9171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af220 con 0x56164c90ed80 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101b0690 con 0x56164c9024d0 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af8f0 con 0x56164c902df0 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c19667740 10 client.0 _ll_register_callbacks cb 0x7f1c0c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1582f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 0x56164c916c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1482d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c9171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1582f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51210/0 (socket says 172.21.15.120:51210) 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1582f640 1 -- 172.21.15.120:0/2619768811 learned_addr learned my addr 172.21.15.120:0/2619768811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 -- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 msgr2=0x56164c9171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c9171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 -- 172.21.15.120:0/2619768811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 msgr2=0x56164c911c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 -- 172.21.15.120:0/2619768811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164c912160 con 0x56164c9024d0 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c9171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1582f640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1c1502e640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 0x56164c916c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c027910 tx=0x7f1c0c00fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22768ea8a1461cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:02.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.430+0000 7f1bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c027b00 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c0c027ca0 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c000ac240 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c036050 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 10 client.26465 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56164c98bee0 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000ac240 con 0x56164c9024d0 2024-09-10T23:01:02.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1c0c003740 con 0x56164c9024d0 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 10 client.26465 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 20 client.26465 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c98c570 0x56164c98e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 4 client.26465 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56164c8768a0 con 0x56164c98c570 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.431+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c98c570 0x56164c98e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.432+0000 7f1c1482d640 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c98c570 0x56164c98e990 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba6b0d66aaf9ec4 server_cookie=916111b4b220dc57 in_seq=0 out_seq=0 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.432+0000 7f1bf8ff9640 10 client.26465 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.432+0000 7f1bf8ff9640 1 -- 172.21.15.120:0/2619768811 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56164c8768a0 con 0x56164c98c570 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.432+0000 7f1bf8ff9640 10 client.26465 handle_command_reply: tid=0 2024-09-10T23:01:02.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.432+0000 7f1c19667740 1 client.26465 shutdown 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 msgr2=0x56164c916c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 0x56164c916c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c027910 tx=0x7f1c0c00fef0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 shutdown_connections 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56164c98c570 0x56164c98e990 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56164c90ed80 0x56164c911c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56164c902df0 0x56164c9171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 --2- 172.21.15.120:0/2619768811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56164c9024d0 0x56164c916c70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1bf8ff9640 0 client.26465 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 >> 172.21.15.120:0/2619768811 conn(0x56164c900300 msgr2=0x56164c90d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 shutdown_connections 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 1 -- 172.21.15.120:0/2619768811 wait complete. 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c19667740 20 client.26465 trim_cache size 0 max 16384 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 msgr2=0x7f1bfc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 0x7f1bfc0b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c08045000 tx=0x7f1c080048d0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 msgr2=0x7f1c1019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 shutdown_connections 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1bfc0b52e0 0x7f1bfc0b77a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c10108020 0x7f1c101a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c101076a0 0x7f1c1019b970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 --2- 172.21.15.120:0/2244725580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10101c90 0x7f1c1019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 >> 172.21.15.120:0/2244725580 conn(0x7f1c100fd700 msgr2=0x7f1c10103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.433+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 shutdown_connections 2024-09-10T23:01:02.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:02.434+0000 7f1c17292640 1 -- 172.21.15.120:0/2244725580 wait complete. 2024-09-10T23:01:02.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1458845 16.29 MB/sec execute 357 sec latency 62.928 ms 2024-09-10T23:01:03.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1461208 16.28 MB/sec execute 358 sec latency 23.802 ms 2024-09-10T23:01:03.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:03 smithi120 ceph-mon[31247]: pgmap v605: 97 pgs: 97 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T23:01:03.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:03 smithi153 ceph-mon[34951]: pgmap v605: 97 pgs: 97 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T23:01:03.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:03 smithi176 ceph-mon[34982]: pgmap v605: 97 pgs: 97 active+clean; 31 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 466 op/s 2024-09-10T23:01:04.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:01:04.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:01:04.475 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:04.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1464291 16.29 MB/sec execute 359 sec latency 7.869 ms 2024-09-10T23:01:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 -- 172.21.15.120:0/1893681623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060103370 msgr2=0x7f1060109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 --2- 172.21.15.120:0/1893681623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060103370 0x7f1060109cf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1058009cd0 tx=0x7f105802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 -- 172.21.15.120:0/1893681623 shutdown_connections 2024-09-10T23:01:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 --2- 172.21.15.120:0/1893681623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060103370 0x7f1060109cf0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 --2- 172.21.15.120:0/1893681623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 0x7f1060102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 --2- 172.21.15.120:0/1893681623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060101860 0x7f1060101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 -- 172.21.15.120:0/1893681623 >> 172.21.15.120:0/1893681623 conn(0x7f10600fd140 msgr2=0x7f10600ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 -- 172.21.15.120:0/1893681623 shutdown_connections 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.646+0000 7f10687ad640 1 -- 172.21.15.120:0/1893681623 wait complete. 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 Processor -- start 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 -- start start 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 0x7f1060199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060103370 0x7f1060199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f106010e600 con 0x7f1060101860 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f106010e480 con 0x7f1060103370 2024-09-10T23:01:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f10687ad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f106010e780 con 0x7f10601029b0 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45724/0 (socket says 172.21.15.120:45724) 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f1066549640 1 -- 172.21.15.120:0/747531580 learned_addr learned my addr 172.21.15.120:0/747531580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f1065d48640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 0x7f1060199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.647+0000 7f1066d4a640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060103370 0x7f1060199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 -- 172.21.15.120:0/747531580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 msgr2=0x7f1060199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 0x7f1060199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 -- 172.21.15.120:0/747531580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060103370 msgr2=0x7f1060199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060103370 0x7f1060199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 -- 172.21.15.120:0/747531580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019a840 con 0x7f1060101860 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f1066549640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c920 tx=0x7f105c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f10577fe640 1 -- 172.21.15.120:0/747531580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c007c70 con 0x7f1060101860 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106019ab30 con 0x7f1060101860 2024-09-10T23:01:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f10577fe640 1 -- 172.21.15.120:0/747531580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f105c007e10 con 0x7f1060101860 2024-09-10T23:01:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f10577fe640 1 -- 172.21.15.120:0/747531580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c018820 con 0x7f1060101860 2024-09-10T23:01:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.648+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a1200 con 0x7f1060101860 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.650+0000 7f10577fe640 1 -- 172.21.15.120:0/747531580 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f105c00f450 con 0x7f1060101860 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.650+0000 7f106b65c740 1 Processor -- start 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 -- start start 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc539d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc539f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbc53086f0 con 0x55fbc539d5a0 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fbc52f7b20 con 0x55fbc539ac70 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f106b65c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fbc52fb930 con 0x55fbc539ffe0 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc539f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc539f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45736/0 (socket says 172.21.15.120:45736) 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066549640 1 -- 172.21.15.120:0/351545412 learned_addr learned my addr 172.21.15.120:0/351545412 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc539d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1065d48640 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 -- 172.21.15.120:0/351545412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 msgr2=0x55fbc53a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f10577fe640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 0x7f104c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 -- 172.21.15.120:0/351545412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 msgr2=0x55fbc539f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc539f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 -- 172.21.15.120:0/351545412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc52bd530 con 0x55fbc539ac70 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f10577fe640 1 -- 172.21.15.120:0/747531580 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f105c0e5c70 con 0x7f1060101860 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.651+0000 7f1066d4a640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 0x7f104c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f1066d4a640 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc539d060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1058009ca0 tx=0x7f10580093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99204adfc9aafc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f1054ff9640 1 -- 172.21.15.120:0/351545412 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105804a070 con 0x55fbc539ac70 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f1054ff9640 1 -- 172.21.15.120:0/351545412 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1058036040 con 0x55fbc539ac70 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f1054ff9640 1 -- 172.21.15.120:0/351545412 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580305a0 con 0x55fbc539ac70 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f1066d4a640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 0x7f104c0b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1058006010 tx=0x7f1058030bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f106b65c740 1 -- 172.21.15.120:0/351545412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 msgr2=0x55fbc539d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.652+0000 7f106b65c740 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc539d060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1058009ca0 tx=0x7f10580093f0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- 172.21.15.120:0/351545412 shutdown_connections 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc539f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- 172.21.15.120:0/351545412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc539d060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- 172.21.15.120:0/351545412 >> 172.21.15.120:0/351545412 conn(0x55fbc53919b0 msgr2=0x55fbc5392740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- 172.21.15.120:0/351545412 shutdown_connections 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- 172.21.15.120:0/351545412 wait complete. 2024-09-10T23:01:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- start start 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 Processor -- start 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc53a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fbc522fab0 con 0x55fbc539d5a0 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fbc529d730 con 0x55fbc539ac70 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fbc53a2f60 con 0x55fbc539ffe0 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f106b65c740 10 client.0 _ll_register_callbacks cb 0x55fbc52f16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f1065d48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f1065d48640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45752/0 (socket says 172.21.15.120:45752) 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f1065d48640 1 -- 172.21.15.120:0/1121886466 learned_addr learned my addr 172.21.15.120:0/1121886466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.653+0000 7f1066d4a640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1066549640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc53a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 -- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 msgr2=0x55fbc53a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 -- 172.21.15.120:0/1121886466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 msgr2=0x55fbc53a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc53a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 -- 172.21.15.120:0/1121886466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc5059790 con 0x55fbc539d5a0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1066549640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc53a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1065d48640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1050012840 tx=0x7f1050012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e4ef33a697451a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105000f8c0 con 0x55fbc539d5a0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1058036040 con 0x55fbc539d5a0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f105000fa60 con 0x55fbc539d5a0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f105000fc60 con 0x55fbc539d5a0 2024-09-10T23:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f106b65c740 10 client.17094 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbc50d2620 con 0x55fbc539d5a0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1058036040 con 0x55fbc539d5a0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.654+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1050027420 con 0x55fbc539d5a0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 10 client.17094 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 20 client.17094 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc53afbf0 0x55fbc53b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 4 client.17094 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc530a640 con 0x55fbc53afbf0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f1066549640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc53afbf0 0x55fbc53b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f1066549640 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc53afbf0 0x55fbc53b2070 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85feca049d0907ca server_cookie=88a1906b2b9cd79f in_seq=0 out_seq=0 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.655+0000 7f1049ffb640 10 client.17094 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f1049ffb640 1 -- 172.21.15.120:0/1121886466 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fbc530a640 con 0x55fbc53afbf0 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f1049ffb640 10 client.17094 handle_command_reply: tid=0 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 client.17094 shutdown 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 msgr2=0x55fbc53a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1050012840 tx=0x7f1050012d00 comp rx=0 tx=0).stop 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 shutdown_connections 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc53afbf0 0x55fbc53b2070 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ffe0 0x55fbc53a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539d5a0 0x55fbc53a48e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f1049ffb640 0 client.17094 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 --2- 172.21.15.120:0/1121886466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ac70 0x55fbc53a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 >> 172.21.15.120:0/1121886466 conn(0x55fbc53919b0 msgr2=0x55fbc5390bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 shutdown_connections 2024-09-10T23:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 1 -- 172.21.15.120:0/1121886466 wait complete. 2024-09-10T23:01:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.656+0000 7f106b65c740 20 client.17094 trim_cache size 0 max 16384 2024-09-10T23:01:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.671+0000 7f106b65c740 1 Processor -- start 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.671+0000 7f106b65c740 1 -- start start 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ac70 0x55fbc5395cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539d5a0 0x55fbc5396200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a5b90 con 0x55fbc539ffe0 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a8410 con 0x55fbc539d5a0 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f106b65c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a6200 con 0x55fbc539ac70 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45768/0 (socket says 172.21.15.120:45768) 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 -- 172.21.15.120:0/2736103245 learned_addr learned my addr 172.21.15.120:0/2736103245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066549640 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ac70 0x55fbc5395cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 -- 172.21.15.120:0/2736103245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ac70 msgr2=0x55fbc5395cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ac70 0x55fbc5395cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1065d48640 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539d5a0 0x55fbc5396200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 -- 172.21.15.120:0/2736103245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539d5a0 msgr2=0x55fbc5396200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539d5a0 0x55fbc5396200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 -- 172.21.15.120:0/2736103245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc539d3d0 con 0x55fbc539ffe0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f1066d4a640 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1058009cd0 tx=0x7f1058050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a4f9e50f922909 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f104bfff640 1 -- 172.21.15.120:0/2736103245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105804a040 con 0x55fbc539ffe0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f104bfff640 1 -- 172.21.15.120:0/2736103245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1058055050 con 0x55fbc539ffe0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.672+0000 7f104bfff640 1 -- 172.21.15.120:0/2736103245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058004430 con 0x55fbc539ffe0 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- 172.21.15.120:0/2736103245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 msgr2=0x55fbc53a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1058009cd0 tx=0x7f1058050000 comp rx=0 tx=0).stop 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- 172.21.15.120:0/2736103245 shutdown_connections 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ffe0 0x55fbc53a03c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539d5a0 0x55fbc5396200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- 172.21.15.120:0/2736103245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539ac70 0x55fbc5395cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- 172.21.15.120:0/2736103245 >> 172.21.15.120:0/2736103245 conn(0x55fbc53919b0 msgr2=0x55fbc538caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- 172.21.15.120:0/2736103245 shutdown_connections 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- 172.21.15.120:0/2736103245 wait complete. 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- start start 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 Processor -- start 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539d5a0 0x55fbc53ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 0x55fbc53aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a68d0 con 0x55fbc539ac70 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a7d40 con 0x55fbc539ffe0 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a6fa0 con 0x55fbc539d5a0 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.673+0000 7f106b65c740 10 client.0 _ll_register_callbacks cb 0x7f1050009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45780/0 (socket says 172.21.15.120:45780) 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 -- 172.21.15.120:0/995292118 learned_addr learned my addr 172.21.15.120:0/995292118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1065d48640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539d5a0 0x55fbc53ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066d4a640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 0x55fbc53aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 -- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539d5a0 msgr2=0x55fbc53ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539d5a0 0x55fbc53ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 -- 172.21.15.120:0/995292118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 msgr2=0x55fbc53aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 0x55fbc53aa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 -- 172.21.15.120:0/995292118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc53ab010 con 0x55fbc539ac70 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066d4a640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 0x55fbc53aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1066549640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f105c00fea0 tx=0x7f105c0ac2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9641afc30c28261 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0b5330 con 0x55fbc539ac70 2024-09-10T23:01:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f105c0a86d0 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1058055050 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f105c0abf70 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f106b65c740 10 client.17100 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbc53a9f30 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.674+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1058055050 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f105c0ed040 con 0x55fbc539ac70 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 10 client.17100 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 20 client.17100 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc541d260 0x55fbc541f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 4 client.17100 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbc53060f0 con 0x55fbc541d260 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f1065d48640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc541d260 0x55fbc541f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f1065d48640 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc541d260 0x55fbc541f680 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d340dcdfc1c9e1ef server_cookie=f4386773f542102d in_seq=0 out_seq=0 2024-09-10T23:01:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.675+0000 7f1049ffb640 10 client.17100 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.958+0000 7f1049ffb640 1 -- 172.21.15.120:0/995292118 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbc53060f0 con 0x55fbc541d260 2024-09-10T23:01:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.958+0000 7f1049ffb640 10 client.17100 handle_command_reply: tid=0 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.958+0000 7f106b65c740 1 client.17100 shutdown 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 msgr2=0x55fbc53ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f105c00fea0 tx=0x7f105c0ac2f0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 shutdown_connections 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fbc541d260 0x55fbc541f680 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f1049ffb640 0 client.17100 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fbc539ffe0 0x55fbc53aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fbc539d5a0 0x55fbc53ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 --2- 172.21.15.120:0/995292118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fbc539ac70 0x55fbc53ab970 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 >> 172.21.15.120:0/995292118 conn(0x55fbc53919b0 msgr2=0x55fbc53a03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 shutdown_connections 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 1 -- 172.21.15.120:0/995292118 wait complete. 2024-09-10T23:01:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.959+0000 7f106b65c740 20 client.17100 trim_cache size 0 max 16384 2024-09-10T23:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 msgr2=0x7f104c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 0x7f104c0b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1058006010 tx=0x7f1058030bf0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 msgr2=0x7f1060198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c920 tx=0x7f105c00cde0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 shutdown_connections 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f104c0b52e0 0x7f104c0b77a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1060103370 0x7f1060199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f10601029b0 0x7f1060199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 --2- 172.21.15.120:0/747531580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1060101860 0x7f1060198d00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 >> 172.21.15.120:0/747531580 conn(0x7f10600fd140 msgr2=0x7f1060105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 shutdown_connections 2024-09-10T23:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:04.960+0000 7f10687ad640 1 -- 172.21.15.120:0/747531580 wait complete. 2024-09-10T23:01:04.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- 172.21.15.120:0/3575791159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 msgr2=0x7f42bc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:04.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 --2- 172.21.15.120:0/3575791159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc10f550 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009ca0 tx=0x7f42b402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- 172.21.15.120:0/3575791159 shutdown_connections 2024-09-10T23:01:04.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 --2- 172.21.15.120:0/3575791159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc10f550 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 --2- 172.21.15.120:0/3575791159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc104570 0x7f42bc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 --2- 172.21.15.120:0/3575791159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc103b70 0x7f42bc103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- 172.21.15.120:0/3575791159 >> 172.21.15.120:0/3575791159 conn(0x7f42bc0fd700 msgr2=0x7f42bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- 172.21.15.120:0/3575791159 shutdown_connections 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- 172.21.15.120:0/3575791159 wait complete. 2024-09-10T23:01:04.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 Processor -- start 2024-09-10T23:01:05.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.985+0000 7f42c366a640 1 -- start start 2024-09-10T23:01:05.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 0x7f42bc1019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc104570 0x7f42bc101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112d60 con 0x7f42bc10d120 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112be0 con 0x7f42bc103b70 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112ee0 con 0x7f42bc104570 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1406640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 0x7f42bc1019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc104570 0x7f42bc101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45790/0 (socket says 172.21.15.120:45790) 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1406640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 0x7f42bc1019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42494/0 (socket says 172.21.15.120:42494) 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 -- 172.21.15.120:0/3465862682 learned_addr learned my addr 172.21.15.120:0/3465862682 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 -- 172.21.15.120:0/3465862682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc104570 msgr2=0x7f42bc101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc104570 0x7f42bc101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 -- 172.21.15.120:0/3465862682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 msgr2=0x7f42bc1019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 0x7f42bc1019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 -- 172.21.15.120:0/3465862682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1a75f0 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c1c07640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f42b4005cc0 tx=0x7f42b402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42b27fc640 1 -- 172.21.15.120:0/3465862682 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4003bf0 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42b27fc640 1 -- 172.21.15.120:0/3465862682 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4003d90 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.986+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc1a7880 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.987+0000 7f42b27fc640 1 -- 172.21.15.120:0/3465862682 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b402fb20 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.987+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc1a7e10 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.989+0000 7f42b27fc640 1 -- 172.21.15.120:0/3465862682 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f42b4004580 con 0x7f42bc10d120 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.989+0000 7f42c5a3f740 1 Processor -- start 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.989+0000 7f42c5a3f740 1 -- start start 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a1f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563514989e60 con 0x563514a12ee0 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563514979290 con 0x563514a1f790 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56351497d0a0 con 0x563514a13800 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45796/0 (socket says 172.21.15.120:45796) 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 -- 172.21.15.120:0/1598324114 learned_addr learned my addr 172.21.15.120:0/1598324114 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a1f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 -- 172.21.15.120:0/1598324114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 msgr2=0x563514a21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a21b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 -- 172.21.15.120:0/1598324114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 msgr2=0x563514a132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42b27fc640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 0x7f42a80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 -- 172.21.15.120:0/1598324114 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635146d7930 con 0x563514a13800 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c1406640 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a1f250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42b800bc20 tx=0x7f42b80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64230190dbed143a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42b27fc640 1 -- 172.21.15.120:0/3465862682 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42b4106e90 con 0x7f42bc10d120 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42c0c05640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 0x7f42a80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.990+0000 7f42a77fe640 1 -- 172.21.15.120:0/1598324114 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004030 con 0x563514a13800 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42a77fe640 1 -- 172.21.15.120:0/1598324114 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b80035f0 con 0x563514a13800 2024-09-10T23:01:05.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42a77fe640 1 -- 172.21.15.120:0/1598324114 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80116a0 con 0x563514a13800 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1598324114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 msgr2=0x563514a1f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c0c05640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 0x7f42a80b7820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x563514a12e90 tx=0x7f42ac014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a1f250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42b800bc20 tx=0x7f42b80107a0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1598324114 shutdown_connections 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a1f250 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1598324114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1598324114 >> 172.21.15.120:0/1598324114 conn(0x563514a10d10 msgr2=0x563514a11eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1598324114 shutdown_connections 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.991+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1598324114 wait complete. 2024-09-10T23:01:05.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 -- start start 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 Processor -- start 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 0x563514a1c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5635147da890 con 0x563514a13800 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563514913d40 con 0x563514a1f790 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563514a22440 con 0x563514a12ee0 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c5a3f740 10 client.0 _ll_register_callbacks cb 0x563514977c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45812/0 (socket says 172.21.15.120:45812) 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 -- 172.21.15.120:0/4118677277 learned_addr learned my addr 172.21.15.120:0/4118677277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c1c07640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c1406640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 0x563514a1c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 -- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 msgr2=0x563514a1c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 0x563514a1c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 -- 172.21.15.120:0/4118677277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 msgr2=0x563514a1d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a1d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.992+0000 7f42c0c05640 1 -- 172.21.15.120:0/4118677277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563514a2b850 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c1406640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 0x563514a1c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c0c05640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f42ac014420 tx=0x7f42ac00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ac2ec927cb931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac025070 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b80035f0 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42ac005220 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42ac005460 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 10 client.17112 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563514a2e2b0 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b80035f0 con 0x563514a13800 2024-09-10T23:01:05.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f42ac01e040 con 0x563514a13800 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 10 client.17112 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 20 client.17112 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.993+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a2f6c0 0x563514a31af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.994+0000 7f42c5a3f740 4 client.17112 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.994+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56351498bdb0 con 0x563514a2f6c0 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.994+0000 7f42c1406640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a2f6c0 0x563514a31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.994+0000 7f42c1406640 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a2f6c0 0x563514a31af0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327a0c0d8b731b7a server_cookie=e9b898008527f385 in_seq=0 out_seq=0 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.994+0000 7f42a4ff9640 10 client.17112 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:05.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42a4ff9640 1 -- 172.21.15.120:0/4118677277 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56351498bdb0 con 0x563514a2f6c0 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42a4ff9640 10 client.17112 handle_command_reply: tid=0 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 client.17112 shutdown 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 msgr2=0x563514a1cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f42ac014420 tx=0x7f42ac00a2d0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 shutdown_connections 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a2f6c0 0x563514a31af0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a1d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a1cc70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/4118677277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a12ee0 0x563514a1c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 >> 172.21.15.120:0/4118677277 conn(0x563514a10d10 msgr2=0x563514a0ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42a4ff9640 0 client.17112 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 shutdown_connections 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 1 -- 172.21.15.120:0/4118677277 wait complete. 2024-09-10T23:01:05.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:04.995+0000 7f42c5a3f740 20 client.17112 trim_cache size 0 max 16384 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.013+0000 7f42c5a3f740 1 Processor -- start 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.013+0000 7f42c5a3f740 1 -- start start 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 0x563514a1e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a1f790 0x563514a19e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ae420 con 0x563514a13800 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b0ca0 con 0x563514a12ee0 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1aea90 con 0x563514a1f790 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c1406640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 0x563514a1e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45820/0 (socket says 172.21.15.120:45820) 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 -- 172.21.15.120:0/1997578540 learned_addr learned my addr 172.21.15.120:0/1997578540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c1c07640 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a1f790 0x563514a19e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 -- 172.21.15.120:0/1997578540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a1f790 msgr2=0x563514a19e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a1f790 0x563514a19e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 -- 172.21.15.120:0/1997578540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 msgr2=0x563514a1e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 0x563514a1e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 -- 172.21.15.120:0/1997578540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563514a10b40 con 0x563514a13800 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c1406640 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 0x563514a1e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c00 tx=0x7f42ac005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf181600b03c028f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42a6ffd640 1 -- 172.21.15.120:0/1997578540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac03a020 con 0x563514a13800 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.014+0000 7f42a6ffd640 1 -- 172.21.15.120:0/1997578540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac021ad0 con 0x563514a13800 2024-09-10T23:01:05.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42a6ffd640 1 -- 172.21.15.120:0/1997578540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac025040 con 0x563514a13800 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1997578540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 msgr2=0x563514a19940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c00 tx=0x7f42ac005ec0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1997578540 shutdown_connections 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a1f790 0x563514a19e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a13800 0x563514a19940 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1997578540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a12ee0 0x563514a1e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1997578540 >> 172.21.15.120:0/1997578540 conn(0x563514a10d10 msgr2=0x563514a1dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1997578540 shutdown_connections 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.015+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1997578540 wait complete. 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 -- start start 2024-09-10T23:01:05.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 Processor -- start 2024-09-10T23:01:05.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1af160 con 0x563514a12ee0 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b05d0 con 0x563514a1f790 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1af830 con 0x563514a13800 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c5a3f740 10 client.0 _ll_register_callbacks cb 0x7f42ac014680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c1c07640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c1406640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c1c07640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42536/0 (socket says 172.21.15.120:42536) 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c1c07640 1 -- 172.21.15.120:0/1654702397 learned_addr learned my addr 172.21.15.120:0/1654702397 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c1406640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45822/0 (socket says 172.21.15.120:45822) 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.016+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 -- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 msgr2=0x563514a246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 -- 172.21.15.120:0/1654702397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 msgr2=0x563514a95640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 -- 172.21.15.120:0/1654702397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563514a11790 con 0x563514a12ee0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1c07640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c1406640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f42b8003e90 tx=0x7f42b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13537b534e72067 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8003a20 con 0x563514a12ee0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42b8003bc0 con 0x563514a12ee0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac021ad0 con 0x563514a12ee0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42b8033050 con 0x563514a12ee0 2024-09-10T23:01:05.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c5a3f740 10 client.17118 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.017+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563514a9d0c0 con 0x563514a12ee0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac021ad0 con 0x563514a12ee0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f42b8011820 con 0x563514a12ee0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 10 client.17118 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 20 client.17118 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a9d5c0 0x563514a9f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 4 client.17118 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563514987860 con 0x563514a9d5c0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a9d5c0 0x563514a9f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42c0c05640 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a9d5c0 0x563514a9f9e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2e4afcabc4a5dc server_cookie=c080b89b03b003c1 in_seq=0 out_seq=0 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.018+0000 7f42a4ff9640 10 client.17118 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:05.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.019+0000 7f42a4ff9640 1 -- 172.21.15.120:0/1654702397 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563514987860 con 0x563514a9d5c0 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.019+0000 7f42a4ff9640 10 client.17118 handle_command_reply: tid=0 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.019+0000 7f42c5a3f740 1 client.17118 shutdown 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 msgr2=0x563514a24110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f42b8003e90 tx=0x7f42b800be20 comp rx=0 tx=0).stop 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 shutdown_connections 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563514a9d5c0 0x563514a9f9e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563514a1f790 0x563514a95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563514a13800 0x563514a246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 --2- 172.21.15.120:0/1654702397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563514a12ee0 0x563514a24110 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 >> 172.21.15.120:0/1654702397 conn(0x563514a10d10 msgr2=0x563514a1c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42a4ff9640 0 client.17118 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 shutdown_connections 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 1 -- 172.21.15.120:0/1654702397 wait complete. 2024-09-10T23:01:05.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.020+0000 7f42c5a3f740 20 client.17118 trim_cache size 0 max 16384 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "2626e072-8186-435c-bd3f-c6171dd25846", 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:05.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 msgr2=0x7f42a80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 0x7f42a80b7820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x563514a12e90 tx=0x7f42ac014040 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 msgr2=0x7f42bc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f42b4005cc0 tx=0x7f42b402f380 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 shutdown_connections 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a80b5360 0x7f42a80b7820 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42bc10d120 0x7f42bc102470 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42bc104570 0x7f42bc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 --2- 172.21.15.120:0/3465862682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42bc103b70 0x7f42bc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 >> 172.21.15.120:0/3465862682 conn(0x7f42bc0fd700 msgr2=0x7f42bc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 shutdown_connections 2024-09-10T23:01:05.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:05.021+0000 7f42c366a640 1 -- 172.21.15.120:0/3465862682 wait complete. 2024-09-10T23:01:05.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1466524 16.27 MB/sec execute 360 sec latency 93.410 ms 2024-09-10T23:01:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:05 smithi153 ceph-mon[34951]: pgmap v606: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.8 MiB/s wr, 417 op/s 2024-09-10T23:01:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:05 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:01:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:05 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:05.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:05 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:01:05.719 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:05 smithi120 ceph-mon[31247]: pgmap v606: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.8 MiB/s wr, 417 op/s 2024-09-10T23:01:05.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:05 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:01:05.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:05 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:05.720 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:05 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:01:05.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:05 smithi176 ceph-mon[34982]: pgmap v606: 97 pgs: 97 active+clean; 33 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 3.8 MiB/s wr, 417 op/s 2024-09-10T23:01:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:05 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:01:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:05 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:05 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:01:06.019 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:06.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 -- 172.21.15.120:0/30911937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 msgr2=0x7f91f010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 --2- 172.21.15.120:0/30911937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f010e820 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009900 tx=0x7f91e802a410 comp rx=0 tx=0).stop 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 -- 172.21.15.120:0/30911937 shutdown_connections 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 --2- 172.21.15.120:0/30911937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f010e820 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 --2- 172.21.15.120:0/30911937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f01076a0 0x7f91f0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 --2- 172.21.15.120:0/30911937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f0101c90 0x7f91f0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 -- 172.21.15.120:0/30911937 >> 172.21.15.120:0/30911937 conn(0x7f91f00fd700 msgr2=0x7f91f00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 -- 172.21.15.120:0/30911937 shutdown_connections 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.556+0000 7f91f754e640 1 -- 172.21.15.120:0/30911937 wait complete. 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 Processor -- start 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 -- start start 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 0x7f91f00fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f01076a0 0x7f91f00fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0111ce0 con 0x7f91f0108020 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0111b60 con 0x7f91f0101c90 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f754e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0111e60 con 0x7f91f01076a0 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 0x7f91f00fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45840/0 (socket says 172.21.15.120:45840) 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 -- 172.21.15.120:0/1299972636 learned_addr learned my addr 172.21.15.120:0/1299972636 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f01076a0 0x7f91f00fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 -- 172.21.15.120:0/1299972636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f01076a0 msgr2=0x7f91f00fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f01076a0 0x7f91f00fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 -- 172.21.15.120:0/1299972636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 msgr2=0x7f91f00fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 0x7f91f00fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f5aeb640 1 -- 172.21.15.120:0/1299972636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f01a7620 con 0x7f91f0108020 2024-09-10T23:01:06.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.557+0000 7f91f52ea640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 0x7f91f00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f91e8004770 tx=0x7f91e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91ee7fc640 1 -- 172.21.15.120:0/1299972636 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e803f030 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01a78b0 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91ee7fc640 1 -- 172.21.15.120:0/1299972636 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e8003aa0 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91ee7fc640 1 -- 172.21.15.120:0/1299972636 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8035070 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.558+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01a7d90 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91ee7fc640 1 -- 172.21.15.120:0/1299972636 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f91e80035f0 con 0x7f91f0108020 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 Processor -- start 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 -- start start 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c5da941e50 con 0x55c5da9cb480 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5da931280 con 0x55c5da9cbda0 2024-09-10T23:01:06.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f9923740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5da935090 con 0x55c5da9d7d30 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f4ae9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f4ae9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45856/0 (socket says 172.21.15.120:45856) 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f4ae9640 1 -- 172.21.15.120:0/2669449948 learned_addr learned my addr 172.21.15.120:0/2669449948 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.561+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 -- 172.21.15.120:0/2669449948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 msgr2=0x55c5da9da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 -- 172.21.15.120:0/2669449948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 msgr2=0x55c5da9d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 -- 172.21.15.120:0/2669449948 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5da68f650 con 0x55c5da9cb480 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91ee7fc640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 0x7f91d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012740 tx=0x7f91e400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf7274a84904e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91f52ea640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 0x7f91d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91d77fe640 1 -- 172.21.15.120:0/2669449948 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e40120d0 con 0x55c5da9cb480 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91ee7fc640 1 -- 172.21.15.120:0/1299972636 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f91e8106380 con 0x7f91f0108020 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91d77fe640 1 -- 172.21.15.120:0/2669449948 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e4004c50 con 0x55c5da9cb480 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.562+0000 7f91d77fe640 1 -- 172.21.15.120:0/2669449948 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e40055e0 con 0x55c5da9cb480 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- 172.21.15.120:0/2669449948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 msgr2=0x55c5da9cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f91e4012740 tx=0x7f91e400b7e0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f52ea640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 0x7f91d80b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55c5da9ca190 tx=0x7f91e0009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- 172.21.15.120:0/2669449948 shutdown_connections 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 --2- 172.21.15.120:0/2669449948 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9cb860 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- 172.21.15.120:0/2669449948 >> 172.21.15.120:0/2669449948 conn(0x55c5da9c92b0 msgr2=0x55c5da9ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- 172.21.15.120:0/2669449948 shutdown_connections 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- 172.21.15.120:0/2669449948 wait complete. 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 -- start start 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.563+0000 7f91f9923740 1 Processor -- start 2024-09-10T23:01:06.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 0x55c5da9d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c5da792700 con 0x55c5da9d7d30 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c5da8cc110 con 0x55c5da9cbda0 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5da9da950 con 0x55c5da9cb480 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 10 client.0 _ll_register_callbacks cb 0x55c5da8e5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 0x55c5da9d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45870/0 (socket says 172.21.15.120:45870) 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 -- 172.21.15.120:0/2449171398 learned_addr learned my addr 172.21.15.120:0/2449171398 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 -- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 msgr2=0x55c5da9d7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 0x55c5da9d7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 -- 172.21.15.120:0/2449171398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 msgr2=0x55c5da9d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 -- 172.21.15.120:0/2449171398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5da8f9c40 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 0x55c5da9d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f91e8031440 tx=0x7f91e80c9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48d36c5d3db9b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8049070 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f91e80cd080 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.564+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e4004c50 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91e80cc390 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 10 client.17130 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c5da9eeac0 con 0x55c5da9d7d30 2024-09-10T23:01:06.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e4004c50 con 0x55c5da9d7d30 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f91e80d5c70 con 0x55c5da9d7d30 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 10 client.17130 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 20 client.17130 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5da9eff70 0x55c5da9f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 4 client.17130 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5da943da0 con 0x55c5da9eff70 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.565+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5da9eff70 0x55c5da9f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.566+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5da9eff70 0x55c5da9f23a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e369962aca0e7 server_cookie=74fcc6112a6db428 in_seq=0 out_seq=0 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.566+0000 7f91d4ff9640 10 client.17130 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2449171398 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5da943da0 con 0x55c5da9eff70 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91d4ff9640 10 client.17130 handle_command_reply: tid=0 2024-09-10T23:01:06.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 client.17130 shutdown 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 msgr2=0x55c5da9d50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f91e8031440 tx=0x7f91e80c9330 comp rx=0 tx=0).stop 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 shutdown_connections 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5da9eff70 0x55c5da9f23a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9d7d30 0x55c5da9d50d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91d4ff9640 0 client.17130 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 --2- 172.21.15.120:0/2449171398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cb480 0x55c5da9d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 >> 172.21.15.120:0/2449171398 conn(0x55c5da9c92b0 msgr2=0x55c5da9cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 shutdown_connections 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 1 -- 172.21.15.120:0/2449171398 wait complete. 2024-09-10T23:01:06.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.567+0000 7f91f9923740 20 client.17130 trim_cache size 0 max 16384 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.584+0000 7f91f9923740 1 Processor -- start 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.584+0000 7f91f9923740 1 -- start start 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cbda0 0x55c5da9caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9d7d30 0x55c5da9d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01a9fd0 con 0x55c5da9cb480 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01ac850 con 0x55c5da9d7d30 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f9923740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01aa640 con 0x55c5da9cbda0 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45886/0 (socket says 172.21.15.120:45886) 2024-09-10T23:01:06.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 -- 172.21.15.120:0/3834201743 learned_addr learned my addr 172.21.15.120:0/3834201743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cbda0 0x55c5da9caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9d7d30 0x55c5da9d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 -- 172.21.15.120:0/3834201743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cbda0 msgr2=0x55c5da9caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cbda0 0x55c5da9caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 -- 172.21.15.120:0/3834201743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9d7d30 msgr2=0x55c5da9d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9d7d30 0x55c5da9d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 -- 172.21.15.120:0/3834201743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5da9d2fa0 con 0x55c5da9cb480 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91f52ea640 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55c5da9f2c10 tx=0x7f91e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ecf2f362d2e61b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91d6ffd640 1 -- 172.21.15.120:0/3834201743 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e001ea20 con 0x55c5da9cb480 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91d6ffd640 1 -- 172.21.15.120:0/3834201743 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e0003e20 con 0x55c5da9cb480 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.585+0000 7f91d6ffd640 1 -- 172.21.15.120:0/3834201743 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e00047b0 con 0x55c5da9cb480 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- 172.21.15.120:0/3834201743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 msgr2=0x55c5da9ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55c5da9f2c10 tx=0x7f91e0019000 comp rx=0 tx=0).stop 2024-09-10T23:01:06.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- 172.21.15.120:0/3834201743 shutdown_connections 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9d7d30 0x55c5da9d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9cbda0 0x55c5da9caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 --2- 172.21.15.120:0/3834201743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9ec410 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- 172.21.15.120:0/3834201743 >> 172.21.15.120:0/3834201743 conn(0x55c5da9c92b0 msgr2=0x55c5da9ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- 172.21.15.120:0/3834201743 shutdown_connections 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- 172.21.15.120:0/3834201743 wait complete. 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.586+0000 7f91f9923740 1 -- start start 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 Processor -- start 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01aad10 con 0x55c5da9cb480 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01ac180 con 0x55c5da9cbda0 2024-09-10T23:01:06.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01ab3e0 con 0x55c5da9d7d30 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f9923740 10 client.0 _ll_register_callbacks cb 0x7f91e8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45888/0 (socket says 172.21.15.120:45888) 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 -- 172.21.15.120:0/2246682701 learned_addr learned my addr 172.21.15.120:0/2246682701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f5aeb640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 -- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 msgr2=0x55c5da9f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 -- 172.21.15.120:0/2246682701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 msgr2=0x55c5da9f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f52ea640 1 -- 172.21.15.120:0/2246682701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5da9f8430 con 0x55c5da9cb480 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.587+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f52ea640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f91e0012010 tx=0x7f91e0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b37e86bda2657c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:06.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0004100 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f91e0003970 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e0003e20 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91e0003b70 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 10 client.17136 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c5daa5ce70 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e0003e20 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f91e002a750 con 0x55c5da9cb480 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 10 client.17136 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 20 client.17136 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:06.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5daa5d580 0x55c5daa5f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 4 client.17136 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.588+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5da93f850 con 0x55c5daa5d580 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5daa5d580 0x55c5daa5f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91d4ff9640 10 client.17136 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f4ae9640 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5daa5d580 0x55c5daa5f9a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3f4d7ec4f5c7e8 server_cookie=3faef4726faa0538 in_seq=0 out_seq=0 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91d4ff9640 1 -- 172.21.15.120:0/2246682701 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55c5da93f850 con 0x55c5daa5d580 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91d4ff9640 10 client.17136 handle_command_reply: tid=0 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 client.17136 shutdown 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 msgr2=0x55c5da9e8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f91e0012010 tx=0x7f91e0005fa0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 shutdown_connections 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c5daa5d580 0x55c5daa5f9a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5da9d7d30 0x55c5da9f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5da9cbda0 0x55c5da9f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 --2- 172.21.15.120:0/2246682701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c5da9cb480 0x55c5da9e8b40 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 >> 172.21.15.120:0/2246682701 conn(0x55c5da9c92b0 msgr2=0x55c5da9c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.589+0000 7f91d4ff9640 0 client.17136 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 shutdown_connections 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f9923740 1 -- 172.21.15.120:0/2246682701 wait complete. 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f9923740 20 client.17136 trim_cache size 0 max 16384 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:01:06.598 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "2626e072-8186-435c-bd3f-c6171dd25846", 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:01:06.599 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.600 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.601 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:01:06.602 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.604 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.605 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.606 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.607 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:01:06.608 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:06.609 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 msgr2=0x7f91d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 0x7f91d80b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55c5da9ca190 tx=0x7f91e0009760 comp rx=0 tx=0).stop 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 msgr2=0x7f91f00fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f91e8004770 tx=0x7f91e8004270 comp rx=0 tx=0).stop 2024-09-10T23:01:06.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 shutdown_connections 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f91d80b52e0 0x7f91d80b77a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f0108020 0x7f91f00fe680 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91f01076a0 0x7f91f00fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 --2- 172.21.15.120:0/1299972636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0101c90 0x7f91f00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.590+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 >> 172.21.15.120:0/1299972636 conn(0x7f91f00fd700 msgr2=0x7f91f006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.591+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 shutdown_connections 2024-09-10T23:01:06.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:06.591+0000 7f91f754e640 1 -- 172.21.15.120:0/1299972636 wait complete. 2024-09-10T23:01:06.633 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1468939 16.26 MB/sec execute 361 sec latency 32.879 ms 2024-09-10T23:01:06.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:06 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:06.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:06 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:06.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:06 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:07.409 INFO:tasks.cephfs.filesystem:scrub status for tag:2626e072-8186-435c-bd3f-c6171dd25846 - {'path': '/', 'tag': '2626e072-8186-435c-bd3f-c6171dd25846', 'options': 'recursive,force'} 2024-09-10T23:01:07.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1471464 16.26 MB/sec execute 362 sec latency 124.277 ms 2024-09-10T23:01:07.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:07 smithi120 ceph-mon[31247]: pgmap v607: 97 pgs: 97 active+clean; 31 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-09-10T23:01:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:07 smithi176 ceph-mon[34982]: pgmap v607: 97 pgs: 97 active+clean; 31 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-09-10T23:01:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:07 smithi153 ceph-mon[34951]: pgmap v607: 97 pgs: 97 active+clean; 31 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2024-09-10T23:01:08.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1474674 16.26 MB/sec execute 363 sec latency 44.997 ms 2024-09-10T23:01:09.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1477934 16.27 MB/sec execute 364 sec latency 12.092 ms 2024-09-10T23:01:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:09 smithi176 ceph-mon[34982]: pgmap v608: 97 pgs: 97 active+clean; 28 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T23:01:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:09 smithi120 ceph-mon[31247]: pgmap v608: 97 pgs: 97 active+clean; 28 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T23:01:09.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:09 smithi153 ceph-mon[34951]: pgmap v608: 97 pgs: 97 active+clean; 28 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 4.0 MiB/s wr, 411 op/s 2024-09-10T23:01:10.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1480837 16.26 MB/sec execute 365 sec latency 23.815 ms 2024-09-10T23:01:10.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:10 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:01:10.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:10 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:01:10.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:10 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:01:11.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1483818 16.25 MB/sec execute 366 sec latency 21.022 ms 2024-09-10T23:01:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:11 smithi176 ceph-mon[34982]: pgmap v609: 97 pgs: 97 active+clean; 40 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-09-10T23:01:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:11 smithi120 ceph-mon[31247]: pgmap v609: 97 pgs: 97 active+clean; 40 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-09-10T23:01:11.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:11 smithi153 ceph-mon[34951]: pgmap v609: 97 pgs: 97 active+clean; 40 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-09-10T23:01:12.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1486006 16.25 MB/sec execute 367 sec latency 63.004 ms 2024-09-10T23:01:13.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1489122 16.25 MB/sec execute 368 sec latency 29.540 ms 2024-09-10T23:01:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:13 smithi176 ceph-mon[34982]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 434 op/s 2024-09-10T23:01:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:13 smithi120 ceph-mon[31247]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 434 op/s 2024-09-10T23:01:13.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:13 smithi153 ceph-mon[34951]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.6 MiB/s wr, 434 op/s 2024-09-10T23:01:14.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1492415 16.26 MB/sec execute 369 sec latency 22.335 ms 2024-09-10T23:01:15.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1495634 16.26 MB/sec execute 370 sec latency 18.500 ms 2024-09-10T23:01:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:15 smithi176 ceph-mon[34982]: pgmap v611: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 5.2 MiB/s wr, 402 op/s 2024-09-10T23:01:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:15 smithi120 ceph-mon[31247]: pgmap v611: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 5.2 MiB/s wr, 402 op/s 2024-09-10T23:01:15.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:15 smithi153 ceph-mon[34951]: pgmap v611: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 5.2 MiB/s wr, 402 op/s 2024-09-10T23:01:16.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1498155 16.24 MB/sec execute 371 sec latency 46.815 ms 2024-09-10T23:01:17.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1500466 16.24 MB/sec execute 372 sec latency 83.731 ms 2024-09-10T23:01:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:17 smithi176 ceph-mon[34982]: pgmap v612: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-10T23:01:17.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:17 smithi120 ceph-mon[31247]: pgmap v612: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-10T23:01:17.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:17 smithi153 ceph-mon[34951]: pgmap v612: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.082+0000 7f98e8990640 1 -- 172.21.15.120:0/1555674321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc006d90 msgr2=0x7f98dc0980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.082+0000 7f98e8990640 1 --2- 172.21.15.120:0/1555674321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc006d90 0x7f98dc0980e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a410 comp rx=0 tx=0).stop 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.082+0000 7f98e8990640 1 -- 172.21.15.120:0/1555674321 shutdown_connections 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 --2- 172.21.15.120:0/1555674321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc09b220 0x7f98dc09d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 --2- 172.21.15.120:0/1555674321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc098670 0x7f98dc09aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 --2- 172.21.15.120:0/1555674321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc006d90 0x7f98dc0980e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 -- 172.21.15.120:0/1555674321 >> 172.21.15.120:0/1555674321 conn(0x7f98dc08fcb0 msgr2=0x7f98dc0920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 -- 172.21.15.120:0/1555674321 shutdown_connections 2024-09-10T23:01:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.083+0000 7f98e8990640 1 -- 172.21.15.120:0/1555674321 wait complete. 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 Processor -- start 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 -- start start 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 0x7f98dc095820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 0x7f98dc094080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0a1180 con 0x7f98dc09b220 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0a1000 con 0x7f98dc006d90 2024-09-10T23:01:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e8990640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0a1300 con 0x7f98dc098670 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 0x7f98dc095820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33788/0 (socket says 172.21.15.120:33788) 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e359e640 1 -- 172.21.15.120:0/2182085221 learned_addr learned my addr 172.21.15.120:0/2182085221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.084+0000 7f98e259c640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 0x7f98dc094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 -- 172.21.15.120:0/2182085221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 msgr2=0x7f98dc094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 0x7f98dc094080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 -- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 msgr2=0x7f98dc0945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 -- 172.21.15.120:0/2182085221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc094e30 con 0x7f98dc006d90 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e259c640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 0x7f98dc094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e359e640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 0x7f98dc095820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006010 tx=0x7f98d40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98cbfff640 1 -- 172.21.15.120:0/2182085221 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d403f030 con 0x7f98dc006d90 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98cbfff640 1 -- 172.21.15.120:0/2182085221 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d40318f0 con 0x7f98dc006d90 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc135920 con 0x7f98dc006d90 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98cbfff640 1 -- 172.21.15.120:0/2182085221 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4035070 con 0x7f98dc006d90 2024-09-10T23:01:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.085+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc135de0 con 0x7f98dc006d90 2024-09-10T23:01:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98cbfff640 1 -- 172.21.15.120:0/2182085221 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f98d403b9d0 con 0x7f98dc006d90 2024-09-10T23:01:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 Processor -- start 2024-09-10T23:01:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 -- start start 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59ef9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59efc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d59e696f0 con 0x562d59efcdd0 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d59e58b20 con 0x562d59ef7a60 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98ec037740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562d59e5c930 con 0x562d59efa390 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59efc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33798/0 (socket says 172.21.15.120:33798) 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e259c640 1 -- 172.21.15.120:0/690392366 learned_addr learned my addr 172.21.15.120:0/690392366 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e359e640 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59ef9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e259c640 1 -- 172.21.15.120:0/690392366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 msgr2=0x562d59efc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.088+0000 7f98e259c640 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59efc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98e259c640 1 -- 172.21.15.120:0/690392366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 msgr2=0x562d59ef9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98e259c640 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59ef9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98e259c640 1 -- 172.21.15.120:0/690392366 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d59e1e530 con 0x562d59efcdd0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98e259c640 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f98d80104d0 tx=0x7f98d8010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bb088bff726e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98cbfff640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 0x7f98cc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98c97fa640 1 -- 172.21.15.120:0/690392366 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8004220 con 0x562d59efcdd0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98e359e640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 0x7f98cc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98c97fa640 1 -- 172.21.15.120:0/690392366 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d80043c0 con 0x562d59efcdd0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98c97fa640 1 -- 172.21.15.120:0/690392366 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8011aa0 con 0x562d59efcdd0 2024-09-10T23:01:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.089+0000 7f98cbfff640 1 -- 172.21.15.120:0/2182085221 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98d4106340 con 0x7f98dc006d90 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- 172.21.15.120:0/690392366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 msgr2=0x562d59eff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98e359e640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 0x7f98cc0b77a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f98e4052890 tx=0x7f98e407b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f98d80104d0 tx=0x7f98d8010990 comp rx=0 tx=0).stop 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- 172.21.15.120:0/690392366 shutdown_connections 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59eff290 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59efc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 --2- 172.21.15.120:0/690392366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59ef9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- 172.21.15.120:0/690392366 >> 172.21.15.120:0/690392366 conn(0x562d59ef2bb0 msgr2=0x562d59ef3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- 172.21.15.120:0/690392366 shutdown_connections 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- 172.21.15.120:0/690392366 wait complete. 2024-09-10T23:01:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 -- start start 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.090+0000 7f98ec037740 1 Processor -- start 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 0x562d59f010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efa390 0x562d59f01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562d59d90ab0 con 0x562d59efcdd0 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562d59dfe730 con 0x562d59efa390 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562d59effdb0 con 0x562d59ef7a60 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 0x562d59f010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33808/0 (socket says 172.21.15.120:33808) 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98ec037740 10 client.0 _ll_register_callbacks cb 0x562d59e526b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 -- 172.21.15.120:0/707707783 learned_addr learned my addr 172.21.15.120:0/707707783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e259c640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efa390 0x562d59f01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 -- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 msgr2=0x562d59f010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 0x562d59f010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 -- 172.21.15.120:0/707707783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efa390 msgr2=0x562d59f01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efa390 0x562d59f01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 -- 172.21.15.120:0/707707783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d59bba790 con 0x562d59efcdd0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 0x562d59f010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98e359e640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f98e4051c90 tx=0x7f98e4068f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c847b346980d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.091+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4098060 con 0x562d59efcdd0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98e406a9b0 con 0x562d59efcdd0 2024-09-10T23:01:18.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d80043c0 con 0x562d59efcdd0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98e4092460 con 0x562d59efcdd0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 10 client.17148 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d59c33620 con 0x562d59efcdd0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d80043c0 con 0x562d59efcdd0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f98e409f610 con 0x562d59efcdd0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 10 client.17148 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 20 client.17148 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f0ca60 0x562d59f0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 4 client.17148 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d59e6b640 con 0x562d59f0ca60 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.092+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f0ca60 0x562d59f0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.093+0000 7f98c6ff5640 10 client.17148 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.093+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f0ca60 0x562d59f0ef00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9b1cf1b8a69500 server_cookie=c639e9586f2918e7 in_seq=0 out_seq=0 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.093+0000 7f98c6ff5640 1 -- 172.21.15.120:0/707707783 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d59e6b640 con 0x562d59f0ca60 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.093+0000 7f98c6ff5640 10 client.17148 handle_command_reply: tid=0 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.093+0000 7f98ec037740 1 client.17148 shutdown 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 msgr2=0x562d59f02330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f98e4051c90 tx=0x7f98e4068f10 comp rx=0 tx=0).stop 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 shutdown_connections 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f0ca60 0x562d59f0ef00 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efcdd0 0x562d59f02330 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98c6ff5640 0 client.17148 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efa390 0x562d59f01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 --2- 172.21.15.120:0/707707783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59ef7a60 0x562d59f010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 >> 172.21.15.120:0/707707783 conn(0x562d59ef2bb0 msgr2=0x562d59ef1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 shutdown_connections 2024-09-10T23:01:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 1 -- 172.21.15.120:0/707707783 wait complete. 2024-09-10T23:01:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.094+0000 7f98ec037740 20 client.17148 trim_cache size 0 max 16384 2024-09-10T23:01:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.108+0000 7f98ec037740 1 Processor -- start 2024-09-10T23:01:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 -- start start 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59ef3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efcdd0 0x562d59ef3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13ca80 con 0x562d59ef7a60 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13f300 con 0x562d59efcdd0 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98ec037740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13d0f0 con 0x562d59efa390 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33816/0 (socket says 172.21.15.120:33816) 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 -- 172.21.15.120:0/1846869721 learned_addr learned my addr 172.21.15.120:0/1846869721 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e259c640 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59ef3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e359e640 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efcdd0 0x562d59ef3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 -- 172.21.15.120:0/1846869721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 msgr2=0x562d59ef3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59ef3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 -- 172.21.15.120:0/1846869721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efcdd0 msgr2=0x562d59ef3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efcdd0 0x562d59ef3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 -- 172.21.15.120:0/1846869721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d59f00e90 con 0x562d59ef7a60 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f98d40c9da0 tx=0x7f98d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa37b056d35950f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.109+0000 7f98c8ff9640 1 -- 172.21.15.120:0/1846869721 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40cde60 con 0x562d59ef7a60 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98c8ff9640 1 -- 172.21.15.120:0/1846869721 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d40cc640 con 0x562d59ef7a60 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98c8ff9640 1 -- 172.21.15.120:0/1846869721 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4049040 con 0x562d59ef7a60 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 -- 172.21.15.120:0/1846869721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 msgr2=0x562d59f02310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f98d40c9da0 tx=0x7f98d40045e0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 -- 172.21.15.120:0/1846869721 shutdown_connections 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59efcdd0 0x562d59ef3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efa390 0x562d59ef3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 --2- 172.21.15.120:0/1846869721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59ef7a60 0x562d59f02310 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 -- 172.21.15.120:0/1846869721 >> 172.21.15.120:0/1846869721 conn(0x562d59ef2bb0 msgr2=0x562d59f0ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 -- 172.21.15.120:0/1846869721 shutdown_connections 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.110+0000 7f98ec037740 1 -- 172.21.15.120:0/1846869721 wait complete. 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 -- start start 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 Processor -- start 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efcdd0 0x562d59f07430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13d7c0 con 0x562d59efa390 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13ec30 con 0x562d59ef7a60 2024-09-10T23:01:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc13de90 con 0x562d59efcdd0 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98ec037740 10 client.0 _ll_register_callbacks cb 0x7f98e406adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33820/0 (socket says 172.21.15.120:33820) 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53920/0 (socket says 172.21.15.120:53920) 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 -- 172.21.15.120:0/2168919008 learned_addr learned my addr 172.21.15.120:0/2168919008 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e359e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efcdd0 0x562d59f07430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 -- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efcdd0 msgr2=0x562d59f07430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efcdd0 0x562d59f07430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 -- 172.21.15.120:0/2168919008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 msgr2=0x562d59f06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e259c640 1 -- 172.21.15.120:0/2168919008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d59f07c30 con 0x562d59efa390 2024-09-10T23:01:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.111+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98e259c640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f98d80104a0 tx=0x7f98d8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487f57d31ec1acf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8004660 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98d8004800 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d40cc640 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d8004a00 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 10 client.17154 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d59f08f70 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d40cc640 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f98d802e030 con 0x562d59efa390 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 10 client.17154 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 20 client.17154 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f79c00 0x562d59f7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 4 client.17154 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d59e670f0 con 0x562d59f79c00 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.112+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f79c00 0x562d59f7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.113+0000 7f98c6ff5640 10 client.17154 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.113+0000 7f98e2d9d640 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f79c00 0x562d59f7bff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80fe1e8d53558f0 server_cookie=278003c0eced849b in_seq=0 out_seq=0 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98c6ff5640 1 -- 172.21.15.120:0/2168919008 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d59e670f0 con 0x562d59f79c00 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98c6ff5640 10 client.17154 handle_command_reply: tid=0 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 client.17154 shutdown 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 msgr2=0x562d59f06cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f98d80104a0 tx=0x7f98d8005270 comp rx=0 tx=0).stop 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 shutdown_connections 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562d59f79c00 0x562d59f7bff0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98c6ff5640 0 client.17154 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562d59efcdd0 0x562d59f07430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562d59efa390 0x562d59f06cd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 --2- 172.21.15.120:0/2168919008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d59ef7a60 0x562d59f06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 >> 172.21.15.120:0/2168919008 conn(0x562d59ef2bb0 msgr2=0x562d59ef3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 shutdown_connections 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 1 -- 172.21.15.120:0/2168919008 wait complete. 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.315+0000 7f98ec037740 20 client.17154 trim_cache size 0 max 16384 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:01:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:01:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 msgr2=0x7f98cc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 0x7f98cc0b77a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f98e4052890 tx=0x7f98e407b040 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 msgr2=0x7f98dc095820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 0x7f98dc095820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006010 tx=0x7f98d40042f0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 shutdown_connections 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98cc0b52e0 0x7f98cc0b77a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98dc09b220 0x7f98dc0945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98dc098670 0x7f98dc094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 --2- 172.21.15.120:0/2182085221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98dc006d90 0x7f98dc095820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 >> 172.21.15.120:0/2182085221 conn(0x7f98dc08fcb0 msgr2=0x7f98dc0968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 shutdown_connections 2024-09-10T23:01:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:18.316+0000 7f98e8990640 1 -- 172.21.15.120:0/2182085221 wait complete. 2024-09-10T23:01:18.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1503015 16.24 MB/sec execute 373 sec latency 134.261 ms 2024-09-10T23:01:19.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1506101 16.23 MB/sec execute 374 sec latency 13.674 ms 2024-09-10T23:01:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:19 smithi176 ceph-mon[34982]: pgmap v613: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-10T23:01:19.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:19 smithi120 ceph-mon[31247]: pgmap v613: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-10T23:01:19.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:19 smithi153 ceph-mon[34951]: pgmap v613: 97 pgs: 97 active+clean; 49 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-10T23:01:20.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1509281 16.24 MB/sec execute 375 sec latency 11.384 ms 2024-09-10T23:01:21.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1512618 16.24 MB/sec execute 376 sec latency 11.644 ms 2024-09-10T23:01:21.748 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:21 smithi120 ceph-mon[31247]: pgmap v614: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 446 op/s 2024-09-10T23:01:21.825 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:21 smithi176 ceph-mon[34982]: pgmap v614: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 446 op/s 2024-09-10T23:01:21.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:21 smithi153 ceph-mon[34951]: pgmap v614: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 446 op/s 2024-09-10T23:01:22.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1515067 16.23 MB/sec execute 377 sec latency 65.390 ms 2024-09-10T23:01:23.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1518011 16.23 MB/sec execute 378 sec latency 21.542 ms 2024-09-10T23:01:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:23 smithi176 ceph-mon[34982]: pgmap v615: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-09-10T23:01:23.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:23 smithi120 ceph-mon[31247]: pgmap v615: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-09-10T23:01:23.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:23 smithi153 ceph-mon[34951]: pgmap v615: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-09-10T23:01:24.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1521404 16.24 MB/sec execute 379 sec latency 11.908 ms 2024-09-10T23:01:25.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1524652 16.24 MB/sec execute 380 sec latency 15.286 ms 2024-09-10T23:01:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:25 smithi176 ceph-mon[34982]: pgmap v616: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2024-09-10T23:01:25.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:25 smithi120 ceph-mon[31247]: pgmap v616: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2024-09-10T23:01:25.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:25 smithi153 ceph-mon[34951]: pgmap v616: 97 pgs: 97 active+clean; 42 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 388 op/s 2024-09-10T23:01:26.634 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1528037 16.25 MB/sec execute 381 sec latency 22.326 ms 2024-09-10T23:01:27.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1531262 16.25 MB/sec execute 382 sec latency 48.406 ms 2024-09-10T23:01:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:27 smithi176 ceph-mon[34982]: pgmap v617: 97 pgs: 97 active+clean; 46 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-09-10T23:01:27.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:27 smithi120 ceph-mon[31247]: pgmap v617: 97 pgs: 97 active+clean; 46 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-09-10T23:01:27.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:27 smithi153 ceph-mon[34951]: pgmap v617: 97 pgs: 97 active+clean; 46 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 467 op/s 2024-09-10T23:01:28.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1534091 16.25 MB/sec execute 383 sec latency 60.043 ms 2024-09-10T23:01:29.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1537089 16.24 MB/sec execute 384 sec latency 62.528 ms 2024-09-10T23:01:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:29 smithi176 ceph-mon[34982]: pgmap v618: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2024-09-10T23:01:29.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:29 smithi120 ceph-mon[31247]: pgmap v618: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2024-09-10T23:01:29.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:29 smithi153 ceph-mon[34951]: pgmap v618: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 409 op/s 2024-09-10T23:01:30.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1539979 16.24 MB/sec execute 385 sec latency 22.397 ms 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/1076429299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 msgr2=0x7fcb68107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/1076429299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb68107f40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009900 tx=0x7fcb5002a410 comp rx=0 tx=0).stop 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/1076429299 shutdown_connections 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/1076429299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/1076429299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb68107f40 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/1076429299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 0x7fcb68105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/1076429299 >> 172.21.15.120:0/1076429299 conn(0x7fcb680fd140 msgr2=0x7fcb680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/1076429299 shutdown_connections 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.409+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/1076429299 wait complete. 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 Processor -- start 2024-09-10T23:01:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 -- start start 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 0x7fcb68198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810e5a0 con 0x7fcb68105ab0 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810e420 con 0x7fcb681086d0 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6c8c6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6810e720 con 0x7fcb6806b430 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59514/0 (socket says 172.21.15.120:59514) 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb5ffff640 1 -- 172.21.15.120:0/3774066845 learned_addr learned my addr 172.21.15.120:0/3774066845 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.410+0000 7fcb6659c640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 0x7fcb68198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 -- 172.21.15.120:0/3774066845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 msgr2=0x7fcb68198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 0x7fcb68198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 -- 172.21.15.120:0/3774066845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 msgr2=0x7fcb6819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 -- 172.21.15.120:0/3774066845 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6819a8c0 con 0x7fcb68105ab0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005940 tx=0x7fcb50031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5f7fe640 1 -- 172.21.15.120:0/3774066845 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5003f030 con 0x7fcb68105ab0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5f7fe640 1 -- 172.21.15.120:0/3774066845 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb50003aa0 con 0x7fcb68105ab0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6819ab50 con 0x7fcb68105ab0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb5f7fe640 1 -- 172.21.15.120:0/3774066845 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50035070 con 0x7fcb68105ab0 2024-09-10T23:01:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.411+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681a1280 con 0x7fcb68105ab0 2024-09-10T23:01:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb5f7fe640 1 -- 172.21.15.120:0/3774066845 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcb5003bbb0 con 0x7fcb68105ab0 2024-09-10T23:01:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 Processor -- start 2024-09-10T23:01:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 -- start start 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1ea3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea6bd0 0x5650a1ea9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650a1e136f0 con 0x5650a1ea1860 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5650a1e02b20 con 0x5650a1ea6bd0 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5650a1e06930 con 0x5650a1ea4190 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1ea3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb6659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1ea3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59516/0 (socket says 172.21.15.120:59516) 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb66d9d640 1 -- 172.21.15.120:0/2708968406 learned_addr learned my addr 172.21.15.120:0/2708968406 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.414+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea6bd0 0x5650a1ea9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 -- 172.21.15.120:0/2708968406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea6bd0 msgr2=0x5650a1ea9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea6bd0 0x5650a1ea9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 -- 172.21.15.120:0/2708968406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 msgr2=0x5650a1ea3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1ea3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 -- 172.21.15.120:0/2708968406 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a1dc8530 con 0x5650a1ea4190 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb5f7fe640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 0x7fcb540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 0x7fcb540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb6659c640 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea6580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009c20 tx=0x7fcb58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee7a5e538de3c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb5f7fe640 1 -- 172.21.15.120:0/3774066845 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb501064d0 con 0x7fcb68105ab0 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb5cff9640 1 -- 172.21.15.120:0/2708968406 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004050 con 0x5650a1ea4190 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb5cff9640 1 -- 172.21.15.120:0/2708968406 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb580054d0 con 0x5650a1ea4190 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.415+0000 7fcb5cff9640 1 -- 172.21.15.120:0/2708968406 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004ca0 con 0x5650a1ea4190 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2708968406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 msgr2=0x5650a1ea6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 0x7fcb540b77a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb60008ec0 tx=0x7fcb60008450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea6580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009c20 tx=0x7fcb58011810 comp rx=0 tx=0).stop 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2708968406 shutdown_connections 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea6bd0 0x5650a1ea9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea6580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2708968406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1ea3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2708968406 >> 172.21.15.120:0/2708968406 conn(0x5650a1e9c9b0 msgr2=0x5650a1e9d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2708968406 shutdown_connections 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2708968406 wait complete. 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- start start 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 Processor -- start 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 0x5650a1eaae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650a1d3aab0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5650a1da8730 con 0x5650a1ea4190 2024-09-10T23:01:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.416+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5650a1ea9bb0 con 0x5650a1ea1860 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6f69b740 10 client.0 _ll_register_callbacks cb 0x5650a1dfc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 0x5650a1eaae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 0x5650a1eaae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55622/0 (socket says 172.21.15.120:55622) 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6659c640 1 -- 172.21.15.120:0/2235590627 learned_addr learned my addr 172.21.15.120:0/2235590627 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 -- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 msgr2=0x5650a1eaae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 0x5650a1eaae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 -- 172.21.15.120:0/2235590627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 msgr2=0x5650a1eab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 -- 172.21.15.120:0/2235590627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a1b64790 con 0x5650a1ea6bd0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eac0a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010ca0 tx=0x7fcb60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dd9c1c20f27465 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6001d420 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb6001d5c0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb580054d0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb6002c070 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6f69b740 10 client.17166 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.417+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650a1bdd620 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb580054d0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb600257d0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 10 client.17166 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 20 client.17166 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1eb6830 0x5650a1eb8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 4 client.17166 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a1e15640 con 0x5650a1eb6830 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6659c640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1eb6830 0x5650a1eb8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb6659c640 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1eb6830 0x5650a1eb8c40 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98eea3f552e46de1 server_cookie=e6def01ed154c344 in_seq=0 out_seq=0 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.418+0000 7fcb4dffb640 10 client.17166 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb4dffb640 1 -- 172.21.15.120:0/2235590627 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650a1e15640 con 0x5650a1eb6830 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb4dffb640 10 client.17166 handle_command_reply: tid=0 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 client.17166 shutdown 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 msgr2=0x5650a1eac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eac0a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010ca0 tx=0x7fcb60004a40 comp rx=0 tx=0).stop 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 shutdown_connections 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1eb6830 0x5650a1eb8c40 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eac0a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb4dffb640 0 client.17166 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/2235590627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea1860 0x5650a1eaae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 >> 172.21.15.120:0/2235590627 conn(0x5650a1e9c9b0 msgr2=0x5650a1e9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 shutdown_connections 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.419+0000 7fcb6f69b740 1 -- 172.21.15.120:0/2235590627 wait complete. 2024-09-10T23:01:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.420+0000 7fcb6f69b740 20 client.17166 trim_cache size 0 max 16384 2024-09-10T23:01:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 Processor -- start 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 -- start start 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea1860 0x5650a1ea0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681a7db0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681aa630 con 0x5650a1ea1860 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681a8420 con 0x5650a1ea4190 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59534/0 (socket says 172.21.15.120:59534) 2024-09-10T23:01:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb66d9d640 1 -- 172.21.15.120:0/3171617354 learned_addr learned my addr 172.21.15.120:0/3171617354 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.434+0000 7fcb6659c640 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea1860 0x5650a1ea0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 -- 172.21.15.120:0/3171617354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 msgr2=0x5650a1ea11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 -- 172.21.15.120:0/3171617354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea1860 msgr2=0x5650a1ea0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea1860 0x5650a1ea0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 -- 172.21.15.120:0/3171617354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a1ea68f0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010010 tx=0x7fcb60008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6fa979120252f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb4ffff640 1 -- 172.21.15.120:0/3171617354 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60004b40 con 0x5650a1ea6bd0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb4ffff640 1 -- 172.21.15.120:0/3171617354 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60004ce0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.435+0000 7fcb4ffff640 1 -- 172.21.15.120:0/3171617354 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb600316e0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- 172.21.15.120:0/3171617354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 msgr2=0x5650a1eaa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcb60010010 tx=0x7fcb60008ca0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- 172.21.15.120:0/3171617354 shutdown_connections 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea6bd0 0x5650a1eaa0a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea4190 0x5650a1ea11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/3171617354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea1860 0x5650a1ea0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- 172.21.15.120:0/3171617354 >> 172.21.15.120:0/3171617354 conn(0x5650a1e9c9b0 msgr2=0x5650a1e97af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- 172.21.15.120:0/3171617354 shutdown_connections 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- 172.21.15.120:0/3171617354 wait complete. 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- start start 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 Processor -- start 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 0x5650a1eb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681a8af0 con 0x5650a1ea1860 2024-09-10T23:01:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681a9f60 con 0x5650a1ea4190 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb681a91c0 con 0x5650a1ea6bd0 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6f69b740 10 client.0 _ll_register_callbacks cb 0x7fcb60008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59544/0 (socket says 172.21.15.120:59544) 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb6659c640 1 -- 172.21.15.120:0/4206135221 learned_addr learned my addr 172.21.15.120:0/4206135221 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.436+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 0x5650a1eb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 -- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 msgr2=0x5650a1eb1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 0x5650a1eb1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 -- 172.21.15.120:0/4206135221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 msgr2=0x5650a1eb1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eb1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 -- 172.21.15.120:0/4206135221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a1ea3fc0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb66d9d640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 0x5650a1eb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6659c640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009820 tx=0x7fcb58011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cd84d58063ff1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58005040 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb580051e0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb60004ce0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb580053e0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 10 client.17172 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650a1eb0ee0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60004ce0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb580037b0 con 0x5650a1ea1860 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 10 client.17172 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 20 client.17172 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1f23da0 0x5650a1f261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 4 client.17172 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.437+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650a1e110f0 con 0x5650a1f23da0 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.438+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1f23da0 0x5650a1f261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.438+0000 7fcb4dffb640 10 client.17172 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.438+0000 7fcb5ffff640 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1f23da0 0x5650a1f261c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee739811143e6ea4 server_cookie=dcc3ca2a810fa2b9 in_seq=0 out_seq=0 2024-09-10T23:01:31.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1542453 16.24 MB/sec execute 386 sec latency 96.801 ms 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.667+0000 7fcb4dffb640 1 -- 172.21.15.120:0/4206135221 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650a1e110f0 con 0x5650a1f23da0 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.667+0000 7fcb4dffb640 10 client.17172 handle_command_reply: tid=0 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.667+0000 7fcb6f69b740 1 client.17172 shutdown 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.667+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 msgr2=0x5650a1eb2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.667+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009820 tx=0x7fcb58011d10 comp rx=0 tx=0).stop 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 shutdown_connections 2024-09-10T23:01:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5650a1f23da0 0x5650a1f261c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5650a1ea6bd0 0x5650a1eb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb4dffb640 0 client.17172 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5650a1ea4190 0x5650a1eb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 --2- 172.21.15.120:0/4206135221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650a1ea1860 0x5650a1eb2890 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 >> 172.21.15.120:0/4206135221 conn(0x5650a1e9c9b0 msgr2=0x5650a1eaa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 shutdown_connections 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 1 -- 172.21.15.120:0/4206135221 wait complete. 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.668+0000 7fcb6f69b740 20 client.17172 trim_cache size 0 max 16384 2024-09-10T23:01:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:01:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:01:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:01:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 msgr2=0x7fcb540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 0x7fcb540b77a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb60008ec0 tx=0x7fcb60008450 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 msgr2=0x7fcb681993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005940 tx=0x7fcb50031a50 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 shutdown_connections 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb540b52e0 0x7fcb540b77a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb681086d0 0x7fcb6819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb68105ab0 0x7fcb681993d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 --2- 172.21.15.120:0/3774066845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb6806b430 0x7fcb68198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 >> 172.21.15.120:0/3774066845 conn(0x7fcb680fd140 msgr2=0x7fcb68103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 shutdown_connections 2024-09-10T23:01:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:31.669+0000 7fcb6c8c6640 1 -- 172.21.15.120:0/3774066845 wait complete. 2024-09-10T23:01:31.750 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:31 smithi153 ceph-mon[34951]: pgmap v619: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 433 op/s 2024-09-10T23:01:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:31 smithi176 ceph-mon[34982]: pgmap v619: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 433 op/s 2024-09-10T23:01:31.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:31 smithi120 ceph-mon[31247]: pgmap v619: 97 pgs: 97 active+clean; 48 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 433 op/s 2024-09-10T23:01:32.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1545680 16.24 MB/sec execute 387 sec latency 44.874 ms 2024-09-10T23:01:33.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1548014 16.23 MB/sec execute 388 sec latency 58.800 ms 2024-09-10T23:01:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:33 smithi176 ceph-mon[34982]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 370 op/s 2024-09-10T23:01:33.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:33 smithi120 ceph-mon[31247]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 370 op/s 2024-09-10T23:01:33.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:33 smithi153 ceph-mon[34951]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 370 op/s 2024-09-10T23:01:34.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1551262 16.23 MB/sec execute 389 sec latency 13.890 ms 2024-09-10T23:01:35.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1554174 16.23 MB/sec execute 390 sec latency 12.668 ms 2024-09-10T23:01:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:35 smithi176 ceph-mon[34982]: pgmap v621: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-10T23:01:35.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:35 smithi120 ceph-mon[31247]: pgmap v621: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-10T23:01:35.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:35 smithi153 ceph-mon[34951]: pgmap v621: 97 pgs: 97 active+clean; 53 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-10T23:01:36.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1557128 16.23 MB/sec execute 391 sec latency 22.357 ms 2024-09-10T23:01:37.411 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:37.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1560273 16.23 MB/sec execute 392 sec latency 14.064 ms 2024-09-10T23:01:37.676 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:37.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:37 smithi120 ceph-mon[31247]: pgmap v622: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-09-10T23:01:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:37 smithi176 ceph-mon[34982]: pgmap v622: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-09-10T23:01:37.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:37 smithi153 ceph-mon[34951]: pgmap v622: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 436 op/s 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 -- 172.21.15.120:0/1886477734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 msgr2=0x7f59f410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 --2- 172.21.15.120:0/1886477734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f410e820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009ca0 tx=0x7f59ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 -- 172.21.15.120:0/1886477734 shutdown_connections 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 --2- 172.21.15.120:0/1886477734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f410e820 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 --2- 172.21.15.120:0/1886477734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 --2- 172.21.15.120:0/1886477734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 0x7f59f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 -- 172.21.15.120:0/1886477734 >> 172.21.15.120:0/1886477734 conn(0x7f59f40fd700 msgr2=0x7f59f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 -- 172.21.15.120:0/1886477734 shutdown_connections 2024-09-10T23:01:38.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.429+0000 7f59fa97f640 1 -- 172.21.15.120:0/1886477734 wait complete. 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 Processor -- start 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 -- start start 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 0x7f59f4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4112e00 con 0x7f59f4108020 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4112c80 con 0x7f59f4101c90 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59fa97f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4112f80 con 0x7f59f41076a0 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57782/0 (socket says 172.21.15.120:57782) 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59f37fe640 1 -- 172.21.15.120:0/3687690155 learned_addr learned my addr 172.21.15.120:0/3687690155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.430+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f3fff640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 0x7f59f4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 -- 172.21.15.120:0/3687690155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 msgr2=0x7f59f4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 0x7f59f4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 -- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 msgr2=0x7f59f419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 -- 172.21.15.120:0/3687690155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f419f2e0 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f37fe640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d950 tx=0x7f59e800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f17fa640 1 -- 172.21.15.120:0/3687690155 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8017030 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f419f5d0 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f17fa640 1 -- 172.21.15.120:0/3687690155 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e8004020 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59f17fa640 1 -- 172.21.15.120:0/3687690155 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80047a0 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.431+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f41a5e60 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59f17fa640 1 -- 172.21.15.120:0/3687690155 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f59e8009a30 con 0x7f59f41076a0 2024-09-10T23:01:38.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 Processor -- start 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 -- start start 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca59410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558aac9c3e50 con 0x558aaca4d0a0 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558aac9b3280 con 0x558aaca59950 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59fcd54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558aac9b7090 con 0x558aaca4d9c0 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59f8f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59f8f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36492/0 (socket says 172.21.15.120:36492) 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59f8f1c640 1 -- 172.21.15.120:0/3900739997 learned_addr learned my addr 172.21.15.120:0/3900739997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.434+0000 7f59f3fff640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca59410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f37fe640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 -- 172.21.15.120:0/3900739997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 msgr2=0x558aaca59410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca59410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 -- 172.21.15.120:0/3900739997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 msgr2=0x558aaca5bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 -- 172.21.15.120:0/3900739997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aac711650 con 0x558aaca4d0a0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f37fe640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f17fa640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 0x7f59dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f59ec035000 tx=0x7f59ec02f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07e0d502544a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f3fff640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 0x7f59dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59da7fc640 1 -- 172.21.15.120:0/3900739997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec031650 con 0x558aaca4d0a0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59da7fc640 1 -- 172.21.15.120:0/3900739997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ec036040 con 0x558aaca4d0a0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59da7fc640 1 -- 172.21.15.120:0/3900739997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec004580 con 0x558aaca4d0a0 2024-09-10T23:01:38.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.435+0000 7f59f17fa640 1 -- 172.21.15.120:0/3687690155 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f59e80e5520 con 0x7f59f41076a0 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- 172.21.15.120:0/3900739997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 msgr2=0x558aaca4d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f59ec035000 tx=0x7f59ec02f430 comp rx=0 tx=0).stop 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59f3fff640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 0x7f59dc0b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x558aaca4bdb0 tx=0x7f59e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- 172.21.15.120:0/3900739997 shutdown_connections 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca59410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 --2- 172.21.15.120:0/3900739997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d0a0 0x558aaca4d480 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- 172.21.15.120:0/3900739997 >> 172.21.15.120:0/3900739997 conn(0x558aaca4aed0 msgr2=0x558aaca4c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- 172.21.15.120:0/3900739997 shutdown_connections 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- 172.21.15.120:0/3900739997 wait complete. 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.436+0000 7f59fcd54740 1 -- start start 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 Processor -- start 2024-09-10T23:01:38.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 0x558aaca591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca56630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558aac814700 con 0x558aaca59950 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558aac94e110 con 0x558aaca4d0a0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558aaca5c3f0 con 0x558aaca4d9c0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59fcd54740 10 client.0 _ll_register_callbacks cb 0x558aac967990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36496/0 (socket says 172.21.15.120:36496) 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 -- 172.21.15.120:0/2025078573 learned_addr learned my addr 172.21.15.120:0/2025078573 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f37fe640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca56630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f3fff640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 0x558aaca591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 -- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 msgr2=0x558aaca56630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca56630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 -- 172.21.15.120:0/2025078573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 msgr2=0x558aaca591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 0x558aaca591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f8f1c640 1 -- 172.21.15.120:0/2025078573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aac97bc40 con 0x558aaca59950 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f37fe640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca56630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.437+0000 7f59f3fff640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 0x558aaca591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005e30 tx=0x7f59ec004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d40d3e7e12dbb31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec031910 con 0x558aaca59950 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f59ec003b80 con 0x558aaca59950 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ec036040 con 0x558aaca59950 2024-09-10T23:01:38.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59ec004850 con 0x558aaca59950 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 10 client.17184 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558aaca67cc0 con 0x558aaca59950 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ec036040 con 0x558aaca59950 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f59ec04f070 con 0x558aaca59950 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 10 client.17184 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 20 client.17184 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aaca6b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 4 client.17184 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.438+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aac9c5da0 con 0x558aaca69010 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.439+0000 7f59f3fff640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aaca6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.439+0000 7f59f3fff640 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aaca6b400 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cf90ce7d1f4e3 server_cookie=2c926def509606c0 in_seq=0 out_seq=0 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.439+0000 7f59d37fe640 10 client.17184 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:38.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59d37fe640 1 -- 172.21.15.120:0/2025078573 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558aac9c5da0 con 0x558aaca69010 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59d37fe640 10 client.17184 handle_command_reply: tid=0 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 client.17184 shutdown 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 msgr2=0x558aaca56bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005e30 tx=0x7f59ec004630 comp rx=0 tx=0).stop 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 shutdown_connections 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aaca6b400 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca56bd0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59d37fe640 0 client.17184 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d9c0 0x558aaca56630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 --2- 172.21.15.120:0/2025078573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d0a0 0x558aaca591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 >> 172.21.15.120:0/2025078573 conn(0x558aaca4aed0 msgr2=0x558aaca50a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 shutdown_connections 2024-09-10T23:01:38.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 1 -- 172.21.15.120:0/2025078573 wait complete. 2024-09-10T23:01:38.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.440+0000 7f59fcd54740 20 client.17184 trim_cache size 0 max 16384 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 Processor -- start 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 -- start start 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41ac410 con 0x558aaca59950 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41aec90 con 0x558aaca4d9c0 2024-09-10T23:01:38.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59fcd54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41aca80 con 0x558aaca4d0a0 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f8f1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48334/0 (socket says 172.21.15.120:48334) 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 -- 172.21.15.120:0/618138744 learned_addr learned my addr 172.21.15.120:0/618138744 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f3fff640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 -- 172.21.15.120:0/618138744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 msgr2=0x558aaca58370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca58370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 -- 172.21.15.120:0/618138744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 msgr2=0x558aaca5b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca5b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f37fe640 1 -- 172.21.15.120:0/618138744 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aaca571b0 con 0x558aaca4d9c0 2024-09-10T23:01:38.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.458+0000 7f59f3fff640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.459+0000 7f59f37fe640 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f59e80e3050 tx=0x7f59e80a8360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de68f1b3fe1e7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.459+0000 7f59d9ffb640 1 -- 172.21.15.120:0/618138744 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80ab660 con 0x558aaca4d9c0 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.459+0000 7f59d9ffb640 1 -- 172.21.15.120:0/618138744 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e80ab820 con 0x558aaca4d9c0 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.459+0000 7f59d9ffb640 1 -- 172.21.15.120:0/618138744 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80ab480 con 0x558aaca4d9c0 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- 172.21.15.120:0/618138744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 msgr2=0x558aaca5ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f59e80e3050 tx=0x7f59e80a8360 comp rx=0 tx=0).stop 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- 172.21.15.120:0/618138744 shutdown_connections 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca59950 0x558aaca5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca4d9c0 0x558aaca5ad40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 --2- 172.21.15.120:0/618138744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- 172.21.15.120:0/618138744 >> 172.21.15.120:0/618138744 conn(0x558aaca4aed0 msgr2=0x558aaca57c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- 172.21.15.120:0/618138744 shutdown_connections 2024-09-10T23:01:38.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- 172.21.15.120:0/618138744 wait complete. 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 -- start start 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.460+0000 7f59fcd54740 1 Processor -- start 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca61690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41ad150 con 0x558aaca4d9c0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41ae5c0 con 0x558aaca59950 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59f41ad820 con 0x558aaca4d0a0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 10 client.0 _ll_register_callbacks cb 0x7f59e400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36518/0 (socket says 172.21.15.120:36518) 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 -- 172.21.15.120:0/666025996 learned_addr learned my addr 172.21.15.120:0/666025996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f3fff640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca61690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 -- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 msgr2=0x558aaca61690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca61690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 -- 172.21.15.120:0/666025996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 msgr2=0x558aaca5c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 -- 172.21.15.120:0/666025996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aaca5cb80 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f8f1c640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59f37fe640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005fd0 tx=0x7f59e80e8c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec917f8cd67c765 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80ac410 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f59e80a7f80 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.461+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e80ab820 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59e80b52a0 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 10 client.17190 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558aacada8c0 con 0x558aaca4d9c0 2024-09-10T23:01:38.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e80ab820 con 0x558aaca4d9c0 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f59e80aaf70 con 0x558aaca4d9c0 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 10 client.17190 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 20 client.17190 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aacadd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 4 client.17190 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558aac9c1850 con 0x558aaca69010 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.462+0000 7f59f3fff640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aacadd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59d37fe640 10 client.17190 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59f3fff640 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aacadd050 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb5708eaf0e861f server_cookie=52960ed828fa2dbf in_seq=0 out_seq=0 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59d37fe640 1 -- 172.21.15.120:0/666025996 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x558aac9c1850 con 0x558aaca69010 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59d37fe640 10 client.17190 handle_command_reply: tid=0 2024-09-10T23:01:38.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 client.17190 shutdown 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 msgr2=0x558aaca61bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005fd0 tx=0x7f59e80e8c20 comp rx=0 tx=0).stop 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 shutdown_connections 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558aaca69010 0x558aacadd050 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558aaca59950 0x558aaca5c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558aaca4d9c0 0x558aaca61bd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 --2- 172.21.15.120:0/666025996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558aaca4d0a0 0x558aaca61690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59d37fe640 0 client.17190 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.463+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 >> 172.21.15.120:0/666025996 conn(0x558aaca4aed0 msgr2=0x558aaca58480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 shutdown_connections 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fcd54740 1 -- 172.21.15.120:0/666025996 wait complete. 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fcd54740 20 client.17190 trim_cache size 0 max 16384 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:01:38.471 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.473 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.474 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.475 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:01:38.476 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.477 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:38.480 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 msgr2=0x7f59dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 0x7f59dc0b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x558aaca4bdb0 tx=0x7f59e400b390 comp rx=0 tx=0).stop 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 msgr2=0x7f59f4199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d950 tx=0x7f59e800de10 comp rx=0 tx=0).stop 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 shutdown_connections 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59dc0b52e0 0x7f59dc0b77a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f4108020 0x7f59f419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59f41076a0 0x7f59f4199770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 --2- 172.21.15.120:0/3687690155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f59f4101c90 0x7f59f4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.464+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 >> 172.21.15.120:0/3687690155 conn(0x7f59f40fd700 msgr2=0x7f59f40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.465+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 shutdown_connections 2024-09-10T23:01:38.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:38.465+0000 7f59fa97f640 1 -- 172.21.15.120:0/3687690155 wait complete. 2024-09-10T23:01:38.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1562844 16.23 MB/sec execute 393 sec latency 131.698 ms 2024-09-10T23:01:39.382 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:01:39.382 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:39.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1565640 16.22 MB/sec execute 394 sec latency 18.032 ms 2024-09-10T23:01:39.655 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:39 smithi120 ceph-mon[31247]: pgmap v623: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 453 op/s 2024-09-10T23:01:39.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:39 smithi176 ceph-mon[34982]: pgmap v623: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 453 op/s 2024-09-10T23:01:39.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:39 smithi153 ceph-mon[34951]: pgmap v623: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 453 op/s 2024-09-10T23:01:40.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 -- 172.21.15.120:0/4245001633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c1076a0 msgr2=0x7f339c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:40.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 --2- 172.21.15.120:0/4245001633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c1076a0 0x7f339c107ae0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f338c009900 tx=0x7f338c02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:40.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 -- 172.21.15.120:0/4245001633 shutdown_connections 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 --2- 172.21.15.120:0/4245001633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 --2- 172.21.15.120:0/4245001633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c1076a0 0x7f339c107ae0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 --2- 172.21.15.120:0/4245001633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c101c90 0x7f339c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 -- 172.21.15.120:0/4245001633 >> 172.21.15.120:0/4245001633 conn(0x7f339c0fd700 msgr2=0x7f339c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 -- 172.21.15.120:0/4245001633 shutdown_connections 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.131+0000 7f33a38e9640 1 -- 172.21.15.120:0/4245001633 wait complete. 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 Processor -- start 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 -- start start 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1076a0 0x7f339c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:40.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111c70 con 0x7f339c101c90 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111af0 con 0x7f339c108020 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a38e9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111df0 con 0x7f339c1076a0 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.132+0000 7f33a1685640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36534/0 (socket says 172.21.15.120:36534) 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 -- 172.21.15.120:0/2860813166 learned_addr learned my addr 172.21.15.120:0/2860813166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a0e84640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1076a0 0x7f339c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1e86640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 -- 172.21.15.120:0/2860813166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1076a0 msgr2=0x7f339c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1076a0 0x7f339c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 -- 172.21.15.120:0/2860813166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 msgr2=0x7f339c19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 -- 172.21.15.120:0/2860813166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c19e2f0 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1e86640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a1685640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3398009900 tx=0x7f3398009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800d030 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339800d1d0 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c19e5e0 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398005280 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.133+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c06ac80 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.135+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c102f40 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.136+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3398005420 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.136+0000 7f33927fc640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 0x7f33880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.137+0000 7f33a0e84640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 0x7f33880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.137+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f33980e7280 con 0x7f339c101c90 2024-09-10T23:01:40.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.137+0000 7f33a0e84640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 0x7f33880b7820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f338c031440 tx=0x7f338c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:40.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.140+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f33980ae170 con 0x7f339c101c90 2024-09-10T23:01:40.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.373+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f339c102070 con 0x7f339c101c90 2024-09-10T23:01:40.376 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.374+0000 7f33927fc640 1 -- 172.21.15.120:0/2860813166 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f33980ada70 con 0x7f339c101c90 2024-09-10T23:01:40.376 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:01:40.376 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:01:40.377 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 msgr2=0x7f33880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 0x7f33880b7820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f338c031440 tx=0x7f338c005b50 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 msgr2=0x7f339c198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3398009900 tx=0x7f3398009dc0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 shutdown_connections 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33880b5360 0x7f33880b7820 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c108020 0x7f339c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f339c1076a0 0x7f339c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 --2- 172.21.15.120:0/2860813166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f339c101c90 0x7f339c198240 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 >> 172.21.15.120:0/2860813166 conn(0x7f339c0fd700 msgr2=0x7f339c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 shutdown_connections 2024-09-10T23:01:40.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:40.376+0000 7f33a38e9640 1 -- 172.21.15.120:0/2860813166 wait complete. 2024-09-10T23:01:40.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1568495 16.22 MB/sec execute 395 sec latency 33.407 ms 2024-09-10T23:01:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:40 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2860813166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:40.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:40 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2860813166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:40.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:40 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2860813166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:41.078 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:01:41.348 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:41.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1571197 16.22 MB/sec execute 396 sec latency 20.135 ms 2024-09-10T23:01:41.783 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:41 smithi120 ceph-mon[31247]: pgmap v624: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-10T23:01:41.783 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:41 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:41 smithi176 ceph-mon[34982]: pgmap v624: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-10T23:01:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:41 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:41.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 -- 172.21.15.120:0/1031243983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c10d100 msgr2=0x7f708c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 --2- 172.21.15.120:0/1031243983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c10d100 0x7f708c10f550 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f707c009900 tx=0x7f707c02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 -- 172.21.15.120:0/1031243983 shutdown_connections 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 --2- 172.21.15.120:0/1031243983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c10d100 0x7f708c10f550 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 --2- 172.21.15.120:0/1031243983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c104550 0x7f708c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 --2- 172.21.15.120:0/1031243983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c103b50 0x7f708c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.876+0000 7f7090be1640 1 -- 172.21.15.120:0/1031243983 >> 172.21.15.120:0/1031243983 conn(0x7f708c0fd700 msgr2=0x7f708c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.877+0000 7f7090be1640 1 -- 172.21.15.120:0/1031243983 shutdown_connections 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.877+0000 7f7090be1640 1 -- 172.21.15.120:0/1031243983 wait complete. 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.877+0000 7f7090be1640 1 Processor -- start 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 -- start start 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 0x7f708c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 0x7f708c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f708c112db0 con 0x7f708c104550 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f708c112c30 con 0x7f708c10d100 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7090be1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f708c112f30 con 0x7f708c103b50 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f708a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 0x7f708c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36556/0 (socket says 172.21.15.120:36556) 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 -- 172.21.15.120:0/2606046964 learned_addr learned my addr 172.21.15.120:0/2606046964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f708ad9d640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 0x7f708c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 -- 172.21.15.120:0/2606046964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 msgr2=0x7f708c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 0x7f708c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 -- 172.21.15.120:0/2606046964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 msgr2=0x7f708c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 0x7f708c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f7089d9b640 1 -- 172.21.15.120:0/2606046964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c19f330 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f708a59c640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 0x7f708c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.878+0000 7f708ad9d640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 0x7f708c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f7089d9b640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f707400d950 tx=0x7f707400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f70837fe640 1 -- 172.21.15.120:0/2606046964 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074017030 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f70837fe640 1 -- 172.21.15.120:0/2606046964 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7074004020 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f70837fe640 1 -- 172.21.15.120:0/2606046964 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70740047a0 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c19f620 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.879+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1a5ce0 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.882+0000 7f70837fe640 1 -- 172.21.15.120:0/2606046964 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7074009a30 con 0x7f708c104550 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.882+0000 7f70837fe640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 0x7f70780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.882+0000 7f7092fb6740 1 Processor -- start 2024-09-10T23:01:41.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708a59c640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 0x7f70780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f70837fe640 1 -- 172.21.15.120:0/2606046964 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f70740e4f30 con 0x7f708c104550 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 -- start start 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708a59c640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 0x7f70780b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7084005fd0 tx=0x7f7084009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9c79a0 0x56213d9c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57862/0 (socket says 172.21.15.120:57862) 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36562/0 (socket says 172.21.15.120:36562) 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708a59c640 1 -- 172.21.15.120:0/936692521 learned_addr learned my addr 172.21.15.120:0/936692521 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:41.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56213d931ea0 con 0x56213d9bba10 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56213d9212d0 con 0x56213d9c79a0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56213d9250e0 con 0x56213d9bb0f0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.883+0000 7f708ad9d640 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9c79a0 0x56213d9c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 -- 172.21.15.120:0/936692521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9c79a0 msgr2=0x56213d9c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9c79a0 0x56213d9c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 -- 172.21.15.120:0/936692521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 msgr2=0x56213d9c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 -- 172.21.15.120:0/936692521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213d67f650 con 0x56213d9bb0f0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f708a59c640 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f708400fd40 tx=0x7f7084014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396462cf75957ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f7080ff9640 1 -- 172.21.15.120:0/936692521 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70840045c0 con 0x56213d9bb0f0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f7080ff9640 1 -- 172.21.15.120:0/936692521 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7084004760 con 0x56213d9bb0f0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.884+0000 7f7080ff9640 1 -- 172.21.15.120:0/936692521 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708401d610 con 0x56213d9bb0f0 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 msgr2=0x56213d9bb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f708400fd40 tx=0x7f7084014000 comp rx=0 tx=0).stop 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 shutdown_connections 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9c79a0 0x56213d9c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 --2- 172.21.15.120:0/936692521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bb0f0 0x56213d9bb4d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 >> 172.21.15.120:0/936692521 conn(0x56213d9b8f20 msgr2=0x56213d9ba0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 shutdown_connections 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.885+0000 7f7092fb6740 1 -- 172.21.15.120:0/936692521 wait complete. 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 -- start start 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 Processor -- start 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 0x56213d9c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56213d782700 con 0x56213d9bb0f0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56213d8bc160 con 0x56213d9bba10 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56213d9ca620 con 0x56213d9c79a0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f7092fb6740 10 client.0 _ll_register_callbacks cb 0x56213d8d59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f708a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f708a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36578/0 (socket says 172.21.15.120:36578) 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f708a59c640 1 -- 172.21.15.120:0/1704679166 learned_addr learned my addr 172.21.15.120:0/1704679166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.886+0000 7f708ad9d640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7089d9b640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 0x56213d9c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 -- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 msgr2=0x56213d9c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 -- 172.21.15.120:0/1704679166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 msgr2=0x56213d9c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 0x56213d9c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 -- 172.21.15.120:0/1704679166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213d8e9c90 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7089d9b640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 0x56213d9c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f708a59c640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7084004950 tx=0x7f708401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e06c84b667651d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70840045c0 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7084004760 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7084004a40 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7084004c40 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7092fb6740 10 client.17208 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.887+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56213d9d5f10 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7084004760 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f708403b420 con 0x56213d9bb0f0 2024-09-10T23:01:41.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 10 client.17208 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 20 client.17208 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213d9d72c0 0x56213d9d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 4 client.17208 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56213d933df0 con 0x56213d9d72c0 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7089d9b640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213d9d72c0 0x56213d9d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7089d9b640 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213d9d72c0 0x56213d9d96b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef2d83d16db939 server_cookie=6f6a1bf980fe7d26 in_seq=0 out_seq=0 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.888+0000 7f7071ffb640 10 client.17208 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.889+0000 7f7071ffb640 1 -- 172.21.15.120:0/1704679166 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56213d933df0 con 0x56213d9d72c0 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.889+0000 7f7071ffb640 10 client.17208 handle_command_reply: tid=0 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.889+0000 7f7092fb6740 1 client.17208 shutdown 2024-09-10T23:01:41.901 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 msgr2=0x56213d9c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7084004950 tx=0x7f708401eb30 comp rx=0 tx=0).stop 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 shutdown_connections 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213d9d72c0 0x56213d9d96b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7071ffb640 0 client.17208 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bba10 0x56213d9c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 --2- 172.21.15.120:0/1704679166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bb0f0 0x56213d9c7240 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 >> 172.21.15.120:0/1704679166 conn(0x56213d9b8f20 msgr2=0x56213d9beac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 shutdown_connections 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 1 -- 172.21.15.120:0/1704679166 wait complete. 2024-09-10T23:01:41.902 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.890+0000 7f7092fb6740 20 client.17208 trim_cache size 0 max 16384 2024-09-10T23:01:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 Processor -- start 2024-09-10T23:01:41.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 -- start start 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 0x56213d9c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1ac390 con 0x56213d9c79a0 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1aec10 con 0x56213d9bb0f0 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7092fb6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1aca00 con 0x56213d9bba10 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36582/0 (socket says 172.21.15.120:36582) 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 -- 172.21.15.120:0/2047367717 learned_addr learned my addr 172.21.15.120:0/2047367717 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7089d9b640 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 0x56213d9c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 -- 172.21.15.120:0/2047367717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 msgr2=0x56213d9c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 0x56213d9c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 -- 172.21.15.120:0/2047367717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 msgr2=0x56213d9c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f708ad9d640 1 -- 172.21.15.120:0/2047367717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213d9bf410 con 0x56213d9c79a0 2024-09-10T23:01:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.903+0000 7f7089d9b640 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 0x56213d9c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.904+0000 7f708ad9d640 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f707c02aa00 tx=0x7f707c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5ffe7dbcfa7a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.904+0000 7f7073fff640 1 -- 172.21.15.120:0/2047367717 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c035040 con 0x56213d9c79a0 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.904+0000 7f7073fff640 1 -- 172.21.15.120:0/2047367717 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c003830 con 0x56213d9c79a0 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.904+0000 7f7073fff640 1 -- 172.21.15.120:0/2047367717 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c004b60 con 0x56213d9c79a0 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- 172.21.15.120:0/2047367717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 msgr2=0x56213d9c18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f707c02aa00 tx=0x7f707c003680 comp rx=0 tx=0).stop 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- 172.21.15.120:0/2047367717 shutdown_connections 2024-09-10T23:01:41.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9c79a0 0x56213d9c18a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9bba10 0x56213d9c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 --2- 172.21.15.120:0/2047367717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- 172.21.15.120:0/2047367717 >> 172.21.15.120:0/2047367717 conn(0x56213d9b8f20 msgr2=0x56213d9c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- 172.21.15.120:0/2047367717 shutdown_connections 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- 172.21.15.120:0/2047367717 wait complete. 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 -- start start 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.905+0000 7f7092fb6740 1 Processor -- start 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36592/0 (socket says 172.21.15.120:36592) 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 -- 172.21.15.120:0/3650484448 learned_addr learned my addr 172.21.15.120:0/3650484448 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1ad0d0 con 0x56213d9bba10 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1ae540 con 0x56213d9bb0f0 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f708c1ad7a0 con 0x56213d9c79a0 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f708a59c640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7092fb6740 10 client.0 _ll_register_callbacks cb 0x7f7074005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f708ad9d640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 -- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 msgr2=0x56213d9d9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9d9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 -- 172.21.15.120:0/3650484448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 msgr2=0x56213d9dcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9dcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213d9d9da0 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7089d9b640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7074005e00 tx=0x7f70740a7720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839a37fef53dff3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.906+0000 7f7071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70740d2070 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f70740a7930 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c003830 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70740ab6c0 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 10 client.17214 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56213da44d40 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c003830 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f70740a7010 con 0x56213d9bba10 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 10 client.17214 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 20 client.17214 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:41.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213da45470 0x56213da47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 4 client.17214 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56213d92f8a0 con 0x56213da45470 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.907+0000 7f708a59c640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213da45470 0x56213da47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f708a59c640 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213da45470 0x56213da47890 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ffe1662e4e8dcbd server_cookie=91127dbc65b22e57 in_seq=0 out_seq=0 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7071ffb640 10 client.17214 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7071ffb640 1 -- 172.21.15.120:0/3650484448 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56213d92f8a0 con 0x56213da45470 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7071ffb640 10 client.17214 handle_command_reply: tid=0 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7092fb6740 1 client.17214 shutdown 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 msgr2=0x56213d9d91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.908+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7074005e00 tx=0x7f70740a7720 comp rx=0 tx=0).stop 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 shutdown_connections 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56213da45470 0x56213da47890 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56213d9c79a0 0x56213d9d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7071ffb640 0 client.17214 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56213d9bba10 0x56213d9d91c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 --2- 172.21.15.120:0/3650484448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56213d9bb0f0 0x56213d9dcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 >> 172.21.15.120:0/3650484448 conn(0x56213d9b8f20 msgr2=0x56213d9b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 shutdown_connections 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 1 -- 172.21.15.120:0/3650484448 wait complete. 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7092fb6740 20 client.17214 trim_cache size 0 max 16384 2024-09-10T23:01:41.916 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 msgr2=0x7f70780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 0x7f70780b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7084005fd0 tx=0x7f7084009500 comp rx=0 tx=0).stop 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 msgr2=0x7f708c1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f707400d950 tx=0x7f707400de10 comp rx=0 tx=0).stop 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 shutdown_connections 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f70780b52e0 0x7f70780b77a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f708c10d100 0x7f708c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f708c104550 0x7f708c1997c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 --2- 172.21.15.120:0/2606046964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f708c103b50 0x7f708c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.909+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 >> 172.21.15.120:0/2606046964 conn(0x7f708c0fd700 msgr2=0x7f708c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.910+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 shutdown_connections 2024-09-10T23:01:41.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:41.910+0000 7f7090be1640 1 -- 172.21.15.120:0/2606046964 wait complete. 2024-09-10T23:01:41.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:41 smithi153 ceph-mon[34951]: pgmap v624: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-09-10T23:01:41.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:41 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:42.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1574290 16.22 MB/sec execute 397 sec latency 12.220 ms 2024-09-10T23:01:43.612 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:01:43.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:01:43.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1577299 16.22 MB/sec execute 398 sec latency 75.937 ms 2024-09-10T23:01:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:43 smithi176 ceph-mon[34982]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-10T23:01:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:43 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:43 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:43 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:01:43.882 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:43 smithi120 ceph-mon[31247]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-10T23:01:43.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:43 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:43 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:43 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:01:43.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:43 smithi153 ceph-mon[34951]: pgmap v625: 97 pgs: 97 active+clean; 49 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-10T23:01:43.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:43 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:43 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:43 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:01:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 -- 172.21.15.120:0/1086580998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694101cd0 msgr2=0x7f96941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 --2- 172.21.15.120:0/1086580998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694101cd0 0x7f96941020b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9684009900 tx=0x7f968402a410 comp rx=0 tx=0).stop 2024-09-10T23:01:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 -- 172.21.15.120:0/1086580998 shutdown_connections 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 --2- 172.21.15.120:0/1086580998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 --2- 172.21.15.120:0/1086580998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96941076e0 0x7f9694107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 --2- 172.21.15.120:0/1086580998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694101cd0 0x7f96941020b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 -- 172.21.15.120:0/1086580998 >> 172.21.15.120:0/1086580998 conn(0x7f96940fd740 msgr2=0x7f96940ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 -- 172.21.15.120:0/1086580998 shutdown_connections 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.392+0000 7f969aef9640 1 -- 172.21.15.120:0/1086580998 wait complete. 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 Processor -- start 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 -- start start 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 0x7f9694199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9694111d20 con 0x7f96941076e0 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9694111ba0 con 0x7f9694101cd0 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f969aef9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9694111ea0 con 0x7f9694108060 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60660/0 (socket says 172.21.15.120:60660) 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 -- 172.21.15.120:0/4146181163 learned_addr learned my addr 172.21.15.120:0/4146181163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9699496640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9698c95640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 0x7f9694199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 -- 172.21.15.120:0/4146181163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 msgr2=0x7f969419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 -- 172.21.15.120:0/4146181163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 msgr2=0x7f9694199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 0x7f9694199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9693fff640 1 -- 172.21.15.120:0/4146181163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969419f3e0 con 0x7f96941076e0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9699496640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.393+0000 7f9698c95640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 0x7f9694199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f9693fff640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d950 tx=0x7f967c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f9691ffb640 1 -- 172.21.15.120:0/4146181163 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c017030 con 0x7f96941076e0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f9691ffb640 1 -- 172.21.15.120:0/4146181163 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c004020 con 0x7f96941076e0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969419f6d0 con 0x7f96941076e0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f9691ffb640 1 -- 172.21.15.120:0/4146181163 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0047a0 con 0x7f96941076e0 2024-09-10T23:01:44.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.394+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96941a5d90 con 0x7f96941076e0 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 Processor -- start 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9691ffb640 1 -- 172.21.15.120:0/4146181163 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f967c009a30 con 0x7f96941076e0 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 -- start start 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d7772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d785b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56048d6ede60 con 0x56048d777800 2024-09-10T23:01:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56048d6dd290 con 0x56048d776ee0 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f969d2ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56048d6e10a0 con 0x56048d783790 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60674/0 (socket says 172.21.15.120:60674) 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9698c95640 1 -- 172.21.15.120:0/843243569 learned_addr learned my addr 172.21.15.120:0/843243569 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9693fff640 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d785b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.397+0000 7f9699496640 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d7772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 -- 172.21.15.120:0/843243569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 msgr2=0x56048d785b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d785b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 -- 172.21.15.120:0/843243569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 msgr2=0x56048d7772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d7772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 -- 172.21.15.120:0/843243569 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048d43b930 con 0x56048d777800 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9691ffb640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 0x7f96800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9699496640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 0x7f96800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9691ffb640 1 -- 172.21.15.120:0/4146181163 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f967c0e5520 con 0x7f96941076e0 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9698c95640 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9684033000 tx=0x7f96840314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e63dfb1c471bb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f9699496640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 0x7f96800b77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f968c005fd0 tx=0x7f968c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f968affd640 1 -- 172.21.15.120:0/843243569 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035040 con 0x56048d777800 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f968affd640 1 -- 172.21.15.120:0/843243569 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9684031890 con 0x56048d777800 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.398+0000 7f968affd640 1 -- 172.21.15.120:0/843243569 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96840049d0 con 0x56048d777800 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 -- 172.21.15.120:0/843243569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 msgr2=0x56048d783250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9684033000 tx=0x7f96840314a0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 -- 172.21.15.120:0/843243569 shutdown_connections 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d785b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d777800 0x56048d783250 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 --2- 172.21.15.120:0/843243569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d7772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 -- 172.21.15.120:0/843243569 >> 172.21.15.120:0/843243569 conn(0x56048d774d10 msgr2=0x56048d775eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 -- 172.21.15.120:0/843243569 shutdown_connections 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.399+0000 7f969d2ce740 1 -- 172.21.15.120:0/843243569 wait complete. 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 -- start start 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 Processor -- start 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d777800 0x56048d780ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d781220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56048d53e890 con 0x56048d776ee0 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56048d677d40 con 0x56048d777800 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56048d7863e0 con 0x56048d783790 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f969d2ce740 10 client.0 _ll_register_callbacks cb 0x56048d6dbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60678/0 (socket says 172.21.15.120:60678) 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f9698c95640 1 -- 172.21.15.120:0/3151128482 learned_addr learned my addr 172.21.15.120:0/3151128482 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f9693fff640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d777800 0x56048d780ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.400+0000 7f9699496640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d781220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 -- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 msgr2=0x56048d781220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d781220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 -- 172.21.15.120:0/3151128482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d777800 msgr2=0x56048d780ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d777800 0x56048d780ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 -- 172.21.15.120:0/3151128482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048d793c20 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9698c95640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f968402a930 tx=0x7f9684005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac33ed2fd964ddb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9673fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035040 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9684004250 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9684031890 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9684057050 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.401+0000 7f969d2ce740 10 client.17226 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56048d7965d0 con 0x56048d776ee0 2024-09-10T23:01:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9684031890 con 0x56048d776ee0 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9684003710 con 0x56048d776ee0 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 10 client.17226 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 20 client.17226 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d797920 0x56048d799d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 4 client.17226 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048d6efdb0 con 0x56048d797920 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f9693fff640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d797920 0x56048d799d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.402+0000 7f9673fff640 10 client.17226 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.403+0000 7f9693fff640 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d797920 0x56048d799d10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ef6b8b29403b9 server_cookie=4252475c464dff38 in_seq=0 out_seq=0 2024-09-10T23:01:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f9673fff640 1 -- 172.21.15.120:0/3151128482 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56048d6efdb0 con 0x56048d797920 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f9673fff640 10 client.17226 handle_command_reply: tid=0 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 client.17226 shutdown 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 msgr2=0x56048d7807a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f968402a930 tx=0x7f9684005580 comp rx=0 tx=0).stop 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 shutdown_connections 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d797920 0x56048d799d10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d783790 0x56048d781220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.405+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d777800 0x56048d780ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f969d2ce740 1 --2- 172.21.15.120:0/3151128482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d7807a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 >> 172.21.15.120:0/3151128482 conn(0x56048d774d10 msgr2=0x56048d773f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f9673fff640 0 client.17226 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 shutdown_connections 2024-09-10T23:01:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f969d2ce740 1 -- 172.21.15.120:0/3151128482 wait complete. 2024-09-10T23:01:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.406+0000 7f969d2ce740 20 client.17226 trim_cache size 0 max 16384 2024-09-10T23:01:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 Processor -- start 2024-09-10T23:01:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 -- start start 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d77df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 0x56048d79f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aa200 con 0x56048d783790 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aca80 con 0x56048d776ee0 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f969d2ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aa870 con 0x56048d777800 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9699496640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 0x56048d79f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9698c95640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9699496640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 0x56048d79f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60682/0 (socket says 172.21.15.120:60682) 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9698c95640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59814/0 (socket says 172.21.15.120:59814) 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9699496640 1 -- 172.21.15.120:0/732466848 learned_addr learned my addr 172.21.15.120:0/732466848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.424+0000 7f9693fff640 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d77df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 -- 172.21.15.120:0/732466848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 msgr2=0x56048d77df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d77df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 -- 172.21.15.120:0/732466848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 msgr2=0x56048d79f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 0x56048d79f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 -- 172.21.15.120:0/732466848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048d774b40 con 0x56048d776ee0 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f9698c95640 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9684003a80 tx=0x7f9684031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57c2d436111154d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f968a7fc640 1 -- 172.21.15.120:0/732466848 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684000e60 con 0x56048d776ee0 2024-09-10T23:01:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f968a7fc640 1 -- 172.21.15.120:0/732466848 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9684005730 con 0x56048d776ee0 2024-09-10T23:01:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.425+0000 7f968a7fc640 1 -- 172.21.15.120:0/732466848 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96840049d0 con 0x56048d776ee0 2024-09-10T23:01:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- 172.21.15.120:0/732466848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 msgr2=0x56048d77d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9684003a80 tx=0x7f9684031400 comp rx=0 tx=0).stop 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- 172.21.15.120:0/732466848 shutdown_connections 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d783790 0x56048d79f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d77df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 --2- 172.21.15.120:0/732466848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d776ee0 0x56048d77d9d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- 172.21.15.120:0/732466848 >> 172.21.15.120:0/732466848 conn(0x56048d774d10 msgr2=0x56048d7825f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- 172.21.15.120:0/732466848 shutdown_connections 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- 172.21.15.120:0/732466848 wait complete. 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 -- start start 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.426+0000 7f969d2ce740 1 Processor -- start 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d78e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d783790 0x56048d8034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aaf40 con 0x56048d776ee0 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96941ac3b0 con 0x56048d783790 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f96941ab610 con 0x56048d777800 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f969d2ce740 10 client.0 _ll_register_callbacks cb 0x7f968c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60696/0 (socket says 172.21.15.120:60696) 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 -- 172.21.15.120:0/2938964282 learned_addr learned my addr 172.21.15.120:0/2938964282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9693fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d78e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9699496640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d783790 0x56048d8034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 -- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 msgr2=0x56048d78e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d78e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 -- 172.21.15.120:0/2938964282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d783790 msgr2=0x56048d8034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d783790 0x56048d8034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 -- 172.21.15.120:0/2938964282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048d803a00 con 0x56048d776ee0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9693fff640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d78e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9698c95640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9684003930 tx=0x7f9684005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff77dc042e3fbefc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.427+0000 7f9673fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035070 con 0x56048d776ee0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9684004450 con 0x56048d776ee0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56048d6eb860 con 0x56048d776ee0 2024-09-10T23:01:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9684031590 con 0x56048d776ee0 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 10 client.17232 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56048d804c40 con 0x56048d776ee0 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56048d6eb860 con 0x56048d776ee0 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f968403c440 con 0x56048d776ee0 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 10 client.17232 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 20 client.17232 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d8050c0 0x56048d8074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 4 client.17232 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.428+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56048d6e8c90 con 0x56048d8050c0 2024-09-10T23:01:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.429+0000 7f9693fff640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d8050c0 0x56048d8074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.429+0000 7f9693fff640 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d8050c0 0x56048d8074b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49d002d8c7c1c1b server_cookie=b600876061704b46 in_seq=0 out_seq=0 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.429+0000 7f9673fff640 10 client.17232 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f9673fff640 1 -- 172.21.15.120:0/2938964282 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56048d6e8c90 con 0x56048d8050c0 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f9673fff640 10 client.17232 handle_command_reply: tid=0 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 client.17232 shutdown 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 msgr2=0x56048d78deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9684003930 tx=0x7f9684005580 comp rx=0 tx=0).stop 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 shutdown_connections 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56048d8050c0 0x56048d8074b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56048d783790 0x56048d8034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56048d777800 0x56048d78e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f969d2ce740 1 --2- 172.21.15.120:0/2938964282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56048d776ee0 0x56048d78deb0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.430+0000 7f9673fff640 0 client.17232 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 >> 172.21.15.120:0/2938964282 conn(0x56048d774d10 msgr2=0x56048d781dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 shutdown_connections 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969d2ce740 1 -- 172.21.15.120:0/2938964282 wait complete. 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969d2ce740 20 client.17232 trim_cache size 0 max 16384 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:01:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1b9690c2-c976-40be-b599-edc68acd7a29", 2024-09-10T23:01:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:01:44.440 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 msgr2=0x7f96800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 0x7f96800b77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f968c005fd0 tx=0x7f968c009600 comp rx=0 tx=0).stop 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 msgr2=0x7f9694199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.431+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d950 tx=0x7f967c00de10 comp rx=0 tx=0).stop 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 shutdown_connections 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96800b52e0 0x7f96800b77a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9694108060 0x7f969419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96941076e0 0x7f9694199870 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 --2- 172.21.15.120:0/4146181163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9694101cd0 0x7f9694199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 >> 172.21.15.120:0/4146181163 conn(0x7f96940fd740 msgr2=0x7f969406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 shutdown_connections 2024-09-10T23:01:44.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:44.432+0000 7f969aef9640 1 -- 172.21.15.120:0/4146181163 wait complete. 2024-09-10T23:01:44.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1579554 16.20 MB/sec execute 399 sec latency 39.686 ms 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.765+0000 7f5e1db16640 1 -- 172.21.15.120:0/1591086269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18101860 msgr2=0x7f5e18101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.765+0000 7f5e1db16640 1 --2- 172.21.15.120:0/1591086269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18101860 0x7f5e18101c60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009930 tx=0x7f5e0802a420 comp rx=0 tx=0).stop 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 -- 172.21.15.120:0/1591086269 shutdown_connections 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 --2- 172.21.15.120:0/1591086269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 0x7f5e18109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 --2- 172.21.15.120:0/1591086269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e181029b0 0x7f5e18102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 --2- 172.21.15.120:0/1591086269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e18101860 0x7f5e18101c60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 -- 172.21.15.120:0/1591086269 >> 172.21.15.120:0/1591086269 conn(0x7f5e180fd140 msgr2=0x7f5e180ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 -- 172.21.15.120:0/1591086269 shutdown_connections 2024-09-10T23:01:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 -- 172.21.15.120:0/1591086269 wait complete. 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 Processor -- start 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.766+0000 7f5e1db16640 1 -- start start 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 0x7f5e181a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810d4d0 con 0x7f5e181029b0 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810d350 con 0x7f5e18103370 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e1db16640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1810d650 con 0x7f5e18101860 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60720/0 (socket says 172.21.15.120:60720) 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47590/0 (socket says 172.21.15.120:47590) 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e16ffd640 1 -- 172.21.15.120:0/623188762 learned_addr learned my addr 172.21.15.120:0/623188762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e17fff640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 0x7f5e181a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 -- 172.21.15.120:0/623188762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 msgr2=0x7f5e181a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 0x7f5e181a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 -- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 msgr2=0x7f5e1819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 -- 172.21.15.120:0/623188762 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e181a0970 con 0x7f5e18101860 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.767+0000 7f5e177fe640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009230 tx=0x7f5e080042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.768+0000 7f5e14ff9640 1 -- 172.21.15.120:0/623188762 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0803f030 con 0x7f5e18101860 2024-09-10T23:01:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.768+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e181a0c00 con 0x7f5e18101860 2024-09-10T23:01:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.768+0000 7f5e14ff9640 1 -- 172.21.15.120:0/623188762 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e08031910 con 0x7f5e18101860 2024-09-10T23:01:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.768+0000 7f5e14ff9640 1 -- 172.21.15.120:0/623188762 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08035070 con 0x7f5e18101860 2024-09-10T23:01:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.768+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e181a10f0 con 0x7f5e18101860 2024-09-10T23:01:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e14ff9640 1 -- 172.21.15.120:0/623188762 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5e0803b9c0 con 0x7f5e18101860 2024-09-10T23:01:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 Processor -- start 2024-09-10T23:01:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 -- start start 2024-09-10T23:01:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 0x55e871090060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e8710954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e870ffb6f0 con 0x55e8710905a0 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e870feab20 con 0x55e87108dc70 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.770+0000 7f5e20522740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e870fee930 con 0x55e871092fe0 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60724/0 (socket says 172.21.15.120:60724) 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 -- 172.21.15.120:0/2765502272 learned_addr learned my addr 172.21.15.120:0/2765502272 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e17fff640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 0x55e871090060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e8710954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 -- 172.21.15.120:0/2765502272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 msgr2=0x55e8710954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e8710954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 -- 172.21.15.120:0/2765502272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 msgr2=0x55e871090060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 0x55e871090060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 -- 172.21.15.120:0/2765502272 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e870fb0530 con 0x55e8710905a0 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e17fff640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 0x55e871090060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e14ff9640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 0x7f5e000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5e0810e4e0 tx=0x7f5e0810e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93492c6c04e47dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e17fff640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 0x7f5e000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5dfdffb640 1 -- 172.21.15.120:0/2765502272 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080c91e0 con 0x55e8710905a0 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5e14ff9640 1 -- 172.21.15.120:0/623188762 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5e081062f0 con 0x7f5e18101860 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5dfdffb640 1 -- 172.21.15.120:0/2765502272 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e080cc6a0 con 0x55e8710905a0 2024-09-10T23:01:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.771+0000 7f5dfdffb640 1 -- 172.21.15.120:0/2765502272 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080cc300 con 0x55e8710905a0 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- 172.21.15.120:0/2765502272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 msgr2=0x55e871092990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e17fff640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 0x7f5e000b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009fd0 tx=0x7f5e10009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5e0810e4e0 tx=0x7f5e0810e5d0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- 172.21.15.120:0/2765502272 shutdown_connections 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e8710954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e871092990 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 --2- 172.21.15.120:0/2765502272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e87108dc70 0x55e871090060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- 172.21.15.120:0/2765502272 >> 172.21.15.120:0/2765502272 conn(0x55e8710849b0 msgr2=0x55e871085740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- 172.21.15.120:0/2765502272 shutdown_connections 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- 172.21.15.120:0/2765502272 wait complete. 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.772+0000 7f5e20522740 1 -- start start 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 Processor -- start 2024-09-10T23:01:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e871096fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 0x55e871098230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e870f22ab0 con 0x55e871092fe0 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e870f90730 con 0x55e8710905a0 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e871095d80 con 0x55e87108dc70 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e20522740 10 client.0 _ll_register_callbacks cb 0x55e870fe46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 0x55e871098230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59854/0 (socket says 172.21.15.120:59854) 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 -- 172.21.15.120:0/2669827113 learned_addr learned my addr 172.21.15.120:0/2669827113 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e871096fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 -- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 msgr2=0x55e871096fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e871096fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 -- 172.21.15.120:0/2669827113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 msgr2=0x55e871098230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 0x55e871098230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e16ffd640 1 -- 172.21.15.120:0/2669827113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e870d4c790 con 0x55e8710905a0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.773+0000 7f5e17fff640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 0x55e871098230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c012d20 tx=0x7f5e0c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35dc6a705c2f1091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c010430 con 0x55e8710905a0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e080cc6a0 con 0x55e8710905a0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5e0c0105d0 con 0x55e8710905a0 2024-09-10T23:01:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e0c0107d0 con 0x55e8710905a0 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5e20522740 10 client.26606 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e870dc5620 con 0x55e8710905a0 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.774+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e080cc6a0 con 0x55e8710905a0 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5e0c0278e0 con 0x55e8710905a0 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 10 client.26606 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 10 client.26606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 20 client.26606 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e8710a2990 0x55e8710a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 4 client.26606 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e870ffd640 con 0x55e8710a2990 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e8710a2990 0x55e8710a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5df6ffd640 10 client.26606 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.775+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e8710a2990 0x55e8710a4df0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dde8effc58318c3 server_cookie=23c75a42dad0c6b8 in_seq=0 out_seq=0 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5df6ffd640 1 -- 172.21.15.120:0/2669827113 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e870ffd640 con 0x55e8710a2990 2024-09-10T23:01:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5df6ffd640 10 client.26606 handle_command_reply: tid=0 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 client.26606 shutdown 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 msgr2=0x55e871097660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c012d20 tx=0x7f5e0c00f790 comp rx=0 tx=0).stop 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 shutdown_connections 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e8710a2990 0x55e8710a4df0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e871092fe0 0x55e871098230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871097660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 --2- 172.21.15.120:0/2669827113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e871096fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5df6ffd640 0 client.26606 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 >> 172.21.15.120:0/2669827113 conn(0x55e8710849b0 msgr2=0x55e871083bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 shutdown_connections 2024-09-10T23:01:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 1 -- 172.21.15.120:0/2669827113 wait complete. 2024-09-10T23:01:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.776+0000 7f5e20522740 20 client.26606 trim_cache size 0 max 16384 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 Processor -- start 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 -- start start 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871093900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e871096fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a5bb0 con 0x55e87108dc70 2024-09-10T23:01:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a8430 con 0x55e8710905a0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e20522740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a6220 con 0x55e871092fe0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.791+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60738/0 (socket says 172.21.15.120:60738) 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 -- 172.21.15.120:0/2500774604 learned_addr learned my addr 172.21.15.120:0/2500774604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e17fff640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e871096fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871093900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 -- 172.21.15.120:0/2500774604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 msgr2=0x55e871096fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e871096fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 -- 172.21.15.120:0/2500774604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 msgr2=0x55e871093900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871093900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 -- 172.21.15.120:0/2500774604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8710903d0 con 0x55e87108dc70 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871093900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5e177fe640 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c9e00 tx=0x7f5e08109ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f38923df9306199 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5dfd7fa640 1 -- 172.21.15.120:0/2500774604 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08049040 con 0x55e87108dc70 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5dfd7fa640 1 -- 172.21.15.120:0/2500774604 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e08109b90 con 0x55e87108dc70 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.792+0000 7f5dfd7fa640 1 -- 172.21.15.120:0/2500774604 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080cc4e0 con 0x55e87108dc70 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- 172.21.15.120:0/2500774604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 msgr2=0x55e8710933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c9e00 tx=0x7f5e08109ef0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- 172.21.15.120:0/2500774604 shutdown_connections 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e871092fe0 0x55e871096fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8710905a0 0x55e871093900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 --2- 172.21.15.120:0/2500774604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e87108dc70 0x55e8710933c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- 172.21.15.120:0/2500774604 >> 172.21.15.120:0/2500774604 conn(0x55e8710849b0 msgr2=0x55e871088cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- 172.21.15.120:0/2500774604 shutdown_connections 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- 172.21.15.120:0/2500774604 wait complete. 2024-09-10T23:01:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.793+0000 7f5e20522740 1 -- start start 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 Processor -- start 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e87109bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e871092fe0 0x55e87109d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a68f0 con 0x55e8710905a0 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a7d60 con 0x55e871092fe0 2024-09-10T23:01:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181a6fc0 con 0x55e87108dc70 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e20522740 10 client.0 _ll_register_callbacks cb 0x7f5e0c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e16ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60746/0 (socket says 172.21.15.120:60746) 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e16ffd640 1 -- 172.21.15.120:0/173770275 learned_addr learned my addr 172.21.15.120:0/173770275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e87109bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e17fff640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e871092fe0 0x55e87109d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 -- 172.21.15.120:0/173770275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e871092fe0 msgr2=0x55e87109d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e871092fe0 0x55e87109d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 -- 172.21.15.120:0/173770275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 msgr2=0x55e87109c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 -- 172.21.15.120:0/173770275 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87109d9b0 con 0x55e87108dc70 2024-09-10T23:01:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5e177fe640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e87109bf10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e0810e4e0 tx=0x7f5e0803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5fac69f2e2746c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.794+0000 7f5df6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08049070 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5e0803bce0 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e08109b90 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e080cc960 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 10 client.26728 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e87109ed20 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e08109b90 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5e080ccdf0 con 0x55e87108dc70 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 10 client.26728 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 20 client.26728 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e87110f910 0x55e871111d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 4 client.26728 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e870ff90f0 con 0x55e87110f910 2024-09-10T23:01:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.795+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e87110f910 0x55e871111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.796+0000 7f5df6ffd640 10 client.26728 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.796+0000 7f5e16ffd640 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e87110f910 0x55e871111d00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7915df78b1af486d server_cookie=35ea383c707bf197 in_seq=0 out_seq=0 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5df6ffd640 1 -- 172.21.15.120:0/173770275 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e870ff90f0 con 0x55e87110f910 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5df6ffd640 10 client.26728 handle_command_reply: tid=0 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 client.26728 shutdown 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 msgr2=0x55e87109bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e87109bf10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e0810e4e0 tx=0x7f5e0803b460 comp rx=0 tx=0).stop 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 shutdown_connections 2024-09-10T23:01:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e87110f910 0x55e871111d00 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5df6ffd640 0 client.26728 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e871092fe0 0x55e87109d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8710905a0 0x55e87109c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 --2- 172.21.15.120:0/173770275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e87108dc70 0x55e87109bf10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 >> 172.21.15.120:0/173770275 conn(0x55e8710849b0 msgr2=0x55e8710933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 shutdown_connections 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.997+0000 7f5e20522740 1 -- 172.21.15.120:0/173770275 wait complete. 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e20522740 20 client.26728 trim_cache size 0 max 16384 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:01:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:01:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:01:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:01:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 msgr2=0x7f5e000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 0x7f5e000b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009fd0 tx=0x7f5e10009500 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 msgr2=0x7f5e1806b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009230 tx=0x7f5e080042f0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 shutdown_connections 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5e000b52e0 0x7f5e000b77a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e18103370 0x7f5e181a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5e181029b0 0x7f5e1819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 --2- 172.21.15.120:0/623188762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e18101860 0x7f5e1806b1e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.998+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 >> 172.21.15.120:0/623188762 conn(0x7f5e180fd140 msgr2=0x7f5e18105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.999+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 shutdown_connections 2024-09-10T23:01:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:44.999+0000 7f5e1db16640 1 -- 172.21.15.120:0/623188762 wait complete. 2024-09-10T23:01:45.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:45.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1581359 16.19 MB/sec execute 400 sec latency 121.585 ms 2024-09-10T23:01:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 380 op/s 2024-09-10T23:01:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:01:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:01:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:45 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 380 op/s 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:45 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:01:45.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: pgmap v626: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 380 op/s 2024-09-10T23:01:45.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:01:45.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:01:45.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:45.950 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:45 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:01:46.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 -- 172.21.15.120:0/2984897945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 msgr2=0x7fd36810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 --2- 172.21.15.120:0/2984897945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36810e840 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd360009ca0 tx=0x7fd36002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 -- 172.21.15.120:0/2984897945 shutdown_connections 2024-09-10T23:01:46.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 --2- 172.21.15.120:0/2984897945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36810e840 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 --2- 172.21.15.120:0/2984897945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3681076c0 0x7fd368107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 --2- 172.21.15.120:0/2984897945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd368101cb0 0x7fd368102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 -- 172.21.15.120:0/2984897945 >> 172.21.15.120:0/2984897945 conn(0x7fd3680fd700 msgr2=0x7fd3680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 -- 172.21.15.120:0/2984897945 shutdown_connections 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.149+0000 7fd3703e2640 1 -- 172.21.15.120:0/2984897945 wait complete. 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 Processor -- start 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 -- start start 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd368101cb0 0x7fd36819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd368111d00 con 0x7fd368108040 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd368111b80 con 0x7fd368101cb0 2024-09-10T23:01:46.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd3703e2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd368111e80 con 0x7fd3681076c0 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.150+0000 7fd36e97f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60764/0 (socket says 172.21.15.120:60764) 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 -- 172.21.15.120:0/395569930 learned_addr learned my addr 172.21.15.120:0/395569930 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47646/0 (socket says 172.21.15.120:47646) 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e17e640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd368101cb0 0x7fd36819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 -- 172.21.15.120:0/395569930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 msgr2=0x7fd36819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 -- 172.21.15.120:0/395569930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd368101cb0 msgr2=0x7fd36819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd368101cb0 0x7fd36819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 -- 172.21.15.120:0/395569930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3681a5430 con 0x7fd368108040 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36d97d640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd36e97f640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098d0 tx=0x7fd3600317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd35f7fe640 1 -- 172.21.15.120:0/395569930 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360003b70 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd35f7fe640 1 -- 172.21.15.120:0/395569930 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd360003d10 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3681a56c0 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd35f7fe640 1 -- 172.21.15.120:0/395569930 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36002fa20 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.151+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3681a5c00 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd35f7fe640 1 -- 172.21.15.120:0/395569930 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd36002fc40 con 0x7fd368108040 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 Processor -- start 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 -- start start 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af90480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 0x56434af9c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56434af06e50 con 0x56434af9c950 2024-09-10T23:01:46.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56434aef6280 con 0x56434af900a0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd3727b7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56434aefa090 con 0x56434af909c0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd36e17e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 0x56434af9c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60778/0 (socket says 172.21.15.120:60778) 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.154+0000 7fd36d97d640 1 -- 172.21.15.120:0/1285784809 learned_addr learned my addr 172.21.15.120:0/1285784809 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36d97d640 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af90480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd35f7fe640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 0x7fd3540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 -- 172.21.15.120:0/1285784809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 msgr2=0x56434af90480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af90480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 -- 172.21.15.120:0/1285784809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 msgr2=0x56434af9ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 -- 172.21.15.120:0/1285784809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434ac54650 con 0x56434af909c0 2024-09-10T23:01:46.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd35f7fe640 1 -- 172.21.15.120:0/395569930 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd360106e70 con 0x7fd368108040 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e17e640 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 0x56434af9c410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd364009c20 tx=0x7fd364011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd01ccb78e4a2441 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd36e97f640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 0x7fd3540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd35cde6640 1 -- 172.21.15.120:0/1285784809 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364004050 con 0x56434af909c0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd35cde6640 1 -- 172.21.15.120:0/1285784809 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3640054d0 con 0x56434af909c0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.155+0000 7fd35cde6640 1 -- 172.21.15.120:0/1285784809 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364004c80 con 0x56434af909c0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- 172.21.15.120:0/1285784809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 msgr2=0x56434af9c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd36e97f640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 0x7fd3540b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ca8d0 tx=0x7fd3600f0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 0x56434af9c410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd364009c20 tx=0x7fd364011810 comp rx=0 tx=0).stop 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- 172.21.15.120:0/1285784809 shutdown_connections 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af909c0 0x56434af9c410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 --2- 172.21.15.120:0/1285784809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af90480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- 172.21.15.120:0/1285784809 >> 172.21.15.120:0/1285784809 conn(0x56434af8ded0 msgr2=0x56434af8f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- 172.21.15.120:0/1285784809 shutdown_connections 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- 172.21.15.120:0/1285784809 wait complete. 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.156+0000 7fd3727b7740 1 -- start start 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 Processor -- start 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af9c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af99c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56434ad57700 con 0x56434af909c0 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56434ae91110 con 0x56434af900a0 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56434af9f5d0 con 0x56434af9c950 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd3727b7740 10 client.0 _ll_register_callbacks cb 0x56434aeaa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60780/0 (socket says 172.21.15.120:60780) 2024-09-10T23:01:46.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 -- 172.21.15.120:0/2886845788 learned_addr learned my addr 172.21.15.120:0/2886845788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af99c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af9c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 -- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 msgr2=0x56434af99c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af99c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 -- 172.21.15.120:0/2886845788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 msgr2=0x56434af9c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af9c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 -- 172.21.15.120:0/2886845788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434aebec40 con 0x56434af909c0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af99c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd364005740 tx=0x7fd364005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378a8ecbf520673d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.157+0000 7fd351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364029070 con 0x56434af909c0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3640054d0 con 0x56434af909c0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd364003a00 con 0x56434af909c0 2024-09-10T23:01:46.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd364035050 con 0x56434af909c0 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 10 client.17262 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56434afaaec0 con 0x56434af909c0 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3640054d0 con 0x56434af909c0 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd364005140 con 0x56434af909c0 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 10 client.17262 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 20 client.17262 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434afac210 0x56434afae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 4 client.17262 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.158+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434af08da0 con 0x56434afac210 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.159+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434afac210 0x56434afae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.159+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434afac210 0x56434afae600 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5a1fbd044b967a server_cookie=85a3648cbdb14abc in_seq=0 out_seq=0 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.159+0000 7fd351ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd351ffb640 1 -- 172.21.15.120:0/2886845788 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56434af08da0 con 0x56434afac210 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd351ffb640 10 client.17262 handle_command_reply: tid=0 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 client.17262 shutdown 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 msgr2=0x56434af99640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd364005740 tx=0x7fd364005770 comp rx=0 tx=0).stop 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 shutdown_connections 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434afac210 0x56434afae600 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af99c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af99640 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd351ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2886845788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434af9c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 >> 172.21.15.120:0/2886845788 conn(0x56434af8ded0 msgr2=0x56434af93a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 shutdown_connections 2024-09-10T23:01:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 1 -- 172.21.15.120:0/2886845788 wait complete. 2024-09-10T23:01:46.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.160+0000 7fd3727b7740 20 client.17262 trim_cache size 0 max 16384 2024-09-10T23:01:46.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.179+0000 7fd3727b7740 1 Processor -- start 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 -- start start 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434afa8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af9e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681ac280 con 0x56434af909c0 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681aeb00 con 0x56434af900a0 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd3727b7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681ac8f0 con 0x56434af9c950 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60792/0 (socket says 172.21.15.120:60792) 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 -- 172.21.15.120:0/2701328155 learned_addr learned my addr 172.21.15.120:0/2701328155 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af9e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434afa8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 -- 172.21.15.120:0/2701328155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 msgr2=0x56434af9e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af9e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 -- 172.21.15.120:0/2701328155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 msgr2=0x56434afa8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434afa8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 -- 172.21.15.120:0/2701328155 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434af907f0 con 0x56434af909c0 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434afa8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.180+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd364009730 tx=0x7fd364011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d16acada328ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.181+0000 7fd353fff640 1 -- 172.21.15.120:0/2701328155 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364029070 con 0x56434af909c0 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.181+0000 7fd353fff640 1 -- 172.21.15.120:0/2701328155 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364009a70 con 0x56434af909c0 2024-09-10T23:01:46.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.181+0000 7fd353fff640 1 -- 172.21.15.120:0/2701328155 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364003dd0 con 0x56434af909c0 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- 172.21.15.120:0/2701328155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 msgr2=0x56434af9dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd364009730 tx=0x7fd364011810 comp rx=0 tx=0).stop 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- 172.21.15.120:0/2701328155 shutdown_connections 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af9c950 0x56434af9e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af909c0 0x56434af9dd40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2701328155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af900a0 0x56434afa8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- 172.21.15.120:0/2701328155 >> 172.21.15.120:0/2701328155 conn(0x56434af8ded0 msgr2=0x56434af9b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- 172.21.15.120:0/2701328155 shutdown_connections 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- 172.21.15.120:0/2701328155 wait complete. 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- start start 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 Processor -- start 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af909c0 0x56434afa5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681acfc0 con 0x56434af9c950 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681ae430 con 0x56434af909c0 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681ad690 con 0x56434af900a0 2024-09-10T23:01:46.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.182+0000 7fd3727b7740 10 client.0 _ll_register_callbacks cb 0x7fd35800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47694/0 (socket says 172.21.15.120:47694) 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 -- 172.21.15.120:0/2914643059 learned_addr learned my addr 172.21.15.120:0/2914643059 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af909c0 0x56434afa5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 -- 172.21.15.120:0/2914643059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af909c0 msgr2=0x56434afa5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af909c0 0x56434afa5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 -- 172.21.15.120:0/2914643059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 msgr2=0x56434af9fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 -- 172.21.15.120:0/2914643059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434afa02a0 con 0x56434af900a0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd36d97d640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd358012720 tx=0x7fd364004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27657af2517b06d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:01:46.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358024820 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd364009a70 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.183+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd3580249c0 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd358024bc0 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 10 client.26746 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56434b019ca0 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364009a70 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd35803f430 con 0x56434af900a0 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 10 client.26746 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 20 client.26746 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434b01a400 0x56434b01c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 4 client.26746 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:46.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56434af04850 con 0x56434b01a400 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434b01a400 0x56434b01c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.184+0000 7fd351ffb640 10 client.26746 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd36e17e640 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434b01a400 0x56434b01c820 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e300bf9301831d server_cookie=309eb6af455d5267 in_seq=0 out_seq=0 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd351ffb640 1 -- 172.21.15.120:0/2914643059 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x56434af04850 con 0x56434b01a400 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd351ffb640 10 client.26746 handle_command_reply: tid=0 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 client.26746 shutdown 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 msgr2=0x56434afa4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd358012720 tx=0x7fd364004360 comp rx=0 tx=0).stop 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 shutdown_connections 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56434b01a400 0x56434b01c820 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:01:46.193 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.196 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.199 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.200 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:01:46.201 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:01:46.202 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:46.203 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56434af9c950 0x56434af9fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56434af909c0 0x56434afa5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 --2- 172.21.15.120:0/2914643059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56434af900a0 0x56434afa4cf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 >> 172.21.15.120:0/2914643059 conn(0x56434af8ded0 msgr2=0x56434af8c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd351ffb640 0 client.26746 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 shutdown_connections 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.185+0000 7fd3727b7740 1 -- 172.21.15.120:0/2914643059 wait complete. 2024-09-10T23:01:46.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3727b7740 20 client.26746 trim_cache size 0 max 16384 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 msgr2=0x7fd3540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 0x7fd3540b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ca8d0 tx=0x7fd3600f0040 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 msgr2=0x7fd36819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098d0 tx=0x7fd3600317b0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 shutdown_connections 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd3540b52e0 0x7fd3540b77a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd368108040 0x7fd36819aaf0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3681076c0 0x7fd36819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 --2- 172.21.15.120:0/395569930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd368101cb0 0x7fd36819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.186+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 >> 172.21.15.120:0/395569930 conn(0x7fd3680fd700 msgr2=0x7fd3680ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.187+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 shutdown_connections 2024-09-10T23:01:46.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:46.187+0000 7fd3703e2640 1 -- 172.21.15.120:0/395569930 wait complete. 2024-09-10T23:01:46.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1584523 16.20 MB/sec execute 401 sec latency 35.454 ms 2024-09-10T23:01:47.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:01:47.279 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:47.548 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:47.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1587611 16.20 MB/sec execute 402 sec latency 12.460 ms 2024-09-10T23:01:47.816 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:47 smithi120 ceph-mon[31247]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T23:01:47.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:47 smithi176 ceph-mon[34982]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T23:01:47.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:47 smithi153 ceph-mon[34951]: pgmap v627: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 568 op/s 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 -- 172.21.15.120:0/3871320435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3480108020 msgr2=0x7f348010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 --2- 172.21.15.120:0/3871320435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3480108020 0x7f348010e820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3478009ca0 tx=0x7f347802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 -- 172.21.15.120:0/3871320435 shutdown_connections 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 --2- 172.21.15.120:0/3871320435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3480108020 0x7f348010e820 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 --2- 172.21.15.120:0/3871320435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34801076a0 0x7f3480107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 --2- 172.21.15.120:0/3871320435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 0x7f3480102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.040+0000 7f3484d1e640 1 -- 172.21.15.120:0/3871320435 >> 172.21.15.120:0/3871320435 conn(0x7f34800fd700 msgr2=0x7f34800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.041+0000 7f3484d1e640 1 -- 172.21.15.120:0/3871320435 shutdown_connections 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.041+0000 7f3484d1e640 1 -- 172.21.15.120:0/3871320435 wait complete. 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.041+0000 7f3484d1e640 1 Processor -- start 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.041+0000 7f3484d1e640 1 -- start start 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 0x7f34800fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3480108020 0x7f34800fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3480111ca0 con 0x7f34801076a0 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3480111b20 con 0x7f3480101c90 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3484d1e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3480111e20 con 0x7f3480108020 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60820/0 (socket says 172.21.15.120:60820) 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 -- 172.21.15.120:0/2793272058 learned_addr learned my addr 172.21.15.120:0/2793272058 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:48.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f347ed9d640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3480108020 0x7f34800fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f347e59c640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 0x7f34800fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 -- 172.21.15.120:0/2793272058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3480108020 msgr2=0x7f34800fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3480108020 0x7f34800fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 -- 172.21.15.120:0/2793272058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 msgr2=0x7f34800fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 0x7f34800fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 -- 172.21.15.120:0/2793272058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34801a76f0 con 0x7f34801076a0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f3477fff640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f346800e9f0 tx=0x7f346800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.042+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346800cd80 con 0x7f34801076a0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.043+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34801a79e0 con 0x7f34801076a0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.043+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34680054d0 con 0x7f34801076a0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.043+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468010690 con 0x7f34801076a0 2024-09-10T23:01:48.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.043+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34801a7ed0 con 0x7f34801076a0 2024-09-10T23:01:48.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.044+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3480102070 con 0x7f34801076a0 2024-09-10T23:01:48.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.045+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3468004e90 con 0x7f34801076a0 2024-09-10T23:01:48.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.046+0000 7f34777fe640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 0x7f346c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:48.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.046+0000 7f347e59c640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 0x7f346c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:48.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.046+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f34680e5f60 con 0x7f34801076a0 2024-09-10T23:01:48.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.046+0000 7f347e59c640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 0x7f346c0b77a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f347000bbe0 tx=0x7f347000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:48.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.050+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f34680ace50 con 0x7f34801076a0 2024-09-10T23:01:48.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.324+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f348002e5c0 con 0x7f34801076a0 2024-09-10T23:01:48.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.324+0000 7f34777fe640 1 -- 172.21.15.120:0/2793272058 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f34680ac750 con 0x7f34801076a0 2024-09-10T23:01:48.327 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:01:48.327 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:01:48.327 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 msgr2=0x7f346c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 0x7f346c0b77a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f347000bbe0 tx=0x7f347000b390 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 msgr2=0x7f34800fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f346800e9f0 tx=0x7f346800edf0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 shutdown_connections 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f346c0b52e0 0x7f346c0b77a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3480108020 0x7f34800fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34801076a0 0x7f34800fe140 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 --2- 172.21.15.120:0/2793272058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3480101c90 0x7f34800fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.326+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 >> 172.21.15.120:0/2793272058 conn(0x7f34800fd700 msgr2=0x7f348006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.327+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 shutdown_connections 2024-09-10T23:01:48.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:48.327+0000 7f3484d1e640 1 -- 172.21.15.120:0/2793272058 wait complete. 2024-09-10T23:01:48.635 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1589754 16.18 MB/sec execute 403 sec latency 55.661 ms 2024-09-10T23:01:48.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:48 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2793272058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:48.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:48 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2793272058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:48.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:48 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2793272058' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:49.041 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:01:49.341 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:49.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1592454 16.18 MB/sec execute 404 sec latency 24.291 ms 2024-09-10T23:01:49.737 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:49 smithi120 ceph-mon[31247]: pgmap v628: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T23:01:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:49 smithi176 ceph-mon[34982]: pgmap v628: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 -- 172.21.15.120:0/287641500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107500 msgr2=0x7f2e5c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/287641500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107500 0x7f2e5c107940 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009900 tx=0x7f2e5002a410 comp rx=0 tx=0).stop 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 -- 172.21.15.120:0/287641500 shutdown_connections 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/287641500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107e80 0x7f2e5c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/287641500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107500 0x7f2e5c107940 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/287641500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 0x7f2e5c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 -- 172.21.15.120:0/287641500 >> 172.21.15.120:0/287641500 conn(0x7f2e5c0fd560 msgr2=0x7f2e5c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 -- 172.21.15.120:0/287641500 shutdown_connections 2024-09-10T23:01:49.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.821+0000 7f2e5b59e640 1 -- 172.21.15.120:0/287641500 wait complete. 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 Processor -- start 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 -- start start 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 0x7f2e5c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107500 0x7f2e5c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111ad0 con 0x7f2e5c107e80 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.822+0000 7f2e5b59e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111950 con 0x7f2e5c101af0 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5b59e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111c50 con 0x7f2e5c107500 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 0x7f2e5c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60844/0 (socket says 172.21.15.120:60844) 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107500 0x7f2e5c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 -- 172.21.15.120:0/3186253174 learned_addr learned my addr 172.21.15.120:0/3186253174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 -- 172.21.15.120:0/3186253174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107500 msgr2=0x7f2e5c19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107500 0x7f2e5c19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 -- 172.21.15.120:0/3186253174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 msgr2=0x7f2e5c199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 0x7f2e5c199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 -- 172.21.15.120:0/3186253174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c1a53a0 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5ad9d640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400eb10 tx=0x7f2e5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e4b7fe640 1 -- 172.21.15.120:0/3186253174 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54004b20 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e4b7fe640 1 -- 172.21.15.120:0/3186253174 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54004cc0 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c1a5570 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e4b7fe640 1 -- 172.21.15.120:0/3186253174 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54010670 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.823+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1a59d0 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.826+0000 7f2e4b7fe640 1 -- 172.21.15.120:0/3186253174 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2e54005020 con 0x7f2e5c107e80 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.826+0000 7f2e62c8f740 1 Processor -- start 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 -- start start 2024-09-10T23:01:49.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 0x5638e4b32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 0x5638e4b42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638e4aa8ea0 con 0x5638e4b32fc0 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638e4a982d0 con 0x5638e4b43360 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5638e4a9c0e0 con 0x5638e4b326a0 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 0x5638e4b42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 0x5638e4b42e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60856/0 (socket says 172.21.15.120:60856) 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e5a59c640 1 -- 172.21.15.120:0/1879313254 learned_addr learned my addr 172.21.15.120:0/1879313254 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 0x5638e4b32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e4b7fe640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 0x7f2e440b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1879313254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 msgr2=0x5638e4b45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e5ad9d640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 0x7f2e440b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1879313254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 msgr2=0x5638e4b42e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 0x5638e4b42e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.827+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1879313254 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e47f6650 con 0x5638e4b326a0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 0x5638e4b32a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009230 tx=0x7f2e50005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73cfecd2e089fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e4b7fe640 1 -- 172.21.15.120:0/3186253174 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2e540e6070 con 0x7f2e5c107e80 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e48ff9640 1 -- 172.21.15.120:0/1879313254 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50035030 con 0x5638e4b326a0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e5ad9d640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 0x7f2e440b7850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e540a8e30 tx=0x7f2e54093e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e48ff9640 1 -- 172.21.15.120:0/1879313254 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e50004030 con 0x5638e4b326a0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.828+0000 7f2e48ff9640 1 -- 172.21.15.120:0/1879313254 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003980 con 0x5638e4b326a0 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1879313254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 msgr2=0x5638e4b32a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 0x5638e4b32a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009230 tx=0x7f2e50005b50 comp rx=0 tx=0).stop 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1879313254 shutdown_connections 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b32fc0 0x5638e4b42e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1879313254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b326a0 0x5638e4b32a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1879313254 >> 172.21.15.120:0/1879313254 conn(0x5638e4b304d0 msgr2=0x5638e4b31670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1879313254 shutdown_connections 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1879313254 wait complete. 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 -- start start 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.829+0000 7f2e62c8f740 1 Processor -- start 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b42cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638e48f9700 con 0x5638e4b326a0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638e4a33160 con 0x5638e4b32fc0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5638e4b45f80 con 0x5638e4b43360 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e62c8f740 10 client.0 _ll_register_callbacks cb 0x5638e4a4c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b42cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b42cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60862/0 (socket says 172.21.15.120:60862) 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59980/0 (socket says 172.21.15.120:59980) 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e59d9b640 1 -- 172.21.15.120:0/3135297105 learned_addr learned my addr 172.21.15.120:0/3135297105 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 -- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 msgr2=0x5638e4b40810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b40810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 -- 172.21.15.120:0/3135297105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 msgr2=0x5638e4b42cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b42cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 -- 172.21.15.120:0/3135297105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e4a60c90 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005d00 tx=0x7f2e4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9e8a401295831d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.830+0000 7f2e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004020 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e50004030 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2e4c0041c0 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c020020 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 10 client.26645 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638e4b51950 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e50004030 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2e4c003850 con 0x5638e4b32fc0 2024-09-10T23:01:49.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 10 client.26645 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 20 client.26645 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b52db0 0x5638e4b55220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 4 client.26645 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e4aaadf0 con 0x5638e4b52db0 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.831+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b52db0 0x5638e4b55220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.832+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b52db0 0x5638e4b55220 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b77940e4154a5ee server_cookie=62fbf31b8a3e0ce in_seq=0 out_seq=0 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.832+0000 7f2e41ffb640 10 client.26645 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.833+0000 7f2e41ffb640 1 -- 172.21.15.120:0/3135297105 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638e4aaadf0 con 0x5638e4b52db0 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.833+0000 7f2e41ffb640 10 client.26645 handle_command_reply: tid=0 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 client.26645 shutdown 2024-09-10T23:01:49.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 msgr2=0x5638e4b40140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005d00 tx=0x7f2e4c0118d0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 shutdown_connections 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b52db0 0x5638e4b55220 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b40140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e41ffb640 0 client.26645 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3135297105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b42cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 >> 172.21.15.120:0/3135297105 conn(0x5638e4b304d0 msgr2=0x5638e4b36070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 shutdown_connections 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3135297105 wait complete. 2024-09-10T23:01:49.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.834+0000 7f2e62c8f740 20 client.26645 trim_cache size 0 max 16384 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 Processor -- start 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 -- start start 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b32fc0 0x5638e4b5ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1a9de0 con 0x5638e4b326a0 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ac660 con 0x5638e4b43360 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.857+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1aa450 con 0x5638e4b32fc0 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60866/0 (socket says 172.21.15.120:60866) 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 -- 172.21.15.120:0/3579958303 learned_addr learned my addr 172.21.15.120:0/3579958303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b32fc0 0x5638e4b5ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 -- 172.21.15.120:0/3579958303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b32fc0 msgr2=0x5638e4b5ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b32fc0 0x5638e4b5ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 -- 172.21.15.120:0/3579958303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 msgr2=0x5638e4b5b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b5b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 -- 172.21.15.120:0/3579958303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e4b32000 con 0x5638e4b326a0 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2e500099f0 tx=0x7f2e50032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc68cae5562a832 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e43fff640 1 -- 172.21.15.120:0/3579958303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50035040 con 0x5638e4b326a0 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e43fff640 1 -- 172.21.15.120:0/3579958303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e500054f0 con 0x5638e4b326a0 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.858+0000 7f2e43fff640 1 -- 172.21.15.120:0/3579958303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5003d450 con 0x5638e4b326a0 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3579958303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 msgr2=0x5638e4b4f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2e500099f0 tx=0x7f2e50032e50 comp rx=0 tx=0).stop 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3579958303 shutdown_connections 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b43360 0x5638e4b5b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b32fc0 0x5638e4b5ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/3579958303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4f1b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3579958303 >> 172.21.15.120:0/3579958303 conn(0x5638e4b304d0 msgr2=0x5638e4b44250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.859+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3579958303 shutdown_connections 2024-09-10T23:01:49.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 -- 172.21.15.120:0/3579958303 wait complete. 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 -- start start 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 Processor -- start 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1aab20 con 0x5638e4b326a0 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1abf90 con 0x5638e4b32fc0 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ab1f0 con 0x5638e4b43360 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.860+0000 7f2e62c8f740 10 client.0 _ll_register_callbacks cb 0x7f2e4c00b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e5a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60868/0 (socket says 172.21.15.120:60868) 2024-09-10T23:01:49.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1490057590 learned_addr learned my addr 172.21.15.120:0/1490057590 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 msgr2=0x5638e4b481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1490057590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 msgr2=0x5638e4b4b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b4b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 -- 172.21.15.120:0/1490057590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e4b48720 con 0x5638e4b326a0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e59d9b640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2e5002a930 tx=0x7f2e50009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d79e8c8475cf7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50035070 con 0x5638e4b326a0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2e50004b50 con 0x5638e4b326a0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.861+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e500054f0 con 0x5638e4b326a0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e50004d50 con 0x5638e4b326a0 2024-09-10T23:01:49.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 10 client.17292 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638e4b390c0 con 0x5638e4b326a0 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e500054f0 con 0x5638e4b326a0 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2e5004d850 con 0x5638e4b326a0 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 10 client.17292 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 20 client.17292 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b39400 0x5638e4b3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 4 client.17292 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638e4aa68a0 con 0x5638e4b39400 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.862+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b39400 0x5638e4b3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.863+0000 7f2e41ffb640 10 client.17292 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.863+0000 7f2e5a59c640 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b39400 0x5638e4b3b820 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240e99c664e96d84 server_cookie=44595962fa92cc53 in_seq=0 out_seq=0 2024-09-10T23:01:49.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e41ffb640 1 -- 172.21.15.120:0/1490057590 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638e4aa68a0 con 0x5638e4b39400 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e41ffb640 10 client.17292 handle_command_reply: tid=0 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 client.17292 shutdown 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 msgr2=0x5638e4b4b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2e5002a930 tx=0x7f2e50009230 comp rx=0 tx=0).stop 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 shutdown_connections 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638e4b39400 0x5638e4b3b820 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638e4b43360 0x5638e4b481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638e4b32fc0 0x5638e4b4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 --2- 172.21.15.120:0/1490057590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638e4b326a0 0x5638e4b4b210 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 >> 172.21.15.120:0/1490057590 conn(0x5638e4b304d0 msgr2=0x5638e4b2ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e41ffb640 0 client.17292 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 shutdown_connections 2024-09-10T23:01:49.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 1 -- 172.21.15.120:0/1490057590 wait complete. 2024-09-10T23:01:49.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.864+0000 7f2e62c8f740 20 client.17292 trim_cache size 0 max 16384 2024-09-10T23:01:49.879 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 msgr2=0x7f2e440b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 0x7f2e440b7850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e540a8e30 tx=0x7f2e54093e60 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 msgr2=0x7f2e5c1a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400eb10 tx=0x7f2e5400c2d0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 shutdown_connections 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2e440b5390 0x7f2e440b7850 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e5c107e80 0x7f2e5c1a4e60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e5c107500 0x7f2e5c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 --2- 172.21.15.120:0/3186253174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e5c101af0 0x7f2e5c199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 >> 172.21.15.120:0/3186253174 conn(0x7f2e5c0fd560 msgr2=0x7f2e5c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 shutdown_connections 2024-09-10T23:01:49.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:49.865+0000 7f2e5b59e640 1 -- 172.21.15.120:0/3186253174 wait complete. 2024-09-10T23:01:49.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:49 smithi153 ceph-mon[34951]: pgmap v628: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 458 op/s 2024-09-10T23:01:50.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1595565 16.18 MB/sec execute 405 sec latency 21.386 ms 2024-09-10T23:01:51.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1598846 16.19 MB/sec execute 406 sec latency 16.425 ms 2024-09-10T23:01:51.664 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:01:51.664 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:01:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:51 smithi176 ceph-mon[34982]: pgmap v629: 97 pgs: 97 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 407 op/s 2024-09-10T23:01:51.847 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:51 smithi120 ceph-mon[31247]: pgmap v629: 97 pgs: 97 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 407 op/s 2024-09-10T23:01:51.916 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:51 smithi153 ceph-mon[34951]: pgmap v629: 97 pgs: 97 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 407 op/s 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 -- 172.21.15.120:0/376649593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 msgr2=0x7f3e5410e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 --2- 172.21.15.120:0/376649593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5410e850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009ca0 tx=0x7f3e4c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 -- 172.21.15.120:0/376649593 shutdown_connections 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 --2- 172.21.15.120:0/376649593 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5410e850 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 --2- 172.21.15.120:0/376649593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e54107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 --2- 172.21.15.120:0/376649593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e541020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 -- 172.21.15.120:0/376649593 >> 172.21.15.120:0/376649593 conn(0x7f3e540fd770 msgr2=0x7f3e540ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 -- 172.21.15.120:0/376649593 shutdown_connections 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.377+0000 7f3e59153640 1 -- 172.21.15.120:0/376649593 wait complete. 2024-09-10T23:01:52.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 Processor -- start 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 -- start start 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54111d10 con 0x7f3e54108050 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54111b90 con 0x7f3e541076d0 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e59153640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54111e90 con 0x7f3e54101cc0 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e5359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e5359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60892/0 (socket says 172.21.15.120:60892) 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:60034/0 (socket says 172.21.15.120:60034) 2024-09-10T23:01:52.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47796/0 (socket says 172.21.15.120:47796) 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.378+0000 7f3e5359e640 1 -- 172.21.15.120:0/682444130 learned_addr learned my addr 172.21.15.120:0/682444130 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 -- 172.21.15.120:0/682444130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 msgr2=0x7f3e5419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 -- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 msgr2=0x7f3e5419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 -- 172.21.15.120:0/682444130 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e541a5420 con 0x7f3e541076d0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5359e640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e5259c640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800e9f0 tx=0x7f3e4800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e3bfff640 1 -- 172.21.15.120:0/682444130 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800ccc0 con 0x7f3e541076d0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e541a5710 con 0x7f3e541076d0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e3bfff640 1 -- 172.21.15.120:0/682444130 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e480108e0 con 0x7f3e541076d0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e3bfff640 1 -- 172.21.15.120:0/682444130 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48018880 con 0x7f3e541076d0 2024-09-10T23:01:52.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.379+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e541a5ca0 con 0x7f3e541076d0 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.382+0000 7f3e5b528740 1 Processor -- start 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.382+0000 7f3e5b528740 1 -- start start 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.382+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbc0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbc0790 0x55973dbc2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5b528740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55973db2ae60 con 0x55973dbc0790 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5b528740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55973db1a290 con 0x55973dbb4800 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5b528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55973db1e0a0 con 0x55973dbb3ee0 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e3bfff640 1 -- 172.21.15.120:0/682444130 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3e48010430 con 0x7f3e541076d0 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47812/0 (socket says 172.21.15.120:47812) 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 -- 172.21.15.120:0/2019859860 learned_addr learned my addr 172.21.15.120:0/2019859860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:52.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbc0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5259c640 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbc0790 0x55973dbc2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 -- 172.21.15.120:0/2019859860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 msgr2=0x55973dbc0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbc0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 -- 172.21.15.120:0/2019859860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbc0790 msgr2=0x55973dbc2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbc0790 0x55973dbc2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.383+0000 7f3e5359e640 1 -- 172.21.15.120:0/2019859860 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973d878930 con 0x55973dbb3ee0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009400 tx=0x7f3e4c02f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e5a5684607513a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e397fa640 1 -- 172.21.15.120:0/2019859860 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c05b070 con 0x55973dbb3ee0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e397fa640 1 -- 172.21.15.120:0/2019859860 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4c036040 con 0x55973dbb3ee0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e397fa640 1 -- 172.21.15.120:0/2019859860 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0042a0 con 0x55973dbb3ee0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e3bfff640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 0x7f3e3c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 0x7f3e3c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.384+0000 7f3e3bfff640 1 -- 172.21.15.120:0/682444130 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3e480e5ec0 con 0x7f3e541076d0 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- 172.21.15.120:0/2019859860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 msgr2=0x55973dbb42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 0x7f3e3c0b77a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55973dbb7060 tx=0x7f3e44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009400 tx=0x7f3e4c02f310 comp rx=0 tx=0).stop 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- 172.21.15.120:0/2019859860 shutdown_connections 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbc0790 0x55973dbc2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbc0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2019859860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb3ee0 0x55973dbb42c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- 172.21.15.120:0/2019859860 >> 172.21.15.120:0/2019859860 conn(0x55973dbb1d10 msgr2=0x55973dbb2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- 172.21.15.120:0/2019859860 shutdown_connections 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- 172.21.15.120:0/2019859860 wait complete. 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- start start 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 Processor -- start 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbbd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbc0790 0x55973dbbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55973d97b890 con 0x55973dbb4800 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55973dab4d40 con 0x55973dbb3ee0 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55973dbc33e0 con 0x55973dbc0790 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.385+0000 7f3e5b528740 10 client.0 _ll_register_callbacks cb 0x55973db18c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60916/0 (socket says 172.21.15.120:60916) 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 -- 172.21.15.120:0/4230617692 learned_addr learned my addr 172.21.15.120:0/4230617692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5359e640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbc0790 0x55973dbbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbbd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 -- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbc0790 msgr2=0x55973dbbe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbc0790 0x55973dbbe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 -- 172.21.15.120:0/4230617692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 msgr2=0x55973dbbd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbbd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 -- 172.21.15.120:0/4230617692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973dbcc840 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5259c640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3e480a99d0 tx=0x7f3e4800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb644b3d1dac6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480a6e60 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3e480e9d50 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4c036040 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e480ac530 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5b528740 10 client.17304 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55973dbcf1d0 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.386+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4c036040 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3e480ab9c0 con 0x55973dbb4800 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 10 client.17304 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:52.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 20 client.17304 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbd0630 0x55973dbd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 4 client.17304 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55973db2cdb0 con 0x55973dbd0630 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbd0630 0x55973dbd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e36ff5640 10 client.17304 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.387+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbd0630 0x55973dbd2a20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105a01f0c9033517 server_cookie=f2a4b9ab6ea4391a in_seq=0 out_seq=0 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e36ff5640 1 -- 172.21.15.120:0/4230617692 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55973db2cdb0 con 0x55973dbd0630 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e36ff5640 10 client.17304 handle_command_reply: tid=0 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 client.17304 shutdown 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 msgr2=0x55973dbbdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3e480a99d0 tx=0x7f3e4800ce60 comp rx=0 tx=0).stop 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 shutdown_connections 2024-09-10T23:01:52.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbd0630 0x55973dbd2a20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbc0790 0x55973dbbe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbbdd00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 --2- 172.21.15.120:0/4230617692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbbd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e36ff5640 0 client.17304 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.389+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 >> 172.21.15.120:0/4230617692 conn(0x55973dbb1d10 msgr2=0x55973dbb0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.390+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 shutdown_connections 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.390+0000 7f3e5b528740 1 -- 172.21.15.120:0/4230617692 wait complete. 2024-09-10T23:01:52.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.390+0000 7f3e5b528740 20 client.17304 trim_cache size 0 max 16384 2024-09-10T23:01:52.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.408+0000 7f3e5b528740 1 Processor -- start 2024-09-10T23:01:52.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.408+0000 7f3e5b528740 1 -- start start 2024-09-10T23:01:52.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbbd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbbaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54072e20 con 0x55973dbb3ee0 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541a62c0 con 0x55973dbb4800 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5b528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54073470 con 0x55973dbb1d10 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60922/0 (socket says 172.21.15.120:60922) 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 -- 172.21.15.120:0/339265247 learned_addr learned my addr 172.21.15.120:0/339265247 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5359e640 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbbaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbbd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 -- 172.21.15.120:0/339265247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 msgr2=0x55973dbbd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbbd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 -- 172.21.15.120:0/339265247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 msgr2=0x55973dbbaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbbaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 -- 172.21.15.120:0/339265247 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973dbb33f0 con 0x55973dbb3ee0 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e5259c640 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e4800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df6c98d927a5dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.409+0000 7f3e38ff9640 1 -- 172.21.15.120:0/339265247 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480a6e60 con 0x55973dbb3ee0 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e38ff9640 1 -- 172.21.15.120:0/339265247 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e480ac270 con 0x55973dbb3ee0 2024-09-10T23:01:52.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e38ff9640 1 -- 172.21.15.120:0/339265247 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480abed0 con 0x55973dbb3ee0 2024-09-10T23:01:52.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 -- 172.21.15.120:0/339265247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 msgr2=0x55973dbba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e4800cef0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 -- 172.21.15.120:0/339265247 shutdown_connections 2024-09-10T23:01:52.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb4800 0x55973dbbaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb3ee0 0x55973dbba9d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 --2- 172.21.15.120:0/339265247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbbd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 -- 172.21.15.120:0/339265247 >> 172.21.15.120:0/339265247 conn(0x55973dbbfa20 msgr2=0x55973dbbdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 -- 172.21.15.120:0/339265247 shutdown_connections 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.410+0000 7f3e5b528740 1 -- 172.21.15.120:0/339265247 wait complete. 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 -- start start 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 Processor -- start 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbc50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbc5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54073b40 con 0x55973dbb4800 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54074fb0 con 0x55973dbb3ee0 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54074210 con 0x55973dbb1d10 2024-09-10T23:01:52.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5b528740 10 client.0 _ll_register_callbacks cb 0x7f3e4800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbc5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e52d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbc8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47836/0 (socket says 172.21.15.120:47836) 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e52d9d640 1 -- 172.21.15.120:0/2152200737 learned_addr learned my addr 172.21.15.120:0/2152200737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbc50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 -- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 msgr2=0x55973dbc8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbc8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 -- 172.21.15.120:0/2152200737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 msgr2=0x55973dbc50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbc50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.411+0000 7f3e5359e640 1 -- 172.21.15.120:0/2152200737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973dbc5c70 con 0x55973dbb4800 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5259c640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbc50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5359e640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbc5630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c059040 tx=0x7f3e4c02f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723413fb2948f4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:52.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c05b040 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3e4c031450 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e480ac270 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c031650 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5b528740 10 client.17310 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55973dc3de80 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e480ac270 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3e4c0043a0 con 0x55973dbb4800 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.412+0000 7f3e5b528740 10 client.17310 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e5b528740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e5b528740 20 client.17310 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbc0790 0x55973dc402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e5b528740 4 client.17310 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:01:52.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55973db28860 con 0x55973dbc0790 2024-09-10T23:01:52.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbc0790 0x55973dc402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:52.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e36ff5640 10 client.17310 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:52.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.413+0000 7f3e52d9d640 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbc0790 0x55973dc402a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1241e2eb7e2a6ef4 server_cookie=ef430d71f9470083 in_seq=0 out_seq=0 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e36ff5640 1 -- 172.21.15.120:0/2152200737 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55973db28860 con 0x55973dbc0790 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e36ff5640 10 client.17310 handle_command_reply: tid=0 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 client.17310 shutdown 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 msgr2=0x55973dbc5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbc5630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c059040 tx=0x7f3e4c02f4a0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 shutdown_connections 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55973dbc0790 0x55973dc402a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55973dbb4800 0x55973dbc5630 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55973dbb3ee0 0x55973dbc50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 --2- 172.21.15.120:0/2152200737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55973dbb1d10 0x55973dbc8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 >> 172.21.15.120:0/2152200737 conn(0x55973dbbfa20 msgr2=0x55973dbb0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e36ff5640 0 client.17310 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 shutdown_connections 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 1 -- 172.21.15.120:0/2152200737 wait complete. 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.415+0000 7f3e5b528740 20 client.17310 trim_cache size 0 max 16384 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0", 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:01:52.422 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 msgr2=0x7f3e3c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 0x7f3e3c0b77a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55973dbb7060 tx=0x7f3e44005f90 comp rx=0 tx=0).stop 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 msgr2=0x7f3e5419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800e9f0 tx=0x7f3e4800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 shutdown_connections 2024-09-10T23:01:52.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3e3c0b52e0 0x7f3e3c0b77a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e54108050 0x7f3e5419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e541076d0 0x7f3e5419a580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 --2- 172.21.15.120:0/682444130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e54101cc0 0x7f3e5419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 >> 172.21.15.120:0/682444130 conn(0x7f3e540fd770 msgr2=0x7f3e540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 shutdown_connections 2024-09-10T23:01:52.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:52.416+0000 7f3e59153640 1 -- 172.21.15.120:0/682444130 wait complete. 2024-09-10T23:01:52.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1601518 16.18 MB/sec execute 407 sec latency 28.380 ms 2024-09-10T23:01:53.234 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:53.503 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:53.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1603756 16.17 MB/sec execute 408 sec latency 95.173 ms 2024-09-10T23:01:53.806 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:53 smithi120 ceph-mon[31247]: pgmap v630: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-09-10T23:01:53.807 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:53 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:01:53.807 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:53 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:53.807 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:53 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:01:53.807 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:53 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:53 smithi176 ceph-mon[34982]: pgmap v630: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-09-10T23:01:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:53 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:01:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:53 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:53 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:01:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:53 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:53.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:53 smithi153 ceph-mon[34951]: pgmap v630: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 471 op/s 2024-09-10T23:01:53.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:53 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:01:53.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:53 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:53.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:53 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:01:53.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:53 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:54.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 -- 172.21.15.120:0/2082932662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 msgr2=0x7fafb810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/2082932662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb810ae10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 -- 172.21.15.120:0/2082932662 shutdown_connections 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/2082932662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb810b350 0x7fafb8111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/2082932662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb810ae10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/2082932662 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb8104fc0 0x7fafb81053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 -- 172.21.15.120:0/2082932662 >> 172.21.15.120:0/2082932662 conn(0x7fafb81009f0 msgr2=0x7fafb8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 -- 172.21.15.120:0/2082932662 shutdown_connections 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.033+0000 7fafbd9fb640 1 -- 172.21.15.120:0/2082932662 wait complete. 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 Processor -- start 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 -- start start 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb8104fc0 0x7fafb819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb810b350 0x7fafb81a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114f70 con 0x7fafb810a9d0 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114df0 con 0x7fafb8104fc0 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafbd9fb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81150f0 con 0x7fafb810b350 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36910/0 (socket says 172.21.15.120:36910) 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafb67fc640 1 -- 172.21.15.120:0/3585991896 learned_addr learned my addr 172.21.15.120:0/3585991896 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb8104fc0 0x7fafb819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.034+0000 7fafb77fe640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb810b350 0x7fafb81a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 -- 172.21.15.120:0/3585991896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb810b350 msgr2=0x7fafb81a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb810b350 0x7fafb81a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 -- 172.21.15.120:0/3585991896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb8104fc0 msgr2=0x7fafb819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb8104fc0 0x7fafb819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 -- 172.21.15.120:0/3585991896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a14f0 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fafac005940 tx=0x7fafac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafbc9f9640 1 -- 172.21.15.120:0/3585991896 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac03f030 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafbc9f9640 1 -- 172.21.15.120:0/3585991896 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafac003aa0 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a1780 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafbc9f9640 1 -- 172.21.15.120:0/3585991896 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac035070 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.035+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a7ea0 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.037+0000 7fafbfdd0740 1 Processor -- start 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.037+0000 7fafbc9f9640 1 -- 172.21.15.120:0/3585991896 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fafac03bce0 con 0x7fafb810a9d0 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.037+0000 7fafbfdd0740 1 -- start start 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fd4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fe07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574a6f4ae50 con 0x5574a6fe0d30 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574a6f3a280 con 0x5574a6fd4da0 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574a6f3e090 con 0x5574a6fd4480 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fe07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fd4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36912/0 (socket says 172.21.15.120:36912) 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb67fc640 1 -- 172.21.15.120:0/1860400365 learned_addr learned my addr 172.21.15.120:0/1860400365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fe07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36504/0 (socket says 172.21.15.120:36504) 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 -- 172.21.15.120:0/1860400365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 msgr2=0x5574a6fe07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fe07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 -- 172.21.15.120:0/1860400365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 msgr2=0x5574a6fe3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 -- 172.21.15.120:0/1860400365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a6c98650 con 0x5574a6fd4480 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb67fc640 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafbc9f9640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 0x7fafa00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb77fe640 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fd4860 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004640 tx=0x7fafb000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df805e804b7a8864 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.038+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 0x7fafa00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbc9f9640 1 -- 172.21.15.120:0/3585991896 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fafac106510 con 0x7fafb810a9d0 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7faf9dffb640 1 -- 172.21.15.120:0/1860400365 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003bb0 con 0x5574a6fd4480 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7faf9dffb640 1 -- 172.21.15.120:0/1860400365 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb0015070 con 0x5574a6fd4480 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7faf9dffb640 1 -- 172.21.15.120:0/1860400365 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0010460 con 0x5574a6fd4480 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1860400365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 msgr2=0x5574a6fd4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fd4860 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004640 tx=0x7fafb000ba90 comp rx=0 tx=0).stop 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 0x7fafa00b77a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5574a6fd3190 tx=0x7fafa8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1860400365 shutdown_connections 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fe07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1860400365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fd4860 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1860400365 >> 172.21.15.120:0/1860400365 conn(0x5574a6fd22b0 msgr2=0x5574a6fd3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1860400365 shutdown_connections 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.039+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1860400365 wait complete. 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 -- start start 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 Processor -- start 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fe0d30 0x5574a6fde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574a6d9b700 con 0x5574a6fd4480 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574a6ed5110 con 0x5574a6fd4da0 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574a6fe39b0 con 0x5574a6fe0d30 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafbfdd0740 10 client.0 _ll_register_callbacks cb 0x5574a6eee990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:54.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.040+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36914/0 (socket says 172.21.15.120:36914) 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 -- 172.21.15.120:0/3546631943 learned_addr learned my addr 172.21.15.120:0/3546631943 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb77fe640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fe0d30 0x5574a6fde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 -- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fe0d30 msgr2=0x5574a6fde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fe0d30 0x5574a6fde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 -- 172.21.15.120:0/3546631943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 msgr2=0x5574a6fdda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 -- 172.21.15.120:0/3546631943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a6f02c40 con 0x5574a6fd4480 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fafa8004070 tx=0x7fafa8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca7d5284a99d3da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7faf96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800f770 con 0x5574a6fd4480 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fafa800f910 con 0x5574a6fd4480 2024-09-10T23:01:54.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb0015070 con 0x5574a6fd4480 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafa800fb50 con 0x5574a6fd4480 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafbfdd0740 10 client.17322 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.041+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574a6fef340 con 0x5574a6fd4480 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb0015070 con 0x5574a6fd4480 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fafa8026070 con 0x5574a6fd4480 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 10 client.17322 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 20 client.17322 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a6ff0800 0x5574a6ff2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 4 client.17322 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a6f4cda0 con 0x5574a6ff0800 2024-09-10T23:01:54.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.042+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a6ff0800 0x5574a6ff2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.043+0000 7fafb67fc640 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a6ff0800 0x5574a6ff2c30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5f673c516aeb6a server_cookie=218d35b4f54ed1b in_seq=0 out_seq=0 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.043+0000 7faf96ffd640 10 client.17322 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7faf96ffd640 1 -- 172.21.15.120:0/3546631943 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574a6f4cda0 con 0x5574a6ff0800 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7faf96ffd640 10 client.17322 handle_command_reply: tid=0 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 client.17322 shutdown 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 msgr2=0x5574a6fe0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fafa8004070 tx=0x7fafa8004ab0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 shutdown_connections 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a6ff0800 0x5574a6ff2c30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fe0d30 0x5574a6fde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/3546631943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fd4480 0x5574a6fe0620 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 >> 172.21.15.120:0/3546631943 conn(0x5574a6fd22b0 msgr2=0x5574a6fd7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7faf96ffd640 0 client.17322 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:54.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 shutdown_connections 2024-09-10T23:01:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 1 -- 172.21.15.120:0/3546631943 wait complete. 2024-09-10T23:01:54.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.044+0000 7fafbfdd0740 20 client.17322 trim_cache size 0 max 16384 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 Processor -- start 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 -- start start 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81ae400 con 0x5574a6fe0d30 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b0c80 con 0x5574a6fd4da0 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81aea70 con 0x5574a6fd4480 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36920/0 (socket says 172.21.15.120:36920) 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53862/0 (socket says 172.21.15.120:53862) 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.062+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb77fe640 1 -- 172.21.15.120:0/4241903913 learned_addr learned my addr 172.21.15.120:0/4241903913 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 -- 172.21.15.120:0/4241903913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 msgr2=0x5574a6fdc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 -- 172.21.15.120:0/4241903913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 msgr2=0x5574a6fdf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 -- 172.21.15.120:0/4241903913 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a6fdc9d0 con 0x5574a6fd4480 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb77fe640 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafa801e670 tx=0x7fafa8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37bf7fb71bb64c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7faf9d7fa640 1 -- 172.21.15.120:0/4241903913 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800f770 con 0x5574a6fd4480 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7faf9d7fa640 1 -- 172.21.15.120:0/4241903913 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa8037030 con 0x5574a6fd4480 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.063+0000 7faf9d7fa640 1 -- 172.21.15.120:0/4241903913 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa801e9e0 con 0x5574a6fd4480 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 -- 172.21.15.120:0/4241903913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 msgr2=0x5574a6fecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafa801e670 tx=0x7fafa8005f90 comp rx=0 tx=0).stop 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 -- 172.21.15.120:0/4241903913 shutdown_connections 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fdf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4da0 0x5574a6fdc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/4241903913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4480 0x5574a6fecbe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 -- 172.21.15.120:0/4241903913 >> 172.21.15.120:0/4241903913 conn(0x5574a6fd22b0 msgr2=0x5574a6fdf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 -- 172.21.15.120:0/4241903913 shutdown_connections 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.064+0000 7fafbfdd0740 1 -- 172.21.15.120:0/4241903913 wait complete. 2024-09-10T23:01:54.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 -- start start 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 Processor -- start 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 0x5574a6fe8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81af140 con 0x5574a6fe0d30 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81b05b0 con 0x5574a6fd4480 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81af810 con 0x5574a6fd4da0 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafbfdd0740 10 client.0 _ll_register_callbacks cb 0x7fafa8003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 0x5574a6fe8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53878/0 (socket says 172.21.15.120:53878) 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb67fc640 1 -- 172.21.15.120:0/1221182396 learned_addr learned my addr 172.21.15.120:0/1221182396 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:54.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.065+0000 7fafb77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36924/0 (socket says 172.21.15.120:36924) 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 -- 172.21.15.120:0/1221182396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 msgr2=0x5574a6fe8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 0x5574a6fe8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 -- 172.21.15.120:0/1221182396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 msgr2=0x5574a6fe3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 -- 172.21.15.120:0/1221182396 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a6fe4570 con 0x5574a6fd4da0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 0x5574a6fe8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb77fe640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafb67fc640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a450 tx=0x7fafac0c8d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d188a35f7d4024b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7faf96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac049040 con 0x5574a6fd4da0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fafac109c80 con 0x5574a6fd4da0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa8037030 con 0x5574a6fd4da0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafac0cc520 con 0x5574a6fd4da0 2024-09-10T23:01:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafbfdd0740 10 client.26806 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.066+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574a705e370 con 0x5574a6fd4da0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa8037030 con 0x5574a6fd4da0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fafac0c9020 con 0x5574a6fd4da0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 10 client.26806 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 10 client.26806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 20 client.26806 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a705eaf0 0x5574a7060f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 4 client.26806 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574a6f48850 con 0x5574a705eaf0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a705eaf0 0x5574a7060f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7faf96ffd640 10 client.26806 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.067+0000 7fafb6ffd640 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a705eaf0 0x5574a7060f10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcae35786fbb912f server_cookie=1d0ac366671a4e92 in_seq=0 out_seq=0 2024-09-10T23:01:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7faf96ffd640 1 -- 172.21.15.120:0/1221182396 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x5574a6f48850 con 0x5574a705eaf0 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7faf96ffd640 10 client.26806 handle_command_reply: tid=0 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 client.26806 shutdown 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 msgr2=0x5574a6fe9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafac02a450 tx=0x7fafac0c8d10 comp rx=0 tx=0).stop 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 shutdown_connections 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574a705eaf0 0x5574a7060f10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574a6fe0d30 0x5574a6fe3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574a6fd4da0 0x5574a6fe9560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 --2- 172.21.15.120:0/1221182396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574a6fd4480 0x5574a6fe8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 >> 172.21.15.120:0/1221182396 conn(0x5574a6fd22b0 msgr2=0x5574a6fdf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7faf96ffd640 0 client.26806 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 shutdown_connections 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 1 -- 172.21.15.120:0/1221182396 wait complete. 2024-09-10T23:01:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.068+0000 7fafbfdd0740 20 client.26806 trim_cache size 0 max 16384 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0", 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:01:54.077 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.078 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.080 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:01:54.084 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:01:54.087 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 msgr2=0x7fafa00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 0x7fafa00b77a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5574a6fd3190 tx=0x7fafa8009500 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 msgr2=0x7fafb819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fafac005940 tx=0x7fafac004270 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 shutdown_connections 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fafa00b52e0 0x7fafa00b77a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafb810b350 0x7fafb81a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fafb810a9d0 0x7fafb819b980 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 --2- 172.21.15.120:0/3585991896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb8104fc0 0x7fafb819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 >> 172.21.15.120:0/3585991896 conn(0x7fafb81009f0 msgr2=0x7fafb8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 shutdown_connections 2024-09-10T23:01:54.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:01:54.069+0000 7fafbd9fb640 1 -- 172.21.15.120:0/3585991896 wait complete. 2024-09-10T23:01:54.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1606426 16.17 MB/sec execute 409 sec latency 19.217 ms 2024-09-10T23:01:54.972 INFO:tasks.cephfs.filesystem:scrub status for tag:c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0 - {'path': '/', 'tag': 'c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0', 'options': 'recursive,force'} 2024-09-10T23:01:55.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1609717 16.17 MB/sec execute 410 sec latency 18.977 ms 2024-09-10T23:01:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:55 smithi176 ceph-mon[34982]: pgmap v631: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-09-10T23:01:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:01:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:55 smithi120 ceph-mon[31247]: pgmap v631: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-09-10T23:01:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:01:55.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:55 smithi153 ceph-mon[34951]: pgmap v631: 97 pgs: 97 active+clean; 47 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-09-10T23:01:55.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:01:56.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1613054 16.18 MB/sec execute 411 sec latency 12.017 ms 2024-09-10T23:01:57.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1616293 16.18 MB/sec execute 412 sec latency 25.216 ms 2024-09-10T23:01:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:57 smithi176 ceph-mon[34982]: pgmap v632: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-10T23:01:57.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:57 smithi120 ceph-mon[31247]: pgmap v632: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-10T23:01:57.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:57 smithi153 ceph-mon[34951]: pgmap v632: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 -- 172.21.15.120:0/2615897193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4950103370 msgr2=0x7f4950109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 --2- 172.21.15.120:0/2615897193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4950103370 0x7f4950109cf0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4948009cd0 tx=0x7f494802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 -- 172.21.15.120:0/2615897193 shutdown_connections 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 --2- 172.21.15.120:0/2615897193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4950103370 0x7f4950109cf0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 --2- 172.21.15.120:0/2615897193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49501029b0 0x7f4950102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 --2- 172.21.15.120:0/2615897193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 0x7f4950101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 -- 172.21.15.120:0/2615897193 >> 172.21.15.120:0/2615897193 conn(0x7f49500fd140 msgr2=0x7f49500ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 -- 172.21.15.120:0/2615897193 shutdown_connections 2024-09-10T23:01:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.098+0000 7f4957dca640 1 -- 172.21.15.120:0/2615897193 wait complete. 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 Processor -- start 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 -- start start 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 0x7f4950197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 0x7f4950198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d4d0 con 0x7f49501029b0 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d350 con 0x7f4950101860 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4957dca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f495010d650 con 0x7f4950103370 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4955365640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4955b66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 0x7f4950197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4956367640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 0x7f4950198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4955365640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36930/0 (socket says 172.21.15.120:36930) 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.099+0000 7f4955365640 1 -- 172.21.15.120:0/3843555066 learned_addr learned my addr 172.21.15.120:0/3843555066 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 -- 172.21.15.120:0/3843555066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 msgr2=0x7f4950197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 0x7f4950197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 -- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 msgr2=0x7f4950198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 -- 172.21.15.120:0/3843555066 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4950199760 con 0x7f4950103370 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4955365640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4956367640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 0x7f4950198ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f494802ae00 tx=0x7f4948004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4946ffd640 1 -- 172.21.15.120:0/3843555066 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948004040 con 0x7f4950103370 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4946ffd640 1 -- 172.21.15.120:0/3843555066 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4948004690 con 0x7f4950103370 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49501999f0 con 0x7f4950103370 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4946ffd640 1 -- 172.21.15.120:0/3843555066 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948030dd0 con 0x7f4950103370 2024-09-10T23:01:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.100+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4950199d00 con 0x7f4950103370 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.103+0000 7f4946ffd640 1 -- 172.21.15.120:0/3843555066 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4948030070 con 0x7f4950103370 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.103+0000 7f495ac77740 1 Processor -- start 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.103+0000 7f495ac77740 1 -- start start 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc640ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc64102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fcc63786f0 con 0x55fcc640dde0 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fcc6367b20 con 0x55fcc6408a70 2024-09-10T23:01:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f495ac77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fcc636b930 con 0x55fcc640b3a0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955365640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc64102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53898/0 (socket says 172.21.15.120:53898) 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 -- 172.21.15.120:0/1271557761 learned_addr learned my addr 172.21.15.120:0/1271557761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955365640 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc640ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 -- 172.21.15.120:0/1271557761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 msgr2=0x55fcc640ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc640ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 -- 172.21.15.120:0/1271557761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 msgr2=0x55fcc64102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc64102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 -- 172.21.15.120:0/1271557761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc632d530 con 0x55fcc640b3a0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4946ffd640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955b66640 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f494c009cf0 tx=0x7f494c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab4e39108bec106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.104+0000 7f4955365640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.105+0000 7f4946ffd640 1 -- 172.21.15.120:0/3843555066 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f494802e030 con 0x7f4950103370 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.105+0000 7f493bfff640 1 -- 172.21.15.120:0/1271557761 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c01e070 con 0x55fcc640b3a0 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.105+0000 7f493bfff640 1 -- 172.21.15.120:0/1271557761 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494c009de0 con 0x55fcc640b3a0 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.105+0000 7f493bfff640 1 -- 172.21.15.120:0/1271557761 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c01e070 con 0x55fcc640b3a0 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- 172.21.15.120:0/1271557761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 msgr2=0x55fcc640d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f4955365640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4940009640 tx=0x7f4940009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f494c009cf0 tx=0x7f494c011810 comp rx=0 tx=0).stop 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- 172.21.15.120:0/1271557761 shutdown_connections 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc64102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc640d790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 --2- 172.21.15.120:0/1271557761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc640ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- 172.21.15.120:0/1271557761 >> 172.21.15.120:0/1271557761 conn(0x55fcc64019b0 msgr2=0x55fcc6402740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- 172.21.15.120:0/1271557761 shutdown_connections 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- 172.21.15.120:0/1271557761 wait complete. 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 -- start start 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.106+0000 7f495ac77740 1 Processor -- start 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6418ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fcc629fab0 con 0x55fcc6408a70 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fcc630d730 con 0x55fcc640dde0 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fcc6406b60 con 0x55fcc640b3a0 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 10 client.0 _ll_register_callbacks cb 0x55fcc63616b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4956367640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36948/0 (socket says 172.21.15.120:36948) 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4956367640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36568/0 (socket says 172.21.15.120:36568) 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955365640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6418ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 -- 172.21.15.120:0/2386982945 learned_addr learned my addr 172.21.15.120:0/2386982945 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 -- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 msgr2=0x55fcc6418ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6418ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 -- 172.21.15.120:0/2386982945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 msgr2=0x55fcc641b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 -- 172.21.15.120:0/2386982945 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc60c9790 con 0x55fcc6408a70 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4956367640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f4955b66640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a010 tx=0x7f494c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9708a3531441f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f49397fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:01:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c01e040 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f494c005470 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f494c009de0 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f494c0046c0 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.107+0000 7f495ac77740 10 client.17340 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fcc6142620 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494c009de0 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f494c021070 con 0x55fcc6408a70 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 10 client.17340 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 20 client.17340 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc641f910 0x55fcc6421d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 4 client.17340 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc637a640 con 0x55fcc641f910 2024-09-10T23:01:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.108+0000 7f4955365640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc641f910 0x55fcc6421d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.109+0000 7f49397fa640 10 client.17340 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.109+0000 7f4955365640 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc641f910 0x55fcc6421d40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d95c6879203dc5 server_cookie=7dfb9573a8355901 in_seq=0 out_seq=0 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f49397fa640 1 -- 172.21.15.120:0/2386982945 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcc637a640 con 0x55fcc641f910 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f49397fa640 10 client.17340 handle_command_reply: tid=0 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 client.17340 shutdown 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 msgr2=0x55fcc6406190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a010 tx=0x7f494c003a50 comp rx=0 tx=0).stop 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 shutdown_connections 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc641f910 0x55fcc6421d40 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc640dde0 0x55fcc641b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f49397fa640 0 client.17340 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6418ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 --2- 172.21.15.120:0/2386982945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc6408a70 0x55fcc6406190 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 >> 172.21.15.120:0/2386982945 conn(0x55fcc64019b0 msgr2=0x55fcc6400bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 shutdown_connections 2024-09-10T23:01:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 1 -- 172.21.15.120:0/2386982945 wait complete. 2024-09-10T23:01:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.110+0000 7f495ac77740 20 client.17340 trim_cache size 0 max 16384 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 Processor -- start 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 -- start start 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 0x55fcc6405400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640dde0 0x55fcc6404250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a5b80 con 0x55fcc640b3a0 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a8400 con 0x55fcc6408a70 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f495ac77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a61f0 con 0x55fcc640dde0 2024-09-10T23:01:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955365640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 0x55fcc6405400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36578/0 (socket says 172.21.15.120:36578) 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 -- 172.21.15.120:0/4077797738 learned_addr learned my addr 172.21.15.120:0/4077797738 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4956367640 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640dde0 0x55fcc6404250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 -- 172.21.15.120:0/4077797738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640dde0 msgr2=0x55fcc6404250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640dde0 0x55fcc6404250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 -- 172.21.15.120:0/4077797738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 msgr2=0x55fcc6405400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 0x55fcc6405400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 -- 172.21.15.120:0/4077797738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc64223c0 con 0x55fcc6408a70 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955365640 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 0x55fcc6405400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.130+0000 7f4955b66640 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f494c005e00 tx=0x7f494c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ef62c8fec0cbf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f493b7fe640 1 -- 172.21.15.120:0/4077797738 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c015070 con 0x55fcc6408a70 2024-09-10T23:01:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f493b7fe640 1 -- 172.21.15.120:0/4077797738 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494c027070 con 0x55fcc6408a70 2024-09-10T23:01:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f493b7fe640 1 -- 172.21.15.120:0/4077797738 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c01e070 con 0x55fcc6408a70 2024-09-10T23:01:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 -- 172.21.15.120:0/4077797738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 msgr2=0x55fcc6404ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f494c005e00 tx=0x7f494c011810 comp rx=0 tx=0).stop 2024-09-10T23:01:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 -- 172.21.15.120:0/4077797738 shutdown_connections 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640dde0 0x55fcc6404250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640b3a0 0x55fcc6405400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.131+0000 7f495ac77740 1 --2- 172.21.15.120:0/4077797738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6404ec0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- 172.21.15.120:0/4077797738 >> 172.21.15.120:0/4077797738 conn(0x55fcc64019b0 msgr2=0x55fcc6402650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- 172.21.15.120:0/4077797738 shutdown_connections 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- 172.21.15.120:0/4077797738 wait complete. 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- start start 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 Processor -- start 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6432900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc6432e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a68c0 con 0x55fcc640dde0 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a7d30 con 0x55fcc6408a70 2024-09-10T23:01:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a6f90 con 0x55fcc640b3a0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f495ac77740 10 client.0 _ll_register_callbacks cb 0x7f494c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f4955b66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f4955b66640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36592/0 (socket says 172.21.15.120:36592) 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f4956367640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc6432e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f4955b66640 1 -- 172.21.15.120:0/2872720497 learned_addr learned my addr 172.21.15.120:0/2872720497 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.132+0000 7f4955365640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6432900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 -- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 msgr2=0x55fcc6432900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6432900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 -- 172.21.15.120:0/2872720497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 msgr2=0x55fcc6432e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc6432e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 -- 172.21.15.120:0/2872720497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc64335b0 con 0x55fcc6408a70 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4956367640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc6432e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f4955b66640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f494c0042a0 tx=0x7f494c003950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b98d7c59098bd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f49397fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c01e030 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f494c005500 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f494c027070 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f494c015070 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f495ac77740 10 client.26705 fetch_fsmap learned FSMap version 26 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.133+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fcc648c7e0 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494c027070 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f494c036070 con 0x55fcc6408a70 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 10 client.26705 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 10 client.26705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 20 client.26705 populate_metadata read hostname 'smithi120' 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc648d070 0x55fcc648f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 4 client.26705 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcc63760f0 con 0x55fcc648d070 2024-09-10T23:01:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f4955365640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc648d070 0x55fcc648f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f49397fa640 10 client.26705 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.134+0000 7f4955365640 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc648d070 0x55fcc648f490 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f6c42ef5b8e4664 server_cookie=3994475a05606c60 in_seq=0 out_seq=0 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f49397fa640 1 -- 172.21.15.120:0/2872720497 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcc63760f0 con 0x55fcc648d070 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f49397fa640 10 client.26705 handle_command_reply: tid=0 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 client.26705 shutdown 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 msgr2=0x55fcc6433c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f494c0042a0 tx=0x7f494c003950 comp rx=0 tx=0).stop 2024-09-10T23:01:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 shutdown_connections 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fcc648d070 0x55fcc648f490 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f49397fa640 0 client.26705 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fcc640dde0 0x55fcc6432e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fcc640b3a0 0x55fcc6432900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 --2- 172.21.15.120:0/2872720497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fcc6408a70 0x55fcc6433c00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.418+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 >> 172.21.15.120:0/2872720497 conn(0x55fcc64019b0 msgr2=0x55fcc6404ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 shutdown_connections 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f495ac77740 1 -- 172.21.15.120:0/2872720497 wait complete. 2024-09-10T23:01:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f495ac77740 20 client.26705 trim_cache size 0 max 16384 2024-09-10T23:01:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:01:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:01:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:01:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 msgr2=0x7f493c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4940009640 tx=0x7f4940009590 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 msgr2=0x7f4950198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 0x7f4950198ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f494802ae00 tx=0x7f4948004ab0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 shutdown_connections 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4950103370 0x7f4950198ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501029b0 0x7f4950198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 --2- 172.21.15.120:0/3843555066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4950101860 0x7f4950197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.419+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 >> 172.21.15.120:0/3843555066 conn(0x7f49500fd140 msgr2=0x7f4950105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.420+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 shutdown_connections 2024-09-10T23:01:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:01:58.420+0000 7f4957dca640 1 -- 172.21.15.120:0/3843555066 wait complete. 2024-09-10T23:01:58.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1617777 16.16 MB/sec execute 413 sec latency 109.318 ms 2024-09-10T23:01:59.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1620495 16.15 MB/sec execute 414 sec latency 32.969 ms 2024-09-10T23:01:59.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:01:59 smithi176 ceph-mon[34982]: pgmap v633: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-09-10T23:01:59.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:01:59 smithi120 ceph-mon[31247]: pgmap v633: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-09-10T23:01:59.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:01:59 smithi153 ceph-mon[34951]: pgmap v633: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-09-10T23:02:00.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1623787 16.16 MB/sec execute 415 sec latency 14.172 ms 2024-09-10T23:02:01.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1627047 16.17 MB/sec execute 416 sec latency 26.050 ms 2024-09-10T23:02:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:01 smithi176 ceph-mon[34982]: pgmap v634: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-10T23:02:01.948 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:01 smithi120 ceph-mon[31247]: pgmap v634: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-10T23:02:01.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:01 smithi153 ceph-mon[34951]: pgmap v634: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-10T23:02:02.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1630117 16.16 MB/sec execute 417 sec latency 17.202 ms 2024-09-10T23:02:03.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1632368 16.15 MB/sec execute 418 sec latency 92.969 ms 2024-09-10T23:02:03.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:03 smithi176 ceph-mon[34982]: pgmap v635: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:02:03.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:03 smithi120 ceph-mon[31247]: pgmap v635: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:02:03.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:03 smithi153 ceph-mon[34951]: pgmap v635: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:02:04.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1634753 16.15 MB/sec execute 419 sec latency 81.189 ms 2024-09-10T23:02:05.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1637961 16.15 MB/sec execute 420 sec latency 13.895 ms 2024-09-10T23:02:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:05 smithi176 ceph-mon[34982]: pgmap v636: 97 pgs: 97 active+clean; 45 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-09-10T23:02:05.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:05 smithi120 ceph-mon[31247]: pgmap v636: 97 pgs: 97 active+clean; 45 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-09-10T23:02:05.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:05 smithi153 ceph-mon[34951]: pgmap v636: 97 pgs: 97 active+clean; 45 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-09-10T23:02:06.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1641228 16.16 MB/sec execute 421 sec latency 12.495 ms 2024-09-10T23:02:07.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1644074 16.15 MB/sec execute 422 sec latency 17.724 ms 2024-09-10T23:02:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:07 smithi176 ceph-mon[34982]: pgmap v637: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-10T23:02:07.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:07 smithi120 ceph-mon[31247]: pgmap v637: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-10T23:02:07.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:07 smithi153 ceph-mon[34951]: pgmap v637: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-10T23:02:08.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1646256 16.14 MB/sec execute 423 sec latency 54.918 ms 2024-09-10T23:02:09.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1648803 16.13 MB/sec execute 424 sec latency 48.482 ms 2024-09-10T23:02:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:09 smithi176 ceph-mon[34982]: pgmap v638: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-10T23:02:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:09 smithi120 ceph-mon[31247]: pgmap v638: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-10T23:02:09.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:09 smithi153 ceph-mon[34951]: pgmap v638: 97 pgs: 97 active+clean; 48 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 411 op/s 2024-09-10T23:02:10.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1652025 16.14 MB/sec execute 425 sec latency 15.552 ms 2024-09-10T23:02:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 -- 172.21.15.120:0/2615282783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2800103ae0 msgr2=0x7f2800103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 --2- 172.21.15.120:0/2615282783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2800103ae0 0x7f2800103f60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009930 tx=0x7f27fc02a420 comp rx=0 tx=0).stop 2024-09-10T23:02:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 -- 172.21.15.120:0/2615282783 shutdown_connections 2024-09-10T23:02:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 --2- 172.21.15.120:0/2615282783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28001044a0 0x7f280010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 --2- 172.21.15.120:0/2615282783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2800103ae0 0x7f2800103f60 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 --2- 172.21.15.120:0/2615282783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 -- 172.21.15.120:0/2615282783 >> 172.21.15.120:0/2615282783 conn(0x7f28000fe250 msgr2=0x7f2800100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 -- 172.21.15.120:0/2615282783 shutdown_connections 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.511+0000 7f2807ec6640 1 -- 172.21.15.120:0/2615282783 wait complete. 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 Processor -- start 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 -- start start 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2800103ae0 0x7f2800075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 0x7f2800075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f280010e600 con 0x7f28001044a0 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f280010e480 con 0x7f2800103ae0 2024-09-10T23:02:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2807ec6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f280010e780 con 0x7f2800102990 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2805c62640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2805461640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2800103ae0 0x7f2800075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2805c62640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48288/0 (socket says 172.21.15.120:48288) 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2805c62640 1 -- 172.21.15.120:0/695322877 learned_addr learned my addr 172.21.15.120:0/695322877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 0x7f2800075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 -- 172.21.15.120:0/695322877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 msgr2=0x7f2800076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 -- 172.21.15.120:0/695322877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2800103ae0 msgr2=0x7f2800075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2800103ae0 0x7f2800075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2806463640 1 -- 172.21.15.120:0/695322877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28000762f0 con 0x7f28001044a0 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.512+0000 7f2805c62640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f2806463640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 0x7f2800075a80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f27f800ec50 tx=0x7f27f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f27f2ffd640 1 -- 172.21.15.120:0/695322877 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003b70 con 0x7f28001044a0 2024-09-10T23:02:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f27f2ffd640 1 -- 172.21.15.120:0/695322877 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f8003d10 con 0x7f28001044a0 2024-09-10T23:02:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28001a2ee0 con 0x7f28001044a0 2024-09-10T23:02:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f27f2ffd640 1 -- 172.21.15.120:0/695322877 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8010830 con 0x7f28001044a0 2024-09-10T23:02:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.513+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a3350 con 0x7f28001044a0 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.515+0000 7f280ac99740 1 Processor -- start 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.515+0000 7f27f2ffd640 1 -- 172.21.15.120:0/695322877 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f27f8003ec0 con 0x7f28001044a0 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 -- start start 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 0x5600c7f8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 0x5600c7f95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600c7eff6f0 con 0x5600c7f90190 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600c7eeeb20 con 0x5600c7f8d860 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f280ac99740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600c7ef2930 con 0x5600c7f92bd0 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805461640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 0x5600c7f8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805461640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 0x5600c7f8fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39656/0 (socket says 172.21.15.120:39656) 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805461640 1 -- 172.21.15.120:0/269357022 learned_addr learned my addr 172.21.15.120:0/269357022 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51742/0 (socket says 172.21.15.120:51742) 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805461640 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 0x5600c7f95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 -- 172.21.15.120:0/269357022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 msgr2=0x5600c7f95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 0x5600c7f95090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 -- 172.21.15.120:0/269357022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 msgr2=0x5600c7f8fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 0x5600c7f8fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805c62640 1 -- 172.21.15.120:0/269357022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c7eb4530 con 0x5600c7f90190 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.516+0000 7f2805461640 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 0x5600c7f95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f2805c62640 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011510 tx=0x7f27f40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d1e298d3dd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f27f2ffd640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 0x7f27ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f27ebfff640 1 -- 172.21.15.120:0/269357022 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f401e070 con 0x5600c7f90190 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f2805461640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 0x7f27ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f27ebfff640 1 -- 172.21.15.120:0/269357022 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f400fa70 con 0x5600c7f90190 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f27ebfff640 1 -- 172.21.15.120:0/269357022 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4019b00 con 0x5600c7f90190 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.517+0000 7f27f2ffd640 1 -- 172.21.15.120:0/695322877 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f27f80e60e0 con 0x7f28001044a0 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- 172.21.15.120:0/269357022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 msgr2=0x5600c7f92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f2805461640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 0x7f27ec0b77a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc02a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011510 tx=0x7f27f40119d0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- 172.21.15.120:0/269357022 shutdown_connections 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f92bd0 0x5600c7f95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f92580 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- 172.21.15.120:0/269357022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f8d860 0x5600c7f8fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- 172.21.15.120:0/269357022 >> 172.21.15.120:0/269357022 conn(0x5600c7f889b0 msgr2=0x5600c7f89740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- 172.21.15.120:0/269357022 shutdown_connections 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- 172.21.15.120:0/269357022 wait complete. 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- start start 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 Processor -- start 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f96ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600c7e26ab0 con 0x5600c7f90190 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600c7e94730 con 0x5600c7f92bd0 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.518+0000 7f280ac99740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600c7f95bb0 con 0x5600c7f8d860 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f280ac99740 10 client.0 _ll_register_callbacks cb 0x5600c7ee86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805c62640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f96ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51758/0 (socket says 172.21.15.120:51758) 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 -- 172.21.15.120:0/3994690508 learned_addr learned my addr 172.21.15.120:0/3994690508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2806463640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 -- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 msgr2=0x5600c7f96ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f96ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 -- 172.21.15.120:0/3994690508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 msgr2=0x5600c7f98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 -- 172.21.15.120:0/3994690508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c7c50790 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2806463640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f2805461640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009230 tx=0x7f27fc0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f32a49a8f09a7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f27e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc035040 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f27fc031d10 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f400fa70 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f27fc003800 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f280ac99740 10 client.17358 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.519+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600c7cc9620 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f400fa70 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f27fc047030 con 0x5600c7f90190 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 10 client.17358 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 20 client.17358 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c7fa27a0 0x5600c7fa4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 4 client.17358 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600c7f01640 con 0x5600c7fa27a0 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f2805c62640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c7fa27a0 0x5600c7fa4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f27e97fa640 10 client.17358 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.520+0000 7f2805c62640 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c7fa27a0 0x5600c7fa4bb0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebc029337aaaaba server_cookie=fc30755eb2155f48 in_seq=0 out_seq=0 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f27e97fa640 1 -- 172.21.15.120:0/3994690508 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600c7f01640 con 0x5600c7fa27a0 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f27e97fa640 10 client.17358 handle_command_reply: tid=0 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 client.17358 shutdown 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 msgr2=0x5600c7f97570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009230 tx=0x7f27fc0314a0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 shutdown_connections 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c7fa27a0 0x5600c7fa4bb0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f90190 0x5600c7f97570 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f27e97fa640 0 client.17358 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 --2- 172.21.15.120:0/3994690508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f96ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 >> 172.21.15.120:0/3994690508 conn(0x5600c7f889b0 msgr2=0x5600c7f87bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 shutdown_connections 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.521+0000 7f280ac99740 1 -- 172.21.15.120:0/3994690508 wait complete. 2024-09-10T23:02:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.522+0000 7f280ac99740 20 client.17358 trim_cache size 0 max 16384 2024-09-10T23:02:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.536+0000 7f280ac99740 1 Processor -- start 2024-09-10T23:02:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.536+0000 7f280ac99740 1 -- start start 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8ccf0 0x5600c7f8d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28001aa030 con 0x5600c7f8d860 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28001ac8b0 con 0x5600c7f92bd0 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f280ac99740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f28001aa6a0 con 0x5600c7f8ccf0 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51770/0 (socket says 172.21.15.120:51770) 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 -- 172.21.15.120:0/2265858138 learned_addr learned my addr 172.21.15.120:0/2265858138 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805461640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2806463640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8ccf0 0x5600c7f8d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 -- 172.21.15.120:0/2265858138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8ccf0 msgr2=0x5600c7f8d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8ccf0 0x5600c7f8d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 -- 172.21.15.120:0/2265858138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 msgr2=0x5600c7f934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f934f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 -- 172.21.15.120:0/2265858138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c7f928f0 con 0x5600c7f8d860 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805461640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f2805c62640 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011510 tx=0x7f27f400f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c7ace0e32f8578 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f27eb7fe640 1 -- 172.21.15.120:0/2265858138 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f401e040 con 0x5600c7f8d860 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f27eb7fe640 1 -- 172.21.15.120:0/2265858138 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f40054f0 con 0x5600c7f8d860 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.537+0000 7f27eb7fe640 1 -- 172.21.15.120:0/2265858138 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f40048e0 con 0x5600c7f8d860 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- 172.21.15.120:0/2265858138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 msgr2=0x5600c7f92fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011510 tx=0x7f27f400f750 comp rx=0 tx=0).stop 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- 172.21.15.120:0/2265858138 shutdown_connections 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8ccf0 0x5600c7f8d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 --2- 172.21.15.120:0/2265858138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8d860 0x5600c7f92fb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- 172.21.15.120:0/2265858138 >> 172.21.15.120:0/2265858138 conn(0x5600c7f889b0 msgr2=0x5600c7f96460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- 172.21.15.120:0/2265858138 shutdown_connections 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- 172.21.15.120:0/2265858138 wait complete. 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.538+0000 7f280ac99740 1 -- start start 2024-09-10T23:02:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 Processor -- start 2024-09-10T23:02:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f28001aad70 con 0x5600c7f8ccf0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28001ac1e0 con 0x5600c7f92bd0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f28001ab440 con 0x5600c7f8d860 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f280ac99740 10 client.0 _ll_register_callbacks cb 0x7f27fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51776/0 (socket says 172.21.15.120:51776) 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 -- 172.21.15.120:0/3985031382 learned_addr learned my addr 172.21.15.120:0/3985031382 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805461640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2806463640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 -- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 msgr2=0x5600c7f9e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f9e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 -- 172.21.15.120:0/3985031382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 msgr2=0x5600c7f9c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f9c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 -- 172.21.15.120:0/3985031382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c7f8ffc0 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f2805c62640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011460 tx=0x7f27f4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803c81a287d5ef8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.539+0000 7f27e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f403b020 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f27f4019530 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f40054f0 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f27f401e050 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 10 client.17364 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600c800fc90 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f40054f0 con 0x5600c7f8ccf0 2024-09-10T23:02:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f27f4036550 con 0x5600c7f8ccf0 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 20 client.17364 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c80104d0 0x5600c80128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 4 client.17364 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600c7efd0f0 con 0x5600c80104d0 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.540+0000 7f2805461640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c80104d0 0x5600c80128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.541+0000 7f27e97fa640 10 client.17364 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.541+0000 7f2805461640 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c80104d0 0x5600c80128f0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de48cf500325519f server_cookie=ed58c20a86f73c21 in_seq=0 out_seq=0 2024-09-10T23:02:11.636 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1654603 16.13 MB/sec execute 426 sec latency 38.496 ms 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f27e97fa640 1 -- 172.21.15.120:0/3985031382 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600c7efd0f0 con 0x5600c80104d0 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f27e97fa640 10 client.17364 handle_command_reply: tid=0 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 client.17364 shutdown 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 msgr2=0x5600c7f9ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011460 tx=0x7f27f4005dc0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 shutdown_connections 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600c80104d0 0x5600c80128f0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600c7f92bd0 0x5600c7f9c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600c7f8d860 0x5600c7f9e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f27e97fa640 0 client.17364 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 --2- 172.21.15.120:0/3985031382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600c7f8ccf0 0x5600c7f9ddc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 >> 172.21.15.120:0/3985031382 conn(0x5600c7f889b0 msgr2=0x5600c7f92fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 shutdown_connections 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 1 -- 172.21.15.120:0/3985031382 wait complete. 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.748+0000 7f280ac99740 20 client.17364 trim_cache size 0 max 16384 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 msgr2=0x7f27ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 0x7f27ec0b77a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc02a420 comp rx=0 tx=0).stop 2024-09-10T23:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 msgr2=0x7f2800075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 0x7f2800075a80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f27f800ec50 tx=0x7f27f800c6a0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 shutdown_connections 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f27ec0b52e0 0x7f27ec0b77a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f28001044a0 0x7f2800075a80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2800103ae0 0x7f2800075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 --2- 172.21.15.120:0/695322877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2800102990 0x7f2800076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 >> 172.21.15.120:0/695322877 conn(0x7f28000fe250 msgr2=0x7f2800104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 shutdown_connections 2024-09-10T23:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:11.749+0000 7f2807ec6640 1 -- 172.21.15.120:0/695322877 wait complete. 2024-09-10T23:02:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:11 smithi176 ceph-mon[34982]: pgmap v639: 97 pgs: 97 active+clean; 53 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-09-10T23:02:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:11 smithi120 ceph-mon[31247]: pgmap v639: 97 pgs: 97 active+clean; 53 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-09-10T23:02:11.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:11 smithi153 ceph-mon[34951]: pgmap v639: 97 pgs: 97 active+clean; 53 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-09-10T23:02:12.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1656762 16.12 MB/sec execute 427 sec latency 114.134 ms 2024-09-10T23:02:13.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1659806 16.12 MB/sec execute 428 sec latency 21.939 ms 2024-09-10T23:02:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:13 smithi176 ceph-mon[34982]: pgmap v640: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-09-10T23:02:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:13 smithi120 ceph-mon[31247]: pgmap v640: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-09-10T23:02:13.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:13 smithi153 ceph-mon[34951]: pgmap v640: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-09-10T23:02:14.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1662744 16.12 MB/sec execute 429 sec latency 72.540 ms 2024-09-10T23:02:15.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1666211 16.13 MB/sec execute 430 sec latency 13.663 ms 2024-09-10T23:02:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:15 smithi176 ceph-mon[34982]: pgmap v641: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-09-10T23:02:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:15 smithi120 ceph-mon[31247]: pgmap v641: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-09-10T23:02:15.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:15 smithi153 ceph-mon[34951]: pgmap v641: 97 pgs: 97 active+clean; 53 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-09-10T23:02:16.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1669570 16.14 MB/sec execute 431 sec latency 14.151 ms 2024-09-10T23:02:17.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1672832 16.13 MB/sec execute 432 sec latency 11.273 ms 2024-09-10T23:02:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:17 smithi176 ceph-mon[34982]: pgmap v642: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 465 op/s 2024-09-10T23:02:17.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:17 smithi120 ceph-mon[31247]: pgmap v642: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 465 op/s 2024-09-10T23:02:17.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:17 smithi153 ceph-mon[34951]: pgmap v642: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 465 op/s 2024-09-10T23:02:18.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1676004 16.14 MB/sec execute 433 sec latency 14.706 ms 2024-09-10T23:02:19.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1678380 16.13 MB/sec execute 434 sec latency 42.496 ms 2024-09-10T23:02:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:19 smithi176 ceph-mon[34982]: pgmap v643: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-09-10T23:02:19.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:19 smithi153 ceph-mon[34951]: pgmap v643: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-09-10T23:02:19.949 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:19 smithi120 ceph-mon[31247]: pgmap v643: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-09-10T23:02:20.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1681223 16.13 MB/sec execute 435 sec latency 21.092 ms 2024-09-10T23:02:21.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1684367 16.13 MB/sec execute 436 sec latency 15.684 ms 2024-09-10T23:02:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:21 smithi176 ceph-mon[34982]: pgmap v644: 97 pgs: 97 active+clean; 49 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T23:02:21.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:21 smithi120 ceph-mon[31247]: pgmap v644: 97 pgs: 97 active+clean; 49 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T23:02:21.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:21 smithi153 ceph-mon[34951]: pgmap v644: 97 pgs: 97 active+clean; 49 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-10T23:02:22.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1687589 16.14 MB/sec execute 437 sec latency 8.929 ms 2024-09-10T23:02:23.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1690804 16.14 MB/sec execute 438 sec latency 12.528 ms 2024-09-10T23:02:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:23 smithi176 ceph-mon[34982]: pgmap v645: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-10T23:02:23.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:23 smithi120 ceph-mon[31247]: pgmap v645: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-10T23:02:23.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:23 smithi153 ceph-mon[34951]: pgmap v645: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 418 op/s 2024-09-10T23:02:24.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1693161 16.12 MB/sec execute 439 sec latency 49.066 ms 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 -- 172.21.15.120:0/937051004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 msgr2=0x7efe68101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/937051004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68101c60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efe58009930 tx=0x7efe5802a420 comp rx=0 tx=0).stop 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 -- 172.21.15.120:0/937051004 shutdown_connections 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/937051004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 0x7efe68109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/937051004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 0x7efe68102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/937051004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68101c60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.835+0000 7efe6e7a0640 1 -- 172.21.15.120:0/937051004 >> 172.21.15.120:0/937051004 conn(0x7efe680fd140 msgr2=0x7efe680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- 172.21.15.120:0/937051004 shutdown_connections 2024-09-10T23:02:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- 172.21.15.120:0/937051004 wait complete. 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 Processor -- start 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- start start 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 0x7efe68199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 0x7efe6819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d4b0 con 0x7efe68101860 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d330 con 0x7efe681029b0 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6e7a0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d630 con 0x7efe68103370 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36876/0 (socket says 172.21.15.120:36876) 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe67fff640 1 -- 172.21.15.120:0/4225668397 learned_addr learned my addr 172.21.15.120:0/4225668397 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe5ffff640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 0x7efe68199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.836+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 0x7efe6819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 -- 172.21.15.120:0/4225668397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 msgr2=0x7efe6819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 0x7efe6819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 -- 172.21.15.120:0/4225668397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 msgr2=0x7efe68199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 0x7efe68199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 -- 172.21.15.120:0/4225668397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6819a920 con 0x7efe68101860 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe67fff640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efe58009a50 tx=0x7efe58031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe65ffb640 1 -- 172.21.15.120:0/4225668397 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5803f030 con 0x7efe68101860 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe65ffb640 1 -- 172.21.15.120:0/4225668397 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe580044b0 con 0x7efe68101860 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6819abb0 con 0x7efe68101860 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe65ffb640 1 -- 172.21.15.120:0/4225668397 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58035070 con 0x7efe68101860 2024-09-10T23:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.837+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a12e0 con 0x7efe68101860 2024-09-10T23:02:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.839+0000 7efe7164f740 1 Processor -- start 2024-09-10T23:02:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.839+0000 7efe7164f740 1 -- start start 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.839+0000 7efe65ffb640 1 -- 172.21.15.120:0/4225668397 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efe5803bbb0 con 0x7efe68101860 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd265860 0x55c9fd267c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd1d76f0 con 0x55c9fd26abd0 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd1c6b20 con 0x55c9fd265860 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe7164f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd1ca930 con 0x55c9fd268190 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe67fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36878/0 (socket says 172.21.15.120:36878) 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 -- 172.21.15.120:0/1525690996 learned_addr learned my addr 172.21.15.120:0/1525690996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe67fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44352/0 (socket says 172.21.15.120:44352) 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd265860 0x55c9fd267c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 -- 172.21.15.120:0/1525690996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 msgr2=0x55c9fd26a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 -- 172.21.15.120:0/1525690996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd265860 msgr2=0x55c9fd267c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd265860 0x55c9fd267c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 -- 172.21.15.120:0/1525690996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fd18c530 con 0x55c9fd26abd0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe67fff640 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5ffff640 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efe5000bd10 tx=0x7efe500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584766cb0e3ef8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe65ffb640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 0x7efe540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe5e7fc640 1 -- 172.21.15.120:0/1525690996 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50004030 con 0x55c9fd26abd0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.840+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 0x7efe540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe5e7fc640 1 -- 172.21.15.120:0/1525690996 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe500035f0 con 0x55c9fd26abd0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe5e7fc640 1 -- 172.21.15.120:0/1525690996 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50011770 con 0x55c9fd26abd0 2024-09-10T23:02:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe65ffb640 1 -- 172.21.15.120:0/4225668397 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe58106590 con 0x7efe68101860 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 -- 172.21.15.120:0/1525690996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 msgr2=0x55c9fd26d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 0x7efe540b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe68199b60 tx=0x7efe600096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efe5000bd10 tx=0x7efe500107a0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 -- 172.21.15.120:0/1525690996 shutdown_connections 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd26abd0 0x55c9fd26d090 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 --2- 172.21.15.120:0/1525690996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd265860 0x55c9fd267c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 -- 172.21.15.120:0/1525690996 >> 172.21.15.120:0/1525690996 conn(0x55c9fd2609b0 msgr2=0x55c9fd261740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 -- 172.21.15.120:0/1525690996 shutdown_connections 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.841+0000 7efe7164f740 1 -- 172.21.15.120:0/1525690996 wait complete. 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 -- start start 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 Processor -- start 2024-09-10T23:02:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd270120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd0feab0 con 0x55c9fd265860 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd16c730 con 0x55c9fd26abd0 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd26dbb0 con 0x55c9fd268190 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe7164f740 10 client.0 _ll_register_callbacks cb 0x55c9fd1c06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36888/0 (socket says 172.21.15.120:36888) 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 -- 172.21.15.120:0/4076505776 learned_addr learned my addr 172.21.15.120:0/4076505776 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd270120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe5ffff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 -- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 msgr2=0x55c9fd26f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 -- 172.21.15.120:0/4076505776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 msgr2=0x55c9fd270120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd270120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.842+0000 7efe67fff640 1 -- 172.21.15.120:0/4076505776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fcf28790 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd270120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe67fff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efe58109d70 tx=0x7efe580c94e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f766005135a99bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58049070 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe580d5e20 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe500035f0 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe580ccba0 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 10 client.17376 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9fcfa1620 con 0x55c9fd265860 2024-09-10T23:02:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe500035f0 con 0x55c9fd265860 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efe581060d0 con 0x55c9fd265860 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 10 client.17376 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 20 client.17376 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd27cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 4 client.17376 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.843+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9fd1d9640 con 0x55c9fd27a6e0 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.844+0000 7efe5ffff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd27cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.844+0000 7efe477fe640 10 client.17376 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.844+0000 7efe5ffff640 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd27cb70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed753ada210456b9 server_cookie=bf66667eba0f6f3b in_seq=0 out_seq=0 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe477fe640 1 -- 172.21.15.120:0/4076505776 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9fd1d9640 con 0x55c9fd27a6e0 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe477fe640 10 client.17376 handle_command_reply: tid=0 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 client.17376 shutdown 2024-09-10T23:02:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 msgr2=0x55c9fd26ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efe58109d70 tx=0x7efe580c94e0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 shutdown_connections 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd27cb70 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd270120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd268190 0x55c9fd26f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe477fe640 0 client.17376 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 --2- 172.21.15.120:0/4076505776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd26ef20 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 >> 172.21.15.120:0/4076505776 conn(0x55c9fd2609b0 msgr2=0x55c9fd25fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 shutdown_connections 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 1 -- 172.21.15.120:0/4076505776 wait complete. 2024-09-10T23:02:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.845+0000 7efe7164f740 20 client.17376 trim_cache size 0 max 16384 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.859+0000 7efe7164f740 1 Processor -- start 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 -- start start 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 0x55c9fd26eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd265290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a7e90 con 0x55c9fd268190 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe681aa710 con 0x55c9fd26abd0 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe7164f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a8500 con 0x55c9fd265860 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36898/0 (socket says 172.21.15.120:36898) 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 -- 172.21.15.120:0/2180032409 learned_addr learned my addr 172.21.15.120:0/2180032409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd265290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe67fff640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 0x55c9fd26eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 -- 172.21.15.120:0/2180032409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 msgr2=0x55c9fd26eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 0x55c9fd26eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 -- 172.21.15.120:0/2180032409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 msgr2=0x55c9fd265290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd265290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 -- 172.21.15.120:0/2180032409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fd267fc0 con 0x55c9fd268190 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe67fff640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 0x55c9fd26eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.860+0000 7efe5ffff640 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efe50000c00 tx=0x7efe5001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef3add9a467306b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe5dffb640 1 -- 172.21.15.120:0/2180032409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50032070 con 0x55c9fd268190 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe5dffb640 1 -- 172.21.15.120:0/2180032409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe500035f0 con 0x55c9fd268190 2024-09-10T23:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe5dffb640 1 -- 172.21.15.120:0/2180032409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe500116a0 con 0x55c9fd268190 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 -- 172.21.15.120:0/2180032409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 msgr2=0x55c9fd264d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efe50000c00 tx=0x7efe5001d040 comp rx=0 tx=0).stop 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 -- 172.21.15.120:0/2180032409 shutdown_connections 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd26abd0 0x55c9fd265290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd268190 0x55c9fd264d50 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 --2- 172.21.15.120:0/2180032409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd265860 0x55c9fd26eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 -- 172.21.15.120:0/2180032409 >> 172.21.15.120:0/2180032409 conn(0x55c9fd2609b0 msgr2=0x55c9fd261450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 -- 172.21.15.120:0/2180032409 shutdown_connections 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.861+0000 7efe7164f740 1 -- 172.21.15.120:0/2180032409 wait complete. 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 -- start start 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 Processor -- start 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 0x55c9fd2742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd26abd0 0x55c9fd274e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a8bd0 con 0x55c9fd265860 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe681aa040 con 0x55c9fd268190 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a92a0 con 0x55c9fd26abd0 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe7164f740 10 client.0 _ll_register_callbacks cb 0x7efe58109cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 0x55c9fd2742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe67fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36900/0 (socket says 172.21.15.120:36900) 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe67fff640 1 -- 172.21.15.120:0/519704427 learned_addr learned my addr 172.21.15.120:0/519704427 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.862+0000 7efe6cd3d640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd26abd0 0x55c9fd274e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 -- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd26abd0 msgr2=0x55c9fd274e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd26abd0 0x55c9fd274e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 -- 172.21.15.120:0/519704427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 msgr2=0x55c9fd273c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 -- 172.21.15.120:0/519704427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fd2756a0 con 0x55c9fd268190 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe67fff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe5ffff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 0x55c9fd2742a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe50010e80 tx=0x7efe5000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3051da0dda3ee6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe477fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5003b060 con 0x55c9fd268190 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe50003bb0 con 0x55c9fd268190 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe500035f0 con 0x55c9fd268190 2024-09-10T23:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe50032070 con 0x55c9fd268190 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe7164f740 10 client.26741 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9fd276a10 con 0x55c9fd268190 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.863+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe500035f0 con 0x55c9fd268190 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efe5002c030 con 0x55c9fd268190 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 10 client.26741 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 10 client.26741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 20 client.26741 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd2e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 4 client.26741 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9fd1d50f0 con 0x55c9fd27a6e0 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe67fff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd2e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe477fe640 10 client.26741 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:24.864+0000 7efe67fff640 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd2e9660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfd9d52e8d4c8a2 server_cookie=82f15cc6dc40b6bc in_seq=0 out_seq=0 2024-09-10T23:02:24.974 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:02:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.093+0000 7efe477fe640 1 -- 172.21.15.120:0/519704427 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9fd1d50f0 con 0x55c9fd27a6e0 2024-09-10T23:02:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.093+0000 7efe477fe640 10 client.26741 handle_command_reply: tid=0 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.093+0000 7efe7164f740 1 client.26741 shutdown 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.093+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 msgr2=0x55c9fd2742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 0x55c9fd2742a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe50010e80 tx=0x7efe5000b410 comp rx=0 tx=0).stop 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 shutdown_connections 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c9fd27a6e0 0x55c9fd2e9660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c9fd26abd0 0x55c9fd274e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9fd268190 0x55c9fd2742a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe477fe640 0 client.26741 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 --2- 172.21.15.120:0/519704427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c9fd265860 0x55c9fd273c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 >> 172.21.15.120:0/519704427 conn(0x55c9fd2609b0 msgr2=0x55c9fd261450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 shutdown_connections 2024-09-10T23:02:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 1 -- 172.21.15.120:0/519704427 wait complete. 2024-09-10T23:02:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe7164f740 20 client.26741 trim_cache size 0 max 16384 2024-09-10T23:02:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:02:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:02:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:02:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 msgr2=0x7efe540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 0x7efe540b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe68199b60 tx=0x7efe600096a0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 msgr2=0x7efe68198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efe58009a50 tx=0x7efe58031a50 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 shutdown_connections 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe540b52e0 0x7efe540b77a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe68103370 0x7efe6819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe681029b0 0x7efe68199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 --2- 172.21.15.120:0/4225668397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe68101860 0x7efe68198de0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 >> 172.21.15.120:0/4225668397 conn(0x7efe680fd140 msgr2=0x7efe680febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 shutdown_connections 2024-09-10T23:02:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:25.094+0000 7efe6e7a0640 1 -- 172.21.15.120:0/4225668397 wait complete. 2024-09-10T23:02:25.284 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:25.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1695191 16.11 MB/sec execute 440 sec latency 133.192 ms 2024-09-10T23:02:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:25 smithi120 ceph-mon[31247]: pgmap v646: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-09-10T23:02:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:25 smithi176 ceph-mon[34982]: pgmap v646: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-09-10T23:02:25.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:25 smithi153 ceph-mon[34951]: pgmap v646: 97 pgs: 97 active+clean; 50 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-09-10T23:02:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 -- 172.21.15.120:0/638346305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 msgr2=0x7fec40107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 --2- 172.21.15.120:0/638346305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec40107ae0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec3002a410 comp rx=0 tx=0).stop 2024-09-10T23:02:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 -- 172.21.15.120:0/638346305 shutdown_connections 2024-09-10T23:02:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 --2- 172.21.15.120:0/638346305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 0x7fec4010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 --2- 172.21.15.120:0/638346305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec40107ae0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 --2- 172.21.15.120:0/638346305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 0x7fec40102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 -- 172.21.15.120:0/638346305 >> 172.21.15.120:0/638346305 conn(0x7fec400fd700 msgr2=0x7fec400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 -- 172.21.15.120:0/638346305 shutdown_connections 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 -- 172.21.15.120:0/638346305 wait complete. 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.948+0000 7fec47cbf640 1 Processor -- start 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 -- start start 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 0x7fec4019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 0x7fec401a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec40111ca0 con 0x7fec401076a0 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec40111b20 con 0x7fec40101c90 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec47cbf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec40111e20 con 0x7fec40108020 2024-09-10T23:02:25.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36924/0 (socket says 172.21.15.120:36924) 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 -- 172.21.15.120:0/4188287417 learned_addr learned my addr 172.21.15.120:0/4188287417 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec45a5b640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 0x7fec4019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4625c640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 0x7fec401a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 -- 172.21.15.120:0/4188287417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 msgr2=0x7fec401a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 0x7fec401a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 -- 172.21.15.120:0/4188287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 msgr2=0x7fec4019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 0x7fec4019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 -- 172.21.15.120:0/4188287417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec401a1550 con 0x7fec401076a0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.949+0000 7fec4525a640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fec30005940 tx=0x7fec30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.950+0000 7fec36ffd640 1 -- 172.21.15.120:0/4188287417 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3003f030 con 0x7fec401076a0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.950+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec401a17e0 con 0x7fec401076a0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.950+0000 7fec36ffd640 1 -- 172.21.15.120:0/4188287417 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec30003aa0 con 0x7fec401076a0 2024-09-10T23:02:25.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.950+0000 7fec36ffd640 1 -- 172.21.15.120:0/4188287417 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30035070 con 0x7fec401076a0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.950+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401a7f00 con 0x7fec401076a0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.952+0000 7fec36ffd640 1 -- 172.21.15.120:0/4188287417 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fec3003bce0 con 0x7fec401076a0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.952+0000 7fec4a094740 1 Processor -- start 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 -- start start 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa507d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574aa46fe50 con 0x5574aa505950 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574aa45f280 con 0x5574aa4f90a0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4a094740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574aa463090 con 0x5574aa4f99c0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4525a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa507d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35122/0 (socket says 172.21.15.120:35122) 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44404/0 (socket says 172.21.15.120:44404) 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 -- 172.21.15.120:0/4046140270 learned_addr learned my addr 172.21.15.120:0/4046140270 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec36ffd640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 0x7fec2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 -- 172.21.15.120:0/4046140270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 msgr2=0x5574aa505410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 -- 172.21.15.120:0/4046140270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 msgr2=0x5574aa507d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa507d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 -- 172.21.15.120:0/4046140270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aa1bd650 con 0x5574aa4f90a0 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec45a5b640 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4525a640 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa507d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec36ffd640 1 -- 172.21.15.120:0/4188287417 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec30106510 con 0x7fec401076a0 2024-09-10T23:02:25.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.953+0000 7fec4625c640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 0x7fec2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.954+0000 7fec4625c640 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec38009fd0 tx=0x7fec380118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471bc82f45793a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.954+0000 7fec2bfff640 1 -- 172.21.15.120:0/4046140270 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38004050 con 0x5574aa4f90a0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.954+0000 7fec2bfff640 1 -- 172.21.15.120:0/4046140270 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38004720 con 0x5574aa4f90a0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.954+0000 7fec2bfff640 1 -- 172.21.15.120:0/4046140270 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38019400 con 0x5574aa4f90a0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- 172.21.15.120:0/4046140270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 msgr2=0x5574aa4f9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4625c640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 0x7fec2c0b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec3801d000 tx=0x7fec380196e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fec38009fd0 tx=0x7fec380118d0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- 172.21.15.120:0/4046140270 shutdown_connections 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa507d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa505410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 --2- 172.21.15.120:0/4046140270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f90a0 0x5574aa4f9480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- 172.21.15.120:0/4046140270 >> 172.21.15.120:0/4046140270 conn(0x5574aa4f6ed0 msgr2=0x5574aa4f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- 172.21.15.120:0/4046140270 shutdown_connections 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- 172.21.15.120:0/4046140270 wait complete. 2024-09-10T23:02:25.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 -- start start 2024-09-10T23:02:25.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.955+0000 7fec4a094740 1 Processor -- start 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa5051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa502610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574aa2c0700 con 0x5574aa4f90a0 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574aa3fa110 con 0x5574aa505950 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574aa508570 con 0x5574aa4f99c0 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4a094740 10 client.0 _ll_register_callbacks cb 0x5574aa413990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:25.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4525a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa502610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa5051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35134/0 (socket says 172.21.15.120:35134) 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 -- 172.21.15.120:0/54908800 learned_addr learned my addr 172.21.15.120:0/54908800 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 -- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 msgr2=0x5574aa502610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa502610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 -- 172.21.15.120:0/54908800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 msgr2=0x5574aa5051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa5051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 -- 172.21.15.120:0/54908800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aa427c40 con 0x5574aa505950 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4525a640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa502610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec45a5b640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa5051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec4625c640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec3800fa00 tx=0x7fec38017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b56ecd26252973b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.956+0000 7fec297fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:02:25.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38034020 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec38004720 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec3800fc00 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec38019400 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 10 client.26753 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574aa513ea0 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38004720 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec38030460 con 0x5574aa505950 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 10 client.26753 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 20 client.26753 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa5152b0 0x5574aa5176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 4 client.26753 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574aa471da0 con 0x5574aa5152b0 2024-09-10T23:02:25.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.957+0000 7fec45a5b640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa5152b0 0x5574aa5176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.958+0000 7fec297fa640 10 client.26753 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.958+0000 7fec45a5b640 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa5152b0 0x5574aa5176a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af16142ae8d7ec9 server_cookie=f85299f80cae3a8a in_seq=0 out_seq=0 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec297fa640 1 -- 172.21.15.120:0/54908800 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574aa471da0 con 0x5574aa5152b0 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec297fa640 10 client.26753 handle_command_reply: tid=0 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 client.26753 shutdown 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 msgr2=0x5574aa502b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec3800fa00 tx=0x7fec38017000 comp rx=0 tx=0).stop 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 shutdown_connections 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa5152b0 0x5574aa5176a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa505950 0x5574aa502b90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa4f99c0 0x5574aa502610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 --2- 172.21.15.120:0/54908800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa5051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec297fa640 0 client.26753 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:25.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 >> 172.21.15.120:0/54908800 conn(0x5574aa4f6ed0 msgr2=0x5574aa4fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 shutdown_connections 2024-09-10T23:02:25.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 1 -- 172.21.15.120:0/54908800 wait complete. 2024-09-10T23:02:25.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.959+0000 7fec4a094740 20 client.26753 trim_cache size 0 max 16384 2024-09-10T23:02:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 Processor -- start 2024-09-10T23:02:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 -- start start 2024-09-10T23:02:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa4f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa505950 0x5574aa504350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec401ae4f0 con 0x5574aa4f90a0 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b0d70 con 0x5574aa4f99c0 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec4a094740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec401aeb60 con 0x5574aa505950 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36950/0 (socket says 172.21.15.120:36950) 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.977+0000 7fec45a5b640 1 -- 172.21.15.120:0/8873400 learned_addr learned my addr 172.21.15.120:0/8873400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec4625c640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa505950 0x5574aa504350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec4525a640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa4f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 -- 172.21.15.120:0/8873400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa505950 msgr2=0x5574aa504350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa505950 0x5574aa504350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 -- 172.21.15.120:0/8873400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 msgr2=0x5574aa4f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa4f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 -- 172.21.15.120:0/8873400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aa4f97f0 con 0x5574aa4f90a0 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec4525a640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa4f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec45a5b640 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005940 tx=0x7fec3c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a779d8bf43251d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec2b7fe640 1 -- 172.21.15.120:0/8873400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c029070 con 0x5574aa4f90a0 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec2b7fe640 1 -- 172.21.15.120:0/8873400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3c003e60 con 0x5574aa4f90a0 2024-09-10T23:02:25.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.978+0000 7fec2b7fe640 1 -- 172.21.15.120:0/8873400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0247b0 con 0x5574aa4f90a0 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 -- 172.21.15.120:0/8873400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 msgr2=0x5574aa4f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005940 tx=0x7fec3c00b390 comp rx=0 tx=0).stop 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 -- 172.21.15.120:0/8873400 shutdown_connections 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa505950 0x5574aa504350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa4f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 --2- 172.21.15.120:0/8873400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa4f90a0 0x5574aa4f7de0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 -- 172.21.15.120:0/8873400 >> 172.21.15.120:0/8873400 conn(0x5574aa4f6ed0 msgr2=0x5574aa503c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.979+0000 7fec4a094740 1 -- 172.21.15.120:0/8873400 shutdown_connections 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.980+0000 7fec4a094740 1 -- 172.21.15.120:0/8873400 wait complete. 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.980+0000 7fec4a094740 1 -- start start 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.980+0000 7fec4a094740 1 Processor -- start 2024-09-10T23:02:25.986 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa50dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa50e2c0 0x5574aa5087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec401af230 con 0x5574aa505950 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4525a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa50dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35144/0 (socket says 172.21.15.120:35144) 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 -- 172.21.15.120:0/1572674175 learned_addr learned my addr 172.21.15.120:0/1572674175 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4625c640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa50e2c0 0x5574aa5087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b06a0 con 0x5574aa4f99c0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec401af900 con 0x5574aa50e2c0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec4a094740 10 client.0 _ll_register_callbacks cb 0x7fec38028070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 -- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa50e2c0 msgr2=0x5574aa5087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa50e2c0 0x5574aa5087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 -- 172.21.15.120:0/1572674175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 msgr2=0x5574aa50dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa50dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.981+0000 7fec45a5b640 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aa508db0 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec45a5b640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57f1fe4e414e317 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec297fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c032060 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec3c005240 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c003e60 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec3c029040 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec4a094740 10 client.26759 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574aa582910 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.982+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3c003e60 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec3c02c070 con 0x5574aa4f99c0 2024-09-10T23:02:25.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 10 client.26759 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 20 client.26759 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa582f80 0x5574aa5853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 4 client.26759 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574aa46d850 con 0x5574aa582f80 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4525a640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa582f80 0x5574aa5853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec297fa640 10 client.26759 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.983+0000 7fec4525a640 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa582f80 0x5574aa5853a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568a8ed96b27f6bd server_cookie=dd53a2c6467c1465 in_seq=0 out_seq=0 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec297fa640 1 -- 172.21.15.120:0/1572674175 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x5574aa46d850 con 0x5574aa582f80 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec297fa640 10 client.26759 handle_command_reply: tid=0 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 client.26759 shutdown 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 msgr2=0x5574aa50d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:25.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c004550 comp rx=0 tx=0).stop 2024-09-10T23:02:25.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 shutdown_connections 2024-09-10T23:02:25.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574aa582f80 0x5574aa5853a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574aa50e2c0 0x5574aa5087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5574aa505950 0x5574aa50dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.990 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 --2- 172.21.15.120:0/1572674175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5574aa4f99c0 0x5574aa50d840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 >> 172.21.15.120:0/1572674175 conn(0x5574aa4f6ed0 msgr2=0x5574aa503c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec297fa640 0 client.26759 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 shutdown_connections 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.984+0000 7fec4a094740 1 -- 172.21.15.120:0/1572674175 wait complete. 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.985+0000 7fec4a094740 20 client.26759 trim_cache size 0 max 16384 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:02:25.991 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:02:25.993 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.994 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.997 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:25.999 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:02:26.000 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:26.001 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:02:26.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 msgr2=0x7fec2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 0x7fec2c0b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fec3801d000 tx=0x7fec380196e0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 msgr2=0x7fec4019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fec30005940 tx=0x7fec30004270 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 shutdown_connections 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec2c0b52e0 0x7fec2c0b77a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec40108020 0x7fec401a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec401076a0 0x7fec4019b9e0 secure :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fec30005940 tx=0x7fec30004270 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 --2- 172.21.15.120:0/4188287417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec40101c90 0x7fec4019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.986+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 >> 172.21.15.120:0/4188287417 conn(0x7fec400fd700 msgr2=0x7fec4006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.987+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 shutdown_connections 2024-09-10T23:02:26.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:25.987+0000 7fec47cbf640 1 -- 172.21.15.120:0/4188287417 wait complete. 2024-09-10T23:02:26.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1698548 16.12 MB/sec execute 441 sec latency 12.714 ms 2024-09-10T23:02:27.054 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:02:27.054 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:02:27.314 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:27.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1701795 16.13 MB/sec execute 442 sec latency 28.914 ms 2024-09-10T23:02:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:27 smithi120 ceph-mon[31247]: pgmap v647: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 570 op/s 2024-09-10T23:02:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:27 smithi176 ceph-mon[34982]: pgmap v647: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 570 op/s 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- 172.21.15.120:0/2261176019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044102da0 msgr2=0x7f6044103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- 172.21.15.120:0/2261176019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044102da0 0x7f6044103180 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c02a410 comp rx=0 tx=0).stop 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- 172.21.15.120:0/2261176019 shutdown_connections 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- 172.21.15.120:0/2261176019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044109130 0x7f604410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- 172.21.15.120:0/2261176019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f6044108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- 172.21.15.120:0/2261176019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044102da0 0x7f6044103180 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- 172.21.15.120:0/2261176019 >> 172.21.15.120:0/2261176019 conn(0x7f60440fe850 msgr2=0x7f6044100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- 172.21.15.120:0/2261176019 shutdown_connections 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- 172.21.15.120:0/2261176019 wait complete. 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 Processor -- start 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 -- start start 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 0x7f6044199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.820+0000 7f604aeaa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f604aeaa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f604aeaa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112df0 con 0x7f6044109130 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f604aeaa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112c70 con 0x7f60441087b0 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f604aeaa640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6044112f70 con 0x7f6044102da0 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6048c46640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 0x7f6044199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6043fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6048c46640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 0x7f6044199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44454/0 (socket says 172.21.15.120:44454) 2024-09-10T23:02:27.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36986/0 (socket says 172.21.15.120:36986) 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6043fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35158/0 (socket says 172.21.15.120:35158) 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6048c46640 1 -- 172.21.15.120:0/234852277 learned_addr learned my addr 172.21.15.120:0/234852277 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 -- 172.21.15.120:0/234852277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 msgr2=0x7f6044199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 0x7f6044199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 -- 172.21.15.120:0/234852277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 msgr2=0x7f60441997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 -- 172.21.15.120:0/234852277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419f330 con 0x7f6044109130 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6043fff640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6049447640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e970 tx=0x7f602c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c00ccc0 con 0x7f6044109130 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c005020 con 0x7f6044109130 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419f620 con 0x7f6044109130 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.821+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c004a00 con 0x7f6044109130 2024-09-10T23:02:27.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.822+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a5eb0 con 0x7f6044109130 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.823+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044103180 con 0x7f6044109130 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.824+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f602c004ba0 con 0x7f6044109130 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.825+0000 7f6041ffb640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 0x7f60300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.825+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f602c0e5a70 con 0x7f6044109130 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.825+0000 7f6048c46640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 0x7f60300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.825+0000 7f6048c46640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 0x7f60300b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f603c031440 tx=0x7f603c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:27.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:27.829+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f602c0ac960 con 0x7f6044109130 2024-09-10T23:02:27.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:27 smithi153 ceph-mon[34951]: pgmap v647: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 570 op/s 2024-09-10T23:02:28.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.049+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60441a6080 con 0x7f6044109130 2024-09-10T23:02:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.050+0000 7f6041ffb640 1 -- 172.21.15.120:0/234852277 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f602c0ac260 con 0x7f6044109130 2024-09-10T23:02:28.052 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:02:28.052 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:02:28.052 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 msgr2=0x7f60300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 0x7f60300b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f603c031440 tx=0x7f603c0058d0 comp rx=0 tx=0).stop 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 msgr2=0x7f604419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e970 tx=0x7f602c00ee30 comp rx=0 tx=0).stop 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 shutdown_connections 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60300b5360 0x7f60300b7820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6044109130 0x7f604419eb60 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:28.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60441087b0 0x7f60441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 --2- 172.21.15.120:0/234852277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6044102da0 0x7f6044199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 >> 172.21.15.120:0/234852277 conn(0x7f60440fe850 msgr2=0x7f60441000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 shutdown_connections 2024-09-10T23:02:28.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:28.052+0000 7f604aeaa640 1 -- 172.21.15.120:0/234852277 wait complete. 2024-09-10T23:02:28.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1705159 16.13 MB/sec execute 443 sec latency 40.021 ms 2024-09-10T23:02:28.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/234852277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:28.919 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/234852277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:28.949 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:02:28.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/234852277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:29.231 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:29.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1707313 16.11 MB/sec execute 444 sec latency 57.604 ms 2024-09-10T23:02:29.672 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:29 smithi120 ceph-mon[31247]: pgmap v648: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- 172.21.15.120:0/2070866027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc710109170 msgr2=0x7fc71010f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 --2- 172.21.15.120:0/2070866027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc710109170 0x7fc71010f970 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc708009900 tx=0x7fc70802a410 comp rx=0 tx=0).stop 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- 172.21.15.120:0/2070866027 shutdown_connections 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 --2- 172.21.15.120:0/2070866027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc710109170 0x7fc71010f970 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 --2- 172.21.15.120:0/2070866027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7101087f0 0x7fc710108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 --2- 172.21.15.120:0/2070866027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710102de0 0x7fc7101031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- 172.21.15.120:0/2070866027 >> 172.21.15.120:0/2070866027 conn(0x7fc7100fe830 msgr2=0x7fc710100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- 172.21.15.120:0/2070866027 shutdown_connections 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- 172.21.15.120:0/2070866027 wait complete. 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 Processor -- start 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.754+0000 7fc71745e640 1 -- start start 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 0x7fc710199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710109170 0x7fc71019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc710112df0 con 0x7fc7101087f0 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc710112c70 con 0x7fc710109170 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc71745e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc710112f70 con 0x7fc710102de0 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7149f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7149f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37002/0 (socket says 172.21.15.120:37002) 2024-09-10T23:02:29.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7149f9640 1 -- 172.21.15.120:0/2555441466 learned_addr learned my addr 172.21.15.120:0/2555441466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 0x7fc710199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7159fb640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710109170 0x7fc71019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 -- 172.21.15.120:0/2555441466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710109170 msgr2=0x7fc71019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710109170 0x7fc71019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 -- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 msgr2=0x7fc7101997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 -- 172.21.15.120:0/2555441466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019f310 con 0x7fc710102de0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7149f9640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 0x7fc710199260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc70000eb00 tx=0x7fc70000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.755+0000 7fc70e7fc640 1 -- 172.21.15.120:0/2555441466 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700010430 con 0x7fc710102de0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.756+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019f600 con 0x7fc710102de0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.756+0000 7fc70e7fc640 1 -- 172.21.15.120:0/2555441466 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc700010a80 con 0x7fc710102de0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.756+0000 7fc70e7fc640 1 -- 172.21.15.120:0/2555441466 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700017a90 con 0x7fc710102de0 2024-09-10T23:02:29.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.756+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a5cc0 con 0x7fc710102de0 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 Processor -- start 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 -- start start 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc70e7fc640 1 -- 172.21.15.120:0/2555441466 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc700017c70 con 0x7fc710102de0 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56499276ba10 0x564992777460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 0x564992779d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5649926e1ea0 con 0x5649927779a0 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5649926d12d0 con 0x56499276ba10 2024-09-10T23:02:29.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.758+0000 7fc719833740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5649926d50e0 con 0x56499276b0f0 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7151fa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56499276ba10 0x564992777460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7149f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 0x564992779d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44480/0 (socket says 172.21.15.120:44480) 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 -- 172.21.15.120:0/1883763697 learned_addr learned my addr 172.21.15.120:0/1883763697 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 -- 172.21.15.120:0/1883763697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56499276ba10 msgr2=0x564992777460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56499276ba10 0x564992777460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 -- 172.21.15.120:0/1883763697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 msgr2=0x564992779d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 0x564992779d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 -- 172.21.15.120:0/1883763697 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499242f650 con 0x56499276b0f0 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7149f9640 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 0x564992779d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc70802a930 tx=0x7fc708032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c33ab0917896ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc70e7fc640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 0x7fc6f80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc6f77fe640 1 -- 172.21.15.120:0/1883763697 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708040070 con 0x56499276b0f0 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 0x7fc6f80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc6f77fe640 1 -- 172.21.15.120:0/1883763697 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708004030 con 0x56499276b0f0 2024-09-10T23:02:29.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.759+0000 7fc6f77fe640 1 -- 172.21.15.120:0/1883763697 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708035040 con 0x56499276b0f0 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc70e7fc640 1 -- 172.21.15.120:0/2555441466 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc7000e65f0 con 0x7fc710102de0 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 -- 172.21.15.120:0/1883763697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 msgr2=0x56499276b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc7151fa640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 0x7fc6f80b77a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc7000e4490 tx=0x7fc7000ec190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc70802a930 tx=0x7fc708032b20 comp rx=0 tx=0).stop 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 -- 172.21.15.120:0/1883763697 shutdown_connections 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5649927779a0 0x564992779d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56499276ba10 0x564992777460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 --2- 172.21.15.120:0/1883763697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x56499276b4d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 -- 172.21.15.120:0/1883763697 >> 172.21.15.120:0/1883763697 conn(0x564992768f20 msgr2=0x56499276a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 -- 172.21.15.120:0/1883763697 shutdown_connections 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.760+0000 7fc719833740 1 -- 172.21.15.120:0/1883763697 wait complete. 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 -- start start 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 Processor -- start 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x5649927772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992774d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564992532700 con 0x56499276ba10 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56499266c160 con 0x5649927779a0 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56499277a5c0 con 0x56499276b0f0 2024-09-10T23:02:29.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc719833740 10 client.0 _ll_register_callbacks cb 0x5649926859e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc7149f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc7149f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37010/0 (socket says 172.21.15.120:37010) 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x5649927772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc7149f9640 1 -- 172.21.15.120:0/4030638080 learned_addr learned my addr 172.21.15.120:0/4030638080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.761+0000 7fc7159fb640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992774d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 -- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 msgr2=0x5649927772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x5649927772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 -- 172.21.15.120:0/4030638080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 msgr2=0x564992774d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992774d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 -- 172.21.15.120:0/4030638080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564992699c90 con 0x56499276ba10 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7159fb640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x5649927772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:29.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc7149f9640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc704012d20 tx=0x7fc70400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01fdf11585f8379 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7040120d0 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc704010430 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc708004030 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc704012270 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 10 client.17418 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564992785fb0 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708004030 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc704027720 con 0x56499276ba10 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 10 client.17418 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 20 client.17418 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:29.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564992787560 0x564992789980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 4 client.17418 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.762+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649926e3df0 con 0x564992787560 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.763+0000 7fc7159fb640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564992787560 0x564992789980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.763+0000 7fc7159fb640 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564992787560 0x564992789980 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f74dae263d38f3 server_cookie=5ab609a3df8f190d in_seq=0 out_seq=0 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.763+0000 7fc6f4ff9640 10 client.17418 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/4030638080 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649926e3df0 con 0x564992787560 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc6f4ff9640 10 client.17418 handle_command_reply: tid=0 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 client.17418 shutdown 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 msgr2=0x564992774710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc704012d20 tx=0x7fc70400f790 comp rx=0 tx=0).stop 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 shutdown_connections 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564992787560 0x564992789980 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992774d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc6f4ff9640 0 client.17418 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276ba10 0x564992774710 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 --2- 172.21.15.120:0/4030638080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276b0f0 0x5649927772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 >> 172.21.15.120:0/4030638080 conn(0x564992768f20 msgr2=0x56499276ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 shutdown_connections 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 1 -- 172.21.15.120:0/4030638080 wait complete. 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.764+0000 7fc719833740 20 client.17418 trim_cache size 0 max 16384 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 Processor -- start 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 -- start start 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x564992768760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992778d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ac290 con 0x56499276b0f0 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101aeb10 con 0x5649927779a0 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc719833740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ac900 con 0x56499276ba10 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37018/0 (socket says 172.21.15.120:37018) 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc7159fb640 1 -- 172.21.15.120:0/2244394301 learned_addr learned my addr 172.21.15.120:0/2244394301 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.777+0000 7fc7149f9640 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x564992768760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 -- 172.21.15.120:0/2244394301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 msgr2=0x564992768760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x564992768760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 -- 172.21.15.120:0/2244394301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 msgr2=0x564992778d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992778d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 -- 172.21.15.120:0/2244394301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564992769c00 con 0x56499276b0f0 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc7159fb640 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc708033000 tx=0x7fc708032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6947f37840647cec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc6f6ffd640 1 -- 172.21.15.120:0/2244394301 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708035040 con 0x56499276b0f0 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc6f6ffd640 1 -- 172.21.15.120:0/2244394301 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708004380 con 0x56499276b0f0 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.778+0000 7fc6f6ffd640 1 -- 172.21.15.120:0/2244394301 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708040040 con 0x56499276b0f0 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- 172.21.15.120:0/2244394301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 msgr2=0x56499276a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc708033000 tx=0x7fc708032b20 comp rx=0 tx=0).stop 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- 172.21.15.120:0/2244394301 shutdown_connections 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992778d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x564992768760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 --2- 172.21.15.120:0/2244394301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499276a5e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- 172.21.15.120:0/2244394301 >> 172.21.15.120:0/2244394301 conn(0x564992768f20 msgr2=0x564992776560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- 172.21.15.120:0/2244394301 shutdown_connections 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- 172.21.15.120:0/2244394301 wait complete. 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 -- start start 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.779+0000 7fc719833740 1 Processor -- start 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x56499277ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992770930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101acfd0 con 0x56499276b0f0 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ae440 con 0x5649927779a0 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101ad6a0 con 0x56499276ba10 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc719833740 10 client.0 _ll_register_callbacks cb 0x7fc704009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37024/0 (socket says 172.21.15.120:37024) 2024-09-10T23:02:29.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 -- 172.21.15.120:0/1174559692 learned_addr learned my addr 172.21.15.120:0/1174559692 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7149f9640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x56499277ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992770930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 -- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 msgr2=0x56499277ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x56499277ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 -- 172.21.15.120:0/1174559692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 msgr2=0x564992770930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992770930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 -- 172.21.15.120:0/1174559692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564992770f20 con 0x56499276b0f0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7149f9640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x56499277ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc7159fb640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc7080096c0 tx=0x7fc708005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b06dcf0f31f5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.780+0000 7fc6f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708040070 con 0x56499276b0f0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc708031690 con 0x56499276b0f0 2024-09-10T23:02:29.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc708004380 con 0x56499276b0f0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc708035040 con 0x56499276b0f0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 10 client.17424 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564992772200 con 0x56499276b0f0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708004380 con 0x56499276b0f0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc70805b410 con 0x56499276b0f0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 10 client.17424 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 20 client.17424 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649927724e0 0x5649927f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 4 client.17424 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649926df8a0 con 0x5649927724e0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.781+0000 7fc7149f9640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649927724e0 0x5649927f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.782+0000 7fc7149f9640 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649927724e0 0x5649927f7450 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97447ddbc6369e0 server_cookie=9cfe67ae109ab086 in_seq=0 out_seq=0 2024-09-10T23:02:29.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.782+0000 7fc6f4ff9640 10 client.17424 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc6f4ff9640 1 -- 172.21.15.120:0/1174559692 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649926df8a0 con 0x5649927724e0 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc6f4ff9640 10 client.17424 handle_command_reply: tid=0 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 client.17424 shutdown 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 msgr2=0x56499277fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc7080096c0 tx=0x7fc708005490 comp rx=0 tx=0).stop 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 shutdown_connections 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5649927724e0 0x5649927f7450 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5649927779a0 0x564992770930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56499276ba10 0x56499277ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 --2- 172.21.15.120:0/1174559692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56499276b0f0 0x56499277fa60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc6f4ff9640 0 client.17424 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 >> 172.21.15.120:0/1174559692 conn(0x564992768f20 msgr2=0x564992772f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 shutdown_connections 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 1 -- 172.21.15.120:0/1174559692 wait complete. 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.783+0000 7fc719833740 20 client.17424 trim_cache size 0 max 16384 2024-09-10T23:02:29.803 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 msgr2=0x7fc6f80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 0x7fc6f80b77a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc7000e4490 tx=0x7fc7000ec190 comp rx=0 tx=0).stop 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 msgr2=0x7fc710199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 0x7fc710199260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc70000eb00 tx=0x7fc70000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 shutdown_connections 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc6f80b52e0 0x7fc6f80b77a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc710109170 0x7fc71019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7101087f0 0x7fc7101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 --2- 172.21.15.120:0/2555441466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710102de0 0x7fc710199260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 >> 172.21.15.120:0/2555441466 conn(0x7fc7100fe830 msgr2=0x7fc710100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 shutdown_connections 2024-09-10T23:02:29.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:29.784+0000 7fc71745e640 1 -- 172.21.15.120:0/2555441466 wait complete. 2024-09-10T23:02:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:29 smithi176 ceph-mon[34982]: pgmap v648: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T23:02:29.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:29 smithi153 ceph-mon[34951]: pgmap v648: 97 pgs: 97 active+clean; 38 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T23:02:30.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1709885 16.11 MB/sec execute 445 sec latency 12.248 ms 2024-09-10T23:02:31.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:02:31.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:02:31.637 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1712685 16.11 MB/sec execute 446 sec latency 27.281 ms 2024-09-10T23:02:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:31 smithi176 ceph-mon[34982]: pgmap v649: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-09-10T23:02:31.848 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:31.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:31 smithi120 ceph-mon[31247]: pgmap v649: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-09-10T23:02:31.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:31 smithi153 ceph-mon[34951]: pgmap v649: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- 172.21.15.120:0/1396320308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 msgr2=0x7efe0410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 --2- 172.21.15.120:0/1396320308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe0410cb10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009900 tx=0x7efdf402a410 comp rx=0 tx=0).stop 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- 172.21.15.120:0/1396320308 shutdown_connections 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 --2- 172.21.15.120:0/1396320308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 0x7efe0410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 --2- 172.21.15.120:0/1396320308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe0410cb10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 --2- 172.21.15.120:0/1396320308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 0x7efe04103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- 172.21.15.120:0/1396320308 >> 172.21.15.120:0/1396320308 conn(0x7efe040fd700 msgr2=0x7efe040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- 172.21.15.120:0/1396320308 shutdown_connections 2024-09-10T23:02:32.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- 172.21.15.120:0/1396320308 wait complete. 2024-09-10T23:02:32.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 Processor -- start 2024-09-10T23:02:32.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.348+0000 7efe0c54f640 1 -- start start 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 0x7efe041992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 0x7efe0419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe04112db0 con 0x7efe04104530 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe04112c30 con 0x7efe0410d0e0 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0c54f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe04112f30 con 0x7efe04103b30 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37050/0 (socket says 172.21.15.120:37050) 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 -- 172.21.15.120:0/3131466485 learned_addr learned my addr 172.21.15.120:0/3131466485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 0x7efe041992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe0aaec640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 0x7efe0419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 -- 172.21.15.120:0/3131466485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 msgr2=0x7efe041992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 0x7efe041992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 -- 172.21.15.120:0/3131466485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 msgr2=0x7efe0419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 0x7efe0419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 -- 172.21.15.120:0/3131466485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0419f390 con 0x7efe04104530 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.349+0000 7efe09aea640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efdf4005940 tx=0x7efdf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.350+0000 7efdfb7fe640 1 -- 172.21.15.120:0/3131466485 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf403f030 con 0x7efe04104530 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.350+0000 7efdfb7fe640 1 -- 172.21.15.120:0/3131466485 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf4003cb0 con 0x7efe04104530 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.350+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0419f620 con 0x7efe04104530 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.350+0000 7efdfb7fe640 1 -- 172.21.15.120:0/3131466485 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4035070 con 0x7efe04104530 2024-09-10T23:02:32.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.350+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a5d40 con 0x7efe04104530 2024-09-10T23:02:32.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 Processor -- start 2024-09-10T23:02:32.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 -- start start 2024-09-10T23:02:32.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efdfb7fe640 1 -- 172.21.15.120:0/3131466485 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efdf403bbf0 con 0x7efe04104530 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828885250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828887b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e8287efe60 con 0x55e828879800 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e8287df290 con 0x55e828885790 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0e924740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e8287e30a0 con 0x55e828878ee0 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0a2eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828885250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0aaec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0aaec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44546/0 (socket says 172.21.15.120:44546) 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0a2eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828885250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37056/0 (socket says 172.21.15.120:37056) 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe0aaec640 1 -- 172.21.15.120:0/730225261 learned_addr learned my addr 172.21.15.120:0/730225261 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.353+0000 7efe09aea640 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828887b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efdfb7fe640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 0x7efdf00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 -- 172.21.15.120:0/730225261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 msgr2=0x55e828887b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828887b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 -- 172.21.15.120:0/730225261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 msgr2=0x55e828885250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828885250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 -- 172.21.15.120:0/730225261 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e82853d930 con 0x55e828878ee0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efdfb7fe640 1 -- 172.21.15.120:0/3131466485 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efdf4106300 con 0x7efe04104530 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 0x7efdf00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.354+0000 7efe0aaec640 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a4b0 tx=0x7efdfc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f54926f95636f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efdf8ff9640 1 -- 172.21.15.120:0/730225261 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0128b0 con 0x55e828878ee0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efdf8ff9640 1 -- 172.21.15.120:0/730225261 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc012d30 con 0x55e828878ee0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efdf8ff9640 1 -- 172.21.15.120:0/730225261 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc014810 con 0x55e828878ee0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 -- 172.21.15.120:0/730225261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 msgr2=0x55e8288792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0aaec640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 0x7efdf00b77a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00afd0 tx=0x7efdfc014f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a4b0 tx=0x7efdfc009650 comp rx=0 tx=0).stop 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 -- 172.21.15.120:0/730225261 shutdown_connections 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828887b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828885250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 --2- 172.21.15.120:0/730225261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288792c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.355+0000 7efe0e924740 1 -- 172.21.15.120:0/730225261 >> 172.21.15.120:0/730225261 conn(0x55e828876d10 msgr2=0x55e828877eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- 172.21.15.120:0/730225261 shutdown_connections 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- 172.21.15.120:0/730225261 wait complete. 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- start start 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 Processor -- start 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828883160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e828640890 con 0x55e828879800 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e828779d40 con 0x55e828885790 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e828888440 con 0x55e828878ee0 2024-09-10T23:02:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0e924740 10 client.0 _ll_register_callbacks cb 0x55e8287ddc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0a2eb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0aaec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828883160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37066/0 (socket says 172.21.15.120:37066) 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe0a2eb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288826e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44558/0 (socket says 172.21.15.120:44558) 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.356+0000 7efe09aea640 1 -- 172.21.15.120:0/1696486409 learned_addr learned my addr 172.21.15.120:0/1696486409 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 -- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 msgr2=0x55e8288826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 -- 172.21.15.120:0/1696486409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 msgr2=0x55e828883160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828883160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 -- 172.21.15.120:0/1696486409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8288917a0 con 0x55e828879800 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe0aaec640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828883160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe09aea640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efdf40ccb30 tx=0x7efdf402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18f03bbe243c033 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4049050 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efdf40cce00 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc012d30 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdf40cc4f0 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe0e924740 10 client.17436 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e828894150 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc012d30 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.357+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efdf4105070 con 0x55e828879800 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 10 client.17436 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 20 client.17436 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e828895560 0x55e828897950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 4 client.17436 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8287f1db0 con 0x55e828895560 2024-09-10T23:02:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e828895560 0x55e828897950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efdedffb640 10 client.17436 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.358+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e828895560 0x55e828897950 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a88c8b16a62849 server_cookie=582f935dd62b143a in_seq=0 out_seq=0 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efdedffb640 1 -- 172.21.15.120:0/1696486409 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8287f1db0 con 0x55e828895560 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efdedffb640 10 client.17436 handle_command_reply: tid=0 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 client.17436 shutdown 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 msgr2=0x55e828882c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efdf40ccb30 tx=0x7efdf402a410 comp rx=0 tx=0).stop 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 shutdown_connections 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e828895560 0x55e828897950 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828885790 0x55e828883160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e828882c20 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efdedffb640 0 client.17436 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 --2- 172.21.15.120:0/1696486409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e8288826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 >> 172.21.15.120:0/1696486409 conn(0x55e828876d10 msgr2=0x55e828875f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 shutdown_connections 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 1 -- 172.21.15.120:0/1696486409 wait complete. 2024-09-10T23:02:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.360+0000 7efe0e924740 20 client.17436 trim_cache size 0 max 16384 2024-09-10T23:02:32.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.379+0000 7efe0e924740 1 Processor -- start 2024-09-10T23:02:32.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.379+0000 7efe0e924740 1 -- start start 2024-09-10T23:02:32.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e828880170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828879800 0x55e828884420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ac230 con 0x55e828885790 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe041aeab0 con 0x55e828879800 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0e924740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ac8a0 con 0x55e828878ee0 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37068/0 (socket says 172.21.15.120:37068) 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 -- 172.21.15.120:0/2444497783 learned_addr learned my addr 172.21.15.120:0/2444497783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e828880170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe09aea640 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828879800 0x55e828884420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 -- 172.21.15.120:0/2444497783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 msgr2=0x55e828880170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e828880170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 -- 172.21.15.120:0/2444497783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828879800 msgr2=0x55e828884420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828879800 0x55e828884420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 -- 172.21.15.120:0/2444497783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e828872640 con 0x55e828885790 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.380+0000 7efe0aaec640 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a010 tx=0x7efdfc026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3395699860e8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efdeffff640 1 -- 172.21.15.120:0/2444497783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc009bc0 con 0x55e828885790 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efdeffff640 1 -- 172.21.15.120:0/2444497783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc012cf0 con 0x55e828885790 2024-09-10T23:02:32.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efdeffff640 1 -- 172.21.15.120:0/2444497783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc014820 con 0x55e828885790 2024-09-10T23:02:32.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 -- 172.21.15.120:0/2444497783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 msgr2=0x55e82889d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00a010 tx=0x7efdfc026000 comp rx=0 tx=0).stop 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 -- 172.21.15.120:0/2444497783 shutdown_connections 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828885790 0x55e82889d220 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828879800 0x55e828884420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 --2- 172.21.15.120:0/2444497783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828878ee0 0x55e828880170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 -- 172.21.15.120:0/2444497783 >> 172.21.15.120:0/2444497783 conn(0x55e828876d10 msgr2=0x55e828883d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 -- 172.21.15.120:0/2444497783 shutdown_connections 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.381+0000 7efe0e924740 1 -- 172.21.15.120:0/2444497783 wait complete. 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 -- start start 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 Processor -- start 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 0x55e828889ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828885790 0x55e828909c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe041acf70 con 0x55e828879800 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ae3e0 con 0x55e828878ee0 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ad640 con 0x55e828885790 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0e924740 10 client.0 _ll_register_callbacks cb 0x7efdf4005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0a2eb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 0x55e828889ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37076/0 (socket says 172.21.15.120:37076) 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 -- 172.21.15.120:0/896679164 learned_addr learned my addr 172.21.15.120:0/896679164 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0aaec640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828885790 0x55e828909c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 -- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828885790 msgr2=0x55e828909c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828885790 0x55e828909c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 -- 172.21.15.120:0/896679164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 msgr2=0x55e828889ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 0x55e828889ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe09aea640 1 -- 172.21.15.120:0/896679164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e82890a2d0 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.382+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 0x55e828889ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe09aea640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efdf40045e0 tx=0x7efdf40d5ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313b63819f16bdbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4049040 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efdf4003a10 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc012cf0 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdf40c8310 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 10 client.17442 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e82890b5f0 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc012cf0 con 0x55e828879800 2024-09-10T23:02:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efdf4105070 con 0x55e828879800 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 10 client.17442 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 20 client.17442 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e82890bc30 0x55e82890e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 4 client.17442 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.383+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8287ed860 con 0x55e82890bc30 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.384+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e82890bc30 0x55e82890e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.384+0000 7efdedffb640 10 client.17442 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:32.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.384+0000 7efe0a2eb640 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e82890bc30 0x55e82890e050 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d433d32f5c0fc463 server_cookie=139aa601cda75857 in_seq=0 out_seq=0 2024-09-10T23:02:32.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.386+0000 7efdedffb640 1 -- 172.21.15.120:0/896679164 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8287ed860 con 0x55e82890bc30 2024-09-10T23:02:32.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.386+0000 7efdedffb640 10 client.17442 handle_command_reply: tid=0 2024-09-10T23:02:32.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.386+0000 7efe0e924740 1 client.17442 shutdown 2024-09-10T23:02:32.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 msgr2=0x55e82888a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efdf40045e0 tx=0x7efdf40d5ef0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 shutdown_connections 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e82890bc30 0x55e82890e050 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e828885790 0x55e828909c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e828879800 0x55e82888a3e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 --2- 172.21.15.120:0/896679164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e828878ee0 0x55e828889ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efdedffb640 0 client.17442 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 >> 172.21.15.120:0/896679164 conn(0x55e828876d10 msgr2=0x55e82887c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 shutdown_connections 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 1 -- 172.21.15.120:0/896679164 wait complete. 2024-09-10T23:02:32.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0e924740 20 client.17442 trim_cache size 0 max 16384 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4", 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:02:32.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 msgr2=0x7efdf00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 0x7efdf00b77a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00afd0 tx=0x7efdfc014f10 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 msgr2=0x7efe04199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.387+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efdf4005940 tx=0x7efdf40042f0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 shutdown_connections 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efdf00b52e0 0x7efdf00b77a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe0410d0e0 0x7efe0419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe04104530 0x7efe04199820 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 --2- 172.21.15.120:0/3131466485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe04103b30 0x7efe041992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 >> 172.21.15.120:0/3131466485 conn(0x7efe040fd700 msgr2=0x7efe040fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 shutdown_connections 2024-09-10T23:02:32.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:32.388+0000 7efe0c54f640 1 -- 172.21.15.120:0/3131466485 wait complete. 2024-09-10T23:02:32.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1715544 16.11 MB/sec execute 447 sec latency 68.880 ms 2024-09-10T23:02:33.147 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:02:33.408 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:33.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1718053 16.10 MB/sec execute 448 sec latency 14.226 ms 2024-09-10T23:02:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:33 smithi176 ceph-mon[34982]: pgmap v650: 97 pgs: 97 active+clean; 42 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-09-10T23:02:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:33 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:02:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:33 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:33 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:02:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:33 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:33 smithi120 ceph-mon[31247]: pgmap v650: 97 pgs: 97 active+clean; 42 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-09-10T23:02:33.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:33 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:02:33.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:33 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:33 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:02:33.839 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:33 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 -- 172.21.15.120:0/1216172784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb16010d0e0 msgr2=0x7fb16010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 --2- 172.21.15.120:0/1216172784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb16010d0e0 0x7fb16010f550 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb158009ca0 tx=0x7fb15802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 -- 172.21.15.120:0/1216172784 shutdown_connections 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 --2- 172.21.15.120:0/1216172784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb16010d0e0 0x7fb16010f550 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 --2- 172.21.15.120:0/1216172784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 0x7fb16010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 --2- 172.21.15.120:0/1216172784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb160103b30 0x7fb160103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.921+0000 7fb16819b640 1 -- 172.21.15.120:0/1216172784 >> 172.21.15.120:0/1216172784 conn(0x7fb1600fd700 msgr2=0x7fb1600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- 172.21.15.120:0/1216172784 shutdown_connections 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- 172.21.15.120:0/1216172784 wait complete. 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 Processor -- start 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- start start 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 0x7fb160199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb16010d0e0 0x7fb16019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb160112d60 con 0x7fb160103b30 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb160112be0 con 0x7fb16010d0e0 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.922+0000 7fb16819b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb160112ee0 con 0x7fb160104530 2024-09-10T23:02:33.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46806/0 (socket says 172.21.15.120:46806) 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 -- 172.21.15.120:0/3218439450 learned_addr learned my addr 172.21.15.120:0/3218439450 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165736640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 0x7fb160199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb166738640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb16010d0e0 0x7fb16019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 -- 172.21.15.120:0/3218439450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 msgr2=0x7fb160199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 0x7fb160199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 -- 172.21.15.120:0/3218439450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb16010d0e0 msgr2=0x7fb16019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb16010d0e0 0x7fb16019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 -- 172.21.15.120:0/3218439450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16019f2c0 con 0x7fb160103b30 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb165f37640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d950 tx=0x7fb15c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb156ffd640 1 -- 172.21.15.120:0/3218439450 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c017070 con 0x7fb160103b30 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb156ffd640 1 -- 172.21.15.120:0/3218439450 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb15c009c30 con 0x7fb160103b30 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb156ffd640 1 -- 172.21.15.120:0/3218439450 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00f620 con 0x7fb160103b30 2024-09-10T23:02:33.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16019f5b0 con 0x7fb160103b30 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.923+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1601a5c70 con 0x7fb160103b30 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb156ffd640 1 -- 172.21.15.120:0/3218439450 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb15c010460 con 0x7fb160103b30 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 Processor -- start 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 -- start start 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220da0 0x558b6622c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b66196e50 con 0x558b66220480 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b66186280 con 0x558b66220da0 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.926+0000 7fb16a570740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b6618a090 con 0x558b6622cd30 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46820/0 (socket says 172.21.15.120:46820) 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 -- 172.21.15.120:0/4027662282 learned_addr learned my addr 172.21.15.120:0/4027662282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:33.952 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb165736640 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb165f37640 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220da0 0x558b6622c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 -- 172.21.15.120:0/4027662282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 msgr2=0x558b6622f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 -- 172.21.15.120:0/4027662282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220da0 msgr2=0x558b6622c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb156ffd640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 0x7fb14c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220da0 0x558b6622c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 -- 172.21.15.120:0/4027662282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b65ee4650 con 0x558b66220480 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb156ffd640 1 -- 172.21.15.120:0/3218439450 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb15c0e5790 con 0x7fb160103b30 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb158035000 tx=0x7fb158009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3529a420ad070e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb166738640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 0x7fb14c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.927+0000 7fb14bfff640 1 -- 172.21.15.120:0/4027662282 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158031450 con 0x558b66220480 2024-09-10T23:02:33.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb14bfff640 1 -- 172.21.15.120:0/4027662282 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb158036030 con 0x558b66220480 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb14bfff640 1 -- 172.21.15.120:0/4027662282 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15802f660 con 0x558b66220480 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 -- 172.21.15.120:0/4027662282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 msgr2=0x558b66220860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb158035000 tx=0x7fb158009800 comp rx=0 tx=0).stop 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb166738640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 0x7fb14c0b77a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb158009a60 tx=0x7fb15804b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 -- 172.21.15.120:0/4027662282 shutdown_connections 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220da0 0x558b6622c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 --2- 172.21.15.120:0/4027662282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66220860 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 -- 172.21.15.120:0/4027662282 >> 172.21.15.120:0/4027662282 conn(0x558b6621e2b0 msgr2=0x558b6621f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 -- 172.21.15.120:0/4027662282 shutdown_connections 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.928+0000 7fb16a570740 1 -- 172.21.15.120:0/4027662282 wait complete. 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 -- start start 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 Processor -- start 2024-09-10T23:02:33.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b6622c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b66220da0 0x558b66229a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b65fe7700 con 0x558b6622cd30 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b66121110 con 0x558b66220480 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b6622f830 con 0x558b66220da0 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb16a570740 10 client.0 _ll_register_callbacks cb 0x558b6613a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb166738640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb166738640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46828/0 (socket says 172.21.15.120:46828) 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb166738640 1 -- 172.21.15.120:0/2563467599 learned_addr learned my addr 172.21.15.120:0/2563467599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb165736640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b66220da0 0x558b66229a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.929+0000 7fb165f37640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b6622c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 -- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b66220da0 msgr2=0x558b66229a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b66220da0 0x558b66229a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 -- 172.21.15.120:0/2563467599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 msgr2=0x558b6622c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b6622c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 -- 172.21.15.120:0/2563467599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6614ec40 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb165f37640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b6622c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb166738640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb158031fb0 tx=0x7fb158004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abd234683baa98d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb1497fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158004670 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb158004b40 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb158036030 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb158004d40 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb16a570740 10 client.17454 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b6623b1c0 con 0x558b6622cd30 2024-09-10T23:02:33.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.930+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb158036030 con 0x558b6622cd30 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb158061030 con 0x558b6622cd30 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 10 client.17454 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 20 client.17454 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b6623c510 0x558b6623e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 4 client.17454 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b66198da0 con 0x558b6623c510 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb165f37640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b6623c510 0x558b6623e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb165f37640 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b6623c510 0x558b6623e900 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa26bab300be4c77 server_cookie=f03a042ba0eb3f9f in_seq=0 out_seq=0 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.931+0000 7fb1497fa640 10 client.17454 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb1497fa640 1 -- 172.21.15.120:0/2563467599 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b66198da0 con 0x558b6623c510 2024-09-10T23:02:33.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb1497fa640 10 client.17454 handle_command_reply: tid=0 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 client.17454 shutdown 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 msgr2=0x558b6622a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb158031fb0 tx=0x7fb158004400 comp rx=0 tx=0).stop 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 shutdown_connections 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b6623c510 0x558b6623e900 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b6622cd30 0x558b6622a0c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b66220da0 0x558b66229a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 --2- 172.21.15.120:0/2563467599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b6622c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb1497fa640 0 client.17454 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.932+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 >> 172.21.15.120:0/2563467599 conn(0x558b6621e2b0 msgr2=0x558b66223e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.933+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 shutdown_connections 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.933+0000 7fb16a570740 1 -- 172.21.15.120:0/2563467599 wait complete. 2024-09-10T23:02:33.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.933+0000 7fb16a570740 20 client.17454 trim_cache size 0 max 16384 2024-09-10T23:02:33.959 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:33 smithi153 ceph-mon[34951]: pgmap v650: 97 pgs: 97 active+clean; 42 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-09-10T23:02:33.959 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:33 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:02:33.959 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:33 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.959 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:33 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:02:33.959 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:33 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:33.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 Processor -- start 2024-09-10T23:02:33.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 -- start start 2024-09-10T23:02:33.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b66238a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601ac270 con 0x558b66220da0 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601aeaf0 con 0x558b66220480 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb16a570740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601ac8e0 con 0x558b6622cd30 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46830/0 (socket says 172.21.15.120:46830) 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 -- 172.21.15.120:0/1219153509 learned_addr learned my addr 172.21.15.120:0/1219153509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb166738640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165f37640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b66238a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 -- 172.21.15.120:0/1219153509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 msgr2=0x558b6622e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 -- 172.21.15.120:0/1219153509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 msgr2=0x558b66238a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b66238a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 -- 172.21.15.120:0/1219153509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b66227f90 con 0x558b66220da0 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165f37640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b66238a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.951+0000 7fb165736640 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb15000bd10 tx=0x7fb150011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2fb3dfced86cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.952+0000 7fb14b7fe640 1 -- 172.21.15.120:0/1219153509 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150004020 con 0x558b66220da0 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.952+0000 7fb14b7fe640 1 -- 172.21.15.120:0/1219153509 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1500054d0 con 0x558b66220da0 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.952+0000 7fb14b7fe640 1 -- 172.21.15.120:0/1219153509 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150004b20 con 0x558b66220da0 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.952+0000 7fb16a570740 1 -- 172.21.15.120:0/1219153509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 msgr2=0x558b6622e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.952+0000 7fb16a570740 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb15000bd10 tx=0x7fb150011810 comp rx=0 tx=0).stop 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- 172.21.15.120:0/1219153509 shutdown_connections 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622cd30 0x558b6622e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220da0 0x558b6622e120 secure :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb15000bd10 tx=0x7fb150011810 comp rx=0 tx=0).stop 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- 172.21.15.120:0/1219153509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b66220480 0x558b66238a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- 172.21.15.120:0/1219153509 >> 172.21.15.120:0/1219153509 conn(0x558b6621e2b0 msgr2=0x558b66224ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- 172.21.15.120:0/1219153509 shutdown_connections 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- 172.21.15.120:0/1219153509 wait complete. 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- start start 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 Processor -- start 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b6622cd30 0x558b66235220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 0x558b6622ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601acfb0 con 0x558b66220480 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601ae420 con 0x558b6622cd30 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.953+0000 7fb16a570740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601ad680 con 0x558b6622fab0 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb16a570740 10 client.0 _ll_register_callbacks cb 0x7fb158049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb165f37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb165f37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46842/0 (socket says 172.21.15.120:46842) 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb165f37640 1 -- 172.21.15.120:0/467174384 learned_addr learned my addr 172.21.15.120:0/467174384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:33.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 0x558b6622ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb165736640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b6622cd30 0x558b66235220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 -- 172.21.15.120:0/467174384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b6622cd30 msgr2=0x558b66235220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b6622cd30 0x558b66235220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 -- 172.21.15.120:0/467174384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 msgr2=0x558b66234ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 -- 172.21.15.120:0/467174384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b662305d0 con 0x558b6622fab0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb165f37640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb166738640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 0x558b6622ff30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb15802ade0 tx=0x7fb158000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c76b36f584eda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb1497fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158009540 con 0x558b6622fab0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1500054d0 con 0x558b6622fab0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb158004b40 con 0x558b6622fab0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb158075070 con 0x558b6622fab0 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb16a570740 10 client.26938 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:33.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.954+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b662aa020 con 0x558b6622fab0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1500054d0 con 0x558b6622fab0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb15806c020 con 0x558b6622fab0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 10 client.26938 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 20 client.26938 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b662aa6e0 0x558b662acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 4 client.26938 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b66194850 con 0x558b662aa6e0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb165f37640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b662aa6e0 0x558b662acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb165f37640 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b662aa6e0 0x558b662acb00 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cd4a733c71574c server_cookie=87d248b55dfffa2e in_seq=0 out_seq=0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.955+0000 7fb1497fa640 10 client.26938 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.956+0000 7fb1497fa640 1 -- 172.21.15.120:0/467174384 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x558b66194850 con 0x558b662aa6e0 2024-09-10T23:02:33.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.956+0000 7fb1497fa640 10 client.26938 handle_command_reply: tid=0 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.956+0000 7fb16a570740 1 client.26938 shutdown 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 msgr2=0x558b6622ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 0x558b6622ff30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb15802ade0 tx=0x7fb158000f40 comp rx=0 tx=0).stop 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 shutdown_connections 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b662aa6e0 0x558b662acb00 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b6622fab0 0x558b6622ff30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b6622cd30 0x558b66235220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 --2- 172.21.15.120:0/467174384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b66220480 0x558b66234ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb1497fa640 0 client.26938 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 >> 172.21.15.120:0/467174384 conn(0x558b6621e2b0 msgr2=0x558b6621cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 shutdown_connections 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 1 -- 172.21.15.120:0/467174384 wait complete. 2024-09-10T23:02:33.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16a570740 20 client.26938 trim_cache size 0 max 16384 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4", 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:02:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.972 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.973 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.974 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:02:33.975 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.976 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:02:33.977 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:02:33.977 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:02:33.980 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:02:33.983 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:02:33.984 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 msgr2=0x7fb14c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 0x7fb14c0b77a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb158009a60 tx=0x7fb15804b000 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 msgr2=0x7fb160199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d950 tx=0x7fb15c00de10 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 shutdown_connections 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb14c0b52e0 0x7fb14c0b77a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb16010d0e0 0x7fb16019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb160104530 0x7fb160199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 --2- 172.21.15.120:0/3218439450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb160103b30 0x7fb160199210 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.957+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 >> 172.21.15.120:0/3218439450 conn(0x7fb1600fd700 msgr2=0x7fb1600fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.958+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 shutdown_connections 2024-09-10T23:02:33.985 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:02:33.958+0000 7fb16819b640 1 -- 172.21.15.120:0/3218439450 wait complete. 2024-09-10T23:02:34.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1720254 16.09 MB/sec execute 449 sec latency 111.336 ms 2024-09-10T23:02:34.899 INFO:tasks.cephfs.filesystem:scrub status for tag:0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4 - {'path': '/', 'tag': '0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4', 'options': 'recursive,force'} 2024-09-10T23:02:35.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1723309 16.09 MB/sec execute 450 sec latency 32.062 ms 2024-09-10T23:02:35.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:35 smithi120 ceph-mon[31247]: pgmap v651: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-09-10T23:02:35.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:02:35.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:35 smithi153 ceph-mon[34951]: pgmap v651: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-09-10T23:02:35.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:02:36.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:35 smithi176 ceph-mon[34982]: pgmap v651: 97 pgs: 97 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-09-10T23:02:36.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:02:36.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1726582 16.10 MB/sec execute 451 sec latency 21.778 ms 2024-09-10T23:02:37.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1729977 16.11 MB/sec execute 452 sec latency 12.986 ms 2024-09-10T23:02:37.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:37 smithi120 ceph-mon[31247]: pgmap v652: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 454 op/s 2024-09-10T23:02:37.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:37 smithi153 ceph-mon[34951]: pgmap v652: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 454 op/s 2024-09-10T23:02:38.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:37 smithi176 ceph-mon[34982]: pgmap v652: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 454 op/s 2024-09-10T23:02:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- 172.21.15.120:0/333002768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 msgr2=0x7f54fc105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 --2- 172.21.15.120:0/333002768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc105590 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ca0 tx=0x7f54ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- 172.21.15.120:0/333002768 shutdown_connections 2024-09-10T23:02:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 --2- 172.21.15.120:0/333002768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 0x7f54fc10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 --2- 172.21.15.120:0/333002768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 0x7f54fc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 --2- 172.21.15.120:0/333002768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc105590 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- 172.21.15.120:0/333002768 >> 172.21.15.120:0/333002768 conn(0x7f54fc0fd140 msgr2=0x7f54fc0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- 172.21.15.120:0/333002768 shutdown_connections 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- 172.21.15.120:0/333002768 wait complete. 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 Processor -- start 2024-09-10T23:02:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.185+0000 7f55037f3640 1 -- start start 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 0x7f54fc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 0x7f54fc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10e600 con 0x7f54fc06b410 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10e480 con 0x7f54fc105ad0 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10e780 con 0x7f54fc108680 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46854/0 (socket says 172.21.15.120:46854) 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 -- 172.21.15.120:0/12949342 learned_addr learned my addr 172.21.15.120:0/12949342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f5500d8e640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 0x7f54fc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f5501d90640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 0x7f54fc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 -- 172.21.15.120:0/12949342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 msgr2=0x7f54fc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 0x7f54fc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 -- 172.21.15.120:0/12949342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 msgr2=0x7f54fc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 0x7f54fc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 -- 172.21.15.120:0/12949342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc19a840 con 0x7f54fc06b410 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f550158f640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54ec006010 tx=0x7f54ec003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f54fa7fc640 1 -- 172.21.15.120:0/12949342 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02f380 con 0x7f54fc06b410 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f54fa7fc640 1 -- 172.21.15.120:0/12949342 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec0317c0 con 0x7f54fc06b410 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.186+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc19aad0 con 0x7f54fc06b410 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.187+0000 7f54fa7fc640 1 -- 172.21.15.120:0/12949342 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02fb90 con 0x7f54fc06b410 2024-09-10T23:02:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.187+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1a1200 con 0x7f54fc06b410 2024-09-10T23:02:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f54fa7fc640 1 -- 172.21.15.120:0/12949342 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f54ec02fd30 con 0x7f54fc06b410 2024-09-10T23:02:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 Processor -- start 2024-09-10T23:02:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 -- start start 2024-09-10T23:02:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc27c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 0x55bbecc2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bbecb976f0 con 0x55bbecc28190 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bbecb86b20 con 0x55bbecc2abd0 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f55065c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bbecb8a930 con 0x55bbecc25860 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.189+0000 7f550158f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 0x55bbecc2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc27c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f550158f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 0x55bbecc2a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46856/0 (socket says 172.21.15.120:46856) 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f550158f640 1 -- 172.21.15.120:0/1128194304 learned_addr learned my addr 172.21.15.120:0/1128194304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5500d8e640 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 -- 172.21.15.120:0/1128194304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 msgr2=0x55bbecc2d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 -- 172.21.15.120:0/1128194304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 msgr2=0x55bbecc2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 0x55bbecc2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 -- 172.21.15.120:0/1128194304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbecb4c530 con 0x55bbecc25860 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5500d8e640 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f54fa7fc640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 0x7f54e40b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f5501d90640 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc27c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009b50 tx=0x7f54f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58150ce0bbae3cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f550158f640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 0x7f54e40b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f54e37fe640 1 -- 172.21.15.120:0/1128194304 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4004030 con 0x55bbecc25860 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f54e37fe640 1 -- 172.21.15.120:0/1128194304 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f4005020 con 0x55bbecc25860 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f54fa7fc640 1 -- 172.21.15.120:0/12949342 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f54ec106eb0 con 0x7f54fc06b410 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.190+0000 7f54e37fe640 1 -- 172.21.15.120:0/1128194304 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4004c60 con 0x55bbecc25860 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- 172.21.15.120:0/1128194304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 msgr2=0x55bbecc27c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f550158f640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 0x7f54e40b7940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000f80 tx=0x7f54ec10a990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc27c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009b50 tx=0x7f54f4011810 comp rx=0 tx=0).stop 2024-09-10T23:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- 172.21.15.120:0/1128194304 shutdown_connections 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc28190 0x55bbecc2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 --2- 172.21.15.120:0/1128194304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc27c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- 172.21.15.120:0/1128194304 >> 172.21.15.120:0/1128194304 conn(0x55bbecc209b0 msgr2=0x55bbecc21740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- 172.21.15.120:0/1128194304 shutdown_connections 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- 172.21.15.120:0/1128194304 wait complete. 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.191+0000 7f55065c4740 1 -- start start 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 Processor -- start 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc2f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc30120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bbecabeab0 con 0x55bbecc25860 2024-09-10T23:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bbecb2c730 con 0x55bbecc2abd0 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bbecc2dbb0 con 0x55bbecc28190 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f55065c4740 10 client.0 _ll_register_callbacks cb 0x55bbecb806b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5500d8e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc2f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46868/0 (socket says 172.21.15.120:46868) 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 -- 172.21.15.120:0/1320093542 learned_addr learned my addr 172.21.15.120:0/1320093542 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc30120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 -- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 msgr2=0x55bbecc2f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc2f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 -- 172.21.15.120:0/1320093542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 msgr2=0x55bbecc30120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.192+0000 7f5501d90640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc30120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f5501d90640 1 -- 172.21.15.120:0/1320093542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbec8e8790 con 0x55bbecc25860 2024-09-10T23:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f5500d8e640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc2f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f5501d90640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005740 tx=0x7f54f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673185692b96fd72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402d060 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f54f40049e0 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4005020 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54f401e040 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f55065c4740 10 client.17472 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbec961620 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f4005020 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f54f4018070 con 0x55bbecc25860 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f55065c4740 10 client.17472 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.193+0000 7f55065c4740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f55065c4740 20 client.17472 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecc3a460 0x55bbecc3c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f55065c4740 4 client.17472 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbecb99640 con 0x55bbecc3a460 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f5500d8e640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecc3a460 0x55bbecc3c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f54e0ff9640 10 client.17472 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.194+0000 7f5500d8e640 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecc3a460 0x55bbecc3c850 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93313292b91c3a05 server_cookie=bc9fb6f6c0ce1712 in_seq=0 out_seq=0 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f54e0ff9640 1 -- 172.21.15.120:0/1320093542 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbecb99640 con 0x55bbecc3a460 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f54e0ff9640 10 client.17472 handle_command_reply: tid=0 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 client.17472 shutdown 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 msgr2=0x55bbecc2ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005740 tx=0x7f54f4004730 comp rx=0 tx=0).stop 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 shutdown_connections 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecc3a460 0x55bbecc3c850 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc30120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc2f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 --2- 172.21.15.120:0/1320093542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc2ef20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f54e0ff9640 0 client.17472 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 >> 172.21.15.120:0/1320093542 conn(0x55bbecc209b0 msgr2=0x55bbecc1fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 shutdown_connections 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 1 -- 172.21.15.120:0/1320093542 wait complete. 2024-09-10T23:02:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.195+0000 7f55065c4740 20 client.17472 trim_cache size 0 max 16384 2024-09-10T23:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.211+0000 7f55065c4740 1 Processor -- start 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.211+0000 7f55065c4740 1 -- start start 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1a7d50 con 0x55bbecc25860 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1aa5d0 con 0x55bbecc2abd0 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f55065c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1a83c0 con 0x55bbecc28190 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46882/0 (socket says 172.21.15.120:46882) 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 -- 172.21.15.120:0/2511080570 learned_addr learned my addr 172.21.15.120:0/2511080570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5500d8e640 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 -- 172.21.15.120:0/2511080570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 msgr2=0x55bbecc25290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc25290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 -- 172.21.15.120:0/2511080570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 msgr2=0x55bbecc2afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 -- 172.21.15.120:0/2511080570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbecc27fc0 con 0x55bbecc25860 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f5501d90640 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f54f4006010 tx=0x7f54f4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b22db4fd076a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f54e2ffd640 1 -- 172.21.15.120:0/2511080570 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402d070 con 0x55bbecc25860 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f54e2ffd640 1 -- 172.21.15.120:0/2511080570 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f4028910 con 0x55bbecc25860 2024-09-10T23:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.212+0000 7f54e2ffd640 1 -- 172.21.15.120:0/2511080570 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4038040 con 0x55bbecc25860 2024-09-10T23:02:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- 172.21.15.120:0/2511080570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 msgr2=0x55bbecc24d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f54f4006010 tx=0x7f54f4014040 comp rx=0 tx=0).stop 2024-09-10T23:02:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- 172.21.15.120:0/2511080570 shutdown_connections 2024-09-10T23:02:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc2abd0 0x55bbecc2afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc28190 0x55bbecc25290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 --2- 172.21.15.120:0/2511080570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc25860 0x55bbecc24d50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- 172.21.15.120:0/2511080570 >> 172.21.15.120:0/2511080570 conn(0x55bbecc209b0 msgr2=0x55bbecc2eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- 172.21.15.120:0/2511080570 shutdown_connections 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- 172.21.15.120:0/2511080570 wait complete. 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.213+0000 7f55065c4740 1 -- start start 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 Processor -- start 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc33bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 0x55bbecc34270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1a8a90 con 0x55bbecc2abd0 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1a9f00 con 0x55bbecc28190 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1a9160 con 0x55bbecc25860 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f55065c4740 10 client.0 _ll_register_callbacks cb 0x7f54ec10a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f5501d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:46896/0 (socket says 172.21.15.120:46896) 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f5500d8e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 0x55bbecc34270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f5501d90640 1 -- 172.21.15.120:0/2872980079 learned_addr learned my addr 172.21.15.120:0/2872980079 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.214+0000 7f5501d90640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc33bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 -- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 msgr2=0x55bbecc33bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc33bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 -- 172.21.15.120:0/2872980079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 msgr2=0x55bbecc34270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 0x55bbecc34270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 -- 172.21.15.120:0/2872980079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbecc35670 con 0x55bbecc2abd0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5500d8e640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 0x55bbecc34270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f5501d90640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009b50 tx=0x7f54f4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ea230ddda161cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402d070 con 0x55bbecc2abd0 2024-09-10T23:02:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f54f40036f0 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4028910 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54f4038040 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 10 client.17478 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbecc369e0 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f4028910 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f54f4018040 con 0x55bbecc2abd0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 10 client.17478 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 20 client.17478 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecca74c0 0x55bbecca98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 4 client.17478 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.215+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbecb950f0 con 0x55bbecca74c0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.216+0000 7f5500d8e640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecca74c0 0x55bbecca98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.216+0000 7f5500d8e640 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecca74c0 0x55bbecca98b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2136a01691bfa1 server_cookie=2e46330ccc6b1b86 in_seq=0 out_seq=0 2024-09-10T23:02:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.216+0000 7f54e0ff9640 10 client.17478 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.433+0000 7f54e0ff9640 1 -- 172.21.15.120:0/2872980079 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbecb950f0 con 0x55bbecca74c0 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.433+0000 7f54e0ff9640 10 client.17478 handle_command_reply: tid=0 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.433+0000 7f55065c4740 1 client.17478 shutdown 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 msgr2=0x55bbecc34e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009b50 tx=0x7f54f4011e40 comp rx=0 tx=0).stop 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 shutdown_connections 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bbecca74c0 0x55bbecca98b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bbecc2abd0 0x55bbecc34e40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bbecc28190 0x55bbecc34270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f54e0ff9640 0 client.17478 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 --2- 172.21.15.120:0/2872980079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bbecc25860 0x55bbecc33bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 >> 172.21.15.120:0/2872980079 conn(0x55bbecc209b0 msgr2=0x55bbecc2afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 shutdown_connections 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 1 -- 172.21.15.120:0/2872980079 wait complete. 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55065c4740 20 client.17478 trim_cache size 0 max 16384 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:02:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:02:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 msgr2=0x7f54e40b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 0x7f54e40b7940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000f80 tx=0x7f54ec10a990 comp rx=0 tx=0).stop 2024-09-10T23:02:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 msgr2=0x7f54fc198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.434+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54ec006010 tx=0x7f54ec003ba0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 shutdown_connections 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f54e40b5480 0x7f54e40b7940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54fc108680 0x7f54fc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54fc105ad0 0x7f54fc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 --2- 172.21.15.120:0/12949342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f54fc06b410 0x7f54fc198d00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 >> 172.21.15.120:0/12949342 conn(0x7f54fc0fd140 msgr2=0x7f54fc103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 shutdown_connections 2024-09-10T23:02:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:38.435+0000 7f55037f3640 1 -- 172.21.15.120:0/12949342 wait complete. 2024-09-10T23:02:38.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1732230 16.09 MB/sec execute 453 sec latency 138.164 ms 2024-09-10T23:02:39.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1734691 16.09 MB/sec execute 454 sec latency 73.214 ms 2024-09-10T23:02:39.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:39 smithi120 ceph-mon[31247]: pgmap v653: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 273 op/s 2024-09-10T23:02:39.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:39 smithi153 ceph-mon[34951]: pgmap v653: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 273 op/s 2024-09-10T23:02:40.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:39 smithi176 ceph-mon[34982]: pgmap v653: 97 pgs: 97 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 273 op/s 2024-09-10T23:02:40.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1737629 16.09 MB/sec execute 455 sec latency 27.117 ms 2024-09-10T23:02:41.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1740771 16.09 MB/sec execute 456 sec latency 78.729 ms 2024-09-10T23:02:41.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:41 smithi120 ceph-mon[31247]: pgmap v654: 97 pgs: 97 active+clean; 45 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T23:02:41.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:41 smithi153 ceph-mon[34951]: pgmap v654: 97 pgs: 97 active+clean; 45 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T23:02:42.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:41 smithi176 ceph-mon[34982]: pgmap v654: 97 pgs: 97 active+clean; 45 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T23:02:42.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1744170 16.10 MB/sec execute 457 sec latency 13.394 ms 2024-09-10T23:02:43.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1747123 16.09 MB/sec execute 458 sec latency 15.807 ms 2024-09-10T23:02:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:43 smithi120 ceph-mon[31247]: pgmap v655: 97 pgs: 97 active+clean; 49 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T23:02:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:43 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:43.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:43 smithi153 ceph-mon[34951]: pgmap v655: 97 pgs: 97 active+clean; 49 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T23:02:43.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:43 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:44.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:43 smithi176 ceph-mon[34982]: pgmap v655: 97 pgs: 97 active+clean; 49 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 427 op/s 2024-09-10T23:02:44.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:43 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:44.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1749112 16.08 MB/sec execute 459 sec latency 66.078 ms 2024-09-10T23:02:45.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1751761 16.08 MB/sec execute 460 sec latency 23.641 ms 2024-09-10T23:02:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: pgmap v656: 97 pgs: 97 active+clean; 50 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: pgmap v656: 97 pgs: 97 active+clean; 50 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: pgmap v656: 97 pgs: 97 active+clean; 50 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:02:46.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1755137 16.08 MB/sec execute 461 sec latency 12.507 ms 2024-09-10T23:02:47.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1758380 16.09 MB/sec execute 462 sec latency 14.445 ms 2024-09-10T23:02:47.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:47 smithi120 ceph-mon[31247]: pgmap v657: 97 pgs: 97 active+clean; 48 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-09-10T23:02:47.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:47 smithi153 ceph-mon[34951]: pgmap v657: 97 pgs: 97 active+clean; 48 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-09-10T23:02:47.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:47 smithi176 ceph-mon[34982]: pgmap v657: 97 pgs: 97 active+clean; 48 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-09-10T23:02:48.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1761510 16.09 MB/sec execute 463 sec latency 12.118 ms 2024-09-10T23:02:49.638 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1763623 16.07 MB/sec execute 464 sec latency 48.975 ms 2024-09-10T23:02:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:49 smithi120 ceph-mon[31247]: pgmap v658: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-10T23:02:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:49 smithi153 ceph-mon[34951]: pgmap v658: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-10T23:02:49.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:49 smithi176 ceph-mon[34982]: pgmap v658: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-10T23:02:50.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1766057 16.07 MB/sec execute 465 sec latency 77.612 ms 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 -- 172.21.15.120:0/4264040234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44006b470 msgr2=0x7fa440105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 --2- 172.21.15.120:0/4264040234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44006b470 0x7fa440105530 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa430009900 tx=0x7fa43002a410 comp rx=0 tx=0).stop 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 -- 172.21.15.120:0/4264040234 shutdown_connections 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 --2- 172.21.15.120:0/4264040234 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa440108670 0x7fa44010ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 --2- 172.21.15.120:0/4264040234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa440107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 --2- 172.21.15.120:0/4264040234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa44006b470 0x7fa440105530 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 -- 172.21.15.120:0/4264040234 >> 172.21.15.120:0/4264040234 conn(0x7fa4400fd140 msgr2=0x7fa4400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 -- 172.21.15.120:0/4264040234 shutdown_connections 2024-09-10T23:02:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.529+0000 7fa444a7f640 1 -- 172.21.15.120:0/4264040234 wait complete. 2024-09-10T23:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.530+0000 7fa444a7f640 1 Processor -- start 2024-09-10T23:02:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 -- start start 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa44006b470 0x7fa440102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa4401014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010e5c0 con 0x7fa440108670 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010e440 con 0x7fa440105ac0 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa444a7f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010e740 con 0x7fa44006b470 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa44006b470 0x7fa440102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56574/0 (socket says 172.21.15.120:56574) 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1282775542 learned_addr learned my addr 172.21.15.120:0/1282775542 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa437fff640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa4401014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1282775542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa44006b470 msgr2=0x7fa440102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa44006b470 0x7fa440102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1282775542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 msgr2=0x7fa4401014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa4401014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1282775542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa440102280 con 0x7fa440108670 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa437fff640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa4401014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.531+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa43800eb10 tx=0x7fa43800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.532+0000 7fa4377fe640 1 -- 172.21.15.120:0/1282775542 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438004b20 con 0x7fa440108670 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.532+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401a2e50 con 0x7fa440108670 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.532+0000 7fa4377fe640 1 -- 172.21.15.120:0/1282775542 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa438004cc0 con 0x7fa440108670 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.532+0000 7fa4377fe640 1 -- 172.21.15.120:0/1282775542 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438018630 con 0x7fa440108670 2024-09-10T23:02:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.532+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4401a3360 con 0x7fa440108670 2024-09-10T23:02:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.534+0000 7fa447857740 1 Processor -- start 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa4377fe640 1 -- 172.21.15.120:0/1282775542 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa438005000 con 0x7fa440108670 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 -- start start 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f407580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f40a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55868f3746f0 con 0x55868f402860 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55868f363b20 con 0x55868f405190 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa447857740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55868f367930 con 0x55868f407bd0 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa437fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa437fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56578/0 (socket says 172.21.15.120:56578) 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa437fff640 1 -- 172.21.15.120:0/1853625217 learned_addr learned my addr 172.21.15.120:0/1853625217 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f407580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.535+0000 7fa437fff640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f40a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 -- 172.21.15.120:0/1853625217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 msgr2=0x55868f40a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f40a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 -- 172.21.15.120:0/1853625217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 msgr2=0x55868f407580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f407580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 -- 172.21.15.120:0/1853625217 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868f329530 con 0x55868f402860 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f407580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa4377fe640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 0x7fa42c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa437fff640 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa428013aa0 tx=0x7fa428009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7714006da828e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 0x7fa42c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa4377fe640 1 -- 172.21.15.120:0/1282775542 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa4380e6230 con 0x7fa440108670 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa434ff9640 1 -- 172.21.15.120:0/1853625217 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800fa40 con 0x55868f402860 2024-09-10T23:02:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa434ff9640 1 -- 172.21.15.120:0/1853625217 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42800fbe0 con 0x55868f402860 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.536+0000 7fa434ff9640 1 -- 172.21.15.120:0/1853625217 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800e3b0 con 0x55868f402860 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- 172.21.15.120:0/1853625217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 msgr2=0x55868f404c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 0x7fa42c0b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa430000c00 tx=0x7fa4300314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa428013aa0 tx=0x7fa428009b90 comp rx=0 tx=0).stop 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- 172.21.15.120:0/1853625217 shutdown_connections 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f40a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f407580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 --2- 172.21.15.120:0/1853625217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f402860 0x55868f404c50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- 172.21.15.120:0/1853625217 >> 172.21.15.120:0/1853625217 conn(0x55868f3fd9b0 msgr2=0x55868f3fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- 172.21.15.120:0/1853625217 shutdown_connections 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- 172.21.15.120:0/1853625217 wait complete. 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.537+0000 7fa447857740 1 -- start start 2024-09-10T23:02:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 Processor -- start 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f402860 0x55868f40be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f40c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55868f29bab0 con 0x55868f407bd0 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55868f309730 con 0x55868f405190 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55868f40abb0 con 0x55868f402860 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa447857740 10 client.0 _ll_register_callbacks cb 0x55868f35d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f402860 0x55868f40be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa437fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f40c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56588/0 (socket says 172.21.15.120:56588) 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1640828185 learned_addr learned my addr 172.21.15.120:0/1640828185 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f402860 msgr2=0x55868f40be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f402860 0x55868f40be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1640828185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 msgr2=0x55868f40c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f40c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 -- 172.21.15.120:0/1640828185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868f0c5790 con 0x55868f407bd0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa437fff640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f40c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ac9b0 tx=0x7fa4380ac7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0db88e9f68f841 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.538+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380a8190 con 0x55868f407bd0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa4380b5b20 con 0x55868f407bd0 2024-09-10T23:02:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42800fbe0 con 0x55868f407bd0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa4380ac530 con 0x55868f407bd0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 10 client.17490 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55868f13e620 con 0x55868f407bd0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42800fbe0 con 0x55868f407bd0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa4380a8370 con 0x55868f407bd0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 10 client.17490 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 20 client.17490 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f419c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 4 client.17490 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868f376640 con 0x55868f417840 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.539+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f419c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.540+0000 7fa425ffb640 10 client.17490 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.540+0000 7fa43e59c640 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f419c30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c44f29cd415cd37 server_cookie=5c7bc0260f1a628 in_seq=0 out_seq=0 2024-09-10T23:02:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.540+0000 7fa425ffb640 1 -- 172.21.15.120:0/1640828185 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55868f376640 con 0x55868f417840 2024-09-10T23:02:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.540+0000 7fa425ffb640 10 client.17490 handle_command_reply: tid=0 2024-09-10T23:02:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.540+0000 7fa447857740 1 client.17490 shutdown 2024-09-10T23:02:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 msgr2=0x55868f40d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ac9b0 tx=0x7fa4380ac7f0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 shutdown_connections 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f419c30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f40d100 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f405190 0x55868f40c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa425ffb640 0 client.17490 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 --2- 172.21.15.120:0/1640828185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f402860 0x55868f40be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 >> 172.21.15.120:0/1640828185 conn(0x55868f3fd9b0 msgr2=0x55868f3fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 shutdown_connections 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 1 -- 172.21.15.120:0/1640828185 wait complete. 2024-09-10T23:02:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.541+0000 7fa447857740 20 client.17490 trim_cache size 0 max 16384 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 Processor -- start 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 -- start start 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f40d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f405190 0x55868f400a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401a9ff0 con 0x55868f407bd0 2024-09-10T23:02:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401ac870 con 0x55868f402860 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa447857740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401aa660 con 0x55868f405190 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56600/0 (socket says 172.21.15.120:56600) 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 -- 172.21.15.120:0/2323487216 learned_addr learned my addr 172.21.15.120:0/2323487216 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa437fff640 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f405190 0x55868f400a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43e59c640 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f40d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 -- 172.21.15.120:0/2323487216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f405190 msgr2=0x55868f400a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f405190 0x55868f400a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 -- 172.21.15.120:0/2323487216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 msgr2=0x55868f40d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f40d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.556+0000 7fa43ed9d640 1 -- 172.21.15.120:0/2323487216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868f40c8a0 con 0x55868f407bd0 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.557+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa4380e40d0 tx=0x7fa43800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869141953a93690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.557+0000 7fa427fff640 1 -- 172.21.15.120:0/2323487216 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380a8190 con 0x55868f407bd0 2024-09-10T23:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.557+0000 7fa427fff640 1 -- 172.21.15.120:0/2323487216 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4380b5fb0 con 0x55868f407bd0 2024-09-10T23:02:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.557+0000 7fa427fff640 1 -- 172.21.15.120:0/2323487216 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380ac370 con 0x55868f407bd0 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.557+0000 7fa447857740 1 -- 172.21.15.120:0/2323487216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 msgr2=0x55868f400f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa4380e40d0 tx=0x7fa43800cee0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- 172.21.15.120:0/2323487216 shutdown_connections 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f407bd0 0x55868f400f40 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f405190 0x55868f400a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- 172.21.15.120:0/2323487216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f40d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- 172.21.15.120:0/2323487216 >> 172.21.15.120:0/2323487216 conn(0x55868f3fd9b0 msgr2=0x55868f401c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- 172.21.15.120:0/2323487216 shutdown_connections 2024-09-10T23:02:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- 172.21.15.120:0/2323487216 wait complete. 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- start start 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 Processor -- start 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f4135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f4124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401aad30 con 0x55868f405190 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401ac1a0 con 0x55868f402860 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401ab400 con 0x55868f407bd0 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa447857740 10 client.0 _ll_register_callbacks cb 0x7fa4380a9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa437fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa437fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56602/0 (socket says 172.21.15.120:56602) 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa437fff640 1 -- 172.21.15.120:0/4044240544 learned_addr learned my addr 172.21.15.120:0/4044240544 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:02:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.558+0000 7fa43ed9d640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f4124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa43e59c640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f4135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 -- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 msgr2=0x55868f4124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f4124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 -- 172.21.15.120:0/4044240544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 msgr2=0x55868f4135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f4135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 -- 172.21.15.120:0/4044240544 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868f404fc0 con 0x55868f405190 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa43e59c640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f4135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa437fff640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa428013fd0 tx=0x7fa428005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4aba31d8e190c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428005460 con 0x55868f405190 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55868f3720f0 con 0x55868f405190 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa428010430 con 0x55868f405190 2024-09-10T23:02:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa428005600 con 0x55868f405190 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa447857740 10 client.17496 fetch_fsmap learned FSMap version 26 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55868f4078f0 con 0x55868f405190 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55868f3720f0 con 0x55868f405190 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa428026070 con 0x55868f405190 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa447857740 10 client.17496 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.559+0000 7fa447857740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa447857740 20 client.17496 populate_metadata read hostname 'smithi120' 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f486860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa447857740 4 client.17496 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55868f36f520 con 0x55868f417840 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa43e59c640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f486860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa43e59c640 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f486860 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef21d0f96dd6116 server_cookie=32c7a47bd6e1b73b in_seq=0 out_seq=0 2024-09-10T23:02:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.560+0000 7fa425ffb640 10 client.17496 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:51.639 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1769159 16.08 MB/sec execute 466 sec latency 44.603 ms 2024-09-10T23:02:51.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:51 smithi120 ceph-mon[31247]: pgmap v659: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-10T23:02:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:51 smithi153 ceph-mon[34951]: pgmap v659: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-10T23:02:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa425ffb640 1 -- 172.21.15.120:0/4044240544 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55868f36f520 con 0x55868f417840 2024-09-10T23:02:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa425ffb640 10 client.17496 handle_command_reply: tid=0 2024-09-10T23:02:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 client.17496 shutdown 2024-09-10T23:02:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 msgr2=0x55868f413c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa428013fd0 tx=0x7fa428005bb0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 shutdown_connections 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55868f417840 0x55868f486860 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa425ffb640 0 client.17496 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55868f407bd0 0x55868f4124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55868f405190 0x55868f413c90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 --2- 172.21.15.120:0/4044240544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55868f402860 0x55868f4135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.792+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 >> 172.21.15.120:0/4044240544 conn(0x55868f3fd9b0 msgr2=0x55868f40afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 shutdown_connections 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa447857740 1 -- 172.21.15.120:0/4044240544 wait complete. 2024-09-10T23:02:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa447857740 20 client.17496 trim_cache size 0 max 16384 2024-09-10T23:02:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:02:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:02:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:02:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 msgr2=0x7fa42c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 0x7fa42c0b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa430000c00 tx=0x7fa4300314a0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 msgr2=0x7fa440101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa43800eb10 tx=0x7fa43800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 shutdown_connections 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa42c0b52e0 0x7fa42c0b77a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa440108670 0x7fa440101a10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa440105ac0 0x7fa4401014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 --2- 172.21.15.120:0/1282775542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa44006b470 0x7fa440102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.793+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 >> 172.21.15.120:0/1282775542 conn(0x7fa4400fd140 msgr2=0x7fa440103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.794+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 shutdown_connections 2024-09-10T23:02:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:02:51.794+0000 7fa444a7f640 1 -- 172.21.15.120:0/1282775542 wait complete. 2024-09-10T23:02:51.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:51 smithi176 ceph-mon[34982]: pgmap v659: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-10T23:02:52.639 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1771395 16.06 MB/sec execute 467 sec latency 143.426 ms 2024-09-10T23:02:53.639 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1774116 16.06 MB/sec execute 468 sec latency 19.721 ms 2024-09-10T23:02:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:53 smithi120 ceph-mon[31247]: pgmap v660: 97 pgs: 97 active+clean; 58 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-09-10T23:02:53.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:53 smithi153 ceph-mon[34951]: pgmap v660: 97 pgs: 97 active+clean; 58 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-09-10T23:02:53.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:53 smithi176 ceph-mon[34982]: pgmap v660: 97 pgs: 97 active+clean; 58 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-09-10T23:02:54.639 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1776834 16.06 MB/sec execute 469 sec latency 30.171 ms 2024-09-10T23:02:55.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1779744 16.06 MB/sec execute 470 sec latency 66.546 ms 2024-09-10T23:02:55.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:55 smithi120 ceph-mon[31247]: pgmap v661: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-09-10T23:02:55.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:55 smithi153 ceph-mon[34951]: pgmap v661: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-09-10T23:02:55.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:55 smithi176 ceph-mon[34982]: pgmap v661: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-09-10T23:02:56.639 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1782930 16.06 MB/sec execute 471 sec latency 12.513 ms 2024-09-10T23:02:57.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1785960 16.06 MB/sec execute 472 sec latency 15.343 ms 2024-09-10T23:02:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:57 smithi120 ceph-mon[31247]: pgmap v662: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 392 op/s 2024-09-10T23:02:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:57 smithi153 ceph-mon[34951]: pgmap v662: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 392 op/s 2024-09-10T23:02:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:57 smithi176 ceph-mon[34982]: pgmap v662: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 392 op/s 2024-09-10T23:02:58.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1789107 16.06 MB/sec execute 473 sec latency 16.325 ms 2024-09-10T23:02:59.625 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:02:59 smithi176 ceph-mon[34982]: pgmap v663: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T23:02:59.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1792256 16.06 MB/sec execute 474 sec latency 20.299 ms 2024-09-10T23:02:59.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:02:59 smithi120 ceph-mon[31247]: pgmap v663: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T23:02:59.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:02:59 smithi153 ceph-mon[34951]: pgmap v663: 97 pgs: 97 active+clean; 48 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-09-10T23:03:00.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1794406 16.05 MB/sec execute 475 sec latency 47.762 ms 2024-09-10T23:03:01.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1797783 16.06 MB/sec execute 476 sec latency 24.584 ms 2024-09-10T23:03:01.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:01 smithi120 ceph-mon[31247]: pgmap v664: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T23:03:01.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:01 smithi153 ceph-mon[34951]: pgmap v664: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T23:03:01.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:01 smithi176 ceph-mon[34982]: pgmap v664: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T23:03:02.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1801101 16.07 MB/sec execute 477 sec latency 12.274 ms 2024-09-10T23:03:03.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1804384 16.07 MB/sec execute 478 sec latency 28.564 ms 2024-09-10T23:03:03.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:03 smithi120 ceph-mon[31247]: pgmap v665: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T23:03:03.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:03 smithi153 ceph-mon[34951]: pgmap v665: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T23:03:03.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:03 smithi176 ceph-mon[34982]: pgmap v665: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-09-10T23:03:04.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1807448 16.07 MB/sec execute 479 sec latency 12.283 ms 2024-09-10T23:03:04.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:03:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 -- 172.21.15.120:0/2331794869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f4109230 msgr2=0x7f65f4109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 --2- 172.21.15.120:0/2331794869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f4109230 0x7f65f4109630 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009930 tx=0x7f65f002a420 comp rx=0 tx=0).stop 2024-09-10T23:03:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 -- 172.21.15.120:0/2331794869 shutdown_connections 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 --2- 172.21.15.120:0/2331794869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 0x7f65f4111420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 --2- 172.21.15.120:0/2331794869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f410a100 0x7f65f410a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.913+0000 7f65fbb84640 1 --2- 172.21.15.120:0/2331794869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f4109230 0x7f65f4109630 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- 172.21.15.120:0/2331794869 >> 172.21.15.120:0/2331794869 conn(0x7f65f40783d0 msgr2=0x7f65f407a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- 172.21.15.120:0/2331794869 shutdown_connections 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- 172.21.15.120:0/2331794869 wait complete. 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 Processor -- start 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- start start 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f4109230 0x7f65f419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 0x7f65f41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114be0 con 0x7f65f410a100 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114a60 con 0x7f65f4109230 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65fbb84640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114d60 con 0x7f65f410aac0 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.914+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45656/0 (socket says 172.21.15.120:45656) 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 -- 172.21.15.120:0/4234433505 learned_addr learned my addr 172.21.15.120:0/4234433505 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65fa121640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 0x7f65f41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f9920640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f4109230 0x7f65f419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 -- 172.21.15.120:0/4234433505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 msgr2=0x7f65f41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 0x7f65f41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 -- 172.21.15.120:0/4234433505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f4109230 msgr2=0x7f65f419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f4109230 0x7f65f419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 -- 172.21.15.120:0/4234433505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a0ed0 con 0x7f65f410a100 2024-09-10T23:03:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65f911f640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65eaffd640 1 -- 172.21.15.120:0/4234433505 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400cd80 con 0x7f65f410a100 2024-09-10T23:03:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a11c0 con 0x7f65f410a100 2024-09-10T23:03:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65eaffd640 1 -- 172.21.15.120:0/4234433505 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40108e0 con 0x7f65f410a100 2024-09-10T23:03:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65eaffd640 1 -- 172.21.15.120:0/4234433505 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4018980 con 0x7f65f410a100 2024-09-10T23:03:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.915+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a7890 con 0x7f65f410a100 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.918+0000 7f65fea27740 1 Processor -- start 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.918+0000 7f65fea27740 1 -- start start 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f9fd6e6f0 con 0x559f9fdfc860 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f9fd5db20 con 0x559f9fdff190 2024-09-10T23:03:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fea27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559f9fd61930 con 0x559f9fe01bd0 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65eaffd640 1 -- 172.21.15.120:0/4234433505 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f65e4018b20 con 0x7f65f410a100 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45664/0 (socket says 172.21.15.120:45664) 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 -- 172.21.15.120:0/2914151673 learned_addr learned my addr 172.21.15.120:0/2914151673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65f9920640 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65f911f640 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 -- 172.21.15.120:0/2914151673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 msgr2=0x559f9fe04090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe04090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 -- 172.21.15.120:0/2914151673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 msgr2=0x559f9fe01580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe01580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 -- 172.21.15.120:0/2914151673 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9fd23530 con 0x559f9fdfc860 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.919+0000 7f65fa121640 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009fc0 tx=0x7f65ec010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbf57a6c32a8de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65eaffd640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 0x7f65e00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65dffff640 1 -- 172.21.15.120:0/2914151673 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec016070 con 0x559f9fdfc860 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65f9920640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 0x7f65e00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65dffff640 1 -- 172.21.15.120:0/2914151673 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ec004d20 con 0x559f9fdfc860 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65dffff640 1 -- 172.21.15.120:0/2914151673 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0053e0 con 0x559f9fdfc860 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.920+0000 7f65eaffd640 1 -- 172.21.15.120:0/4234433505 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f65e40e5ee0 con 0x7f65f410a100 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- 172.21.15.120:0/2914151673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 msgr2=0x559f9fdfec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65f9920640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 0x7f65e00b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009fc0 tx=0x7f65ec010800 comp rx=0 tx=0).stop 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- 172.21.15.120:0/2914151673 shutdown_connections 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe04090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 --2- 172.21.15.120:0/2914151673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fdfec50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- 172.21.15.120:0/2914151673 >> 172.21.15.120:0/2914151673 conn(0x559f9fdf79b0 msgr2=0x559f9fdf8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- 172.21.15.120:0/2914151673 shutdown_connections 2024-09-10T23:03:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- 172.21.15.120:0/2914151673 wait complete. 2024-09-10T23:03:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 -- start start 2024-09-10T23:03:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.921+0000 7f65fea27740 1 Processor -- start 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fe071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559f9fc95ab0 con 0x559f9fdff190 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559f9fd03730 con 0x559f9fe01bd0 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559f9fe04bb0 con 0x559f9fdfc860 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f9920640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45666/0 (socket says 172.21.15.120:45666) 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 -- 172.21.15.120:0/1101922957 learned_addr learned my addr 172.21.15.120:0/1101922957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f9920640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe05f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48294/0 (socket says 172.21.15.120:48294) 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 10 client.0 _ll_register_callbacks cb 0x559f9fd576b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fa121640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fe071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 -- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 msgr2=0x559f9fe05f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe05f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 -- 172.21.15.120:0/1101922957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 msgr2=0x559f9fe071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fe071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 -- 172.21.15.120:0/1101922957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9fabf790 con 0x559f9fdff190 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fa121640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fe071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65f911f640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9c0 tx=0x7f65e4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb697acb2682cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40ac550 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f65e4010430 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.922+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65ec004d20 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65e40abef0 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 10 client.17508 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f9fb38620 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ec004d20 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f65e40e9020 con 0x559f9fdff190 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 10 client.17508 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 20 client.17508 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe119e0 0x559f9fe13dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 4 client.17508 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f9fd70640 con 0x559f9fe119e0 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.923+0000 7f65f9920640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe119e0 0x559f9fe13dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.924+0000 7f65f9920640 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe119e0 0x559f9fe13dd0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=106442e10b34ca81 server_cookie=8978034e26643719 in_seq=0 out_seq=0 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.924+0000 7f65dd7fa640 10 client.17508 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65dd7fa640 1 -- 172.21.15.120:0/1101922957 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f9fd70640 con 0x559f9fe119e0 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65dd7fa640 10 client.17508 handle_command_reply: tid=0 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 client.17508 shutdown 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 msgr2=0x559f9fe065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9c0 tx=0x7f65e4018420 comp rx=0 tx=0).stop 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 shutdown_connections 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe119e0 0x559f9fe13dd0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fe071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fe065d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65dd7fa640 0 client.17508 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 --2- 172.21.15.120:0/1101922957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe05f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.925+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 >> 172.21.15.120:0/1101922957 conn(0x559f9fdf79b0 msgr2=0x559f9fdf6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.926+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 shutdown_connections 2024-09-10T23:03:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.926+0000 7f65fea27740 1 -- 172.21.15.120:0/1101922957 wait complete. 2024-09-10T23:03:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.926+0000 7f65fea27740 20 client.17508 trim_cache size 0 max 16384 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 Processor -- start 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 -- start start 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe13dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fdf8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41ae4e0 con 0x559f9fdff190 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41b0d60 con 0x559f9fe01bd0 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65fea27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41aeb50 con 0x559f9fdfc860 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65f911f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45676/0 (socket says 172.21.15.120:45676) 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65f911f640 1 -- 172.21.15.120:0/3487532594 learned_addr learned my addr 172.21.15.120:0/3487532594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.944+0000 7f65f9920640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe13dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65fa121640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fdf8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 -- 172.21.15.120:0/3487532594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 msgr2=0x559f9fe13dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe13dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 -- 172.21.15.120:0/3487532594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 msgr2=0x559f9fdf8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fdf8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 -- 172.21.15.120:0/3487532594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9fdfefc0 con 0x559f9fdff190 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65fa121640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fdf8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65f911f640 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f65e40a99f0 tx=0x7f65e4009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37133adcc2aed6ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65df7fe640 1 -- 172.21.15.120:0/3487532594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40ac0d0 con 0x559f9fdff190 2024-09-10T23:03:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65df7fe640 1 -- 172.21.15.120:0/3487532594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40ac290 con 0x559f9fdff190 2024-09-10T23:03:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.945+0000 7f65df7fe640 1 -- 172.21.15.120:0/3487532594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40a8210 con 0x559f9fdff190 2024-09-10T23:03:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- 172.21.15.120:0/3487532594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 msgr2=0x559f9fdf8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f65e40a99f0 tx=0x7f65e4009700 comp rx=0 tx=0).stop 2024-09-10T23:03:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- 172.21.15.120:0/3487532594 shutdown_connections 2024-09-10T23:03:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fe01bd0 0x559f9fdf8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdff190 0x559f9fdf8210 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 --2- 172.21.15.120:0/3487532594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fdfc860 0x559f9fe13dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- 172.21.15.120:0/3487532594 >> 172.21.15.120:0/3487532594 conn(0x559f9fdf79b0 msgr2=0x559f9fe01fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- 172.21.15.120:0/3487532594 shutdown_connections 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- 172.21.15.120:0/3487532594 wait complete. 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 -- start start 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.946+0000 7f65fea27740 1 Processor -- start 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe0ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41af220 con 0x559f9fdfc860 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41b0690 con 0x559f9fdff190 2024-09-10T23:03:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41af8f0 con 0x559f9fe01bd0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fea27740 10 client.0 _ll_register_callbacks cb 0x7f65e40e96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45684/0 (socket says 172.21.15.120:45684) 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 -- 172.21.15.120:0/3815166823 learned_addr learned my addr 172.21.15.120:0/3815166823 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65fa121640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe0ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f911f640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 -- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 msgr2=0x559f9fe0ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe0ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 -- 172.21.15.120:0/3815166823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 msgr2=0x559f9fe0c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe0c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 -- 172.21.15.120:0/3815166823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9fe0d030 con 0x559f9fdfc860 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65f9920640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f65f0005ec0 tx=0x7f65f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12e0ec068e8923 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.947+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f005d060 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f65f0003980 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e40ac290 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65f0035040 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 10 client.17514 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f9fe7e710 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40ac290 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f65f0050070 con 0x559f9fdfc860 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 10 client.17514 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 20 client.17514 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe7efd0 0x559f9fe813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 4 client.17514 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:03:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f9fd6c0f0 con 0x559f9fe7efd0 2024-09-10T23:03:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.948+0000 7f65f911f640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe7efd0 0x559f9fe813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.949+0000 7f65f911f640 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe7efd0 0x559f9fe813f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6a31578df36f2f server_cookie=1757ca78ac0022 in_seq=0 out_seq=0 2024-09-10T23:03:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:04.949+0000 7f65dd7fa640 10 client.17514 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.217+0000 7f65dd7fa640 1 -- 172.21.15.120:0/3815166823 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f9fd6c0f0 con 0x559f9fe7efd0 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.217+0000 7f65dd7fa640 10 client.17514 handle_command_reply: tid=0 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.217+0000 7f65fea27740 1 client.17514 shutdown 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.217+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 msgr2=0x559f9fe0d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.217+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f65f0005ec0 tx=0x7f65f0009260 comp rx=0 tx=0).stop 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 shutdown_connections 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559f9fe7efd0 0x559f9fe813f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559f9fe01bd0 0x559f9fe0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559f9fdff190 0x559f9fe0c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 --2- 172.21.15.120:0/3815166823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559f9fdfc860 0x559f9fe0d7e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 >> 172.21.15.120:0/3815166823 conn(0x559f9fdf79b0 msgr2=0x559f9fe01fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65dd7fa640 0 client.17514 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 shutdown_connections 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 1 -- 172.21.15.120:0/3815166823 wait complete. 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.218+0000 7f65fea27740 20 client.17514 trim_cache size 0 max 16384 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:03:05.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 msgr2=0x7f65e00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 0x7f65e00b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f00314a0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 msgr2=0x7f65f419f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 shutdown_connections 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65e00b52e0 0x7f65e00b77a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65f410aac0 0x7f65f41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65f410a100 0x7f65f419f9e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 --2- 172.21.15.120:0/4234433505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f4109230 0x7f65f419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 >> 172.21.15.120:0/4234433505 conn(0x7f65f40783d0 msgr2=0x7f65f410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 shutdown_connections 2024-09-10T23:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:05.219+0000 7f65fbb84640 1 -- 172.21.15.120:0/4234433505 wait complete. 2024-09-10T23:03:05.260 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:05.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1808535 16.05 MB/sec execute 480 sec latency 134.167 ms 2024-09-10T23:03:05.671 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:05 smithi120 ceph-mon[31247]: pgmap v666: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-09-10T23:03:05.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:05 smithi153 ceph-mon[34951]: pgmap v666: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-09-10T23:03:05.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 -- 172.21.15.120:0/2308924869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918101c90 msgr2=0x7f8918102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2308924869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918101c90 0x7f8918102070 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f890c009900 tx=0x7f890c02a410 comp rx=0 tx=0).stop 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 -- 172.21.15.120:0/2308924869 shutdown_connections 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2308924869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8918108020 0x7f891810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2308924869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89181076a0 0x7f8918107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2308924869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918101c90 0x7f8918102070 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 -- 172.21.15.120:0/2308924869 >> 172.21.15.120:0/2308924869 conn(0x7f89180fd700 msgr2=0x7f89180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 -- 172.21.15.120:0/2308924869 shutdown_connections 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.760+0000 7f891fdc5640 1 -- 172.21.15.120:0/2308924869 wait complete. 2024-09-10T23:03:05.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 Processor -- start 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 -- start start 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8918101c90 0x7f891819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89181076a0 0x7f891819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8918111ce0 con 0x7f8918108020 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8918111b60 con 0x7f89181076a0 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891fdc5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8918111e60 con 0x7f8918101c90 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891db61640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8918101c90 0x7f891819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891d360640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89181076a0 0x7f891819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45704/0 (socket says 172.21.15.120:45704) 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 -- 172.21.15.120:0/2787549581 learned_addr learned my addr 172.21.15.120:0/2787549581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 -- 172.21.15.120:0/2787549581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8918101c90 msgr2=0x7f891819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8918101c90 0x7f891819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 -- 172.21.15.120:0/2787549581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89181076a0 msgr2=0x7f891819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89181076a0 0x7f891819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 -- 172.21.15.120:0/2787549581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89181a15e0 con 0x7f8918108020 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.761+0000 7f891e362640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f891000b970 tx=0x7f891000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.762+0000 7f890affd640 1 -- 172.21.15.120:0/2787549581 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910019070 con 0x7f8918108020 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.762+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89181a18d0 con 0x7f8918108020 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.762+0000 7f890affd640 1 -- 172.21.15.120:0/2787549581 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8910014440 con 0x7f8918108020 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.762+0000 7f890affd640 1 -- 172.21.15.120:0/2787549581 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89100075d0 con 0x7f8918108020 2024-09-10T23:03:05.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.762+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89181a8010 con 0x7f8918108020 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.764+0000 7f892219a740 1 Processor -- start 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.764+0000 7f890affd640 1 -- 172.21.15.120:0/2787549581 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f891000cb40 con 0x7f8918108020 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.764+0000 7f892219a740 1 -- start start 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 0x561143f7c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561143ee6e50 con 0x561143f709c0 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561143ed6280 con 0x561143f700a0 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f892219a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561143eda090 con 0x561143f7c950 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891db61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 0x561143f7c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58982/0 (socket says 172.21.15.120:58982) 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891db61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 0x561143f7c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45716/0 (socket says 172.21.15.120:45716) 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 -- 172.21.15.120:0/2105243621 learned_addr learned my addr 172.21.15.120:0/2105243621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 -- 172.21.15.120:0/2105243621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 msgr2=0x561143f7ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f890affd640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 0x7f89040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 -- 172.21.15.120:0/2105243621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 msgr2=0x561143f7c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 0x561143f7c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 -- 172.21.15.120:0/2105243621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561143c34650 con 0x561143f700a0 2024-09-10T23:03:05.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891e362640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 0x7f89040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.765+0000 7f891d360640 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89140127a0 tx=0x7f891400b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa17c77255f1b89d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f8903fff640 1 -- 172.21.15.120:0/2105243621 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914012d20 con 0x561143f700a0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f890affd640 1 -- 172.21.15.120:0/2787549581 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f89100e5380 con 0x7f8918108020 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f8903fff640 1 -- 172.21.15.120:0/2105243621 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8914005020 con 0x561143f700a0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f8903fff640 1 -- 172.21.15.120:0/2105243621 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140049a0 con 0x561143f700a0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f892219a740 1 -- 172.21.15.120:0/2105243621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 msgr2=0x561143f70480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f891e362640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 0x7f89040b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89100abbb0 tx=0x7f89100790f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.766+0000 7f892219a740 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89140127a0 tx=0x7f891400b7c0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- 172.21.15.120:0/2105243621 shutdown_connections 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f709c0 0x561143f7c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- 172.21.15.120:0/2105243621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f70480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- 172.21.15.120:0/2105243621 >> 172.21.15.120:0/2105243621 conn(0x561143f6ded0 msgr2=0x561143f6f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- 172.21.15.120:0/2105243621 shutdown_connections 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- 172.21.15.120:0/2105243621 wait complete. 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- start start 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 Processor -- start 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f700a0 0x561143f7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561143d37700 con 0x561143f7c950 2024-09-10T23:03:05.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561143e71110 con 0x561143f709c0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.767+0000 7f892219a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561143f7f5d0 con 0x561143f700a0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f892219a740 10 client.0 _ll_register_callbacks cb 0x561143e8a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45730/0 (socket says 172.21.15.120:45730) 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 -- 172.21.15.120:0/2648029607 learned_addr learned my addr 172.21.15.120:0/2648029607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f700a0 0x561143f7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891db61640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 -- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f700a0 msgr2=0x561143f7c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f700a0 0x561143f7c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 -- 172.21.15.120:0/2648029607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 msgr2=0x561143f79630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f79630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 -- 172.21.15.120:0/2648029607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561143e9ec40 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f891d360640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8914005fd0 tx=0x7f891401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eb9f477c9dfb5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f89017fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914005540 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8914010770 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.768+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8914005020 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8914010970 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 10 client.17526 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561143f8aea0 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8914005020 con 0x561143f7c950 2024-09-10T23:03:05.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f891402e020 con 0x561143f7c950 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 10 client.17526 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 20 client.17526 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143f8c1f0 0x561143f8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 4 client.17526 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561143ee8da0 con 0x561143f8c1f0 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.769+0000 7f891db61640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143f8c1f0 0x561143f8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f89017fa640 10 client.17526 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f891db61640 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143f8c1f0 0x561143f8e5e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c995c9abecb2d3 server_cookie=f4989bf5cbefc424 in_seq=0 out_seq=0 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f89017fa640 1 -- 172.21.15.120:0/2648029607 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561143ee8da0 con 0x561143f8c1f0 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f89017fa640 10 client.17526 handle_command_reply: tid=0 2024-09-10T23:03:05.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 client.17526 shutdown 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 msgr2=0x561143f79bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8914005fd0 tx=0x7f891401b040 comp rx=0 tx=0).stop 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 shutdown_connections 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143f8c1f0 0x561143f8e5e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f79bd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 --2- 172.21.15.120:0/2648029607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f700a0 0x561143f7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f89017fa640 0 client.17526 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 >> 172.21.15.120:0/2648029607 conn(0x561143f6ded0 msgr2=0x561143f73a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.770+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 shutdown_connections 2024-09-10T23:03:05.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.771+0000 7f892219a740 1 -- 172.21.15.120:0/2648029607 wait complete. 2024-09-10T23:03:05.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.771+0000 7f892219a740 20 client.17526 trim_cache size 0 max 16384 2024-09-10T23:03:05.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 Processor -- start 2024-09-10T23:03:05.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 -- start start 2024-09-10T23:03:05.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 0x561143f6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89181ae570 con 0x561143f700a0 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b0df0 con 0x561143f709c0 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f892219a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f89181aebe0 con 0x561143f7c950 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f891d360640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 0x561143f6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f891d360640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 0x561143f6f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45742/0 (socket says 172.21.15.120:45742) 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.790+0000 7f891d360640 1 -- 172.21.15.120:0/4011524313 learned_addr learned my addr 172.21.15.120:0/4011524313 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891d360640 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 -- 172.21.15.120:0/4011524313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 msgr2=0x561143f7e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 -- 172.21.15.120:0/4011524313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 msgr2=0x561143f6f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 0x561143f6f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 -- 172.21.15.120:0/4011524313 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561143f7a4a0 con 0x561143f709c0 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f891db61640 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f7dd40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f890c009230 tx=0x7f890c005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a238f0a0879a637c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f89037fe640 1 -- 172.21.15.120:0/4011524313 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c04a020 con 0x561143f709c0 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f89037fe640 1 -- 172.21.15.120:0/4011524313 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c032a50 con 0x561143f709c0 2024-09-10T23:03:05.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.791+0000 7f89037fe640 1 -- 172.21.15.120:0/4011524313 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c035040 con 0x561143f709c0 2024-09-10T23:03:05.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- 172.21.15.120:0/4011524313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 msgr2=0x561143f7dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f7dd40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f890c009230 tx=0x7f890c005e60 comp rx=0 tx=0).stop 2024-09-10T23:03:05.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- 172.21.15.120:0/4011524313 shutdown_connections 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f7c950 0x561143f7e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f709c0 0x561143f7dd40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 --2- 172.21.15.120:0/4011524313 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f700a0 0x561143f6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- 172.21.15.120:0/4011524313 >> 172.21.15.120:0/4011524313 conn(0x561143f6ded0 msgr2=0x561143f6ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- 172.21.15.120:0/4011524313 shutdown_connections 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- 172.21.15.120:0/4011524313 wait complete. 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.792+0000 7f892219a740 1 -- start start 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 Processor -- start 2024-09-10T23:03:05.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f84870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f7f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89181af2b0 con 0x561143f7c950 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b0720 con 0x561143f700a0 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f89181af980 con 0x561143f709c0 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f892219a740 10 client.0 _ll_register_callbacks cb 0x7f890c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891d360640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f84870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891db61640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891db61640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48344/0 (socket says 172.21.15.120:48344) 2024-09-10T23:03:05.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891db61640 1 -- 172.21.15.120:0/1467793241 learned_addr learned my addr 172.21.15.120:0/1467793241 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891d360640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f7f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891db61640 1 -- 172.21.15.120:0/1467793241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 msgr2=0x561143f84870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.793+0000 7f891db61640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f84870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f891db61640 1 -- 172.21.15.120:0/1467793241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 msgr2=0x561143f7f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f891db61640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f7f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f891db61640 1 -- 172.21.15.120:0/1467793241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561143f7fd60 con 0x561143f709c0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f891db61640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f890c032980 tx=0x7f890c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec83a121b9c1854 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f89017fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c04a030 con 0x561143f709c0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c032a50 con 0x561143f709c0 2024-09-10T23:03:05.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f890c004240 con 0x561143f709c0 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f890c060050 con 0x561143f709c0 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f892219a740 10 client.27010 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561143ff9710 con 0x561143f709c0 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.794+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c032a50 con 0x561143f709c0 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f890c04f450 con 0x561143f709c0 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 10 client.27010 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 20 client.27010 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143ff9e90 0x561143ffc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 4 client.27010 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:03:05.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561143ee4850 con 0x561143ff9e90 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f891d360640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143ff9e90 0x561143ffc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f891d360640 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143ff9e90 0x561143ffc2b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5db4f3a6b19678 server_cookie=67c323a559a76f5f in_seq=0 out_seq=0 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.795+0000 7f89017fa640 10 client.27010 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f89017fa640 1 -- 172.21.15.120:0/1467793241 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x561143ee4850 con 0x561143ff9e90 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f89017fa640 10 client.27010 handle_command_reply: tid=0 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 client.27010 shutdown 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 msgr2=0x561143f84db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f890c032980 tx=0x7f890c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 shutdown_connections 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561143ff9e90 0x561143ffc2b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561143f7c950 0x561143f7f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561143f709c0 0x561143f84db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f89017fa640 0 client.27010 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 --2- 172.21.15.120:0/1467793241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561143f700a0 0x561143f84870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 >> 172.21.15.120:0/1467793241 conn(0x561143f6ded0 msgr2=0x561143f6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 shutdown_connections 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 1 -- 172.21.15.120:0/1467793241 wait complete. 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.796+0000 7f892219a740 20 client.27010 trim_cache size 0 max 16384 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:05.805 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.806 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:03:05.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.809 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:03:05.814 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.815 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:03:05.816 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.817 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:03:05.818 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:05.819 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 msgr2=0x7f89040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 0x7f89040b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89100abbb0 tx=0x7f89100790f0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 msgr2=0x7f89181a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f891000b970 tx=0x7f891000be30 comp rx=0 tx=0).stop 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 shutdown_connections 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89040b52e0 0x7f89040b77a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8918108020 0x7f89181a0e10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89181076a0 0x7f891819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 --2- 172.21.15.120:0/2787549581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8918101c90 0x7f891819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 >> 172.21.15.120:0/2787549581 conn(0x7f89180fd700 msgr2=0x7f891806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 shutdown_connections 2024-09-10T23:03:05.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:05.797+0000 7f891fdc5640 1 -- 172.21.15.120:0/2787549581 wait complete. 2024-09-10T23:03:05.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:05 smithi176 ceph-mon[34982]: pgmap v666: 97 pgs: 97 active+clean; 54 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-09-10T23:03:06.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1811717 16.05 MB/sec execute 481 sec latency 13.763 ms 2024-09-10T23:03:06.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:03:06.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:07.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:07.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:07 smithi120 ceph-mon[31247]: pgmap v667: 97 pgs: 97 active+clean; 53 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- 172.21.15.120:0/3496068671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75581045c0 msgr2=0x7f755810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 --2- 172.21.15.120:0/3496068671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75581045c0 0x7f755810cab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7548009900 tx=0x7f754802a410 comp rx=0 tx=0).stop 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- 172.21.15.120:0/3496068671 shutdown_connections 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 --2- 172.21.15.120:0/3496068671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 0x7f755810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 --2- 172.21.15.120:0/3496068671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75581045c0 0x7f755810cab0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 --2- 172.21.15.120:0/3496068671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7558103ca0 0x7f7558104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- 172.21.15.120:0/3496068671 >> 172.21.15.120:0/3496068671 conn(0x7f75580fd700 msgr2=0x7f75580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- 172.21.15.120:0/3496068671 shutdown_connections 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- 172.21.15.120:0/3496068671 wait complete. 2024-09-10T23:03:07.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 Processor -- start 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.548+0000 7f755f837640 1 -- start start 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 0x7f7558199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 0x7f755819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7558112d70 con 0x7f7558103ca0 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7558112bf0 con 0x7f755810d100 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755f837640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7558112ef0 con 0x7f75581045c0 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45772/0 (socket says 172.21.15.120:45772) 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 -- 172.21.15.120:0/1024846762 learned_addr learned my addr 172.21.15.120:0/1024846762 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755cdd2640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 0x7f7558199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755ddd4640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 0x7f755819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 -- 172.21.15.120:0/1024846762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 msgr2=0x7f7558199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 0x7f7558199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 -- 172.21.15.120:0/1024846762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 msgr2=0x7f755819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 0x7f755819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755d5d3640 1 -- 172.21.15.120:0/1024846762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755819f3d0 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.549+0000 7f755cdd2640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 0x7f7558199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f755d5d3640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f755400eb90 tx=0x7f755400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554004d20 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755400ee90 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755819f6c0 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554010670 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.550+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75581a5d80 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.551+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c0052c0 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.552+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7554010850 con 0x7f7558103ca0 2024-09-10T23:03:07.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.553+0000 7f754e7fc640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 0x7f75440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.554+0000 7f755cdd2640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 0x7f75440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.554+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f75540e6290 con 0x7f7558103ca0 2024-09-10T23:03:07.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.554+0000 7f755cdd2640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 0x7f75440b77a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.577 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.556+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7554056220 con 0x7f7558103ca0 2024-09-10T23:03:07.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1814897 16.05 MB/sec execute 482 sec latency 18.345 ms 2024-09-10T23:03:07.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:07 smithi153 ceph-mon[34951]: pgmap v667: 97 pgs: 97 active+clean; 53 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.785+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753c0055b0 con 0x7f7558103ca0 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.786+0000 7f754e7fc640 1 -- 172.21.15.120:0/1024846762 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f75540ad180 con 0x7f7558103ca0 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 msgr2=0x7f75440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 0x7f75440b77a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548031400 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 msgr2=0x7f7558199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f755400eb90 tx=0x7f755400c6a0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 shutdown_connections 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75440b52e0 0x7f75440b77a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f755810d100 0x7f755819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75581045c0 0x7f7558199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 --2- 172.21.15.120:0/1024846762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7558103ca0 0x7f7558199320 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 >> 172.21.15.120:0/1024846762 conn(0x7f75580fd700 msgr2=0x7f75580fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.788+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 shutdown_connections 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:07.789+0000 7f755f837640 1 -- 172.21.15.120:0/1024846762 wait complete. 2024-09-10T23:03:07.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:07 smithi176 ceph-mon[34982]: pgmap v667: 97 pgs: 97 active+clean; 53 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-10T23:03:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:08 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1024846762' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:08.532 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:03:08.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1817483 16.04 MB/sec execute 483 sec latency 71.354 ms 2024-09-10T23:03:08.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:08 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1024846762' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:08.791 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:08.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:08 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1024846762' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:09.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 -- 172.21.15.120:0/1017348995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c101c90 msgr2=0x7f8d2c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 --2- 172.21.15.120:0/1017348995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c101c90 0x7f8d2c102070 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-09-10T23:03:09.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 -- 172.21.15.120:0/1017348995 shutdown_connections 2024-09-10T23:03:09.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 --2- 172.21.15.120:0/1017348995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c108020 0x7f8d2c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 --2- 172.21.15.120:0/1017348995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 --2- 172.21.15.120:0/1017348995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c101c90 0x7f8d2c102070 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.291+0000 7f8d30f15640 1 -- 172.21.15.120:0/1017348995 >> 172.21.15.120:0/1017348995 conn(0x7f8d2c0fd700 msgr2=0x7f8d2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 -- 172.21.15.120:0/1017348995 shutdown_connections 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 -- 172.21.15.120:0/1017348995 wait complete. 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 Processor -- start 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 -- start start 2024-09-10T23:03:09.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c101c90 0x7f8d2c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.292+0000 7f8d30f15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d30f15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111ce0 con 0x7f8d2c108020 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d30f15640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111b60 con 0x7f8d2c101c90 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d30f15640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111e60 con 0x7f8d2c1076a0 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45776/0 (socket says 172.21.15.120:45776) 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/2853789012 learned_addr learned my addr 172.21.15.120:0/2853789012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c101c90 0x7f8d2c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/2853789012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 msgr2=0x7f8d2c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/2853789012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c101c90 msgr2=0x7f8d2c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c101c90 0x7f8d2c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/2853789012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c19f3e0 con 0x7f8d2c108020 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:09.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009fd0 tx=0x7f8d2400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d1b7fe640 1 -- 172.21.15.120:0/2853789012 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24017070 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c19f6d0 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d1b7fe640 1 -- 172.21.15.120:0/2853789012 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d24003e20 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d1b7fe640 1 -- 172.21.15.120:0/2853789012 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24010460 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.293+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a5d90 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 Processor -- start 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d1b7fe640 1 -- 172.21.15.120:0/2853789012 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8d24003970 con 0x7f8d2c108020 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 -- start start 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bdca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a1bd40ea0 con 0x557a1bdcaa10 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a1bd302d0 con 0x557a1bdca0f0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.296+0000 7f8d332ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a1bd340e0 con 0x557a1bdd69a0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48382/0 (socket says 172.21.15.120:48382) 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 -- 172.21.15.120:0/873829178 learned_addr learned my addr 172.21.15.120:0/873829178 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bdca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 -- 172.21.15.120:0/873829178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 msgr2=0x557a1bdca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bdca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 -- 172.21.15.120:0/873829178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 msgr2=0x557a1bdd6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.297+0000 7f8d29d9b640 1 -- 172.21.15.120:0/873829178 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1ba8e650 con 0x557a1bdd69a0 2024-09-10T23:03:09.309 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d20004650 tx=0x7f8d20009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad3017a3ac07a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d1b7fe640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d18ff9640 1 -- 172.21.15.120:0/873829178 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2000f910 con 0x557a1bdd69a0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d18ff9640 1 -- 172.21.15.120:0/873829178 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2002f3b0 con 0x557a1bdd69a0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d18ff9640 1 -- 172.21.15.120:0/873829178 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2000e620 con 0x557a1bdd69a0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.298+0000 7f8d1b7fe640 1 -- 172.21.15.120:0/2853789012 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d240e5890 con 0x7f8d2c108020 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- 172.21.15.120:0/873829178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 msgr2=0x557a1bdd8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 0x7f8d140b77a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c031440 tx=0x7f8d1c005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d20004650 tx=0x7f8d20009b80 comp rx=0 tx=0).stop 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- 172.21.15.120:0/873829178 shutdown_connections 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdd69a0 0x557a1bdd8d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.310 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 --2- 172.21.15.120:0/873829178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bdca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- 172.21.15.120:0/873829178 >> 172.21.15.120:0/873829178 conn(0x557a1bdc7f20 msgr2=0x557a1bdc90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- 172.21.15.120:0/873829178 shutdown_connections 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- 172.21.15.120:0/873829178 wait complete. 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 -- start start 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.299+0000 7f8d332ea740 1 Processor -- start 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bdd62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 0x557a1bdd3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a1bb91700 con 0x557a1bdcaa10 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a1bccb160 con 0x557a1bdd69a0 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a1bdd9620 con 0x557a1bdca0f0 2024-09-10T23:03:09.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 10 client.0 _ll_register_callbacks cb 0x557a1bce49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bdd62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45792/0 (socket says 172.21.15.120:45792) 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d29d9b640 1 -- 172.21.15.120:0/2212032323 learned_addr learned my addr 172.21.15.120:0/2212032323 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 0x557a1bdd3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 -- 172.21.15.120:0/2212032323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 msgr2=0x557a1bdd3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 0x557a1bdd3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 -- 172.21.15.120:0/2212032323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 msgr2=0x557a1bdd3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 -- 172.21.15.120:0/2212032323 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1bcf8c90 con 0x557a1bdca0f0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 0x557a1bdd3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bdd62e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d782642284f20705 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c04d070 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d2002f3b0 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.300+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d1c004450 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c059050 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 10 client.27028 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a1bde4ff0 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d2002f3b0 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8d1c0035f0 con 0x557a1bdca0f0 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 10 client.27028 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 20 client.27028 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:09.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1bde6470 0x557a1bde88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 4 client.27028 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1bd42df0 con 0x557a1bde6470 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.301+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1bde6470 0x557a1bde88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.302+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1bde6470 0x557a1bde88d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa58f9d9e5b28da server_cookie=37ec4ff5173a387b in_seq=0 out_seq=0 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.302+0000 7f8d11ffb640 10 client.27028 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d11ffb640 1 -- 172.21.15.120:0/2212032323 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a1bd42df0 con 0x557a1bde6470 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d11ffb640 10 client.27028 handle_command_reply: tid=0 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 client.27028 shutdown 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 msgr2=0x557a1bdd62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bdd62e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c009260 comp rx=0 tx=0).stop 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 shutdown_connections 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1bde6470 0x557a1bde88d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdd69a0 0x557a1bdd3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d11ffb640 0 client.27028 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdcaa10 0x557a1bdd3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 --2- 172.21.15.120:0/2212032323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bdd62e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 >> 172.21.15.120:0/2212032323 conn(0x557a1bdc7f20 msgr2=0x557a1bdcdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 shutdown_connections 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 1 -- 172.21.15.120:0/2212032323 wait complete. 2024-09-10T23:03:09.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.304+0000 7f8d332ea740 20 client.27028 trim_cache size 0 max 16384 2024-09-10T23:03:09.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 Processor -- start 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 -- start start 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bde2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdcaa10 0x557a1bdd7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c072e70 con 0x557a1bdd69a0 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1a6330 con 0x557a1bdcaa10 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d332ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c0734e0 con 0x557a1bdca0f0 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.317+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bde2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45798/0 (socket says 172.21.15.120:45798) 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/1320771521 learned_addr learned my addr 172.21.15.120:0/1320771521 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d29d9b640 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdcaa10 0x557a1bdd7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 -- 172.21.15.120:0/1320771521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdcaa10 msgr2=0x557a1bdd7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdcaa10 0x557a1bdd7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 -- 172.21.15.120:0/1320771521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 msgr2=0x557a1bdd82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 -- 172.21.15.120:0/1320771521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1bdc8dc0 con 0x557a1bdca0f0 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bde2850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0039c0 tx=0x7f8d1c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57877e13a2b00e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d13fff640 1 -- 172.21.15.120:0/1320771521 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c04d040 con 0x557a1bdca0f0 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d13fff640 1 -- 172.21.15.120:0/1320771521 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1c062070 con 0x557a1bdca0f0 2024-09-10T23:03:09.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.318+0000 7f8d13fff640 1 -- 172.21.15.120:0/1320771521 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035040 con 0x557a1bdca0f0 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- 172.21.15.120:0/1320771521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 msgr2=0x557a1bde2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bde2850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0039c0 tx=0x7f8d1c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- 172.21.15.120:0/1320771521 shutdown_connections 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdcaa10 0x557a1bdd7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 --2- 172.21.15.120:0/1320771521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdca0f0 0x557a1bde2850 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- 172.21.15.120:0/1320771521 >> 172.21.15.120:0/1320771521 conn(0x557a1bdc7f20 msgr2=0x557a1bdc9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- 172.21.15.120:0/1320771521 shutdown_connections 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- 172.21.15.120:0/1320771521 wait complete. 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 -- start start 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.319+0000 7f8d332ea740 1 Processor -- start 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bddeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdcaa10 0x557a1bddefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c073bb0 con 0x557a1bdd69a0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c075020 con 0x557a1bdca0f0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c074280 con 0x557a1bdcaa10 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 10 client.0 _ll_register_callbacks cb 0x7f8d1c032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45800/0 (socket says 172.21.15.120:45800) 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/295487658 learned_addr learned my addr 172.21.15.120:0/295487658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d29d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdcaa10 0x557a1bddefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bddeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdcaa10 msgr2=0x557a1bddefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdcaa10 0x557a1bddefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/295487658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 msgr2=0x557a1bddeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bddeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 -- 172.21.15.120:0/295487658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1bdd9f60 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bddeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d2ad9d640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8d24005c40 tx=0x7f8d240aba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52567b070df58014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24020040 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d240ab390 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.320+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1c062070 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d240ac130 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 10 client.17556 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a1be53e30 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1c062070 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8d240a7b10 con 0x557a1bdd69a0 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 10 client.17556 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:09.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 20 client.17556 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1be54600 0x557a1be56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 4 client.17556 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a1bd3e8a0 con 0x557a1be54600 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.321+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1be54600 0x557a1be56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.322+0000 7f8d2a59c640 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1be54600 0x557a1be56a20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f787c79a089cea server_cookie=c0248ad73d9f7cba in_seq=0 out_seq=0 2024-09-10T23:03:09.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.322+0000 7f8d11ffb640 10 client.17556 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d11ffb640 1 -- 172.21.15.120:0/295487658 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a1bd3e8a0 con 0x557a1be54600 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d11ffb640 10 client.17556 handle_command_reply: tid=0 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 client.17556 shutdown 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 msgr2=0x557a1bdd9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8d24005c40 tx=0x7f8d240aba80 comp rx=0 tx=0).stop 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 shutdown_connections 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a1be54600 0x557a1be56a20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1bdd69a0 0x557a1bdd9870 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bdcaa10 0x557a1bddefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 --2- 172.21.15.120:0/295487658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1bdca0f0 0x557a1bddeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 >> 172.21.15.120:0/295487658 conn(0x557a1bdc7f20 msgr2=0x557a1bdc6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d11ffb640 0 client.17556 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 shutdown_connections 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 1 -- 172.21.15.120:0/295487658 wait complete. 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.323+0000 7f8d332ea740 20 client.17556 trim_cache size 0 max 16384 2024-09-10T23:03:09.331 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 msgr2=0x7f8d140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 0x7f8d140b77a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c031440 tx=0x7f8d1c005b90 comp rx=0 tx=0).stop 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 msgr2=0x7f8d2c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009fd0 tx=0x7f8d2400b260 comp rx=0 tx=0).stop 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 shutdown_connections 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d2c108020 0x7f8d2c19ec10 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d2c1076a0 0x7f8d2c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 --2- 172.21.15.120:0/2853789012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d2c101c90 0x7f8d2c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 >> 172.21.15.120:0/2853789012 conn(0x7f8d2c0fd700 msgr2=0x7f8d2c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 shutdown_connections 2024-09-10T23:03:09.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:09.324+0000 7f8d30f15640 1 -- 172.21.15.120:0/2853789012 wait complete. 2024-09-10T23:03:09.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1819814 16.04 MB/sec execute 484 sec latency 18.153 ms 2024-09-10T23:03:09.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:09 smithi120 ceph-mon[31247]: pgmap v668: 97 pgs: 97 active+clean; 51 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-10T23:03:09.700 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:09 smithi153 ceph-mon[34951]: pgmap v668: 97 pgs: 97 active+clean; 51 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-10T23:03:09.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:09 smithi176 ceph-mon[34982]: pgmap v668: 97 pgs: 97 active+clean; 51 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 433 op/s 2024-09-10T23:03:10.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1822290 16.04 MB/sec execute 485 sec latency 70.889 ms 2024-09-10T23:03:11.061 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:03:11.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:03:11.323 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:11.402 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:11 smithi120 ceph-mon[31247]: pgmap v669: 97 pgs: 97 active+clean; 47 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2024-09-10T23:03:11.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1825341 16.03 MB/sec execute 486 sec latency 14.113 ms 2024-09-10T23:03:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:11 smithi153 ceph-mon[34951]: pgmap v669: 97 pgs: 97 active+clean; 47 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2024-09-10T23:03:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:11 smithi176 ceph-mon[34982]: pgmap v669: 97 pgs: 97 active+clean; 47 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 445 op/s 2024-09-10T23:03:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.850+0000 7f782476f640 1 -- 172.21.15.120:0/4004309357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c1076c0 msgr2=0x7f781c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.850+0000 7f782476f640 1 --2- 172.21.15.120:0/4004309357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c1076c0 0x7f781c107b00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f780c009900 tx=0x7f780c02a410 comp rx=0 tx=0).stop 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 -- 172.21.15.120:0/4004309357 shutdown_connections 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- 172.21.15.120:0/4004309357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c108040 0x7f781c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- 172.21.15.120:0/4004309357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c1076c0 0x7f781c107b00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- 172.21.15.120:0/4004309357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 -- 172.21.15.120:0/4004309357 >> 172.21.15.120:0/4004309357 conn(0x7f781c0fd700 msgr2=0x7f781c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 -- 172.21.15.120:0/4004309357 shutdown_connections 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 -- 172.21.15.120:0/4004309357 wait complete. 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 Processor -- start 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 -- start start 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 0x7f781c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.851+0000 7f782476f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782476f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781c111d50 con 0x7f781c108040 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782476f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c111bd0 con 0x7f781c101cb0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782476f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f781c111ed0 con 0x7f781c1076c0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 0x7f781c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 0x7f781c0fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48418/0 (socket says 172.21.15.120:48418) 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7821d0a640 1 -- 172.21.15.120:0/3907837751 learned_addr learned my addr 172.21.15.120:0/3907837751 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45822/0 (socket says 172.21.15.120:45822) 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 -- 172.21.15.120:0/3907837751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 msgr2=0x7f781c0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 0x7f781c0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 -- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 msgr2=0x7f781c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 -- 172.21.15.120:0/3907837751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c1a7690 con 0x7f781c101cb0 2024-09-10T23:03:11.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782250b640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c0fdc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f781800e9f0 tx=0x7f781800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f78137fe640 1 -- 172.21.15.120:0/3907837751 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781800ccc0 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1a7980 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f78137fe640 1 -- 172.21.15.120:0/3907837751 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78180108e0 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.852+0000 7f78137fe640 1 -- 172.21.15.120:0/3907837751 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818018880 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.853+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a7f10 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 Processor -- start 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f78137fe640 1 -- 172.21.15.120:0/3907837751 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7818010430 con 0x7f781c101cb0 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 -- start start 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cb2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cb4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab06c1ce60 con 0x55ab06cb2790 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab06c0c290 con 0x55ab06ca6800 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7826b44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab06c100a0 con 0x55ab06ca5ee0 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f782250b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cb2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48424/0 (socket says 172.21.15.120:48424) 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7822d0c640 1 -- 172.21.15.120:0/2545232370 learned_addr learned my addr 172.21.15.120:0/2545232370 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.855+0000 7f7821d0a640 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cb4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 -- 172.21.15.120:0/2545232370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 msgr2=0x55ab06cb2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cb2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 -- 172.21.15.120:0/2545232370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 msgr2=0x55ab06cb4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cb4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 -- 172.21.15.120:0/2545232370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0696a930 con 0x55ab06ca5ee0 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f78137fe640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 0x7f78080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7822d0c640 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7814011760 tx=0x7f7814011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6f97481eace5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f782250b640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 0x7f78080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7810ff9640 1 -- 172.21.15.120:0/2545232370 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400fb40 con 0x55ab06ca5ee0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7810ff9640 1 -- 172.21.15.120:0/2545232370 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7814011e70 con 0x55ab06ca5ee0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f7810ff9640 1 -- 172.21.15.120:0/2545232370 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781401a650 con 0x55ab06ca5ee0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.856+0000 7f78137fe640 1 -- 172.21.15.120:0/3907837751 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f78180e5f00 con 0x7f781c101cb0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- 172.21.15.120:0/2545232370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 msgr2=0x55ab06ca62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7814011760 tx=0x7f7814011c20 comp rx=0 tx=0).stop 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f782250b640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 0x7f78080b77a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78180a99b0 tx=0x7f78180eb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- 172.21.15.120:0/2545232370 shutdown_connections 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cb4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cb2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 --2- 172.21.15.120:0/2545232370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06ca62c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- 172.21.15.120:0/2545232370 >> 172.21.15.120:0/2545232370 conn(0x55ab06ca3d10 msgr2=0x55ab06ca4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- 172.21.15.120:0/2545232370 shutdown_connections 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- 172.21.15.120:0/2545232370 wait complete. 2024-09-10T23:03:11.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.857+0000 7f7826b44740 1 -- start start 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 Processor -- start 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca6800 0x55ab06cafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06cb2790 0x55ab06cb0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab06a6d890 con 0x55ab06ca5ee0 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab06ba6d40 con 0x55ab06cb2790 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab06cb5440 con 0x55ab06ca6800 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7826b44740 10 client.0 _ll_register_callbacks cb 0x55ab06c0ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45854/0 (socket says 172.21.15.120:45854) 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 -- 172.21.15.120:0/1419482811 learned_addr learned my addr 172.21.15.120:0/1419482811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7821d0a640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca6800 0x55ab06cafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06cb2790 0x55ab06cb0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 -- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca6800 msgr2=0x55ab06cafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca6800 0x55ab06cafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 -- 172.21.15.120:0/1419482811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06cb2790 msgr2=0x55ab06cb0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06cb2790 0x55ab06cb0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 -- 172.21.15.120:0/1419482811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab06cbe8a0 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7822d0c640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f781400fb30 tx=0x7f7814012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac3e07f1883a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.858+0000 7f7805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781402d060 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7814011e70 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7814004c40 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7814012450 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 10 client.17568 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab06cc1280 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7814011e70 con 0x55ab06ca5ee0 2024-09-10T23:03:11.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7814026040 con 0x55ab06ca5ee0 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 10 client.17568 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 20 client.17568 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06cc2690 0x55ab06cc4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 4 client.17568 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.859+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab06c1edb0 con 0x55ab06cc2690 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.860+0000 7f7821d0a640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06cc2690 0x55ab06cc4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.860+0000 7f7805ffb640 10 client.17568 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.860+0000 7f7821d0a640 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06cc2690 0x55ab06cc4a80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd15dd8978ca1f2 server_cookie=40b8a4553178488e in_seq=0 out_seq=0 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.861+0000 7f7805ffb640 1 -- 172.21.15.120:0/1419482811 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab06c1edb0 con 0x55ab06cc2690 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.861+0000 7f7805ffb640 10 client.17568 handle_command_reply: tid=0 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 client.17568 shutdown 2024-09-10T23:03:11.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 msgr2=0x55ab06caf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f781400fb30 tx=0x7f7814012c90 comp rx=0 tx=0).stop 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 shutdown_connections 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06cc2690 0x55ab06cc4a80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06cb2790 0x55ab06cb0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca6800 0x55ab06cafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7805ffb640 0 client.17568 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 --2- 172.21.15.120:0/1419482811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca5ee0 0x55ab06caf7c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 >> 172.21.15.120:0/1419482811 conn(0x55ab06ca3d10 msgr2=0x55ab06ca2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 shutdown_connections 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 1 -- 172.21.15.120:0/1419482811 wait complete. 2024-09-10T23:03:11.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.862+0000 7f7826b44740 20 client.17568 trim_cache size 0 max 16384 2024-09-10T23:03:11.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 Processor -- start 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 -- start start 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ae530 con 0x55ab06cb2790 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1b0db0 con 0x55ab06ca6800 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7826b44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aeba0 con 0x55ab06ca5ee0 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59142/0 (socket says 172.21.15.120:59142) 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7822d0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48452/0 (socket says 172.21.15.120:48452) 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 -- 172.21.15.120:0/3149776987 learned_addr learned my addr 172.21.15.120:0/3149776987 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:11.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 -- 172.21.15.120:0/3149776987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 msgr2=0x55ab06cac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 -- 172.21.15.120:0/3149776987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 msgr2=0x55ab06cc7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cc7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7821d0a640 1 -- 172.21.15.120:0/3149776987 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab06ca4790 con 0x55ab06ca6800 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.881+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f780c005ab0 tx=0x7f780c004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73716eac6aa3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7807fff640 1 -- 172.21.15.120:0/3149776987 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c035040 con 0x55ab06ca6800 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7807fff640 1 -- 172.21.15.120:0/3149776987 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c0047a0 con 0x55ab06ca6800 2024-09-10T23:03:11.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7807fff640 1 -- 172.21.15.120:0/3149776987 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c003850 con 0x55ab06ca6800 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7826b44740 1 -- 172.21.15.120:0/3149776987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 msgr2=0x55ab06cacf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7826b44740 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f780c005ab0 tx=0x7f780c004410 comp rx=0 tx=0).stop 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.882+0000 7f7826b44740 1 -- 172.21.15.120:0/3149776987 shutdown_connections 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06cb2790 0x55ab06cc7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca6800 0x55ab06cacf10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- 172.21.15.120:0/3149776987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06ca5ee0 0x55ab06cac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- 172.21.15.120:0/3149776987 >> 172.21.15.120:0/3149776987 conn(0x55ab06ca3d10 msgr2=0x55ab06cb1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- 172.21.15.120:0/3149776987 shutdown_connections 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- 172.21.15.120:0/3149776987 wait complete. 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- start start 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 Processor -- start 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca5ee0 0x55ab06cc6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06cb2790 0x55ab06d28240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1af270 con 0x55ab06ca6800 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1b06e0 con 0x55ab06ca5ee0 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1af940 con 0x55ab06cb2790 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7826b44740 10 client.0 _ll_register_callbacks cb 0x7f780c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.883+0000 7f7821d0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45874/0 (socket says 172.21.15.120:45874) 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 -- 172.21.15.120:0/3802240393 learned_addr learned my addr 172.21.15.120:0/3802240393 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06cb2790 0x55ab06d28240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca5ee0 0x55ab06cc6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 -- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06cb2790 msgr2=0x55ab06d28240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06cb2790 0x55ab06d28240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 -- 172.21.15.120:0/3802240393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca5ee0 msgr2=0x55ab06cc6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca5ee0 0x55ab06cc6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 -- 172.21.15.120:0/3802240393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab06d28980 con 0x55ab06ca6800 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7821d0a640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f780c005940 tx=0x7f780c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d1036d7b435e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c055060 con 0x55ab06ca6800 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f780c0037f0 con 0x55ab06ca6800 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c0047a0 con 0x55ab06ca6800 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f780c035070 con 0x55ab06ca6800 2024-09-10T23:03:11.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7826b44740 10 client.17574 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.884+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab06d2fe40 con 0x55ab06ca6800 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c0047a0 con 0x55ab06ca6800 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f780c031c30 con 0x55ab06ca6800 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 10 client.17574 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 20 client.17574 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06d30500 0x55ab06d32920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 4 client.17574 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab06c1a860 con 0x55ab06d30500 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06d30500 0x55ab06d32920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7805ffb640 10 client.17574 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:11.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.885+0000 7f7822d0c640 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06d30500 0x55ab06d32920 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c8e25f61eab451 server_cookie=b6061c5bbe4a0e7f in_seq=0 out_seq=0 2024-09-10T23:03:11.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7805ffb640 1 -- 172.21.15.120:0/3802240393 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab06c1a860 con 0x55ab06d30500 2024-09-10T23:03:11.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7805ffb640 10 client.17574 handle_command_reply: tid=0 2024-09-10T23:03:11.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 client.17574 shutdown 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 msgr2=0x55ab06cc6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f780c005940 tx=0x7f780c0095a0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 shutdown_connections 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab06d30500 0x55ab06d32920 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab06cb2790 0x55ab06d28240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7805ffb640 0 client.17574 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab06ca6800 0x55ab06cc6aa0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 --2- 172.21.15.120:0/3802240393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab06ca5ee0 0x55ab06cc6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 >> 172.21.15.120:0/3802240393 conn(0x55ab06ca3d10 msgr2=0x55ab06ca9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 shutdown_connections 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 1 -- 172.21.15.120:0/3802240393 wait complete. 2024-09-10T23:03:11.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.887+0000 7f7826b44740 20 client.17574 trim_cache size 0 max 16384 2024-09-10T23:03:11.894 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:11.894 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:03:11.894 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "bed6772e-b2a1-4157-8568-445845a41147", 2024-09-10T23:03:11.894 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:03:11.894 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 msgr2=0x7f78080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 0x7f78080b77a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78180a99b0 tx=0x7f78180eb040 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 msgr2=0x7f781c0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c0fdc00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f781800e9f0 tx=0x7f781800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 shutdown_connections 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78080b52e0 0x7f78080b77a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f781c108040 0x7f781c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f781c1076c0 0x7f781c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 --2- 172.21.15.120:0/3907837751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f781c101cb0 0x7f781c0fdc00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 >> 172.21.15.120:0/3907837751 conn(0x7f781c0fd700 msgr2=0x7f781c06b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 shutdown_connections 2024-09-10T23:03:11.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:11.888+0000 7f782476f640 1 -- 172.21.15.120:0/3907837751 wait complete. 2024-09-10T23:03:12.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1827886 16.02 MB/sec execute 487 sec latency 44.328 ms 2024-09-10T23:03:12.681 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:12 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:03:12.681 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:12 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.682 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:12 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:03:12.682 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:12 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:12 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:03:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:12 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:12 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:03:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:12 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:03:12.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:12 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:03:12.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:12 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:12 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:03:12.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:12 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:12.976 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 -- 172.21.15.120:0/2966973919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c108020 msgr2=0x7fbc2c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 --2- 172.21.15.120:0/2966973919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c108020 0x7fbc2c10e820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ca0 tx=0x7fbc2402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 -- 172.21.15.120:0/2966973919 shutdown_connections 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 --2- 172.21.15.120:0/2966973919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c108020 0x7fbc2c10e820 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 --2- 172.21.15.120:0/2966973919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 --2- 172.21.15.120:0/2966973919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c101c90 0x7fbc2c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 -- 172.21.15.120:0/2966973919 >> 172.21.15.120:0/2966973919 conn(0x7fbc2c0fd700 msgr2=0x7fbc2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 -- 172.21.15.120:0/2966973919 shutdown_connections 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 -- 172.21.15.120:0/2966973919 wait complete. 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.493+0000 7fbc310ab640 1 Processor -- start 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 -- start start 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c108020 0x7fbc2c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111ce0 con 0x7fbc2c101c90 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111b60 con 0x7fbc2c108020 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc310ab640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111e60 con 0x7fbc2c1076a0 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47400/0 (socket says 172.21.15.120:47400) 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 -- 172.21.15.120:0/1976909588 learned_addr learned my addr 172.21.15.120:0/1976909588 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c108020 0x7fbc2c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 -- 172.21.15.120:0/1976909588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 msgr2=0x7fbc2c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 -- 172.21.15.120:0/1976909588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c108020 msgr2=0x7fbc2c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c108020 0x7fbc2c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2ad9d640 1 -- 172.21.15.120:0/1976909588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1a5440 con 0x7fbc2c101c90 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.494+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc13fff640 1 -- 172.21.15.120:0/1976909588 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c017070 con 0x7fbc2c101c90 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1a5730 con 0x7fbc2c101c90 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc13fff640 1 -- 172.21.15.120:0/1976909588 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc1c009c30 con 0x7fbc2c101c90 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc13fff640 1 -- 172.21.15.120:0/1976909588 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00f620 con 0x7fbc2c101c90 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.495+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a5cc0 con 0x7fbc2c101c90 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.497+0000 7fbc13fff640 1 -- 172.21.15.120:0/1976909588 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbc1c010460 con 0x7fbc2c101c90 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 Processor -- start 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 -- start start 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc13fff640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 0x7fbc140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x56530002b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc13fff640 1 -- 172.21.15.120:0/1976909588 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbc1c0e5780 con 0x7fbc2c101c90 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x56530003e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652fffa1e50 con 0x56530003c140 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5652fff91280 con 0x56530002b480 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc33480740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5652fff95090 con 0x56530002bda0 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 0x7fbc140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50048/0 (socket says 172.21.15.120:50048) 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.498+0000 7fbc2a59c640 1 -- 172.21.15.120:0/3623757492 learned_addr learned my addr 172.21.15.120:0/3623757492 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x56530003e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x56530002b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 -- 172.21.15.120:0/3623757492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 msgr2=0x56530002b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x56530002b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 -- 172.21.15.120:0/3623757492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 msgr2=0x56530003e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x56530003e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 -- 172.21.15.120:0/3623757492 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ffcef650 con 0x56530002bda0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 0x7fbc140b77a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc24031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc200114f0 tx=0x7fbc200119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b79385518de7776 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc117fa640 1 -- 172.21.15.120:0/3623757492 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001e070 con 0x56530002bda0 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc117fa640 1 -- 172.21.15.120:0/3623757492 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200035f0 con 0x56530002bda0 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.499+0000 7fbc117fa640 1 -- 172.21.15.120:0/3623757492 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000f7c0 con 0x56530002bda0 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 -- 172.21.15.120:0/3623757492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 msgr2=0x56530003bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc200114f0 tx=0x7fbc200119b0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 -- 172.21.15.120:0/3623757492 shutdown_connections 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x56530003e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x56530003bc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 --2- 172.21.15.120:0/3623757492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x56530002b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 -- 172.21.15.120:0/3623757492 >> 172.21.15.120:0/3623757492 conn(0x5653000292b0 msgr2=0x56530002a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 -- 172.21.15.120:0/3623757492 shutdown_connections 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.500+0000 7fbc33480740 1 -- 172.21.15.120:0/3623757492 wait complete. 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 -- start start 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 Processor -- start 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002bda0 0x565300038e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x565300039430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5652ffdf2700 con 0x56530002b480 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5652fff2c110 con 0x56530002bda0 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56530003edf0 con 0x56530003c140 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc33480740 10 client.0 _ll_register_callbacks cb 0x5652fff45990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47422/0 (socket says 172.21.15.120:47422) 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc2ad9d640 1 -- 172.21.15.120:0/2169148085 learned_addr learned my addr 172.21.15.120:0/2169148085 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x565300039430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.501+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002bda0 0x565300038e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 -- 172.21.15.120:0/2169148085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002bda0 msgr2=0x565300038e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002bda0 0x565300038e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 -- 172.21.15.120:0/2169148085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 msgr2=0x56530003ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 -- 172.21.15.120:0/2169148085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652fff59c40 con 0x56530003c140 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x565300039430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc24005e30 tx=0x7fbc2402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520342095fbd4df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2404f070 con 0x56530003c140 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc200035f0 con 0x56530003c140 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbc24004440 con 0x56530003c140 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc240318e0 con 0x56530003c140 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc33480740 10 client.27064 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56530004a6f0 con 0x56530003c140 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.502+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200035f0 con 0x56530003c140 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbc2402fa20 con 0x56530003c140 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 10 client.27064 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 20 client.27064 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x56530004dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 4 client.27064 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652fffa3da0 con 0x56530004bb50 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x56530004dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc0eff5640 10 client.27064 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.503+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x56530004dfb0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff52a4d7187bb96 server_cookie=7868552eb1c17057 in_seq=0 out_seq=0 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc0eff5640 1 -- 172.21.15.120:0/2169148085 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652fffa3da0 con 0x56530004bb50 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc0eff5640 10 client.27064 handle_command_reply: tid=0 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 client.27064 shutdown 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 msgr2=0x565300039430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x565300039430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc24005e30 tx=0x7fbc2402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 shutdown_connections 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x56530004dfb0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x565300039430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc0eff5640 0 client.27064 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002bda0 0x565300038e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 --2- 172.21.15.120:0/2169148085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002b480 0x56530003ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 >> 172.21.15.120:0/2169148085 conn(0x5653000292b0 msgr2=0x56530002ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 shutdown_connections 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 1 -- 172.21.15.120:0/2169148085 wait complete. 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.504+0000 7fbc33480740 20 client.27064 trim_cache size 0 max 16384 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.524+0000 7fbc33480740 1 Processor -- start 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.524+0000 7fbc33480740 1 -- start start 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x565300048020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x56530003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1aa170 con 0x56530002bda0 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1ac9f0 con 0x56530002b480 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc33480740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1aa7e0 con 0x56530003c140 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47436/0 (socket says 172.21.15.120:47436) 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 -- 172.21.15.120:0/1807966961 learned_addr learned my addr 172.21.15.120:0/1807966961 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x56530003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x565300048020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 -- 172.21.15.120:0/1807966961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 msgr2=0x56530003a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x56530003a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 -- 172.21.15.120:0/1807966961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 msgr2=0x565300048020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x565300048020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 -- 172.21.15.120:0/1807966961 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56530002bbd0 con 0x56530002bda0 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x565300048020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.525+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009730 tx=0x7fbc200096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3b92896f3517e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc10ff9640 1 -- 172.21.15.120:0/1807966961 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20027020 con 0x56530002bda0 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc10ff9640 1 -- 172.21.15.120:0/1807966961 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc2001f040 con 0x56530002bda0 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc10ff9640 1 -- 172.21.15.120:0/1807966961 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001e050 con 0x56530002bda0 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 -- 172.21.15.120:0/1807966961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 msgr2=0x56530003a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009730 tx=0x7fbc200096c0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 -- 172.21.15.120:0/1807966961 shutdown_connections 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530003c140 0x56530003a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530002bda0 0x56530003a480 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 --2- 172.21.15.120:0/1807966961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x565300048020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.526+0000 7fbc33480740 1 -- 172.21.15.120:0/1807966961 >> 172.21.15.120:0/1807966961 conn(0x5653000292b0 msgr2=0x56530003cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- 172.21.15.120:0/1807966961 shutdown_connections 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- 172.21.15.120:0/1807966961 wait complete. 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- start start 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 Processor -- start 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x5653000448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x565300053570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1aaeb0 con 0x56530003c140 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1ac320 con 0x56530002b480 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1ab580 con 0x56530002bda0 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc33480740 10 client.0 _ll_register_callbacks cb 0x7fbc24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc2b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc2b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47442/0 (socket says 172.21.15.120:47442) 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc2b59e640 1 -- 172.21.15.120:0/3758741153 learned_addr learned my addr 172.21.15.120:0/3758741153 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.527+0000 7fbc2a59c640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x565300053570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x5653000448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 -- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 msgr2=0x565300053570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x565300053570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 -- 172.21.15.120:0/3758741153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 msgr2=0x5653000448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x5653000448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 -- 172.21.15.120:0/3758741153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653000b7a80 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc2b59e640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbc2402ade0 tx=0x7fbc2402f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959ae65582a6ba37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2404f070 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbc24004240 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2001f040 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc24066070 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc33480740 10 client.17592 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.528+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653000b8b80 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc2001f040 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbc2405e030 con 0x56530003c140 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 10 client.17592 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 20 client.17592 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x5653000bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 4 client.17592 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652fff9f850 con 0x56530004bb50 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x5653000bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc0eff5640 10 client.17592 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.529+0000 7fbc2ad9d640 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x5653000bb340 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fbea2627a341ca8 server_cookie=bb0b0097b27ec0fd in_seq=0 out_seq=0 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc0eff5640 1 -- 172.21.15.120:0/3758741153 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x5652fff9f850 con 0x56530004bb50 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc0eff5640 10 client.17592 handle_command_reply: tid=0 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 client.17592 shutdown 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 msgr2=0x565300053b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbc2402ade0 tx=0x7fbc2402f910 comp rx=0 tx=0).stop 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 shutdown_connections 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56530004bb50 0x5653000bb340 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56530003c140 0x565300053b50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56530002bda0 0x565300053570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 --2- 172.21.15.120:0/3758741153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56530002b480 0x5653000448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 >> 172.21.15.120:0/3758741153 conn(0x5653000292b0 msgr2=0x56530003a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc0eff5640 0 client.17592 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 shutdown_connections 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 1 -- 172.21.15.120:0/3758741153 wait complete. 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.531+0000 7fbc33480740 20 client.17592 trim_cache size 0 max 16384 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "bed6772e-b2a1-4157-8568-445845a41147", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 msgr2=0x7fbc140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 0x7fbc140b77a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc24031af0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 msgr2=0x7fbc2c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00de10 comp rx=0 tx=0).stop 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 shutdown_connections 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbc140b52e0 0x7fbc140b77a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c108020 0x7fbc2c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc2c1076a0 0x7fbc2c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 --2- 172.21.15.120:0/1976909588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbc2c101c90 0x7fbc2c19a060 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 >> 172.21.15.120:0/1976909588 conn(0x7fbc2c0fd700 msgr2=0x7fbc2c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 shutdown_connections 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:13.532+0000 7fbc310ab640 1 -- 172.21.15.120:0/1976909588 wait complete. 2024-09-10T23:03:13.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1830252 16.02 MB/sec execute 488 sec latency 20.158 ms 2024-09-10T23:03:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:13 smithi120 ceph-mon[31247]: pgmap v670: 97 pgs: 97 active+clean; 49 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-09-10T23:03:13.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:13 smithi153 ceph-mon[34951]: pgmap v670: 97 pgs: 97 active+clean; 49 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-09-10T23:03:13.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:13 smithi176 ceph-mon[34982]: pgmap v670: 97 pgs: 97 active+clean; 49 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-09-10T23:03:14.380 INFO:tasks.cephfs.filesystem:scrub status for tag:bed6772e-b2a1-4157-8568-445845a41147 - {'path': '/', 'tag': 'bed6772e-b2a1-4157-8568-445845a41147', 'options': 'recursive,force'} 2024-09-10T23:03:14.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1833376 16.02 MB/sec execute 489 sec latency 20.494 ms 2024-09-10T23:03:15.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1834857 16.04 MB/sec execute 490 sec latency 55.336 ms 2024-09-10T23:03:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:15 smithi120 ceph-mon[31247]: pgmap v671: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 395 op/s 2024-09-10T23:03:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:03:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:15 smithi153 ceph-mon[34951]: pgmap v671: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 395 op/s 2024-09-10T23:03:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:03:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:15 smithi176 ceph-mon[34982]: pgmap v671: 97 pgs: 97 active+clean; 50 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 395 op/s 2024-09-10T23:03:15.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:03:16.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1837938 16.04 MB/sec execute 491 sec latency 18.496 ms 2024-09-10T23:03:17.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1841212 16.05 MB/sec execute 492 sec latency 15.003 ms 2024-09-10T23:03:17.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:17 smithi120 ceph-mon[31247]: pgmap v672: 97 pgs: 97 active+clean; 46 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-09-10T23:03:17.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:17 smithi153 ceph-mon[34951]: pgmap v672: 97 pgs: 97 active+clean; 46 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-09-10T23:03:17.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:17 smithi176 ceph-mon[34982]: pgmap v672: 97 pgs: 97 active+clean; 46 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2024-09-10T23:03:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/103437937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc103ae0 msgr2=0x7f7fbc103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/103437937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc103f60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009930 tx=0x7f7fb002a420 comp rx=0 tx=0).stop 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/103437937 shutdown_connections 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/103437937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc1044a0 0x7f7fbc10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/103437937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc103f60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/103437937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc102990 0x7f7fbc102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/103437937 >> 172.21.15.120:0/103437937 conn(0x7f7fbc0fe250 msgr2=0x7f7fbc100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/103437937 shutdown_connections 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/103437937 wait complete. 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.310+0000 7f7fc1f6b640 1 Processor -- start 2024-09-10T23:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 -- start start 2024-09-10T23:03:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc1044a0 0x7f7fbc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e5e0 con 0x7f7fbc102990 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e460 con 0x7f7fbc1044a0 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fc1f6b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc10e760 con 0x7f7fbc103ae0 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50110/0 (socket says 172.21.15.120:50110) 2024-09-10T23:03:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47458/0 (socket says 172.21.15.120:47458) 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbaffd640 1 -- 172.21.15.120:0/4061332596 learned_addr learned my addr 172.21.15.120:0/4061332596 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc1044a0 0x7f7fbc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 -- 172.21.15.120:0/4061332596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 msgr2=0x7f7fbc075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 -- 172.21.15.120:0/4061332596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc1044a0 msgr2=0x7f7fbc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc1044a0 0x7f7fbc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 -- 172.21.15.120:0/4061332596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0762f0 con 0x7f7fbc102990 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.311+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00c970 tx=0x7f7fac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.312+0000 7f7fb8ff9640 1 -- 172.21.15.120:0/4061332596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac007bb0 con 0x7f7fbc102990 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.312+0000 7f7fb8ff9640 1 -- 172.21.15.120:0/4061332596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fac007d50 con 0x7f7fbc102990 2024-09-10T23:03:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.312+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1a2e40 con 0x7f7fbc102990 2024-09-10T23:03:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.312+0000 7f7fb8ff9640 1 -- 172.21.15.120:0/4061332596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac018780 con 0x7f7fbc102990 2024-09-10T23:03:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.312+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a32a0 con 0x7f7fbc102990 2024-09-10T23:03:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.314+0000 7f7fb8ff9640 1 -- 172.21.15.120:0/4061332596 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7fac00f450 con 0x7f7fbc102990 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.314+0000 7f7fc4e19740 1 Processor -- start 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.314+0000 7f7fc4e19740 1 -- start start 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56524aa986f0 con 0x56524ab29190 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56524aa87b20 con 0x56524ab26860 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56524aa8b930 con 0x56524ab2bbd0 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59444/0 (socket says 172.21.15.120:59444) 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47468/0 (socket says 172.21.15.120:47468) 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1201040839 learned_addr learned my addr 172.21.15.120:0/1201040839 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1201040839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 msgr2=0x56524ab2e090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab2e090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1201040839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 msgr2=0x56524ab2b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1201040839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524aa4d530 con 0x56524ab26860 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fb8ff9640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 0x7f7fa40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2017d6b7f9c49bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.315+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 0x7f7fa40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.316+0000 7f7fa1ffb640 1 -- 172.21.15.120:0/1201040839 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4004d20 con 0x56524ab26860 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.316+0000 7f7fb8ff9640 1 -- 172.21.15.120:0/4061332596 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fac0e5b80 con 0x7f7fbc102990 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.316+0000 7f7fa1ffb640 1 -- 172.21.15.120:0/1201040839 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb4004050 con 0x56524ab26860 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.316+0000 7f7fa1ffb640 1 -- 172.21.15.120:0/1201040839 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4005280 con 0x56524ab26860 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1201040839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 msgr2=0x56524ab28c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fbb7fe640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 0x7f7fa40b77a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac0e9660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb40118d0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1201040839 shutdown_connections 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1201040839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab28c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1201040839 >> 172.21.15.120:0/1201040839 conn(0x56524ab219b0 msgr2=0x56524ab22740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1201040839 shutdown_connections 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1201040839 wait complete. 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- start start 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 Processor -- start 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab26860 0x56524ab2fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab2bbd0 0x56524ab31100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.317+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56524a9bfab0 con 0x56524ab29190 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56524aa2d730 con 0x56524ab2bbd0 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56524ab2ebb0 con 0x56524ab26860 2024-09-10T23:03:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fc4e19740 10 client.0 _ll_register_callbacks cb 0x56524aa816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47470/0 (socket says 172.21.15.120:47470) 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 -- 172.21.15.120:0/2934284534 learned_addr learned my addr 172.21.15.120:0/2934284534 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab26860 0x56524ab2fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab2bbd0 0x56524ab31100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 -- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab26860 msgr2=0x56524ab2fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab26860 0x56524ab2fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 -- 172.21.15.120:0/2934284534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab2bbd0 msgr2=0x56524ab31100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab2bbd0 0x56524ab31100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 -- 172.21.15.120:0/2934284534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524a7e9790 con 0x56524ab29190 2024-09-10T23:03:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0006010 tx=0x7f7fb0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb4a1e891127d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7f9affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0035070 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fb0004230 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb4004050 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.318+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fb0004430 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 10 client.17604 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56524a862620 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb4004050 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7fb00455b0 con 0x56524ab29190 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 10 client.17604 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 20 client.17604 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524ab3dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 4 client.17604 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524aa9a640 con 0x56524ab3b7e0 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.319+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524ab3dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524ab3dbd0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68978f25ca1144de server_cookie=ed94faf96981fe75 in_seq=0 out_seq=0 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7f9affd640 10 client.17604 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7f9affd640 1 -- 172.21.15.120:0/2934284534 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56524aa9a640 con 0x56524ab3b7e0 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7f9affd640 10 client.17604 handle_command_reply: tid=0 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 client.17604 shutdown 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 msgr2=0x56524ab30530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0006010 tx=0x7f7fb0009680 comp rx=0 tx=0).stop 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 shutdown_connections 2024-09-10T23:03:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524ab3dbd0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab2bbd0 0x56524ab31100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7f9affd640 0 client.17604 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab29190 0x56524ab30530 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/2934284534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab26860 0x56524ab2fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.320+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 >> 172.21.15.120:0/2934284534 conn(0x56524ab219b0 msgr2=0x56524ab20bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.321+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 shutdown_connections 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.321+0000 7f7fc4e19740 1 -- 172.21.15.120:0/2934284534 wait complete. 2024-09-10T23:03:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.321+0000 7f7fc4e19740 20 client.17604 trim_cache size 0 max 16384 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 Processor -- start 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 -- start start 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab22450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab29190 0x56524ab2fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1a9f50 con 0x56524ab2bbd0 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ac7d0 con 0x56524ab26860 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1aa5c0 con 0x56524ab29190 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47478/0 (socket says 172.21.15.120:47478) 2024-09-10T23:03:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 -- 172.21.15.120:0/3084126958 learned_addr learned my addr 172.21.15.120:0/3084126958 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab22450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab29190 0x56524ab2fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 -- 172.21.15.120:0/3084126958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab29190 msgr2=0x56524ab2fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab29190 0x56524ab2fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 -- 172.21.15.120:0/3084126958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 msgr2=0x56524ab22450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab22450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.335+0000 7f7fbbfff640 1 -- 172.21.15.120:0/3084126958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524ab3df40 con 0x56524ab2bbd0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.336+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009540 tx=0x7f7fb4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf759fad28dcfb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.336+0000 7f7fa17fa640 1 -- 172.21.15.120:0/3084126958 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40042d0 con 0x56524ab2bbd0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.336+0000 7f7fa17fa640 1 -- 172.21.15.120:0/3084126958 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb4004da0 con 0x56524ab2bbd0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.336+0000 7f7fa17fa640 1 -- 172.21.15.120:0/3084126958 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4005450 con 0x56524ab2bbd0 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- 172.21.15.120:0/3084126958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 msgr2=0x56524ab303d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009540 tx=0x7f7fb4005fd0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- 172.21.15.120:0/3084126958 shutdown_connections 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab2bbd0 0x56524ab303d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab29190 0x56524ab2fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/3084126958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab26860 0x56524ab22450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- 172.21.15.120:0/3084126958 >> 172.21.15.120:0/3084126958 conn(0x56524ab219b0 msgr2=0x56524ab2bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- 172.21.15.120:0/3084126958 shutdown_connections 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- 172.21.15.120:0/3084126958 wait complete. 2024-09-10T23:03:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- start start 2024-09-10T23:03:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 Processor -- start 2024-09-10T23:03:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 0x56524ab35580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1aac90 con 0x56524ab26860 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ac100 con 0x56524ab29190 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ab360 con 0x56524ab2bbd0 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.337+0000 7f7fc4e19740 10 client.0 _ll_register_callbacks cb 0x7f7fb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47482/0 (socket says 172.21.15.120:47482) 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 0x56524ab35580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1123481299 learned_addr learned my addr 172.21.15.120:0/1123481299 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 msgr2=0x56524ab360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1123481299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 msgr2=0x56524ab35580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 0x56524ab35580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 -- 172.21.15.120:0/1123481299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524ab2b8f0 con 0x56524ab26860 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 0x56524ab35580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fbbfff640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fa0 tx=0x7f7fb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef02afd776ee669 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7f9affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4005750 con 0x56524ab26860 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fb40058f0 con 0x56524ab26860 2024-09-10T23:03:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb4004da0 con 0x56524ab26860 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fb400f4e0 con 0x56524ab26860 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fc4e19740 10 client.17610 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56524ab37ad0 con 0x56524ab26860 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.338+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb4004da0 con 0x56524ab26860 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7fb40043a0 con 0x56524ab26860 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 10 client.17610 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 20 client.17610 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524abaa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 4 client.17610 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56524aa960f0 con 0x56524ab3b7e0 2024-09-10T23:03:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524abaa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7f9affd640 10 client.17610 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.339+0000 7f7fbaffd640 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524abaa8e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e3d4f421d5b4b7 server_cookie=3bb9be37d00864c0 in_seq=0 out_seq=0 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.620+0000 7f7f9affd640 1 -- 172.21.15.120:0/1123481299 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56524aa960f0 con 0x56524ab3b7e0 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.620+0000 7f7f9affd640 10 client.17610 handle_command_reply: tid=0 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.620+0000 7f7fc4e19740 1 client.17610 shutdown 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 msgr2=0x56524ab34ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fa0 tx=0x7f7fb4014000 comp rx=0 tx=0).stop 2024-09-10T23:03:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 shutdown_connections 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56524ab3b7e0 0x56524abaa8e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56524ab2bbd0 0x56524ab360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7f9affd640 0 client.17610 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56524ab29190 0x56524ab35580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 --2- 172.21.15.120:0/1123481299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56524ab26860 0x56524ab34ee0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 >> 172.21.15.120:0/1123481299 conn(0x56524ab219b0 msgr2=0x56524ab2fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 shutdown_connections 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 1 -- 172.21.15.120:0/1123481299 wait complete. 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.621+0000 7f7fc4e19740 20 client.17610 trim_cache size 0 max 16384 2024-09-10T23:03:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:03:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:03:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:03:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 msgr2=0x7f7fa40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 0x7f7fa40b77a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac0e9660 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 msgr2=0x7f7fbc076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00c970 tx=0x7f7fac00ce30 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 shutdown_connections 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fa40b52e0 0x7f7fa40b77a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fbc1044a0 0x7f7fbc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fbc103ae0 0x7f7fbc075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 --2- 172.21.15.120:0/4061332596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fbc102990 0x7f7fbc076ce0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 >> 172.21.15.120:0/4061332596 conn(0x7f7fbc0fe250 msgr2=0x7f7fbc106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 shutdown_connections 2024-09-10T23:03:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:18.622+0000 7f7fc1f6b640 1 -- 172.21.15.120:0/4061332596 wait complete. 2024-09-10T23:03:18.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1843804 16.04 MB/sec execute 493 sec latency 169.045 ms 2024-09-10T23:03:19.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1846119 16.03 MB/sec execute 494 sec latency 34.829 ms 2024-09-10T23:03:19.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:19 smithi120 ceph-mon[31247]: pgmap v673: 97 pgs: 97 active+clean; 45 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-09-10T23:03:19.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:19 smithi153 ceph-mon[34951]: pgmap v673: 97 pgs: 97 active+clean; 45 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-09-10T23:03:19.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:19 smithi176 ceph-mon[34982]: pgmap v673: 97 pgs: 97 active+clean; 45 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-09-10T23:03:20.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1848128 16.03 MB/sec execute 495 sec latency 56.889 ms 2024-09-10T23:03:21.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1851360 16.03 MB/sec execute 496 sec latency 17.582 ms 2024-09-10T23:03:21.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:21 smithi120 ceph-mon[31247]: pgmap v674: 97 pgs: 97 active+clean; 29 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-09-10T23:03:21.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:21 smithi153 ceph-mon[34951]: pgmap v674: 97 pgs: 97 active+clean; 29 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-09-10T23:03:21.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:21 smithi176 ceph-mon[34982]: pgmap v674: 97 pgs: 97 active+clean; 29 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-09-10T23:03:22.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1854380 16.03 MB/sec execute 497 sec latency 16.624 ms 2024-09-10T23:03:23.640 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1857047 16.02 MB/sec execute 498 sec latency 36.711 ms 2024-09-10T23:03:23.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:23 smithi120 ceph-mon[31247]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-09-10T23:03:23.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:23 smithi153 ceph-mon[34951]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-09-10T23:03:23.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:23 smithi176 ceph-mon[34982]: pgmap v675: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-09-10T23:03:24.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1859507 16.02 MB/sec execute 499 sec latency 12.080 ms 2024-09-10T23:03:25.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1862226 16.02 MB/sec execute 500 sec latency 53.410 ms 2024-09-10T23:03:25.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:25 smithi120 ceph-mon[31247]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-10T23:03:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:25 smithi153 ceph-mon[34951]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-10T23:03:25.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:25 smithi176 ceph-mon[34982]: pgmap v676: 97 pgs: 97 active+clean; 25 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-10T23:03:26.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1864686 16.01 MB/sec execute 501 sec latency 196.926 ms 2024-09-10T23:03:27.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1866602 16.00 MB/sec execute 502 sec latency 323.123 ms 2024-09-10T23:03:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:27 smithi120 ceph-mon[31247]: pgmap v677: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 494 op/s 2024-09-10T23:03:27.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:27 smithi153 ceph-mon[34951]: pgmap v677: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 494 op/s 2024-09-10T23:03:27.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:27 smithi176 ceph-mon[34982]: pgmap v677: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 494 op/s 2024-09-10T23:03:28.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1869495 16.00 MB/sec execute 503 sec latency 32.479 ms 2024-09-10T23:03:29.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1872749 16.00 MB/sec execute 504 sec latency 11.603 ms 2024-09-10T23:03:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:29 smithi120 ceph-mon[31247]: pgmap v678: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 407 op/s 2024-09-10T23:03:29.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:29 smithi153 ceph-mon[34951]: pgmap v678: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 407 op/s 2024-09-10T23:03:29.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:29 smithi176 ceph-mon[34982]: pgmap v678: 97 pgs: 97 active+clean; 32 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 407 op/s 2024-09-10T23:03:30.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1875361 16.00 MB/sec execute 505 sec latency 88.291 ms 2024-09-10T23:03:31.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1877503 15.99 MB/sec execute 506 sec latency 39.845 ms 2024-09-10T23:03:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:31 smithi120 ceph-mon[31247]: pgmap v679: 97 pgs: 97 active+clean; 36 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T23:03:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:31 smithi153 ceph-mon[34951]: pgmap v679: 97 pgs: 97 active+clean; 36 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 -- 172.21.15.120:0/3869321526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4101860 msgr2=0x7f20c4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 --2- 172.21.15.120:0/3869321526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4101860 0x7f20c4101c60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009cd0 tx=0x7f20b402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 -- 172.21.15.120:0/3869321526 shutdown_connections 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 --2- 172.21.15.120:0/3869321526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c4103370 0x7f20c4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 --2- 172.21.15.120:0/3869321526 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c41029b0 0x7f20c4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.714+0000 7f20cb717640 1 --2- 172.21.15.120:0/3869321526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4101860 0x7f20c4101c60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- 172.21.15.120:0/3869321526 >> 172.21.15.120:0/3869321526 conn(0x7f20c40fd140 msgr2=0x7f20c40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- 172.21.15.120:0/3869321526 shutdown_connections 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- 172.21.15.120:0/3869321526 wait complete. 2024-09-10T23:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 Processor -- start 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- start start 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c4101860 0x7f20c406b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c41029b0 0x7f20c419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410d4b0 con 0x7f20c4103370 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410d330 con 0x7f20c4101860 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20cb717640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410d630 con 0x7f20c41029b0 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20c9cb4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.715+0000 7f20c9cb4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38532/0 (socket says 172.21.15.120:38532) 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 -- 172.21.15.120:0/514800109 learned_addr learned my addr 172.21.15.120:0/514800109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c94b3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c4101860 0x7f20c406b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c41029b0 0x7f20c419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 -- 172.21.15.120:0/514800109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c41029b0 msgr2=0x7f20c419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c41029b0 0x7f20c419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 -- 172.21.15.120:0/514800109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c4101860 msgr2=0x7f20c406b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c4101860 0x7f20c406b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 -- 172.21.15.120:0/514800109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c41a0970 con 0x7f20c4103370 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e970 tx=0x7f20bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c27fc640 1 -- 172.21.15.120:0/514800109 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00ccc0 con 0x7f20c4103370 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c27fc640 1 -- 172.21.15.120:0/514800109 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc00ce60 con 0x7f20c4103370 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c41a0c60 con 0x7f20c4103370 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20c27fc640 1 -- 172.21.15.120:0/514800109 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc010430 con 0x7f20c4103370 2024-09-10T23:03:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.716+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c41a1160 con 0x7f20c4103370 2024-09-10T23:03:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 Processor -- start 2024-09-10T23:03:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c27fc640 1 -- 172.21.15.120:0/514800109 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f20bc004ff0 con 0x7f20c4103370 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 -- start start 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ced1196f0 con 0x564ced1aa190 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ced108b20 con 0x564ced1acbd0 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ced10c930 con 0x564ced1a7860 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c94b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:33844/0 (socket says 172.21.15.120:33844) 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c8cb2640 1 -- 172.21.15.120:0/743958598 learned_addr learned my addr 172.21.15.120:0/743958598 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.719+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 -- 172.21.15.120:0/743958598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 msgr2=0x564ced1a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 -- 172.21.15.120:0/743958598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 msgr2=0x564ced1ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 -- 172.21.15.120:0/743958598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ced0ce530 con 0x564ced1acbd0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c27fc640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 0x7f20ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20b8011760 tx=0x7f20b800a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b13df6bff2c169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c94b3640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 0x7f20ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20ab7fe640 1 -- 172.21.15.120:0/743958598 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8004020 con 0x564ced1acbd0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20ab7fe640 1 -- 172.21.15.120:0/743958598 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b80054d0 con 0x564ced1acbd0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20ab7fe640 1 -- 172.21.15.120:0/743958598 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8004a80 con 0x564ced1acbd0 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.720+0000 7f20c27fc640 1 -- 172.21.15.120:0/514800109 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f20bc0e5f80 con 0x7f20c4103370 2024-09-10T23:03:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- 172.21.15.120:0/743958598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 msgr2=0x564ced1af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20c94b3640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 0x7f20ac0b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20b4031780 tx=0x7f20b403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20b8011760 tx=0x7f20b800a6e0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- 172.21.15.120:0/743958598 shutdown_connections 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1af090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/743958598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- 172.21.15.120:0/743958598 >> 172.21.15.120:0/743958598 conn(0x564ced1a29b0 msgr2=0x564ced1a3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- 172.21.15.120:0/743958598 shutdown_connections 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- 172.21.15.120:0/743958598 wait complete. 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 -- start start 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.721+0000 7f20ce5c0740 1 Processor -- start 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 0x564ced1b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 0x564ced1b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ced040ab0 con 0x564ced1a7860 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ced0ae730 con 0x564ced1aa190 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ced1b3fc0 con 0x564ced1acbd0 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20ce5c0740 10 client.0 _ll_register_callbacks cb 0x564ced1026b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38548/0 (socket says 172.21.15.120:38548) 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 -- 172.21.15.120:0/311494545 learned_addr learned my addr 172.21.15.120:0/311494545 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 0x564ced1b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 0x564ced1b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 -- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 msgr2=0x564ced1b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 0x564ced1b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 -- 172.21.15.120:0/311494545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 msgr2=0x564ced1b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 0x564ced1b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 -- 172.21.15.120:0/311494545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cece6a790 con 0x564ced1a7860 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 0x564ced1b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 0x564ced1b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20c94b3640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009cd0 tx=0x7f20b402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84767adb6b1fe7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4056060 con 0x564ced1a7860 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.722+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f20b40305a0 con 0x564ced1a7860 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b80054d0 con 0x564ced1a7860 2024-09-10T23:03:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20b40307a0 con 0x564ced1a7860 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 10 client.17622 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cecee3620 con 0x564ced1a7860 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b80054d0 con 0x564ced1a7860 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f20b4030070 con 0x564ced1a7860 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 10 client.17622 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 20 client.17622 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced1c0ba0 0x564ced1c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 4 client.17622 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ced11b640 con 0x564ced1c0ba0 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.723+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced1c0ba0 0x564ced1c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.724+0000 7f20a8ff9640 10 client.17622 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.724+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced1c0ba0 0x564ced1c3020 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4838e309fca10f92 server_cookie=f4208d2939a8d70c in_seq=0 out_seq=0 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.725+0000 7f20a8ff9640 1 -- 172.21.15.120:0/311494545 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ced11b640 con 0x564ced1c0ba0 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.725+0000 7f20a8ff9640 10 client.17622 handle_command_reply: tid=0 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.725+0000 7f20ce5c0740 1 client.17622 shutdown 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 msgr2=0x564ced1b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009cd0 tx=0x7f20b402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 shutdown_connections 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced1c0ba0 0x564ced1c3020 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1acbd0 0x564ced1b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20a8ff9640 0 client.17622 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1aa190 0x564ced1b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/311494545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1a7860 0x564ced1b5360 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 >> 172.21.15.120:0/311494545 conn(0x564ced1a29b0 msgr2=0x564ced1a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 shutdown_connections 2024-09-10T23:03:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 1 -- 172.21.15.120:0/311494545 wait complete. 2024-09-10T23:03:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.726+0000 7f20ce5c0740 20 client.17622 trim_cache size 0 max 16384 2024-09-10T23:03:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.740+0000 7f20ce5c0740 1 Processor -- start 2024-09-10T23:03:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.740+0000 7f20ce5c0740 1 -- start start 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a5c20 con 0x564ced1aa190 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a8480 con 0x564ced1acbd0 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a6270 con 0x564ced1a7860 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38552/0 (socket says 172.21.15.120:38552) 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c8cb2640 1 -- 172.21.15.120:0/307253715 learned_addr learned my addr 172.21.15.120:0/307253715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 -- 172.21.15.120:0/307253715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 msgr2=0x564ced1a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 -- 172.21.15.120:0/307253715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 msgr2=0x564ced1ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 -- 172.21.15.120:0/307253715 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ced19e2d0 con 0x564ced1a7860 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20c94b3640 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1ad000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20b4033040 tx=0x7f20b4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d4dc1b642c1bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.741+0000 7f20aaffd640 1 -- 172.21.15.120:0/307253715 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4062030 con 0x564ced1a7860 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20aaffd640 1 -- 172.21.15.120:0/307253715 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b4036040 con 0x564ced1a7860 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20aaffd640 1 -- 172.21.15.120:0/307253715 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4056070 con 0x564ced1a7860 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 -- 172.21.15.120:0/307253715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 msgr2=0x564ced1ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1ad000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20b4033040 tx=0x7f20b4005b50 comp rx=0 tx=0).stop 2024-09-10T23:03:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 -- 172.21.15.120:0/307253715 shutdown_connections 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/307253715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1ad000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 -- 172.21.15.120:0/307253715 >> 172.21.15.120:0/307253715 conn(0x564ced1a29b0 msgr2=0x564ced19daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 -- 172.21.15.120:0/307253715 shutdown_connections 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.742+0000 7f20ce5c0740 1 -- 172.21.15.120:0/307253715 wait complete. 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 -- start start 2024-09-10T23:03:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 Processor -- start 2024-09-10T23:03:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a6940 con 0x564ced1aa190 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a7db0 con 0x564ced1acbd0 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41a7010 con 0x564ced1a7860 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20ce5c0740 10 client.0 _ll_register_callbacks cb 0x7f20b4031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20c8cb2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38568/0 (socket says 172.21.15.120:38568) 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20c8cb2640 1 -- 172.21.15.120:0/3736023266 learned_addr learned my addr 172.21.15.120:0/3736023266 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20c94b3640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.743+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 -- 172.21.15.120:0/3736023266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 msgr2=0x564ced1bbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1bbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 -- 172.21.15.120:0/3736023266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 msgr2=0x564ced1bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 -- 172.21.15.120:0/3736023266 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ced1bc2f0 con 0x564ced1a7860 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c9cb4640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20c94b3640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1bcb70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20b40305e0 tx=0x7f20b4030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d718c474d9b415f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4062070 con 0x564ced1a7860 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f20b40514c0 con 0x564ced1a7860 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b4036040 con 0x564ced1a7860 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20b406e070 con 0x564ced1a7860 2024-09-10T23:03:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 10 client.27103 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ced1bb160 con 0x564ced1a7860 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b4036040 con 0x564ced1a7860 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f20b4051b70 con 0x564ced1a7860 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 10 client.27103 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 10 client.27103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.744+0000 7f20ce5c0740 20 client.27103 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced22e140 0x564ced230560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20ce5c0740 4 client.27103 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ced1170f0 con 0x564ced22e140 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced22e140 0x564ced230560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20c8cb2640 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced22e140 0x564ced230560 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e72a261163b45fb server_cookie=1382faeeaa44ea1b in_seq=0 out_seq=0 2024-09-10T23:03:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.745+0000 7f20a8ff9640 10 client.27103 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:31.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:31 smithi176 ceph-mon[34982]: pgmap v679: 97 pgs: 97 active+clean; 36 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 483 op/s 2024-09-10T23:03:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20a8ff9640 1 -- 172.21.15.120:0/3736023266 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ced1170f0 con 0x564ced22e140 2024-09-10T23:03:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20a8ff9640 10 client.27103 handle_command_reply: tid=0 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 client.27103 shutdown 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 msgr2=0x564ced1bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1bcb70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20b40305e0 tx=0x7f20b4030b70 comp rx=0 tx=0).stop 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 shutdown_connections 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ced22e140 0x564ced230560 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ced1acbd0 0x564ced1bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20a8ff9640 0 client.27103 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ced1aa190 0x564ced1bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 --2- 172.21.15.120:0/3736023266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ced1a7860 0x564ced1bcb70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 >> 172.21.15.120:0/3736023266 conn(0x564ced1a29b0 msgr2=0x564ced1b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 shutdown_connections 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 1 -- 172.21.15.120:0/3736023266 wait complete. 2024-09-10T23:03:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20ce5c0740 20 client.27103 trim_cache size 0 max 16384 2024-09-10T23:03:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:03:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:03:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:03:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:03:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 msgr2=0x7f20ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 0x7f20ac0b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20b4031780 tx=0x7f20b403d040 comp rx=0 tx=0).stop 2024-09-10T23:03:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 msgr2=0x7f20c41a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.976+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e970 tx=0x7f20bc00ed70 comp rx=0 tx=0).stop 2024-09-10T23:03:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 shutdown_connections 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20ac0b52e0 0x7f20ac0b77a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20c4103370 0x7f20c41a00b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20c41029b0 0x7f20c419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 --2- 172.21.15.120:0/514800109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20c4101860 0x7f20c406b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 >> 172.21.15.120:0/514800109 conn(0x7f20c40fd140 msgr2=0x7f20c4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 shutdown_connections 2024-09-10T23:03:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:31.977+0000 7f20cb717640 1 -- 172.21.15.120:0/514800109 wait complete. 2024-09-10T23:03:32.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1879955 15.99 MB/sec execute 507 sec latency 128.625 ms 2024-09-10T23:03:33.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1883326 15.99 MB/sec execute 508 sec latency 11.900 ms 2024-09-10T23:03:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:33 smithi120 ceph-mon[31247]: pgmap v680: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-10T23:03:33.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:33 smithi153 ceph-mon[34951]: pgmap v680: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-10T23:03:33.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:33 smithi176 ceph-mon[34982]: pgmap v680: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.4 MiB/s wr, 306 op/s 2024-09-10T23:03:34.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1886428 15.99 MB/sec execute 509 sec latency 36.744 ms 2024-09-10T23:03:35.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1889189 15.98 MB/sec execute 510 sec latency 41.604 ms 2024-09-10T23:03:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:35 smithi120 ceph-mon[31247]: pgmap v681: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-10T23:03:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:35 smithi153 ceph-mon[34951]: pgmap v681: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-10T23:03:35.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:35 smithi176 ceph-mon[34982]: pgmap v681: 97 pgs: 97 active+clean; 35 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-10T23:03:36.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1891326 15.98 MB/sec execute 511 sec latency 45.779 ms 2024-09-10T23:03:37.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1894443 15.98 MB/sec execute 512 sec latency 12.526 ms 2024-09-10T23:03:37.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:37 smithi120 ceph-mon[31247]: pgmap v682: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T23:03:37.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:37 smithi153 ceph-mon[34951]: pgmap v682: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T23:03:37.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:37 smithi176 ceph-mon[34982]: pgmap v682: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T23:03:38.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1897622 15.99 MB/sec execute 513 sec latency 26.033 ms 2024-09-10T23:03:39.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1900815 15.99 MB/sec execute 514 sec latency 15.341 ms 2024-09-10T23:03:39.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:39 smithi120 ceph-mon[31247]: pgmap v683: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:03:39.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:39 smithi153 ceph-mon[34951]: pgmap v683: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:03:39.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:39 smithi176 ceph-mon[34982]: pgmap v683: 97 pgs: 97 active+clean; 33 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.4 MiB/s wr, 362 op/s 2024-09-10T23:03:40.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1903349 15.98 MB/sec execute 515 sec latency 20.953 ms 2024-09-10T23:03:41.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1905165 15.97 MB/sec execute 516 sec latency 77.305 ms 2024-09-10T23:03:41.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:41 smithi120 ceph-mon[31247]: pgmap v684: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T23:03:41.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:41 smithi153 ceph-mon[34951]: pgmap v684: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T23:03:41.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:41 smithi176 ceph-mon[34982]: pgmap v684: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-09-10T23:03:42.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1908382 15.97 MB/sec execute 517 sec latency 7.441 ms 2024-09-10T23:03:43.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1911685 15.98 MB/sec execute 518 sec latency 20.058 ms 2024-09-10T23:03:43.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:43 smithi120 ceph-mon[31247]: pgmap v685: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T23:03:43.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:43 smithi153 ceph-mon[34951]: pgmap v685: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T23:03:43.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:43 smithi176 ceph-mon[34982]: pgmap v685: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.7 MiB/s wr, 385 op/s 2024-09-10T23:03:44.381 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:03:44.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1914628 15.97 MB/sec execute 519 sec latency 37.078 ms 2024-09-10T23:03:44.668 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 -- 172.21.15.120:0/2205839406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a01029b0 msgr2=0x7ff2a0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/2205839406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a01029b0 0x7ff2a0102e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff294009930 tx=0x7ff29402a420 comp rx=0 tx=0).stop 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 -- 172.21.15.120:0/2205839406 shutdown_connections 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/2205839406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/2205839406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a01029b0 0x7ff2a0102e30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/2205839406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a0101860 0x7ff2a0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.068+0000 7ff2a7d52640 1 -- 172.21.15.120:0/2205839406 >> 172.21.15.120:0/2205839406 conn(0x7ff2a00fd140 msgr2=0x7ff2a00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- 172.21.15.120:0/2205839406 shutdown_connections 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- 172.21.15.120:0/2205839406 wait complete. 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 Processor -- start 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- start start 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a0101860 0x7ff2a0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a01029b0 0x7ff2a0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010d4d0 con 0x7ff2a0103370 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010d350 con 0x7ff2a01029b0 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a7d52640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010d650 con 0x7ff2a0101860 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.069+0000 7ff2a62ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a0101860 0x7ff2a0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51594/0 (socket says 172.21.15.120:51594) 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 -- 172.21.15.120:0/1604279555 learned_addr learned my addr 172.21.15.120:0/1604279555 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a01029b0 0x7ff2a0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 -- 172.21.15.120:0/1604279555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a0101860 msgr2=0x7ff2a0198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a0101860 0x7ff2a0198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 -- 172.21.15.120:0/1604279555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a01029b0 msgr2=0x7ff2a0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a01029b0 0x7ff2a0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 -- 172.21.15.120:0/1604279555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a019a920 con 0x7ff2a0103370 2024-09-10T23:03:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff29000efd0 tx=0x7ff29000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff29effd640 1 -- 172.21.15.120:0/1604279555 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290003b70 con 0x7ff2a0103370 2024-09-10T23:03:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a019ac10 con 0x7ff2a0103370 2024-09-10T23:03:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff29effd640 1 -- 172.21.15.120:0/1604279555 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff290003d10 con 0x7ff2a0103370 2024-09-10T23:03:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff29effd640 1 -- 172.21.15.120:0/1604279555 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900108a0 con 0x7ff2a0103370 2024-09-10T23:03:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.070+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01a12e0 con 0x7ff2a0103370 2024-09-10T23:03:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 Processor -- start 2024-09-10T23:03:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff29effd640 1 -- 172.21.15.120:0/1604279555 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff29001f020 con 0x7ff2a0103370 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 -- start start 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddf4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddf94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed8dd5f6f0 con 0x55ed8ddf6fe0 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed8dd4eb20 con 0x55ed8ddf45a0 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ed8dd52930 con 0x55ed8ddf1c70 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddf94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54940/0 (socket says 172.21.15.120:54940) 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddf4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.073+0000 7ff2a5aee640 1 -- 172.21.15.120:0/1233249837 learned_addr learned my addr 172.21.15.120:0/1233249837 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 -- 172.21.15.120:0/1233249837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 msgr2=0x55ed8ddf4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddf4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 -- 172.21.15.120:0/1233249837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 msgr2=0x55ed8ddf94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddf94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 -- 172.21.15.120:0/1233249837 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8dd14530 con 0x55ed8ddf45a0 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff29effd640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 0x7ff2880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 0x7ff2880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff298009d20 tx=0x7ff298011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce8247d05c7d89e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff29effd640 1 -- 172.21.15.120:0/1604279555 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff2900e5c80 con 0x7ff2a0103370 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff287fff640 1 -- 172.21.15.120:0/1233249837 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29801e070 con 0x55ed8ddf45a0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff287fff640 1 -- 172.21.15.120:0/1233249837 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298009e10 con 0x55ed8ddf45a0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.074+0000 7ff287fff640 1 -- 172.21.15.120:0/1233249837 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298018780 con 0x55ed8ddf45a0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1233249837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 msgr2=0x55ed8ddf6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff298009d20 tx=0x7ff298011810 comp rx=0 tx=0).stop 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 0x7ff2880b77a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff294000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1233249837 shutdown_connections 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddf94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddf6990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1233249837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddf4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1233249837 >> 172.21.15.120:0/1233249837 conn(0x55ed8dde89b0 msgr2=0x55ed8dde9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1233249837 shutdown_connections 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.075+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1233249837 wait complete. 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 -- start start 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 Processor -- start 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddfb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ed8dc86ab0 con 0x55ed8ddf6fe0 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ed8dcf4730 con 0x55ed8ddf1c70 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ed8ddf9fc0 con 0x55ed8ddf45a0 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2aabfc740 10 client.0 _ll_register_callbacks cb 0x55ed8dd486b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a62ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddfb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34924/0 (socket says 172.21.15.120:34924) 2024-09-10T23:03:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a52ed640 1 -- 172.21.15.120:0/1359162398 learned_addr learned my addr 172.21.15.120:0/1359162398 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.076+0000 7ff2a62ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51618/0 (socket says 172.21.15.120:51618) 2024-09-10T23:03:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 -- 172.21.15.120:0/1359162398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 msgr2=0x55ed8ddfb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddfb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 -- 172.21.15.120:0/1359162398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 msgr2=0x55ed8ddfc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 -- 172.21.15.120:0/1359162398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8dab0790 con 0x55ed8ddf45a0 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddfb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff294033000 tx=0x7ff29402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0ceb14180a9193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2857fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294059060 con 0x55ed8ddf45a0 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff298009e10 con 0x55ed8ddf45a0 2024-09-10T23:03:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff294046030 con 0x55ed8ddf45a0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff294035040 con 0x55ed8ddf45a0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2aabfc740 10 client.27115 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed8db29620 con 0x55ed8ddf45a0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.077+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff298009e10 con 0x55ed8ddf45a0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff29404a820 con 0x55ed8ddf45a0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 10 client.27115 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 10 client.27115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 20 client.27115 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 4 client.27115 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8dd61640 con 0x55ed8de06bd0 2024-09-10T23:03:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2857fa640 10 client.27115 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.078+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de08fc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a354244b45b958 server_cookie=1a36846579829261 in_seq=0 out_seq=0 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2857fa640 1 -- 172.21.15.120:0/1359162398 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed8dd61640 con 0x55ed8de06bd0 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2857fa640 10 client.27115 handle_command_reply: tid=0 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 client.27115 shutdown 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 msgr2=0x55ed8ddfb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff294033000 tx=0x7ff29402a420 comp rx=0 tx=0).stop 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 shutdown_connections 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de08fc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddfb970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2857fa640 0 client.27115 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/1359162398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddfb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 >> 172.21.15.120:0/1359162398 conn(0x55ed8dde89b0 msgr2=0x55ed8dde7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 shutdown_connections 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 1 -- 172.21.15.120:0/1359162398 wait complete. 2024-09-10T23:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.079+0000 7ff2aabfc740 20 client.27115 trim_cache size 0 max 16384 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 Processor -- start 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 -- start start 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddeba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a5c40 con 0x55ed8ddf45a0 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a84a0 con 0x55ed8ddf6fe0 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a6290 con 0x55ed8ddf1c70 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddeba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51620/0 (socket says 172.21.15.120:51620) 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 -- 172.21.15.120:0/4033269312 learned_addr learned my addr 172.21.15.120:0/4033269312 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 -- 172.21.15.120:0/4033269312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 msgr2=0x55ed8ddeba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddeba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 -- 172.21.15.120:0/4033269312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 msgr2=0x55ed8ddfa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a52ed640 1 -- 172.21.15.120:0/4033269312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8ddfc8b0 con 0x55ed8ddf45a0 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.094+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff294009230 tx=0x7ff2940319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f7f0cd23e52d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2877fe640 1 -- 172.21.15.120:0/4033269312 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294064070 con 0x55ed8ddf45a0 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2877fe640 1 -- 172.21.15.120:0/4033269312 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29404a8f0 con 0x55ed8ddf45a0 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2877fe640 1 -- 172.21.15.120:0/4033269312 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294059040 con 0x55ed8ddf45a0 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 -- 172.21.15.120:0/4033269312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 msgr2=0x55ed8ddebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff294009230 tx=0x7ff2940319c0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 -- 172.21.15.120:0/4033269312 shutdown_connections 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf6fe0 0x55ed8ddfa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf45a0 0x55ed8ddebf40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/4033269312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf1c70 0x55ed8ddeba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 -- 172.21.15.120:0/4033269312 >> 172.21.15.120:0/4033269312 conn(0x55ed8dde89b0 msgr2=0x55ed8dde9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 -- 172.21.15.120:0/4033269312 shutdown_connections 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.095+0000 7ff2aabfc740 1 -- 172.21.15.120:0/4033269312 wait complete. 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 -- start start 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 Processor -- start 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8de00890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf6fe0 0x55ed8de013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a6960 con 0x55ed8ddf1c70 2024-09-10T23:03:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a7dd0 con 0x55ed8ddf45a0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a7030 con 0x55ed8ddf6fe0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2aabfc740 10 client.0 _ll_register_callbacks cb 0x7ff294005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51624/0 (socket says 172.21.15.120:51624) 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 -- 172.21.15.120:0/3562718669 learned_addr learned my addr 172.21.15.120:0/3562718669 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a62ef640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf6fe0 0x55ed8de013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8de00890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 -- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf6fe0 msgr2=0x55ed8de013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf6fe0 0x55ed8de013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 -- 172.21.15.120:0/3562718669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 msgr2=0x55ed8de00890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8de00890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a5aee640 1 -- 172.21.15.120:0/3562718669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8de01bf0 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.096+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8de00890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2a5aee640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff298013010 tx=0x7ff298009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d90b45b585e49a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298031070 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff298009e10 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff29404a8f0 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff29801e040 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 10 client.17646 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed8de02f40 con 0x55ed8ddf1c70 2024-09-10T23:03:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff29404a8f0 con 0x55ed8ddf1c70 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff298034020 con 0x55ed8ddf1c70 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 10 client.17646 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 20 client.17646 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 4 client.17646 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed8dd5d0f0 con 0x55ed8de06bd0 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.097+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.098+0000 7ff2857fa640 10 client.17646 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.098+0000 7ff2a52ed640 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de75c30 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea9664f0c53a92e server_cookie=51e34316f707843e in_seq=0 out_seq=0 2024-09-10T23:03:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2857fa640 1 -- 172.21.15.120:0/3562718669 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed8dd5d0f0 con 0x55ed8de06bd0 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2857fa640 10 client.17646 handle_command_reply: tid=0 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 client.17646 shutdown 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 msgr2=0x55ed8de001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff298013010 tx=0x7ff298009620 comp rx=0 tx=0).stop 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 shutdown_connections 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ed8de06bd0 0x55ed8de75c30 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed8ddf6fe0 0x55ed8de013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed8ddf45a0 0x55ed8de00890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 --2- 172.21.15.120:0/3562718669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ed8ddf1c70 0x55ed8de001f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 >> 172.21.15.120:0/3562718669 conn(0x55ed8dde89b0 msgr2=0x55ed8ddfa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2857fa640 0 client.17646 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 shutdown_connections 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 1 -- 172.21.15.120:0/3562718669 wait complete. 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.412+0000 7ff2aabfc740 20 client.17646 trim_cache size 0 max 16384 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:03:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:03:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 msgr2=0x7ff2880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 0x7ff2880b77a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff294000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 msgr2=0x7ff2a019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff29000efd0 tx=0x7ff29000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 shutdown_connections 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff2880b52e0 0x7ff2880b77a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2a0103370 0x7ff2a019a060 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2a01029b0 0x7ff2a0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 --2- 172.21.15.120:0/1604279555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a0101860 0x7ff2a0198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 >> 172.21.15.120:0/1604279555 conn(0x7ff2a00fd140 msgr2=0x7ff2a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 shutdown_connections 2024-09-10T23:03:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:45.413+0000 7ff2a7d52640 1 -- 172.21.15.120:0/1604279555 wait complete. 2024-09-10T23:03:45.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 -- 172.21.15.120:0/684386020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 msgr2=0x7f162010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 --2- 172.21.15.120:0/684386020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f162010cb50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1614009900 tx=0x7f161402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 -- 172.21.15.120:0/684386020 shutdown_connections 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 --2- 172.21.15.120:0/684386020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 0x7f162010f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 --2- 172.21.15.120:0/684386020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f162010cb50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 --2- 172.21.15.120:0/684386020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f1620103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 -- 172.21.15.120:0/684386020 >> 172.21.15.120:0/684386020 conn(0x7f16200fd740 msgr2=0x7f16200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 -- 172.21.15.120:0/684386020 shutdown_connections 2024-09-10T23:03:45.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.453+0000 7f1624b73640 1 -- 172.21.15.120:0/684386020 wait complete. 2024-09-10T23:03:45.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.454+0000 7f1624b73640 1 Processor -- start 2024-09-10T23:03:45.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.454+0000 7f1624b73640 1 -- start start 2024-09-10T23:03:45.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f16201992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 0x7f162019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1620112df0 con 0x7f1620104570 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1620112c70 con 0x7f162010d120 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f1624b73640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1620112f70 con 0x7f1620103b70 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f16201992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51632/0 (socket says 172.21.15.120:51632) 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 -- 172.21.15.120:0/1222781372 learned_addr learned my addr 172.21.15.120:0/1222781372 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161ed9d640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 0x7f162019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 -- 172.21.15.120:0/1222781372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 msgr2=0x7f16201992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f16201992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 -- 172.21.15.120:0/1222781372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 msgr2=0x7f162019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 0x7f162019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161dd9b640 1 -- 172.21.15.120:0/1222781372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162019f350 con 0x7f1620104570 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.455+0000 7f161e59c640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f16201992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.456+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1614005ab0 tx=0x7f1614003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.456+0000 7f160f7fe640 1 -- 172.21.15.120:0/1222781372 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161403f070 con 0x7f1620104570 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.456+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162019f5e0 con 0x7f1620104570 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.456+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201a5d00 con 0x7f1620104570 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.457+0000 7f160f7fe640 1 -- 172.21.15.120:0/1222781372 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1614031b60 con 0x7f1620104570 2024-09-10T23:03:45.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.457+0000 7f160f7fe640 1 -- 172.21.15.120:0/1222781372 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614035030 con 0x7f1620104570 2024-09-10T23:03:45.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 Processor -- start 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 -- start start 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9d00a0 0x5600ff9d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9dc950 0x5600ff9ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600ff946e50 con 0x5600ff9d00a0 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600ff936280 con 0x5600ff9d09c0 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f1626f48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600ff93a090 con 0x5600ff9dc950 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f161e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.460+0000 7f161e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54988/0 (socket says 172.21.15.120:54988) 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 -- 172.21.15.120:0/2576660743 learned_addr learned my addr 172.21.15.120:0/2576660743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161ed9d640 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9d00a0 0x5600ff9d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 -- 172.21.15.120:0/2576660743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9dc950 msgr2=0x5600ff9ded40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9dc950 0x5600ff9ded40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 -- 172.21.15.120:0/2576660743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9d00a0 msgr2=0x5600ff9d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9d00a0 0x5600ff9d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 -- 172.21.15.120:0/2576660743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ff694650 con 0x5600ff9d09c0 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f160f7fe640 1 -- 172.21.15.120:0/1222781372 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f16140316b0 con 0x7f1620104570 2024-09-10T23:03:45.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f161e59c640 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16100115f0 tx=0x7f1610011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71784093d5d0236b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f160cff9640 1 -- 172.21.15.120:0/2576660743 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161001e070 con 0x5600ff9d09c0 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f160cff9640 1 -- 172.21.15.120:0/2576660743 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161000faf0 con 0x5600ff9d09c0 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.461+0000 7f160cff9640 1 -- 172.21.15.120:0/2576660743 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610019bc0 con 0x5600ff9d09c0 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.462+0000 7f160f7fe640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 0x7f16080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.462+0000 7f161ed9d640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 0x7f16080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.462+0000 7f160f7fe640 1 -- 172.21.15.120:0/1222781372 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1614106510 con 0x7f1620104570 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- 172.21.15.120:0/2576660743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 msgr2=0x5600ff9dc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16100115f0 tx=0x7f1610011ab0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f161ed9d640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 0x7f16080b7820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- 172.21.15.120:0/2576660743 shutdown_connections 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9dc950 0x5600ff9ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d09c0 0x5600ff9dc410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 --2- 172.21.15.120:0/2576660743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9d00a0 0x5600ff9d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- 172.21.15.120:0/2576660743 >> 172.21.15.120:0/2576660743 conn(0x5600ff9cded0 msgr2=0x5600ff9cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- 172.21.15.120:0/2576660743 shutdown_connections 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- 172.21.15.120:0/2576660743 wait complete. 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.463+0000 7f1626f48740 1 -- start start 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 Processor -- start 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5600ff797700 con 0x5600ff9dc950 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600ff8d1110 con 0x5600ff9d00a0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600ff9df5d0 con 0x5600ff9d09c0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f1626f48740 10 client.0 _ll_register_callbacks cb 0x5600ff8ea990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34976/0 (socket says 172.21.15.120:34976) 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55000/0 (socket says 172.21.15.120:55000) 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51646/0 (socket says 172.21.15.120:51646) 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.464+0000 7f161dd9b640 1 -- 172.21.15.120:0/2841838559 learned_addr learned my addr 172.21.15.120:0/2841838559 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 -- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 msgr2=0x5600ff9d9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 -- 172.21.15.120:0/2841838559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 msgr2=0x5600ff9dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 -- 172.21.15.120:0/2841838559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ff8fec40 con 0x5600ff9dc950 2024-09-10T23:03:45.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161dd9b640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:45.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161e59c640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f161ed9d640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1618012d70 tx=0x7f1618004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fe96a18caee7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f1605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004b20 con 0x5600ff9dc950 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1618018030 con 0x5600ff9dc950 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.465+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f161000faf0 con 0x5600ff9dc950 2024-09-10T23:03:45.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1618004cc0 con 0x5600ff9dc950 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 10 client.17658 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600ff9eb040 con 0x5600ff9dc950 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161000faf0 con 0x5600ff9dc950 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1618035680 con 0x5600ff9dc950 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 10 client.17658 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 20 client.17658 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9ec510 0x5600ff9ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 4 client.17658 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.466+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ff948da0 con 0x5600ff9ec510 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.467+0000 7f161e59c640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9ec510 0x5600ff9ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.467+0000 7f161e59c640 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9ec510 0x5600ff9ee920 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df52004303da1393 server_cookie=2f77a000ef757747 in_seq=0 out_seq=0 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.467+0000 7f1605ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1605ffb640 1 -- 172.21.15.120:0/2841838559 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600ff948da0 con 0x5600ff9ec510 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1605ffb640 10 client.17658 handle_command_reply: tid=0 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 client.17658 shutdown 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 msgr2=0x5600ff9d9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1618012d70 tx=0x7f1618004ab0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 shutdown_connections 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9ec510 0x5600ff9ee920 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1605ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d9df0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9d9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 --2- 172.21.15.120:0/2841838559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 >> 172.21.15.120:0/2841838559 conn(0x5600ff9cded0 msgr2=0x5600ff9d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 shutdown_connections 2024-09-10T23:03:45.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.468+0000 7f1626f48740 1 -- 172.21.15.120:0/2841838559 wait complete. 2024-09-10T23:03:45.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.469+0000 7f1626f48740 20 client.17658 trim_cache size 0 max 16384 2024-09-10T23:03:45.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.488+0000 7f1626f48740 1 Processor -- start 2024-09-10T23:03:45.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 -- start start 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16201aa0f0 con 0x5600ff9dc950 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ac950 con 0x5600ff9d00a0 2024-09-10T23:03:45.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f1626f48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16201aa740 con 0x5600ff9d09c0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34986/0 (socket says 172.21.15.120:34986) 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f161dd9b640 1 -- 172.21.15.120:0/1428901501 learned_addr learned my addr 172.21.15.120:0/1428901501 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.489+0000 7f161ed9d640 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 -- 172.21.15.120:0/1428901501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 msgr2=0x5600ff9cf000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 -- 172.21.15.120:0/1428901501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 msgr2=0x5600ff9db480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9db480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 -- 172.21.15.120:0/1428901501 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ff9c9790 con 0x5600ff9d09c0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161ed9d640 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9db480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f161dd9b640 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1614104090 tx=0x7f16140045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cdb2371bacad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f1607fff640 1 -- 172.21.15.120:0/1428901501 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614049040 con 0x5600ff9d09c0 2024-09-10T23:03:45.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f1607fff640 1 -- 172.21.15.120:0/1428901501 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16140cc810 con 0x5600ff9d09c0 2024-09-10T23:03:45.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.490+0000 7f1607fff640 1 -- 172.21.15.120:0/1428901501 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140d6290 con 0x5600ff9d09c0 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 -- 172.21.15.120:0/1428901501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 msgr2=0x5600ff9cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1614104090 tx=0x7f16140045e0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 -- 172.21.15.120:0/1428901501 shutdown_connections 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9db480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9cd710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 --2- 172.21.15.120:0/1428901501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.491+0000 7f1626f48740 1 -- 172.21.15.120:0/1428901501 >> 172.21.15.120:0/1428901501 conn(0x5600ff9cded0 msgr2=0x5600ff9dad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.492+0000 7f1626f48740 1 -- 172.21.15.120:0/1428901501 shutdown_connections 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.492+0000 7f1626f48740 1 -- 172.21.15.120:0/1428901501 wait complete. 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.492+0000 7f1626f48740 1 -- start start 2024-09-10T23:03:45.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.492+0000 7f1626f48740 1 Processor -- start 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16201aae10 con 0x5600ff9dc950 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ac280 con 0x5600ff9d00a0 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16201ab4e0 con 0x5600ff9d09c0 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34996/0 (socket says 172.21.15.120:34996) 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 -- 172.21.15.120:0/3345856335 learned_addr learned my addr 172.21.15.120:0/3345856335 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f1626f48740 10 client.0 _ll_register_callbacks cb 0x7f1618009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161ed9d640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161e59c640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 -- 172.21.15.120:0/3345856335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 msgr2=0x5600ff9e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 -- 172.21.15.120:0/3345856335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 msgr2=0x5600ff9d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161dd9b640 1 -- 172.21.15.120:0/3345856335 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ff9e5410 con 0x5600ff9d09c0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.493+0000 7f161ed9d640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f161e59c640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f161dd9b640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1614005c20 tx=0x7f16140ccdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65d818d3055f5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614049070 con 0x5600ff9d09c0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f16140d6000 con 0x5600ff9d09c0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16140cc810 con 0x5600ff9d09c0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16140cc470 con 0x5600ff9d09c0 2024-09-10T23:03:45.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1626f48740 10 client.27136 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:45.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.494+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600ff9d6da0 con 0x5600ff9d09c0 2024-09-10T23:03:45.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16140cd400 con 0x5600ff9d09c0 2024-09-10T23:03:45.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f16140d4bf0 con 0x5600ff9d09c0 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1626f48740 10 client.27136 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1626f48740 10 client.27136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1626f48740 20 client.27136 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.495+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9d7430 0x5600ffa5c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.496+0000 7f1626f48740 4 client.27136 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.496+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f16140cc810 con 0x5600ff9d7430 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.496+0000 7f161e59c640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9d7430 0x5600ffa5c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.496+0000 7f161e59c640 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9d7430 0x5600ffa5c410 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1446871bf3c5092 server_cookie=e133cecc9bd025c0 in_seq=0 out_seq=0 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.496+0000 7f1605ffb640 10 client.27136 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1605ffb640 1 -- 172.21.15.120:0/3345856335 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x7f16140cc810 con 0x5600ff9d7430 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1605ffb640 10 client.27136 handle_command_reply: tid=0 2024-09-10T23:03:45.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 client.27136 shutdown 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 msgr2=0x5600ff9e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1614005c20 tx=0x7f16140ccdd0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 shutdown_connections 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600ff9d7430 0x5600ffa5c410 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600ff9dc950 0x5600ff9d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600ff9d09c0 0x5600ff9e4dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 --2- 172.21.15.120:0/3345856335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600ff9d00a0 0x5600ff9e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 >> 172.21.15.120:0/3345856335 conn(0x5600ff9cded0 msgr2=0x5600ff9db590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1605ffb640 0 client.27136 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 shutdown_connections 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 1 -- 172.21.15.120:0/3345856335 wait complete. 2024-09-10T23:03:45.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.498+0000 7f1626f48740 20 client.27136 trim_cache size 0 max 16384 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:03:45.504 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:03:45.506 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:03:45.509 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.510 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:03:45.513 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:03:45.514 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:45.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:45.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:45.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:45.516 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:45.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.499+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 msgr2=0x7f16080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.499+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 0x7f16080b7820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618009500 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.499+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 msgr2=0x7f16201997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.499+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1614005ab0 tx=0x7f1614003c10 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 shutdown_connections 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16080b5360 0x7f16080b7820 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f162010d120 0x7f162019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1620104570 0x7f16201997e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 --2- 172.21.15.120:0/1222781372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1620103b70 0x7f16201992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 >> 172.21.15.120:0/1222781372 conn(0x7f16200fd740 msgr2=0x7f16200fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 shutdown_connections 2024-09-10T23:03:45.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:45.500+0000 7f1624b73640 1 -- 172.21.15.120:0/1222781372 wait complete. 2024-09-10T23:03:45.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1915955 15.96 MB/sec execute 520 sec latency 187.543 ms 2024-09-10T23:03:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:45 smithi120 ceph-mon[31247]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-09-10T23:03:45.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:45 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:45 smithi176 ceph-mon[34982]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-09-10T23:03:45.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:45 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:45.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:45 smithi153 ceph-mon[34951]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-09-10T23:03:45.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:45 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:46.305 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:03:46.305 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:46.620 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:46.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1918777 15.96 MB/sec execute 521 sec latency 103.900 ms 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:46.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:46.894 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:46.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:03:47.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 -- 172.21.15.120:0/647917807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818101c90 msgr2=0x7f6818102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 --2- 172.21.15.120:0/647917807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818101c90 0x7f6818102070 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6808009900 tx=0x7f680802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 -- 172.21.15.120:0/647917807 shutdown_connections 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 --2- 172.21.15.120:0/647917807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818108020 0x7f681810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 --2- 172.21.15.120:0/647917807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68181076a0 0x7f6818107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 --2- 172.21.15.120:0/647917807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818101c90 0x7f6818102070 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 -- 172.21.15.120:0/647917807 >> 172.21.15.120:0/647917807 conn(0x7f68180fd700 msgr2=0x7f68180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 -- 172.21.15.120:0/647917807 shutdown_connections 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.139+0000 7f681cbd2640 1 -- 172.21.15.120:0/647917807 wait complete. 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 Processor -- start 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 -- start start 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:47.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68181076a0 0x7f681819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 0x7f681819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6818111ca0 con 0x7f6818108020 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6818111b20 con 0x7f68181076a0 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681cbd2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6818111e20 con 0x7f6818101c90 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.140+0000 7f681659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 0x7f681819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:47.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f681659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35022/0 (socket says 172.21.15.120:35022) 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f681659c640 1 -- 172.21.15.120:0/3823599850 learned_addr learned my addr 172.21.15.120:0/3823599850 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6815d9b640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68181076a0 0x7f681819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 -- 172.21.15.120:0/3823599850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 msgr2=0x7f681819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 -- 172.21.15.120:0/3823599850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68181076a0 msgr2=0x7f681819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68181076a0 0x7f681819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 -- 172.21.15.120:0/3823599850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68181a5450 con 0x7f6818108020 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f681659c640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:47.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f6816d9d640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 0x7f681819ab10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f681000cab0 tx=0x7f681000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000ae20 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f681000f900 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68181a5740 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.141+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100189f0 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.142+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68181a5c80 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.143+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818102070 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.145+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f681000f450 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.145+0000 7f68077fe640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 0x7f68000b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.146+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f68100e64a0 con 0x7f6818108020 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.146+0000 7f681659c640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 0x7f68000b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:47.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.146+0000 7f681659c640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 0x7f68000b7820 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6808031440 tx=0x7f68080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:47.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.149+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6810014030 con 0x7f6818108020 2024-09-10T23:03:47.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.386+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681806aee0 con 0x7f6818108020 2024-09-10T23:03:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.387+0000 7f68077fe640 1 -- 172.21.15.120:0/3823599850 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f681806aee0 con 0x7f6818108020 2024-09-10T23:03:47.388 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:03:47.388 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:03:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 msgr2=0x7f68000b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 0x7f68000b7820 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6808031440 tx=0x7f68080058d0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 msgr2=0x7f681819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 0x7f681819ab10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f681000cab0 tx=0x7f681000cf70 comp rx=0 tx=0).stop 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 shutdown_connections 2024-09-10T23:03:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f68000b5360 0x7f68000b7820 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6818108020 0x7f681819ab10 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68181076a0 0x7f681819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 --2- 172.21.15.120:0/3823599850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101c90 0x7f681819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 >> 172.21.15.120:0/3823599850 conn(0x7f68180fd700 msgr2=0x7f68180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 shutdown_connections 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:47.389+0000 7f681cbd2640 1 -- 172.21.15.120:0/3823599850 wait complete. 2024-09-10T23:03:47.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1922081 15.96 MB/sec execute 522 sec latency 19.876 ms 2024-09-10T23:03:48.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:47 smithi176 ceph-mon[34982]: pgmap v687: 97 pgs: 97 active+clean; 38 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-10T23:03:48.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:47 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3823599850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:48.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:47 smithi120 ceph-mon[31247]: pgmap v687: 97 pgs: 97 active+clean; 38 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-10T23:03:48.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:47 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3823599850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:48.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:47 smithi153 ceph-mon[34951]: pgmap v687: 97 pgs: 97 active+clean; 38 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-09-10T23:03:48.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:47 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3823599850' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:48.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:03:48.540 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:48.641 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1924978 15.96 MB/sec execute 523 sec latency 21.437 ms 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 -- 172.21.15.120:0/2159523228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c103ca0 msgr2=0x7f7f4c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 --2- 172.21.15.120:0/2159523228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c104080 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009ca0 tx=0x7f7f3c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 -- 172.21.15.120:0/2159523228 shutdown_connections 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 --2- 172.21.15.120:0/2159523228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 0x7f7f4c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 --2- 172.21.15.120:0/2159523228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 --2- 172.21.15.120:0/2159523228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c104080 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 -- 172.21.15.120:0/2159523228 >> 172.21.15.120:0/2159523228 conn(0x7f7f4c0fd700 msgr2=0x7f7f4c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 -- 172.21.15.120:0/2159523228 shutdown_connections 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.095+0000 7f7f50a06640 1 -- 172.21.15.120:0/2159523228 wait complete. 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 Processor -- start 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 -- start start 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 0x7f7f4c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112dc0 con 0x7f7f4c1045c0 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112c40 con 0x7f7f4c103ca0 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f50a06640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112f40 con 0x7f7f4c10d100 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51704/0 (socket says 172.21.15.120:51704) 2024-09-10T23:03:49.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f49d9b640 1 -- 172.21.15.120:0/459093861 learned_addr learned my addr 172.21.15.120:0/459093861 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 0x7f7f4c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.096+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 -- 172.21.15.120:0/459093861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 msgr2=0x7f7f4c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 0x7f7f4c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 -- 172.21.15.120:0/459093861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c103ca0 msgr2=0x7f7f4c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 -- 172.21.15.120:0/459093861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19f360 con 0x7f7f4c1045c0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d750 tx=0x7f7f3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f437fe640 1 -- 172.21.15.120:0/459093861 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34017070 con 0x7f7f4c1045c0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19f650 con 0x7f7f4c1045c0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f437fe640 1 -- 172.21.15.120:0/459093861 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f34004020 con 0x7f7f4c1045c0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f437fe640 1 -- 172.21.15.120:0/459093861 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34003c00 con 0x7f7f4c1045c0 2024-09-10T23:03:49.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.097+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a5ee0 con 0x7f7f4c1045c0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.100+0000 7f7f52ddb740 1 Processor -- start 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.100+0000 7f7f52ddb740 1 -- start start 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad37840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da8aca1ea0 con 0x55da8ad2b4d0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da8ac912d0 con 0x55da8ad37d80 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55da8ac950e0 con 0x55da8ad2bdf0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f437fe640 1 -- 172.21.15.120:0/459093861 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7f34009a30 con 0x7f7f4c1045c0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad37840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51706/0 (socket says 172.21.15.120:51706) 2024-09-10T23:03:49.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4ad9d640 1 -- 172.21.15.120:0/1286726770 learned_addr learned my addr 172.21.15.120:0/1286726770 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 -- 172.21.15.120:0/1286726770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 msgr2=0x55da8ad3a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 -- 172.21.15.120:0/1286726770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 msgr2=0x55da8ad2b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 -- 172.21.15.120:0/1286726770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8a9ef650 con 0x55da8ad2bdf0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.101+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad37840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000f80 tx=0x7f7f3c02f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0c4c8a00b8f44e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f40ff9640 1 -- 172.21.15.120:0/1286726770 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c031df0 con 0x55da8ad2bdf0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f437fe640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 0x7f7f380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f40ff9640 1 -- 172.21.15.120:0/1286726770 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c036030 con 0x55da8ad2bdf0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f40ff9640 1 -- 172.21.15.120:0/1286726770 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c004260 con 0x55da8ad2bdf0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 0x7f7f380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f437fe640 1 -- 172.21.15.120:0/459093861 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f340e53b0 con 0x7f7f4c1045c0 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 0x7f7f380b7820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400b820 tx=0x7f7f44005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.102+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1286726770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 msgr2=0x55da8ad37840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad37840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000f80 tx=0x7f7f3c02f450 comp rx=0 tx=0).stop 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1286726770 shutdown_connections 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad37840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1286726770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1286726770 >> 172.21.15.120:0/1286726770 conn(0x55da8ad29300 msgr2=0x55da8ad2a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1286726770 shutdown_connections 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1286726770 wait complete. 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- start start 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 Processor -- start 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da8aaf2700 con 0x55da8ad2bdf0 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55da8ac2c160 con 0x55da8ad2b4d0 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.103+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55da8ad3a820 con 0x55da8ad37d80 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f52ddb740 10 client.0 _ll_register_callbacks cb 0x55da8ac459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad3c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51716/0 (socket says 172.21.15.120:51716) 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f49d9b640 1 -- 172.21.15.120:0/95240700 learned_addr learned my addr 172.21.15.120:0/95240700 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:49.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 -- 172.21.15.120:0/95240700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 msgr2=0x55da8ad3be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad3be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 -- 172.21.15.120:0/95240700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 msgr2=0x55da8ad3c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad3c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 -- 172.21.15.120:0/95240700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8ac59c90 con 0x55da8ad37d80 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad414c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f440039c0 tx=0x7f7f44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5ac7993a63acd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4401fc90 con 0x55da8ad37d80 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f44017030 con 0x55da8ad37d80 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.104+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c036030 con 0x55da8ad37d80 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f4401fe30 con 0x55da8ad37d80 2024-09-10T23:03:49.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 10 client.27154 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da8ad4a5b0 con 0x55da8ad37d80 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c036030 con 0x55da8ad37d80 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7f44004cb0 con 0x55da8ad37d80 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 10 client.27154 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 10 client.27154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 20 client.27154 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8ad4ba80 0x55da8ad4deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 4 client.27154 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da8aca3df0 con 0x55da8ad4ba80 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.105+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8ad4ba80 0x55da8ad4deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.106+0000 7f7f31ffb640 10 client.27154 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.106+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8ad4ba80 0x55da8ad4deb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194f199a4af7d1c6 server_cookie=b6276b48534de47e in_seq=0 out_seq=0 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.106+0000 7f7f31ffb640 1 -- 172.21.15.120:0/95240700 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da8aca3df0 con 0x55da8ad4ba80 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.106+0000 7f7f31ffb640 10 client.27154 handle_command_reply: tid=0 2024-09-10T23:03:49.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.106+0000 7f7f52ddb740 1 client.27154 shutdown 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 msgr2=0x55da8ad414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad414c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f440039c0 tx=0x7f7f44004730 comp rx=0 tx=0).stop 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 shutdown_connections 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8ad4ba80 0x55da8ad4deb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad414c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad3c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f31ffb640 0 client.27154 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/95240700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 >> 172.21.15.120:0/95240700 conn(0x55da8ad29300 msgr2=0x55da8ad2ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 shutdown_connections 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 1 -- 172.21.15.120:0/95240700 wait complete. 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.107+0000 7f7f52ddb740 20 client.27154 trim_cache size 0 max 16384 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.120+0000 7f7f52ddb740 1 Processor -- start 2024-09-10T23:03:49.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 -- start start 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aa0f0 con 0x55da8ad2b4d0 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ac950 con 0x55da8ad37d80 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aa740 con 0x55da8ad2bdf0 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51730/0 (socket says 172.21.15.120:51730) 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 -- 172.21.15.120:0/3264912268 learned_addr learned my addr 172.21.15.120:0/3264912268 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 -- 172.21.15.120:0/3264912268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 msgr2=0x55da8ad28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad28c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 -- 172.21.15.120:0/3264912268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 msgr2=0x55da8ad3aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 -- 172.21.15.120:0/3264912268 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8ad24bb0 con 0x55da8ad2b4d0 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004af0 tx=0x7f7f3c02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c6a7a7e93827ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.121+0000 7f7f33fff640 1 -- 172.21.15.120:0/3264912268 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c05e070 con 0x55da8ad2b4d0 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.122+0000 7f7f33fff640 1 -- 172.21.15.120:0/3264912268 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c055070 con 0x55da8ad2b4d0 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.122+0000 7f7f33fff640 1 -- 172.21.15.120:0/3264912268 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c02fbd0 con 0x55da8ad2b4d0 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.122+0000 7f7f52ddb740 1 -- 172.21.15.120:0/3264912268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 msgr2=0x55da8ad2a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.122+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004af0 tx=0x7f7f3c02b040 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- 172.21.15.120:0/3264912268 shutdown_connections 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad37d80 0x55da8ad3aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad2bdf0 0x55da8ad28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/3264912268 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2b4d0 0x55da8ad2a920 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- 172.21.15.120:0/3264912268 >> 172.21.15.120:0/3264912268 conn(0x55da8ad29300 msgr2=0x55da8ad2a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- 172.21.15.120:0/3264912268 shutdown_connections 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- 172.21.15.120:0/3264912268 wait complete. 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- start start 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 Processor -- start 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aae10 con 0x55da8ad2bdf0 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.123+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ac280 con 0x55da8ad2b4d0 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f52ddb740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ab4e0 con 0x55da8ad37d80 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f52ddb740 10 client.0 _ll_register_callbacks cb 0x7f7f4400be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f4ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51744/0 (socket says 172.21.15.120:51744) 2024-09-10T23:03:49.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 -- 172.21.15.120:0/1384275379 learned_addr learned my addr 172.21.15.120:0/1384275379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 -- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 msgr2=0x55da8ad472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 -- 172.21.15.120:0/1384275379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 msgr2=0x55da8ad42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad42430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 -- 172.21.15.120:0/1384275379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8ad47910 con 0x55da8ad2bdf0 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f4ad9d640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f49d9b640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005f70 tx=0x7f7f340e89f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1554b28e81dae039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:49.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34020040 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c055070 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.124+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f340a7e60 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f340ab540 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 10 client.17685 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da8adb8cf0 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c055070 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7f340a7040 con 0x55da8ad2bdf0 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 10 client.17685 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 20 client.17685 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8adb94a0 0x55da8adbb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 4 client.17685 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:49.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da8ac9f8a0 con 0x55da8adb94a0 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.125+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8adb94a0 0x55da8adbb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f31ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f4a59c640 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8adb94a0 0x55da8adbb8c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b963416f64a22d server_cookie=2a685d95aa0f3c83 in_seq=0 out_seq=0 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f31ffb640 1 -- 172.21.15.120:0/1384275379 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da8ac9f8a0 con 0x55da8adb94a0 2024-09-10T23:03:49.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f31ffb640 10 client.17685 handle_command_reply: tid=0 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 client.17685 shutdown 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 msgr2=0x55da8ad42970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005f70 tx=0x7f7f340e89f0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 shutdown_connections 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55da8adb94a0 0x55da8adbb8c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55da8ad37d80 0x55da8ad472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da8ad2bdf0 0x55da8ad42970 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 --2- 172.21.15.120:0/1384275379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55da8ad2b4d0 0x55da8ad42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 >> 172.21.15.120:0/1384275379 conn(0x55da8ad29300 msgr2=0x55da8ad3aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f31ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 shutdown_connections 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 1 -- 172.21.15.120:0/1384275379 wait complete. 2024-09-10T23:03:49.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.126+0000 7f7f52ddb740 20 client.17685 trim_cache size 0 max 16384 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 msgr2=0x7f7f380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 0x7f7f380b7820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400b820 tx=0x7f7f44005cc0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 msgr2=0x7f7f4c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d750 tx=0x7f7f3400dc10 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 shutdown_connections 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f380b5360 0x7f7f380b7820 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f4c10d100 0x7f7f4c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f4c1045c0 0x7f7f4c1997f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 --2- 172.21.15.120:0/459093861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f4c103ca0 0x7f7f4c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 >> 172.21.15.120:0/459093861 conn(0x7f7f4c0fd700 msgr2=0x7f7f4c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 shutdown_connections 2024-09-10T23:03:49.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:49.127+0000 7f7f50a06640 1 -- 172.21.15.120:0/459093861 wait complete. 2024-09-10T23:03:49.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1927331 15.95 MB/sec execute 524 sec latency 38.175 ms 2024-09-10T23:03:50.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:49 smithi176 ceph-mon[34982]: pgmap v688: 97 pgs: 97 active+clean; 41 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-10T23:03:50.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:49 smithi120 ceph-mon[31247]: pgmap v688: 97 pgs: 97 active+clean; 41 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-10T23:03:50.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:49 smithi153 ceph-mon[34951]: pgmap v688: 97 pgs: 97 active+clean; 41 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-10T23:03:50.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1930182 15.95 MB/sec execute 525 sec latency 28.470 ms 2024-09-10T23:03:50.854 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:03:50.854 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:03:51.139 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:51.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1933065 15.95 MB/sec execute 526 sec latency 68.693 ms 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- 172.21.15.120:0/3962383939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 msgr2=0x7fade4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 --2- 172.21.15.120:0/3962383939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade4102070 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- 172.21.15.120:0/3962383939 shutdown_connections 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 --2- 172.21.15.120:0/3962383939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade4108020 0x7fade410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 --2- 172.21.15.120:0/3962383939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade41076a0 0x7fade4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 --2- 172.21.15.120:0/3962383939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade4102070 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- 172.21.15.120:0/3962383939 >> 172.21.15.120:0/3962383939 conn(0x7fade40fd700 msgr2=0x7fade40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- 172.21.15.120:0/3962383939 shutdown_connections 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- 172.21.15.120:0/3962383939 wait complete. 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 Processor -- start 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.668+0000 7fadeb990640 1 -- start start 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 0x7fade419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4108020 0x7fade41a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade4110cc0 con 0x7fade4101c90 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade4110b40 con 0x7fade41076a0 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade4110e40 con 0x7fade4108020 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 0x7fade419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 0x7fade419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55110/0 (socket says 172.21.15.120:55110) 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade8f2b640 1 -- 172.21.15.120:0/3482582649 learned_addr learned my addr 172.21.15.120:0/3482582649 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade9f2d640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4108020 0x7fade41a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 -- 172.21.15.120:0/3482582649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4108020 msgr2=0x7fade41a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4108020 0x7fade41a6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 -- 172.21.15.120:0/3482582649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 msgr2=0x7fade419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 0x7fade419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 -- 172.21.15.120:0/3482582649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade41a76c0 con 0x7fade4101c90 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fade972c640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade419c040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fade0006010 tx=0x7fade0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadda7fc640 1 -- 172.21.15.120:0/3482582649 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade003f070 con 0x7fade4101c90 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadda7fc640 1 -- 172.21.15.120:0/3482582649 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0003ca0 con 0x7fade4101c90 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41a7950 con 0x7fade4101c90 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.669+0000 7fadda7fc640 1 -- 172.21.15.120:0/3482582649 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0035070 con 0x7fade4101c90 2024-09-10T23:03:51.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.670+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a7e30 con 0x7fade4101c90 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.672+0000 7fadedd65740 1 Processor -- start 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.672+0000 7fadedd65740 1 -- start start 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.672+0000 7fadedd65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 0x5621246dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.672+0000 7fadedd65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.672+0000 7fadedd65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 0x5621246ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadedd65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562124653e60 con 0x5621246dcee0 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadedd65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562124643290 con 0x5621246e9790 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadedd65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621246470a0 con 0x5621246dd800 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade9f2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 0x5621246dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade9f2d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 0x5621246dd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51766/0 (socket says 172.21.15.120:51766) 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade9f2d640 1 -- 172.21.15.120:0/1615595866 learned_addr learned my addr 172.21.15.120:0/1615595866 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 0x5621246ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 -- 172.21.15.120:0/1615595866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 msgr2=0x5621246e9250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e9250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 -- 172.21.15.120:0/1615595866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 msgr2=0x5621246dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 0x5621246dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 -- 172.21.15.120:0/1615595866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621243a1930 con 0x5621246e9790 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadda7fc640 1 -- 172.21.15.120:0/3482582649 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fade004b060 con 0x7fade4101c90 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fade8f2b640 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 0x5621246ebb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fade4100c60 tx=0x7fadd4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f75db9a87723e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1615595866 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004230 con 0x5621246e9790 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1615595866 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40043d0 con 0x5621246e9790 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.673+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1615595866 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004ce0 con 0x5621246e9790 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadda7fc640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 0x7fadd00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fade9f2d640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 0x7fadd00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadda7fc640 1 -- 172.21.15.120:0/3482582649 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fade0105b90 con 0x7fade4101c90 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 -- 172.21.15.120:0/1615595866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 msgr2=0x5621246ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 0x5621246ebb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fade4100c60 tx=0x7fadd4011810 comp rx=0 tx=0).stop 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 -- 172.21.15.120:0/1615595866 shutdown_connections 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246e9790 0x5621246ebb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 --2- 172.21.15.120:0/1615595866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dcee0 0x5621246dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 -- 172.21.15.120:0/1615595866 >> 172.21.15.120:0/1615595866 conn(0x5621246dad10 msgr2=0x5621246dbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 -- 172.21.15.120:0/1615595866 shutdown_connections 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fadedd65740 1 -- 172.21.15.120:0/1615595866 wait complete. 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.674+0000 7fade9f2d640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 0x7fadd00b78f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faddc005f40 tx=0x7faddc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 -- start start 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 Processor -- start 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246e9790 0x5621246e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621244a4890 con 0x5621246e9790 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5621245ddd40 con 0x5621246dcee0 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5621246ec440 con 0x5621246dd800 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fadedd65740 10 client.0 _ll_register_callbacks cb 0x562124641c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fade972c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fade972c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55118/0 (socket says 172.21.15.120:55118) 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fade972c640 1 -- 172.21.15.120:0/761676123 learned_addr learned my addr 172.21.15.120:0/761676123 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fade9f2d640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246e9790 0x5621246e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.675+0000 7fade8f2b640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 -- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 msgr2=0x5621246e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 -- 172.21.15.120:0/761676123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246e9790 msgr2=0x5621246e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246e9790 0x5621246e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 -- 172.21.15.120:0/761676123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621246f5850 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fade972c640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fade0031440 tx=0x7fade00c82d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab699f48993fe99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadccff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0048040 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fade00cc690 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd40043d0 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fade00cbba0 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadedd65740 10 client.27020 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.676+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562124700a10 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00cbf40 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fade00d5480 con 0x5621246dcee0 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 10 client.27020 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 20 client.27020 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:51.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124701f10 0x562124704300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 4 client.27020 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:51.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fadd40043d0 con 0x562124701f10 2024-09-10T23:03:51.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.677+0000 7fade8f2b640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124701f10 0x562124704300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fade8f2b640 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124701f10 0x562124704300 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af9f7e0961be27e server_cookie=de4c865e6ae4a754 in_seq=0 out_seq=0 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadccff9640 10 client.27020 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadccff9640 1 -- 172.21.15.120:0/761676123 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fadd40043d0 con 0x562124701f10 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadccff9640 10 client.27020 handle_command_reply: tid=0 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadedd65740 1 client.27020 shutdown 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 msgr2=0x5621246e6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.678+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fade0031440 tx=0x7fade00c82d0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 shutdown_connections 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124701f10 0x562124704300 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246e9790 0x5621246e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246dd800 0x5621246e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadccff9640 0 client.27020 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 --2- 172.21.15.120:0/761676123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e6730 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 >> 172.21.15.120:0/761676123 conn(0x5621246dad10 msgr2=0x5621246d9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 shutdown_connections 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 1 -- 172.21.15.120:0/761676123 wait complete. 2024-09-10T23:03:51.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.679+0000 7fadedd65740 20 client.27020 trim_cache size 0 max 16384 2024-09-10T23:03:51.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 Processor -- start 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 -- start start 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade4116d80 con 0x5621246dd800 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ac830 con 0x5621246dcee0 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fadedd65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aa620 con 0x5621246e9790 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55130/0 (socket says 172.21.15.120:55130) 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51772/0 (socket says 172.21.15.120:51772) 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 -- 172.21.15.120:0/3900835580 learned_addr learned my addr 172.21.15.120:0/3900835580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:51.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade9f2d640 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 -- 172.21.15.120:0/3900835580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 msgr2=0x5621246e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 -- 172.21.15.120:0/3900835580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 msgr2=0x5621246e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade972c640 1 -- 172.21.15.120:0/3900835580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621246db790 con 0x5621246dcee0 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.697+0000 7fade8f2b640 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fade972c640 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fade00cc3c0 tx=0x7fade0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0028ea0afbc1da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fadceffd640 1 -- 172.21.15.120:0/3900835580 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0048030 con 0x5621246dcee0 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fadceffd640 1 -- 172.21.15.120:0/3900835580 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00cbf40 con 0x5621246dcee0 2024-09-10T23:03:51.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fadceffd640 1 -- 172.21.15.120:0/3900835580 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00d5480 con 0x5621246dcee0 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fadedd65740 1 -- 172.21.15.120:0/3900835580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 msgr2=0x5621246e5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.698+0000 7fadedd65740 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fade00cc3c0 tx=0x7fade0000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 -- 172.21.15.120:0/3900835580 shutdown_connections 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- 172.21.15.120:0/3900835580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246e5e90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 -- 172.21.15.120:0/3900835580 >> 172.21.15.120:0/3900835580 conn(0x5621246dad10 msgr2=0x5621246e6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 -- 172.21.15.120:0/3900835580 shutdown_connections 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 -- 172.21.15.120:0/3900835580 wait complete. 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 -- start start 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 Processor -- start 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.699+0000 7fadedd65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246ef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadedd65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aacf0 con 0x5621246dd800 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadedd65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ac160 con 0x5621246dcee0 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadedd65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ab3c0 con 0x5621246e9790 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadedd65740 10 client.0 _ll_register_callbacks cb 0x7fade00c8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:51.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51788/0 (socket says 172.21.15.120:51788) 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 -- 172.21.15.120:0/3650069565 learned_addr learned my addr 172.21.15.120:0/3650069565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade972c640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade9f2d640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 -- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 msgr2=0x5621246ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246ef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 -- 172.21.15.120:0/3650069565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 msgr2=0x5621246ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 -- 172.21.15.120:0/3650069565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621246e75d0 con 0x5621246dd800 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade972c640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fade8f2b640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009bd0 tx=0x7fadd40048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c2b91263004da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadccff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd401e040 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fadd4004450 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.700+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade00cbf40 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadd403c050 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 10 client.17703 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621247700a0 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00cbf40 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fadd4024040 con 0x5621246dd800 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 10 client.17703 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 20 client.17703 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124770630 0x562124772a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 4 client.17703 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:03:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562124655db0 con 0x562124770630 2024-09-10T23:03:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fade972c640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124770630 0x562124772a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fade972c640 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124770630 0x562124772a50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdc81ab0419b350 server_cookie=2241a6b9ff4c0634 in_seq=0 out_seq=0 2024-09-10T23:03:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.701+0000 7fadccff9640 10 client.17703 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadccff9640 1 -- 172.21.15.120:0/3650069565 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562124655db0 con 0x562124770630 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadccff9640 10 client.17703 handle_command_reply: tid=0 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 client.17703 shutdown 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 msgr2=0x5621246eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009bd0 tx=0x7fadd40048e0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 shutdown_connections 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562124770630 0x562124772a50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5621246e9790 0x5621246ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621246dd800 0x5621246eecf0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 --2- 172.21.15.120:0/3650069565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621246dcee0 0x5621246ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 >> 172.21.15.120:0/3650069565 conn(0x5621246dad10 msgr2=0x5621246d9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadccff9640 0 client.17703 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 shutdown_connections 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 1 -- 172.21.15.120:0/3650069565 wait complete. 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.703+0000 7fadedd65740 20 client.17703 trim_cache size 0 max 16384 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "756a95b7-b141-4232-b641-f7ca49c6ddcd", 2024-09-10T23:03:51.709 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 msgr2=0x7fadd00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 0x7fadd00b78f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faddc005f40 tx=0x7faddc009600 comp rx=0 tx=0).stop 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 msgr2=0x7fade419c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade419c040 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fade0006010 tx=0x7fade0004060 comp rx=0 tx=0).stop 2024-09-10T23:03:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 shutdown_connections 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b5430 0x7fadd00b78f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4108020 0x7fade41a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41076a0 0x7fade419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 --2- 172.21.15.120:0/3482582649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101c90 0x7fade419c040 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 >> 172.21.15.120:0/3482582649 conn(0x7fade40fd700 msgr2=0x7fade40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 shutdown_connections 2024-09-10T23:03:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:51.704+0000 7fadeb990640 1 -- 172.21.15.120:0/3482582649 wait complete. 2024-09-10T23:03:51.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:51 smithi120 ceph-mon[31247]: pgmap v689: 97 pgs: 97 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-09-10T23:03:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:51 smithi176 ceph-mon[34982]: pgmap v689: 97 pgs: 97 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-09-10T23:03:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:51 smithi153 ceph-mon[34951]: pgmap v689: 97 pgs: 97 active+clean; 42 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 5.3 MiB/s wr, 409 op/s 2024-09-10T23:03:52.438 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:03:52.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1935715 15.95 MB/sec execute 527 sec latency 24.524 ms 2024-09-10T23:03:52.707 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:53.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:52 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:03:53.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:52 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:52 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:03:53.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:52 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.145 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:52 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:03:53.145 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:52 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.145 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:52 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:03:53.145 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:52 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:52 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:03:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:52 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:52 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:03:53.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:52 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 -- 172.21.15.120:0/3980593718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 msgr2=0x7f0b5010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3980593718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5010f570 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009ca0 tx=0x7f0b4802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 -- 172.21.15.120:0/3980593718 shutdown_connections 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3980593718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5010f570 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3980593718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50104550 0x7f0b5010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3980593718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50103b50 0x7f0b50103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 -- 172.21.15.120:0/3980593718 >> 172.21.15.120:0/3980593718 conn(0x7f0b500fd700 msgr2=0x7f0b500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 -- 172.21.15.120:0/3980593718 shutdown_connections 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.227+0000 7f0b576e3640 1 -- 172.21.15.120:0/3980593718 wait complete. 2024-09-10T23:03:53.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 Processor -- start 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 -- start start 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50103b50 0x7f0b50199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50104550 0x7f0b501996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50112d80 con 0x7f0b5010d100 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50112c00 con 0x7f0b50104550 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b576e3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50112f00 con 0x7f0b50103b50 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b55c80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b5547f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50103b50 0x7f0b50199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b55c80640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47778/0 (socket says 172.21.15.120:47778) 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.228+0000 7f0b55c80640 1 -- 172.21.15.120:0/3184364785 learned_addr learned my addr 172.21.15.120:0/3184364785 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:53.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50104550 0x7f0b501996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 -- 172.21.15.120:0/3184364785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50103b50 msgr2=0x7f0b50199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50103b50 0x7f0b50199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 -- 172.21.15.120:0/3184364785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50104550 msgr2=0x7f0b501996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50104550 0x7f0b501996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 -- 172.21.15.120:0/3184364785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5019f240 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b55c80640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005cc0 tx=0x7f0b4802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b467fc640 1 -- 172.21.15.120:0/3184364785 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48003bf0 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b467fc640 1 -- 172.21.15.120:0/3184364785 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b48003d90 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5019f4d0 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b467fc640 1 -- 172.21.15.120:0/3184364785 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4802fb20 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.229+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b501a5bf0 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b467fc640 1 -- 172.21.15.120:0/3184364785 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0b48004580 con 0x7f0b5010d100 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 Processor -- start 2024-09-10T23:03:53.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 -- start start 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559af1e80e50 con 0x559af1f1b140 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559af1e70280 con 0x559af1f0a480 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559af1e74090 con 0x559af1f0ada0 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b5547f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f1d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47784/0 (socket says 172.21.15.120:47784) 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.232+0000 7f0b54c7e640 1 -- 172.21.15.120:0/2810209377 learned_addr learned my addr 172.21.15.120:0/2810209377 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b467fc640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 0x7f0b3c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 -- 172.21.15.120:0/2810209377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 msgr2=0x559af1f0a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f0a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 -- 172.21.15.120:0/2810209377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 msgr2=0x559af1f1d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f1d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 -- 172.21.15.120:0/2810209377 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af1bce650 con 0x559af1f0ada0 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f0a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b55c80640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 0x7f0b3c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b5547f640 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f1ac00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00ac20 tx=0x7f0b4c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbedc0541c27f5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b467fc640 1 -- 172.21.15.120:0/3184364785 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b48106fa0 con 0x7f0b5010d100 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/2810209377 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c005220 con 0x559af1f0ada0 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/2810209377 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4c004c50 con 0x559af1f0ada0 2024-09-10T23:03:53.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.233+0000 7f0b3b7fe640 1 -- 172.21.15.120:0/2810209377 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0187c0 con 0x559af1f0ada0 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- 172.21.15.120:0/2810209377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 msgr2=0x559af1f1ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f1ac00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00ac20 tx=0x7f0b4c00f790 comp rx=0 tx=0).stop 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b55c80640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 0x7f0b3c0b77a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b48049000 tx=0x7f0b48097a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- 172.21.15.120:0/2810209377 shutdown_connections 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f1ac00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/2810209377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f0a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- 172.21.15.120:0/2810209377 >> 172.21.15.120:0/2810209377 conn(0x559af1f082b0 msgr2=0x559af1f09450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- 172.21.15.120:0/2810209377 shutdown_connections 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- 172.21.15.120:0/2810209377 wait complete. 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.234+0000 7f0b59ab8740 1 -- start start 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 Processor -- start 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0ada0 0x559af1f17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f1b140 0x559af1f18440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559af1cd1700 con 0x559af1f0a480 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559af1e0b110 con 0x559af1f0ada0 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559af1f1ddf0 con 0x559af1f1b140 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b59ab8740 10 client.0 _ll_register_callbacks cb 0x559af1e24990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47792/0 (socket says 172.21.15.120:47792) 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 -- 172.21.15.120:0/3123195729 learned_addr learned my addr 172.21.15.120:0/3123195729 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b5547f640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0ada0 0x559af1f17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f1b140 0x559af1f18440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 -- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f1b140 msgr2=0x559af1f18440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f1b140 0x559af1f18440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 -- 172.21.15.120:0/3123195729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0ada0 msgr2=0x559af1f17e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0ada0 0x559af1f17e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 -- 172.21.15.120:0/3123195729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af1e38c40 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bfe0 tx=0x7f0b40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21794b58beca0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.235+0000 7f0b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40011b80 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b40011d20 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4c004c50 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b40026020 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 10 client.17715 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559af1f29790 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4c004c50 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0b40003c60 con 0x559af1f0a480 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 10 client.17715 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 20 client.17715 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f2abf0 0x559af1f2d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 4 client.17715 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.236+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af1e82da0 con 0x559af1f2abf0 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.237+0000 7f0b5547f640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f2abf0 0x559af1f2d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.237+0000 7f0b5547f640 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f2abf0 0x559af1f2d030 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b246fd4adc255cbd server_cookie=3b9186c630094948 in_seq=0 out_seq=0 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.237+0000 7f0b38ff9640 10 client.17715 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b38ff9640 1 -- 172.21.15.120:0/3123195729 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559af1e82da0 con 0x559af1f2abf0 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b38ff9640 10 client.17715 handle_command_reply: tid=0 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 client.17715 shutdown 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 msgr2=0x559af1f1aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000bfe0 tx=0x7f0b40004730 comp rx=0 tx=0).stop 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 shutdown_connections 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f2abf0 0x559af1f2d030 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f1b140 0x559af1f18440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0ada0 0x559af1f17e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/3123195729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f0a480 0x559af1f1aa10 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 >> 172.21.15.120:0/3123195729 conn(0x559af1f082b0 msgr2=0x559af1f0de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b38ff9640 0 client.17715 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 shutdown_connections 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 1 -- 172.21.15.120:0/3123195729 wait complete. 2024-09-10T23:03:53.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.238+0000 7f0b59ab8740 20 client.17715 trim_cache size 0 max 16384 2024-09-10T23:03:53.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 Processor -- start 2024-09-10T23:03:53.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 -- start start 2024-09-10T23:03:53.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f27050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501aa000 con 0x559af1f1b140 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501ac860 con 0x559af1f0a480 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501aa650 con 0x559af1f0ada0 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47800/0 (socket says 172.21.15.120:47800) 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 -- 172.21.15.120:0/143399063 learned_addr learned my addr 172.21.15.120:0/143399063 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b5547f640 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 -- 172.21.15.120:0/143399063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 msgr2=0x559af1f032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 -- 172.21.15.120:0/143399063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 msgr2=0x559af1f27050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f27050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 -- 172.21.15.120:0/143399063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af1f0ab90 con 0x559af1f1b140 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.260+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005d00 tx=0x7f0b40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a688ad7b144ace91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b3affd640 1 -- 172.21.15.120:0/143399063 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000b520 con 0x559af1f1b140 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b3affd640 1 -- 172.21.15.120:0/143399063 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b40011110 con 0x559af1f1b140 2024-09-10T23:03:53.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b3affd640 1 -- 172.21.15.120:0/143399063 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40011e60 con 0x559af1f1b140 2024-09-10T23:03:53.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 -- 172.21.15.120:0/143399063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 msgr2=0x559af1f09130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005d00 tx=0x7f0b40000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 -- 172.21.15.120:0/143399063 shutdown_connections 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f09130 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.261+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/143399063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f27050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- 172.21.15.120:0/143399063 >> 172.21.15.120:0/143399063 conn(0x559af1f082b0 msgr2=0x559af1f1cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- 172.21.15.120:0/143399063 shutdown_connections 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- 172.21.15.120:0/143399063 wait complete. 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- start start 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 Processor -- start 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501aad20 con 0x559af1f1b140 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501ac190 con 0x559af1f0a480 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501ab3f0 con 0x559af1f0ada0 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.262+0000 7f0b59ab8740 10 client.0 _ll_register_callbacks cb 0x7f0b40012680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b5547f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55544/0 (socket says 172.21.15.120:55544) 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 -- 172.21.15.120:0/1480411284 learned_addr learned my addr 172.21.15.120:0/1480411284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 -- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 msgr2=0x559af1f326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 -- 172.21.15.120:0/1480411284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 msgr2=0x559af1f32e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f32e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 -- 172.21.15.120:0/1480411284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af1f96b00 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b5547f640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b54c7e640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b40011370 tx=0x7f0b4000bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=784cf2c22679775e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000b520 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b40004220 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.263+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b40011110 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b40042050 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 10 client.27041 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559af1f97d30 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b40011110 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0b40004660 con 0x559af1f0a480 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 10 client.27041 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:53.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 20 client.27041 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f98450 0x559af1f9a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 4 client.27041 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559af1e7e850 con 0x559af1f98450 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.264+0000 7f0b5547f640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f98450 0x559af1f9a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.265+0000 7f0b38ff9640 10 client.27041 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:53.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.265+0000 7f0b5547f640 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f98450 0x559af1f9a870 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b5f496e0c4b2e5 server_cookie=f12b007e37b267ef in_seq=0 out_seq=0 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b38ff9640 1 -- 172.21.15.120:0/1480411284 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x559af1e7e850 con 0x559af1f98450 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b38ff9640 10 client.27041 handle_command_reply: tid=0 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 client.27041 shutdown 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 msgr2=0x559af1f23a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b40011370 tx=0x7f0b4000bd20 comp rx=0 tx=0).stop 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 shutdown_connections 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559af1f98450 0x559af1f9a870 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559af1f1b140 0x559af1f32e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559af1f0ada0 0x559af1f326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 --2- 172.21.15.120:0/1480411284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559af1f0a480 0x559af1f23a40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b38ff9640 0 client.27041 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 >> 172.21.15.120:0/1480411284 conn(0x559af1f082b0 msgr2=0x559af1f09130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 shutdown_connections 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 1 -- 172.21.15.120:0/1480411284 wait complete. 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.266+0000 7f0b59ab8740 20 client.27041 trim_cache size 0 max 16384 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:03:53.272 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "756a95b7-b141-4232-b641-f7ca49c6ddcd", 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:03:53.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.275 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:03:53.278 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:03:53.281 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:03:53.285 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:03:53.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 msgr2=0x7f0b3c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 0x7f0b3c0b77a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0b48049000 tx=0x7f0b48097a10 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 msgr2=0x7f0b5019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0b48005cc0 tx=0x7f0b4802f380 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 shutdown_connections 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b3c0b52e0 0x7f0b3c0b77a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b5010d100 0x7f0b5019ea70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b50104550 0x7f0b501996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 --2- 172.21.15.120:0/3184364785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b50103b50 0x7f0b50199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 >> 172.21.15.120:0/3184364785 conn(0x7f0b500fd700 msgr2=0x7f0b5010f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 shutdown_connections 2024-09-10T23:03:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:03:53.267+0000 7f0b576e3640 1 -- 172.21.15.120:0/3184364785 wait complete. 2024-09-10T23:03:53.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1938265 15.94 MB/sec execute 528 sec latency 26.870 ms 2024-09-10T23:03:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:53 smithi120 ceph-mon[31247]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 420 op/s 2024-09-10T23:03:53.996 INFO:tasks.cephfs.filesystem:scrub status for tag:756a95b7-b141-4232-b641-f7ca49c6ddcd - {'path': '/', 'tag': '756a95b7-b141-4232-b641-f7ca49c6ddcd', 'options': 'recursive,force'} 2024-09-10T23:03:54.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:53 smithi176 ceph-mon[34982]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 420 op/s 2024-09-10T23:03:54.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:53 smithi153 ceph-mon[34951]: pgmap v690: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 420 op/s 2024-09-10T23:03:54.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1940876 15.94 MB/sec execute 529 sec latency 20.964 ms 2024-09-10T23:03:55.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1943922 15.94 MB/sec execute 530 sec latency 9.255 ms 2024-09-10T23:03:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:55 smithi176 ceph-mon[34982]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 387 op/s 2024-09-10T23:03:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:03:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:55 smithi120 ceph-mon[31247]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 387 op/s 2024-09-10T23:03:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:03:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:55 smithi153 ceph-mon[34951]: pgmap v691: 97 pgs: 97 active+clean; 49 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 387 op/s 2024-09-10T23:03:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:03:56.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1946882 15.94 MB/sec execute 531 sec latency 71.306 ms 2024-09-10T23:03:57.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1949818 15.94 MB/sec execute 532 sec latency 29.132 ms 2024-09-10T23:03:58.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:57 smithi176 ceph-mon[34982]: pgmap v692: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 434 op/s 2024-09-10T23:03:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:57 smithi120 ceph-mon[31247]: pgmap v692: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 434 op/s 2024-09-10T23:03:58.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:57 smithi153 ceph-mon[34951]: pgmap v692: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 434 op/s 2024-09-10T23:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 -- 172.21.15.120:0/3587771439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 msgr2=0x7f115810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 --2- 172.21.15.120:0/3587771439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f115810ab40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1150009ca0 tx=0x7f115002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 -- 172.21.15.120:0/3587771439 shutdown_connections 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 --2- 172.21.15.120:0/3587771439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f115810ab40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 --2- 172.21.15.120:0/3587771439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 --2- 172.21.15.120:0/3587771439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 0x7f11581055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 -- 172.21.15.120:0/3587771439 >> 172.21.15.120:0/3587771439 conn(0x7f11580fd140 msgr2=0x7f11580ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 -- 172.21.15.120:0/3587771439 shutdown_connections 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.511+0000 7f115dc52640 1 -- 172.21.15.120:0/3587771439 wait complete. 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.512+0000 7f115dc52640 1 Processor -- start 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.512+0000 7f115dc52640 1 -- start start 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 0x7f1158102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f115810e5f0 con 0x7f1158108680 2024-09-10T23:03:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f115810e470 con 0x7f115806b3d0 2024-09-10T23:03:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f115dc52640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f115810e770 con 0x7f1158105b10 2024-09-10T23:03:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47828/0 (socket says 172.21.15.120:47828) 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 -- 172.21.15.120:0/2776572533 learned_addr learned my addr 172.21.15.120:0/2776572533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f11577fe640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 0x7f1158102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 -- 172.21.15.120:0/2776572533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 msgr2=0x7f1158101570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158101570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 -- 172.21.15.120:0/2776572533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 msgr2=0x7f1158102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 0x7f1158102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 -- 172.21.15.120:0/2776572533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1158102320 con 0x7f1158108680 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1156ffd640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1157fff640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f11500097b0 tx=0x7f1150031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1154ff9640 1 -- 172.21.15.120:0/2776572533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150003b70 con 0x7f1158108680 2024-09-10T23:03:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.513+0000 7f1154ff9640 1 -- 172.21.15.120:0/2776572533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1150003d10 con 0x7f1158108680 2024-09-10T23:03:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.514+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11581a2dc0 con 0x7f1158108680 2024-09-10T23:03:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.514+0000 7f1154ff9640 1 -- 172.21.15.120:0/2776572533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115002fc10 con 0x7f1158108680 2024-09-10T23:03:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.514+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11581a3280 con 0x7f1158108680 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.516+0000 7f1160a21740 1 Processor -- start 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1154ff9640 1 -- 172.21.15.120:0/2776572533 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1150004040 con 0x7f1158108680 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 -- start start 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fb7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a38f216f0 con 0x562a38faf860 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a38f10b20 con 0x562a38fb4bd0 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1160a21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a38f14930 con 0x562a38fb2190 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47840/0 (socket says 172.21.15.120:47840) 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f11577fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 -- 172.21.15.120:0/254727503 learned_addr learned my addr 172.21.15.120:0/254727503 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fb7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 -- 172.21.15.120:0/254727503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 msgr2=0x562a38fb4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 -- 172.21.15.120:0/254727503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 msgr2=0x562a38fb7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fb7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1156ffd640 1 -- 172.21.15.120:0/254727503 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a38ed6530 con 0x562a38faf860 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f11577fe640 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.517+0000 7f1154ff9640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 0x7f11440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f1156ffd640 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1140013a10 tx=0x7f1140009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8cc8a67f5fe6ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f11577fe640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 0x7f11440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f1154ff9640 1 -- 172.21.15.120:0/2776572533 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1150106ef0 con 0x7f1158108680 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f114dffb640 1 -- 172.21.15.120:0/254727503 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140004050 con 0x562a38faf860 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f114dffb640 1 -- 172.21.15.120:0/254727503 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114000fe30 con 0x562a38faf860 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f114dffb640 1 -- 172.21.15.120:0/254727503 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140003a80 con 0x562a38faf860 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f11577fe640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 0x7f11440b77a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1148009f70 tx=0x7f1148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.518+0000 7f1160a21740 1 -- 172.21.15.120:0/254727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 msgr2=0x562a38fb1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1140013a10 tx=0x7f1140009c00 comp rx=0 tx=0).stop 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- 172.21.15.120:0/254727503 shutdown_connections 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fb7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- 172.21.15.120:0/254727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb1c50 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- 172.21.15.120:0/254727503 >> 172.21.15.120:0/254727503 conn(0x562a38faa9b0 msgr2=0x562a38fab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- 172.21.15.120:0/254727503 shutdown_connections 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- 172.21.15.120:0/254727503 wait complete. 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- start start 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 Processor -- start 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a38e48ab0 con 0x562a38faf860 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a38eb6730 con 0x562a38fb4bd0 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a38fb7bb0 con 0x562a38fb2190 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f1160a21740 10 client.0 _ll_register_callbacks cb 0x562a38f0a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.519+0000 7f11577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f11577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47842/0 (socket says 172.21.15.120:47842) 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f11577fe640 1 -- 172.21.15.120:0/854219138 learned_addr learned my addr 172.21.15.120:0/854219138 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1157fff640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 -- 172.21.15.120:0/854219138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 msgr2=0x562a38fba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 -- 172.21.15.120:0/854219138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 msgr2=0x562a38fb8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 -- 172.21.15.120:0/854219138 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a38c72790 con 0x562a38fb2190 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f11577fe640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1156ffd640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb9590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1140009710 tx=0x7f11400135b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f9afea2d31eab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400054d0 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114000fe30 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11400046c0 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11400048c0 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1160a21740 10 client.27205 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.520+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a38ceb620 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114000fe30 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f114001e070 con 0x562a38fb2190 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 10 client.27205 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 20 client.27205 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a38fc4530 0x562a38fc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 4 client.27205 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a38f23640 con 0x562a38fc4530 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.521+0000 7f11577fe640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a38fc4530 0x562a38fc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.522+0000 7f11577fe640 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a38fc4530 0x562a38fc6920 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67992bedd3c2eda9 server_cookie=76069afd7f3a4716 in_seq=0 out_seq=0 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.522+0000 7f1136ffd640 10 client.27205 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1136ffd640 1 -- 172.21.15.120:0/854219138 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a38f23640 con 0x562a38fc4530 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1136ffd640 10 client.27205 handle_command_reply: tid=0 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 client.27205 shutdown 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 msgr2=0x562a38fb9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb9590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1140009710 tx=0x7f11400135b0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 shutdown_connections 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a38fc4530 0x562a38fc6920 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fb9590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 --2- 172.21.15.120:0/854219138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1136ffd640 0 client.27205 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 >> 172.21.15.120:0/854219138 conn(0x562a38faa9b0 msgr2=0x562a38fa9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 shutdown_connections 2024-09-10T23:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 1 -- 172.21.15.120:0/854219138 wait complete. 2024-09-10T23:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.524+0000 7f1160a21740 20 client.27205 trim_cache size 0 max 16384 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 Processor -- start 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 -- start start 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fcb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fcb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11581a9ff0 con 0x562a38faf860 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11581ac870 con 0x562a38fb4bd0 2024-09-10T23:03:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1160a21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11581aa660 con 0x562a38fb2190 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47854/0 (socket says 172.21.15.120:47854) 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fcb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 -- 172.21.15.120:0/1560871838 learned_addr learned my addr 172.21.15.120:0/1560871838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1157fff640 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fcb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 -- 172.21.15.120:0/1560871838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 msgr2=0x562a38fcb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fcb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 -- 172.21.15.120:0/1560871838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 msgr2=0x562a38fcb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fcb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 -- 172.21.15.120:0/1560871838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a38fc6c90 con 0x562a38faf860 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f1157fff640 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fcb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.543+0000 7f11577fe640 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f11480044e0 tx=0x7f114801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08951bfac8951b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f114d7fa640 1 -- 172.21.15.120:0/1560871838 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114801f710 con 0x562a38faf860 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f114d7fa640 1 -- 172.21.15.120:0/1560871838 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114801fd60 con 0x562a38faf860 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f114d7fa640 1 -- 172.21.15.120:0/1560871838 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148032450 con 0x562a38faf860 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 -- 172.21.15.120:0/1560871838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 msgr2=0x562a38fb8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f11480044e0 tx=0x7f114801b000 comp rx=0 tx=0).stop 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 -- 172.21.15.120:0/1560871838 shutdown_connections 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38fb4bd0 0x562a38fcb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb2190 0x562a38fcb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 --2- 172.21.15.120:0/1560871838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38faf860 0x562a38fb8840 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 -- 172.21.15.120:0/1560871838 >> 172.21.15.120:0/1560871838 conn(0x562a38faa9b0 msgr2=0x562a38fb8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 -- 172.21.15.120:0/1560871838 shutdown_connections 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.544+0000 7f1160a21740 1 -- 172.21.15.120:0/1560871838 wait complete. 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 -- start start 2024-09-10T23:03:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 Processor -- start 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38faf860 0x562a38fbfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb4bd0 0x562a38fbe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11581aad30 con 0x562a38fb2190 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11581ac1a0 con 0x562a38faf860 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11581ab400 con 0x562a38fb4bd0 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1160a21740 10 client.0 _ll_register_callbacks cb 0x7f11480097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1156ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47856/0 (socket says 172.21.15.120:47856) 2024-09-10T23:03:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1156ffd640 1 -- 172.21.15.120:0/3748400852 learned_addr learned my addr 172.21.15.120:0/3748400852 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f1157fff640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb4bd0 0x562a38fbe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.545+0000 7f11577fe640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38faf860 0x562a38fbfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 -- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb4bd0 msgr2=0x562a38fbe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb4bd0 0x562a38fbe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 -- 172.21.15.120:0/3748400852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38faf860 msgr2=0x562a38fbfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38faf860 0x562a38fbfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 -- 172.21.15.120:0/3748400852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a38fbede0 con 0x562a38fb2190 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1156ffd640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f11400059f0 tx=0x7f114000ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c49a56afd081c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114003a060 con 0x562a38fb2190 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f114002c0f0 con 0x562a38fb2190 2024-09-10T23:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114801fd60 con 0x562a38fb2190 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f114002c310 con 0x562a38fb2190 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1160a21740 10 client.17739 fetch_fsmap learned FSMap version 26 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a390319f0 con 0x562a38fb2190 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114801fd60 con 0x562a38fb2190 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.546+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f114001e040 con 0x562a38fb2190 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 10 client.17739 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 20 client.17739 populate_metadata read hostname 'smithi120' 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a39032370 0x562a39034790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 4 client.17739 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a38f1f0f0 con 0x562a39032370 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f11577fe640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a39032370 0x562a39034790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f1136ffd640 10 client.17739 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.547+0000 7f11577fe640 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a39032370 0x562a39034790 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa3fe70fe9cb2d3 server_cookie=4c1a3739e9d7f96d in_seq=0 out_seq=0 2024-09-10T23:03:58.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1952835 15.94 MB/sec execute 533 sec latency 44.891 ms 2024-09-10T23:03:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1136ffd640 1 -- 172.21.15.120:0/3748400852 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a38f1f0f0 con 0x562a39032370 2024-09-10T23:03:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1136ffd640 10 client.17739 handle_command_reply: tid=0 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 client.17739 shutdown 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 msgr2=0x562a38fc0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f11400059f0 tx=0x7f114000ffa0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 shutdown_connections 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a39032370 0x562a39034790 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a38fb4bd0 0x562a38fbe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a38fb2190 0x562a38fc0220 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 --2- 172.21.15.120:0/3748400852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a38faf860 0x562a38fbfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 >> 172.21.15.120:0/3748400852 conn(0x562a38faa9b0 msgr2=0x562a38fa93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1136ffd640 0 client.17739 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:03:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 shutdown_connections 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 1 -- 172.21.15.120:0/3748400852 wait complete. 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.781+0000 7f1160a21740 20 client.17739 trim_cache size 0 max 16384 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:03:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:03:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 msgr2=0x7f11440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 0x7f11440b77a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1148009f70 tx=0x7f1148005f90 comp rx=0 tx=0).stop 2024-09-10T23:03:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 msgr2=0x7f1158101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f11500097b0 tx=0x7f1150031770 comp rx=0 tx=0).stop 2024-09-10T23:03:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 shutdown_connections 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11440b52e0 0x7f11440b77a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1158108680 0x7f1158101ab0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1158105b10 0x7f1158101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 --2- 172.21.15.120:0/2776572533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f115806b3d0 0x7f1158102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 >> 172.21.15.120:0/2776572533 conn(0x7f11580fd140 msgr2=0x7f1158103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 shutdown_connections 2024-09-10T23:03:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:03:58.782+0000 7f115dc52640 1 -- 172.21.15.120:0/2776572533 wait complete. 2024-09-10T23:03:59.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1954893 15.93 MB/sec execute 534 sec latency 105.802 ms 2024-09-10T23:04:00.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:03:59 smithi176 ceph-mon[34982]: pgmap v693: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-10T23:04:00.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:03:59 smithi120 ceph-mon[31247]: pgmap v693: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-10T23:04:00.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:03:59 smithi153 ceph-mon[34951]: pgmap v693: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-10T23:04:00.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1957988 15.94 MB/sec execute 535 sec latency 27.725 ms 2024-09-10T23:04:01.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1960923 15.93 MB/sec execute 536 sec latency 38.411 ms 2024-09-10T23:04:02.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:01 smithi176 ceph-mon[34982]: pgmap v694: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-09-10T23:04:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:01 smithi120 ceph-mon[31247]: pgmap v694: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-09-10T23:04:02.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:01 smithi153 ceph-mon[34951]: pgmap v694: 97 pgs: 97 active+clean; 54 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-09-10T23:04:02.642 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1963533 15.93 MB/sec execute 537 sec latency 29.589 ms 2024-09-10T23:04:03.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1966182 15.93 MB/sec execute 538 sec latency 12.508 ms 2024-09-10T23:04:04.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:03 smithi176 ceph-mon[34982]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-10T23:04:04.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:03 smithi120 ceph-mon[31247]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-10T23:04:04.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:03 smithi153 ceph-mon[34951]: pgmap v695: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-10T23:04:04.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1969189 15.93 MB/sec execute 539 sec latency 18.399 ms 2024-09-10T23:04:05.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1972476 15.93 MB/sec execute 540 sec latency 14.796 ms 2024-09-10T23:04:06.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:05 smithi120 ceph-mon[31247]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-09-10T23:04:06.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:05 smithi153 ceph-mon[34951]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-09-10T23:04:06.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:05 smithi176 ceph-mon[34982]: pgmap v696: 97 pgs: 97 active+clean; 44 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 383 op/s 2024-09-10T23:04:06.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1975515 15.94 MB/sec execute 541 sec latency 110.200 ms 2024-09-10T23:04:07.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1978214 15.93 MB/sec execute 542 sec latency 26.343 ms 2024-09-10T23:04:08.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:07 smithi120 ceph-mon[31247]: pgmap v697: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-10T23:04:08.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:07 smithi153 ceph-mon[34951]: pgmap v697: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-10T23:04:08.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:07 smithi176 ceph-mon[34982]: pgmap v697: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-10T23:04:08.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1981315 15.93 MB/sec execute 543 sec latency 12.373 ms 2024-09-10T23:04:09.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1984080 15.93 MB/sec execute 544 sec latency 58.042 ms 2024-09-10T23:04:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:09 smithi120 ceph-mon[31247]: pgmap v698: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 452 op/s 2024-09-10T23:04:10.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:09 smithi153 ceph-mon[34951]: pgmap v698: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 452 op/s 2024-09-10T23:04:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:09 smithi176 ceph-mon[34982]: pgmap v698: 97 pgs: 97 active+clean; 50 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 452 op/s 2024-09-10T23:04:10.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1986850 15.93 MB/sec execute 545 sec latency 20.484 ms 2024-09-10T23:04:11.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1989837 15.93 MB/sec execute 546 sec latency 60.134 ms 2024-09-10T23:04:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- 172.21.15.120:0/719347008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb80103370 msgr2=0x7fcb80109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/719347008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb80103370 0x7fcb80109cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009930 tx=0x7fcb7802a420 comp rx=0 tx=0).stop 2024-09-10T23:04:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- 172.21.15.120:0/719347008 shutdown_connections 2024-09-10T23:04:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/719347008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb80103370 0x7fcb80109cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/719347008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb801029b0 0x7fcb80102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/719347008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 0x7fcb80101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- 172.21.15.120:0/719347008 >> 172.21.15.120:0/719347008 conn(0x7fcb800fd140 msgr2=0x7fcb800ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- 172.21.15.120:0/719347008 shutdown_connections 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- 172.21.15.120:0/719347008 wait complete. 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 Processor -- start 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.872+0000 7fcb87fd3640 1 -- start start 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 0x7fcb80198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb801029b0 0x7fcb801993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e600 con 0x7fcb80103370 2024-09-10T23:04:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e480 con 0x7fcb80101860 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e780 con 0x7fcb801029b0 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37092/0 (socket says 172.21.15.120:37092) 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb8556e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb801029b0 0x7fcb801993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 -- 172.21.15.120:0/2083173078 learned_addr learned my addr 172.21.15.120:0/2083173078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 0x7fcb80198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 -- 172.21.15.120:0/2083173078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb801029b0 msgr2=0x7fcb801993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb801029b0 0x7fcb801993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 -- 172.21.15.120:0/2083173078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 msgr2=0x7fcb80198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 0x7fcb80198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 -- 172.21.15.120:0/2083173078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8019a8d0 con 0x7fcb80103370 2024-09-10T23:04:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb86570640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fcb78031500 tx=0x7fcb780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb7effd640 1 -- 172.21.15.120:0/2083173078 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7803f030 con 0x7fcb80103370 2024-09-10T23:04:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb7effd640 1 -- 172.21.15.120:0/2083173078 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb78003cb0 con 0x7fcb80103370 2024-09-10T23:04:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8019ab60 con 0x7fcb80103370 2024-09-10T23:04:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb7effd640 1 -- 172.21.15.120:0/2083173078 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78035070 con 0x7fcb80103370 2024-09-10T23:04:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.873+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801a1290 con 0x7fcb80103370 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb7effd640 1 -- 172.21.15.120:0/2083173078 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcb7803bb70 con 0x7fcb80103370 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 Processor -- start 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 -- start start 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3a7c70 0x558f0b3aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 0x558f0b3af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f0b3156f0 con 0x558f0b3aa5a0 2024-09-10T23:04:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558f0b304b20 con 0x558f0b3acfe0 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.876+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558f0b308930 con 0x558f0b3a7c70 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37098/0 (socket says 172.21.15.120:37098) 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1750023192 learned_addr learned my addr 172.21.15.120:0/1750023192 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb8556e640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 0x558f0b3af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb8556e640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3a7c70 0x558f0b3aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1750023192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3a7c70 msgr2=0x558f0b3aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3a7c70 0x558f0b3aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1750023192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 msgr2=0x558f0b3af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 0x558f0b3af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1750023192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0b2ca530 con 0x558f0b3aa5a0 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb8556e640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 0x558f0b3af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb7effd640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 0x7fcb680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009c50 tx=0x7fcb70010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfdd00162d1a392 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb8556e640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 0x7fcb680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb7effd640 1 -- 172.21.15.120:0/2083173078 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb781063f0 con 0x7fcb80103370 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb67fff640 1 -- 172.21.15.120:0/1750023192 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7001b480 con 0x558f0b3aa5a0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb67fff640 1 -- 172.21.15.120:0/1750023192 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70012900 con 0x558f0b3aa5a0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.877+0000 7fcb67fff640 1 -- 172.21.15.120:0/1750023192 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70011840 con 0x558f0b3aa5a0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1750023192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 msgr2=0x558f0b3ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8556e640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 0x7fcb680b77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005fd0 tx=0x7fcb74004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009c50 tx=0x7fcb70010740 comp rx=0 tx=0).stop 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1750023192 shutdown_connections 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3acfe0 0x558f0b3af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3aa5a0 0x558f0b3ac990 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1750023192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3a7c70 0x558f0b3aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1750023192 >> 172.21.15.120:0/1750023192 conn(0x558f0b39e9b0 msgr2=0x558f0b39f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1750023192 shutdown_connections 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.878+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1750023192 wait complete. 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 -- start start 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 Processor -- start 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f0b23cab0 con 0x558f0b3acfe0 2024-09-10T23:04:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558f0b2aa730 con 0x558f0b3a7c70 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558f0b3affc0 con 0x558f0b3aa5a0 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8ae80740 10 client.0 _ll_register_callbacks cb 0x558f0b2fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37110/0 (socket says 172.21.15.120:37110) 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb85d6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb8556e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.879+0000 7fcb86570640 1 -- 172.21.15.120:0/3330109860 learned_addr learned my addr 172.21.15.120:0/3330109860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 -- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 msgr2=0x558f0b3b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 -- 172.21.15.120:0/3330109860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 msgr2=0x558f0b3b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 -- 172.21.15.120:0/3330109860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0b066790 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb86570640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcb7802a460 tx=0x7fcb780ccaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5e425199324334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78049040 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb780d5ee0 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb70012900 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb78003e60 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 10 client.17751 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558f0b0df620 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70012900 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb780cbe40 con 0x558f0b3acfe0 2024-09-10T23:04:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 10 client.17751 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.880+0000 7fcb8ae80740 20 client.17751 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b3bcb90 0x558f0b3bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb8ae80740 4 client.17751 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f0b317640 con 0x558f0b3bcb90 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b3bcb90 0x558f0b3bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb657fa640 10 client.17751 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.881+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b3bcb90 0x558f0b3bf020 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e970fc5acbcf033f server_cookie=14b9ff882b86cc39 in_seq=0 out_seq=0 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.882+0000 7fcb657fa640 1 -- 172.21.15.120:0/3330109860 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f0b317640 con 0x558f0b3bcb90 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.882+0000 7fcb657fa640 10 client.17751 handle_command_reply: tid=0 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.882+0000 7fcb8ae80740 1 client.17751 shutdown 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.882+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 msgr2=0x558f0b3b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.882+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcb7802a460 tx=0x7fcb780ccaa0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 shutdown_connections 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b3bcb90 0x558f0b3bf020 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b2580 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb657fa640 0 client.17751 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/3330109860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 >> 172.21.15.120:0/3330109860 conn(0x558f0b39e9b0 msgr2=0x558f0b39dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 shutdown_connections 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 1 -- 172.21.15.120:0/3330109860 wait complete. 2024-09-10T23:04:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.883+0000 7fcb8ae80740 20 client.17751 trim_cache size 0 max 16384 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 Processor -- start 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 -- start start 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 0x558f0b3a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3acfe0 0x558f0b3a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a5be0 con 0x558f0b3a7c70 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a8460 con 0x558f0b3aa5a0 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a6250 con 0x558f0b3acfe0 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37118/0 (socket says 172.21.15.120:37118) 2024-09-10T23:04:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1380602479 learned_addr learned my addr 172.21.15.120:0/1380602479 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb86570640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3acfe0 0x558f0b3a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8556e640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 0x558f0b3a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1380602479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3acfe0 msgr2=0x558f0b3a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3acfe0 0x558f0b3a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1380602479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 msgr2=0x558f0b3a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 0x558f0b3a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 -- 172.21.15.120:0/1380602479 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0b3bf600 con 0x558f0b3a7c70 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb8556e640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 0x558f0b3a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009c50 tx=0x7fcb70010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e36fcf99f59d162 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb677fe640 1 -- 172.21.15.120:0/1380602479 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb700050a0 con 0x558f0b3a7c70 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb677fe640 1 -- 172.21.15.120:0/1380602479 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70012760 con 0x558f0b3a7c70 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.899+0000 7fcb677fe640 1 -- 172.21.15.120:0/1380602479 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003850 con 0x558f0b3a7c70 2024-09-10T23:04:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1380602479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 msgr2=0x558f0b3b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009c50 tx=0x7fcb70010fb0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1380602479 shutdown_connections 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3acfe0 0x558f0b3a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3aa5a0 0x558f0b3a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1380602479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3a7c70 0x558f0b3b2560 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1380602479 >> 172.21.15.120:0/1380602479 conn(0x558f0b39e9b0 msgr2=0x558f0b3a2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1380602479 shutdown_connections 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1380602479 wait complete. 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- start start 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 Processor -- start 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a6920 con 0x558f0b3acfe0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a7d90 con 0x558f0b3a7c70 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a6ff0 con 0x558f0b3aa5a0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.900+0000 7fcb8ae80740 10 client.0 _ll_register_callbacks cb 0x7fcb78005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8556e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8556e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55342/0 (socket says 172.21.15.120:55342) 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37128/0 (socket says 172.21.15.120:37128) 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8556e640 1 -- 172.21.15.120:0/1505902788 learned_addr learned my addr 172.21.15.120:0/1505902788 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 -- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 msgr2=0x558f0b3b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 -- 172.21.15.120:0/1505902788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 msgr2=0x558f0b3b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 -- 172.21.15.120:0/1505902788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0b3b84f0 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb86570640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcb780c9e50 tx=0x7fcb78009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20763f7be4ab30cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb657fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78049070 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb78003800 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb70012760 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb780d1710 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8ae80740 10 client.17757 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558f0b429a90 con 0x558f0b3acfe0 2024-09-10T23:04:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.901+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70012760 con 0x558f0b3acfe0 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb780c8530 con 0x558f0b3acfe0 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 10 client.17757 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 20 client.17757 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b42a130 0x558f0b42c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 4 client.17757 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f0b3130f0 con 0x558f0b42a130 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b42a130 0x558f0b42c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb657fa640 10 client.17757 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:11.902+0000 7fcb85d6f640 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b42a130 0x558f0b42c550 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff445153bf86855 server_cookie=73960e3fb2ac9d4f in_seq=0 out_seq=0 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb657fa640 1 -- 172.21.15.120:0/1505902788 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f0b3130f0 con 0x558f0b42a130 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb657fa640 10 client.17757 handle_command_reply: tid=0 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 client.17757 shutdown 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 msgr2=0x558f0b3b7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcb780c9e50 tx=0x7fcb78009580 comp rx=0 tx=0).stop 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 shutdown_connections 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558f0b42a130 0x558f0b42c550 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f0b3acfe0 0x558f0b3b7e80 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f0b3aa5a0 0x558f0b3b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb657fa640 0 client.17757 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 --2- 172.21.15.120:0/1505902788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558f0b3a7c70 0x558f0b3b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.123+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 >> 172.21.15.120:0/1505902788 conn(0x558f0b39e9b0 msgr2=0x558f0b3a2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 shutdown_connections 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb8ae80740 1 -- 172.21.15.120:0/1505902788 wait complete. 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb8ae80740 20 client.17757 trim_cache size 0 max 16384 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 msgr2=0x7fcb680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 0x7fcb680b77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005fd0 tx=0x7fcb74004ab0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 msgr2=0x7fcb8019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fcb78031500 tx=0x7fcb780042f0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 shutdown_connections 2024-09-10T23:04:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb680b52e0 0x7fcb680b77a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb80103370 0x7fcb8019a010 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb801029b0 0x7fcb801993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 --2- 172.21.15.120:0/2083173078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb80101860 0x7fcb80198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.124+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 >> 172.21.15.120:0/2083173078 conn(0x7fcb800fd140 msgr2=0x7fcb80105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.125+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 shutdown_connections 2024-09-10T23:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:12.125+0000 7fcb87fd3640 1 -- 172.21.15.120:0/2083173078 wait complete. 2024-09-10T23:04:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:11 smithi120 ceph-mon[31247]: pgmap v699: 97 pgs: 97 active+clean; 51 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-10T23:04:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:11 smithi153 ceph-mon[34951]: pgmap v699: 97 pgs: 97 active+clean; 51 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-10T23:04:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:11 smithi176 ceph-mon[34982]: pgmap v699: 97 pgs: 97 active+clean; 51 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-10T23:04:12.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1992028 15.92 MB/sec execute 547 sec latency 120.433 ms 2024-09-10T23:04:13.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1995192 15.92 MB/sec execute 548 sec latency 13.112 ms 2024-09-10T23:04:14.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:13 smithi120 ceph-mon[31247]: pgmap v700: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-09-10T23:04:14.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:13 smithi153 ceph-mon[34951]: pgmap v700: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-09-10T23:04:14.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:13 smithi176 ceph-mon[34982]: pgmap v700: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-09-10T23:04:14.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 1998089 15.92 MB/sec execute 549 sec latency 13.368 ms 2024-09-10T23:04:15.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2001077 15.92 MB/sec execute 550 sec latency 19.177 ms 2024-09-10T23:04:16.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:15 smithi120 ceph-mon[31247]: pgmap v701: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-09-10T23:04:16.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:15 smithi153 ceph-mon[34951]: pgmap v701: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-09-10T23:04:16.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:15 smithi176 ceph-mon[34982]: pgmap v701: 97 pgs: 97 active+clean; 54 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-09-10T23:04:16.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2004196 15.92 MB/sec execute 551 sec latency 33.569 ms 2024-09-10T23:04:17.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2007347 15.93 MB/sec execute 552 sec latency 52.251 ms 2024-09-10T23:04:18.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:17 smithi120 ceph-mon[31247]: pgmap v702: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 440 op/s 2024-09-10T23:04:18.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:17 smithi153 ceph-mon[34951]: pgmap v702: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 440 op/s 2024-09-10T23:04:18.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:17 smithi176 ceph-mon[34982]: pgmap v702: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 440 op/s 2024-09-10T23:04:18.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2009957 15.92 MB/sec execute 553 sec latency 33.103 ms 2024-09-10T23:04:19.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2012657 15.92 MB/sec execute 554 sec latency 14.392 ms 2024-09-10T23:04:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:19 smithi120 ceph-mon[31247]: pgmap v703: 97 pgs: 97 active+clean; 44 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-09-10T23:04:20.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:19 smithi153 ceph-mon[34951]: pgmap v703: 97 pgs: 97 active+clean; 44 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-09-10T23:04:20.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:19 smithi176 ceph-mon[34982]: pgmap v703: 97 pgs: 97 active+clean; 44 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-09-10T23:04:20.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2015262 15.91 MB/sec execute 555 sec latency 23.720 ms 2024-09-10T23:04:21.643 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2018303 15.92 MB/sec execute 556 sec latency 12.529 ms 2024-09-10T23:04:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:21 smithi120 ceph-mon[31247]: pgmap v704: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-10T23:04:22.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:21 smithi153 ceph-mon[34951]: pgmap v704: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-10T23:04:22.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:21 smithi176 ceph-mon[34982]: pgmap v704: 97 pgs: 97 active+clean; 49 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-10T23:04:22.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2021368 15.92 MB/sec execute 557 sec latency 54.380 ms 2024-09-10T23:04:23.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2023873 15.91 MB/sec execute 558 sec latency 30.677 ms 2024-09-10T23:04:24.000 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:04:24.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:23 smithi120 ceph-mon[31247]: pgmap v705: 97 pgs: 97 active+clean; 49 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T23:04:24.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:23 smithi153 ceph-mon[34951]: pgmap v705: 97 pgs: 97 active+clean; 49 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T23:04:24.259 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:23 smithi176 ceph-mon[34982]: pgmap v705: 97 pgs: 97 active+clean; 49 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-09-10T23:04:24.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2026561 15.91 MB/sec execute 559 sec latency 39.596 ms 2024-09-10T23:04:24.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 -- 172.21.15.120:0/2653577885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 msgr2=0x7f663410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 --2- 172.21.15.120:0/2653577885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f663410f560 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f662c009ca0 tx=0x7f662c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 -- 172.21.15.120:0/2653577885 shutdown_connections 2024-09-10T23:04:24.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 --2- 172.21.15.120:0/2653577885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f663410f560 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 --2- 172.21.15.120:0/2653577885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66341045c0 0x7f663410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 --2- 172.21.15.120:0/2653577885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6634103ca0 0x7f6634104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.757+0000 7f663a6a7640 1 -- 172.21.15.120:0/2653577885 >> 172.21.15.120:0/2653577885 conn(0x7f66340fd700 msgr2=0x7f66340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.758+0000 7f663a6a7640 1 -- 172.21.15.120:0/2653577885 shutdown_connections 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.758+0000 7f663a6a7640 1 -- 172.21.15.120:0/2653577885 wait complete. 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.758+0000 7f663a6a7640 1 Processor -- start 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.758+0000 7f663a6a7640 1 -- start start 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6634103ca0 0x7f663419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 0x7f663419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6634112d70 con 0x7f663410d100 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6634112bf0 con 0x7f6634103ca0 2024-09-10T23:04:24.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6634112ef0 con 0x7f66341045c0 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36182/0 (socket says 172.21.15.120:36182) 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 -- 172.21.15.120:0/1744273318 learned_addr learned my addr 172.21.15.120:0/1744273318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f66337fe640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 0x7f663419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6633fff640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6634103ca0 0x7f663419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 -- 172.21.15.120:0/1744273318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 msgr2=0x7f663419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 0x7f663419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 -- 172.21.15.120:0/1744273318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6634103ca0 msgr2=0x7f663419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6634103ca0 0x7f663419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 -- 172.21.15.120:0/1744273318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66341a1500 con 0x7f663410d100 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f66337fe640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 0x7f663419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:24.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f6638c44640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f662c005cc0 tx=0x7f662c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f66317fa640 1 -- 172.21.15.120:0/1744273318 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c003b70 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f66317fa640 1 -- 172.21.15.120:0/1744273318 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c0318f0 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.759+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66341a1790 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.760+0000 7f66317fa640 1 -- 172.21.15.120:0/1744273318 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c02f950 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.760+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66341a7eb0 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 Processor -- start 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f66317fa640 1 -- 172.21.15.120:0/1744273318 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f662c02fb50 con 0x7f663410d100 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 -- start start 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f69d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55801f613e50 con 0x55801f69dda0 2024-09-10T23:04:24.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55801f603280 con 0x55801f69d480 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f663ca7c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55801f607090 con 0x55801f6a9d30 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36188/0 (socket says 172.21.15.120:36188) 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 -- 172.21.15.120:0/1654330752 learned_addr learned my addr 172.21.15.120:0/1654330752 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f66337fe640 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f66337fe640 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f69d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 -- 172.21.15.120:0/1654330752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 msgr2=0x55801f6ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 -- 172.21.15.120:0/1654330752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 msgr2=0x55801f69d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f69d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.763+0000 7f6633fff640 1 -- 172.21.15.120:0/1654330752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801f361650 con 0x55801f69dda0 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f6633fff640 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6624009cf0 tx=0x7f6624011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae575557161781d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f66317fa640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 0x7f661c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f661a7fc640 1 -- 172.21.15.120:0/1654330752 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662401e070 con 0x55801f69dda0 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f66337fe640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 0x7f661c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f661a7fc640 1 -- 172.21.15.120:0/1654330752 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6624009de0 con 0x55801f69dda0 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f661a7fc640 1 -- 172.21.15.120:0/1654330752 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66240196b0 con 0x55801f69dda0 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.764+0000 7f66317fa640 1 -- 172.21.15.120:0/1744273318 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f662c106c20 con 0x7f663410d100 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- 172.21.15.120:0/1654330752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 msgr2=0x55801f6a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f66337fe640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 0x7f661c0b7820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6628009fd0 tx=0x7f6628005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6624009cf0 tx=0x7f6624011810 comp rx=0 tx=0).stop 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- 172.21.15.120:0/1654330752 shutdown_connections 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69dda0 0x55801f6a97f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 --2- 172.21.15.120:0/1654330752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f69d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- 172.21.15.120:0/1654330752 >> 172.21.15.120:0/1654330752 conn(0x55801f69b2b0 msgr2=0x55801f69c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- 172.21.15.120:0/1654330752 shutdown_connections 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- 172.21.15.120:0/1654330752 wait complete. 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 -- start start 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.765+0000 7f663ca7c740 1 Processor -- start 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55801f464700 con 0x55801f69d480 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55801f59e110 con 0x55801f69dda0 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55801f6ac9b0 con 0x55801f6a9d30 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f663ca7c740 10 client.0 _ll_register_callbacks cb 0x55801f5b7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f66337fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36202/0 (socket says 172.21.15.120:36202) 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 -- 172.21.15.120:0/2847688713 learned_addr learned my addr 172.21.15.120:0/2847688713 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6638c44640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 -- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 msgr2=0x55801f6a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 -- 172.21.15.120:0/2847688713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 msgr2=0x55801f6a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 -- 172.21.15.120:0/2847688713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801f5cbc40 con 0x55801f69d480 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f66337fe640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f6633fff640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f66240058c0 tx=0x7f66240052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89baac3cd61b17ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f66137fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662401e040 con 0x55801f69d480 2024-09-10T23:04:24.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.766+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6624016070 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6624009de0 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6624005c50 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 10 client.17769 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55801f6b8340 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6624009de0 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6624021070 con 0x55801f69d480 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 10 client.17769 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 10 client.17769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 20 client.17769 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f6b9740 0x55801f6bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 4 client.17769 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55801f615da0 con 0x55801f6b9740 2024-09-10T23:04:24.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.767+0000 7f66337fe640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f6b9740 0x55801f6bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.768+0000 7f66137fe640 10 client.17769 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.768+0000 7f66337fe640 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f6b9740 0x55801f6bbb30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47271f9596ab6f8 server_cookie=c4a6aba8a2083269 in_seq=0 out_seq=0 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.768+0000 7f66137fe640 1 -- 172.21.15.120:0/2847688713 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55801f615da0 con 0x55801f6b9740 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.768+0000 7f66137fe640 10 client.17769 handle_command_reply: tid=0 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.768+0000 7f663ca7c740 1 client.17769 shutdown 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 msgr2=0x55801f6a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f66240058c0 tx=0x7f66240052f0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 shutdown_connections 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f6b9740 0x55801f6bbb30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f6a9d30 0x55801f6a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f66137fe640 0 client.17769 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2847688713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f69d480 0x55801f6a9640 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 >> 172.21.15.120:0/2847688713 conn(0x55801f69b2b0 msgr2=0x55801f6a0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 shutdown_connections 2024-09-10T23:04:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 1 -- 172.21.15.120:0/2847688713 wait complete. 2024-09-10T23:04:24.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.769+0000 7f663ca7c740 20 client.17769 trim_cache size 0 max 16384 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 Processor -- start 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 -- start start 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f6b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 0x55801f6a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66341ae4f0 con 0x55801f6a9d30 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b0d70 con 0x55801f69d480 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f663ca7c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66341aeb60 con 0x55801f69dda0 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f6633fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f6b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36216/0 (socket says 172.21.15.120:36216) 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f66337fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 0x55801f6a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.788+0000 7f6638c44640 1 -- 172.21.15.120:0/2546627767 learned_addr learned my addr 172.21.15.120:0/2546627767 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 -- 172.21.15.120:0/2546627767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 msgr2=0x55801f6a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 0x55801f6a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 -- 172.21.15.120:0/2546627767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 msgr2=0x55801f6b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f6b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 -- 172.21.15.120:0/2546627767 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801f696b60 con 0x55801f6a9d30 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6633fff640 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f6b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f66337fe640 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 0x55801f6a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6638c44640 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f662c104ac0 tx=0x7f662c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7059354b2010842a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6619ffb640 1 -- 172.21.15.120:0/2546627767 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0cdb10 con 0x55801f6a9d30 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6619ffb640 1 -- 172.21.15.120:0/2546627767 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c036070 con 0x55801f6a9d30 2024-09-10T23:04:24.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.789+0000 7f6619ffb640 1 -- 172.21.15.120:0/2546627767 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0c9680 con 0x55801f6a9d30 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- 172.21.15.120:0/2546627767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 msgr2=0x55801f6a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f662c104ac0 tx=0x7f662c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- 172.21.15.120:0/2546627767 shutdown_connections 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6a5850 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69dda0 0x55801f6a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 --2- 172.21.15.120:0/2546627767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69d480 0x55801f6b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- 172.21.15.120:0/2546627767 >> 172.21.15.120:0/2546627767 conn(0x55801f69b2b0 msgr2=0x55801f69c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- 172.21.15.120:0/2546627767 shutdown_connections 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- 172.21.15.120:0/2546627767 wait complete. 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.790+0000 7f663ca7c740 1 -- start start 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 Processor -- start 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 0x55801f6adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66341af230 con 0x55801f6a9d30 2024-09-10T23:04:24.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b06a0 con 0x55801f69dda0 2024-09-10T23:04:24.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66341af900 con 0x55801f69d480 2024-09-10T23:04:24.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f663ca7c740 10 client.0 _ll_register_callbacks cb 0x7f662401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:24.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f6633fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 0x55801f6adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f6638c44640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36226/0 (socket says 172.21.15.120:36226) 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f6638c44640 1 -- 172.21.15.120:0/226808098 learned_addr learned my addr 172.21.15.120:0/226808098 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 -- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 msgr2=0x55801f6adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 0x55801f6adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 -- 172.21.15.120:0/226808098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 msgr2=0x55801f6ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f66337fe640 1 -- 172.21.15.120:0/226808098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801f725520 con 0x55801f69dda0 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.791+0000 7f6633fff640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 0x55801f6adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66337fe640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6ae1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55801f6bc1c0 tx=0x7f66280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6d95dd775e69c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:24.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628039060 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f66280221f0 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c036070 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6628022a80 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 10 client.27089 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55801f726890 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c036070 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6628033030 con 0x55801f69dda0 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 10 client.27089 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 10 client.27089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 20 client.27089 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:24.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.792+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f726f60 0x55801f729350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.793+0000 7f663ca7c740 4 client.27089 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.793+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55801f611850 con 0x55801f726f60 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.793+0000 7f6633fff640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f726f60 0x55801f729350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.793+0000 7f6633fff640 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f726f60 0x55801f729350 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc9ebb223fdff8d server_cookie=4a3dc8a9138bce9a in_seq=0 out_seq=0 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.793+0000 7f66137fe640 10 client.27089 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.794+0000 7f66137fe640 1 -- 172.21.15.120:0/226808098 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x55801f611850 con 0x55801f726f60 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.794+0000 7f66137fe640 10 client.27089 handle_command_reply: tid=0 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.794+0000 7f663ca7c740 1 client.27089 shutdown 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 msgr2=0x55801f6ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6ae1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55801f6bc1c0 tx=0x7f66280046c0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 shutdown_connections 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55801f726f60 0x55801f729350 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55801f6a9d30 0x55801f6ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55801f69dda0 0x55801f6ae1d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 --2- 172.21.15.120:0/226808098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801f69d480 0x55801f6adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f66137fe640 0 client.27089 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 >> 172.21.15.120:0/226808098 conn(0x55801f69b2b0 msgr2=0x55801f699bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 shutdown_connections 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 1 -- 172.21.15.120:0/226808098 wait complete. 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663ca7c740 20 client.27089 trim_cache size 0 max 16384 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:04:24.801 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.803 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.805 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:04:24.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:04:24.809 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.810 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.813 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:24.814 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:24.816 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:24.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:04:24.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:04:24.816 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:04:24.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 msgr2=0x7f661c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 0x7f661c0b7820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6628009fd0 tx=0x7f6628005f90 comp rx=0 tx=0).stop 2024-09-10T23:04:24.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 msgr2=0x7f66341a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:24.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.795+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f662c005cc0 tx=0x7f662c031700 comp rx=0 tx=0).stop 2024-09-10T23:04:24.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 shutdown_connections 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f661c0b5360 0x7f661c0b7820 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f663410d100 0x7f66341a0d30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66341045c0 0x7f663419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 --2- 172.21.15.120:0/1744273318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6634103ca0 0x7f663419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 >> 172.21.15.120:0/1744273318 conn(0x7f66340fd700 msgr2=0x7f663410f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 shutdown_connections 2024-09-10T23:04:24.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:24.796+0000 7f663a6a7640 1 -- 172.21.15.120:0/1744273318 wait complete. 2024-09-10T23:04:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 -- 172.21.15.120:0/1853635285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc105b10 msgr2=0x7f45fc107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 --2- 172.21.15.120:0/1853635285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc105b10 0x7f45fc107f80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009900 tx=0x7f45e402a410 comp rx=0 tx=0).stop 2024-09-10T23:04:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 -- 172.21.15.120:0/1853635285 shutdown_connections 2024-09-10T23:04:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 --2- 172.21.15.120:0/1853635285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 0x7f45fc10ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 --2- 172.21.15.120:0/1853635285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc105b10 0x7f45fc107f80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 --2- 172.21.15.120:0/1853635285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc06bff0 0x7f45fc105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.216+0000 7f46028bd640 1 -- 172.21.15.120:0/1853635285 >> 172.21.15.120:0/1853635285 conn(0x7f45fc0fd150 msgr2=0x7f45fc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- 172.21.15.120:0/1853635285 shutdown_connections 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- 172.21.15.120:0/1853635285 wait complete. 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 Processor -- start 2024-09-10T23:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- start start 2024-09-10T23:04:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc105b10 0x7f45fc101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 0x7f45fc101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e610 con 0x7f45fc06bff0 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e490 con 0x7f45fc1086c0 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f46028bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10e790 con 0x7f45fc105b10 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36254/0 (socket says 172.21.15.120:36254) 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f45fbfff640 1 -- 172.21.15.120:0/474326126 learned_addr learned my addr 172.21.15.120:0/474326126 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f4600e5a640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 0x7f45fc101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.217+0000 7f45f3fff640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc105b10 0x7f45fc101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 -- 172.21.15.120:0/474326126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc105b10 msgr2=0x7f45fc101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc105b10 0x7f45fc101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 -- 172.21.15.120:0/474326126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 msgr2=0x7f45fc101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 0x7f45fc101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 -- 172.21.15.120:0/474326126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1022d0 con 0x7f45fc06bff0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45fbfff640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00e9f0 tx=0x7f45ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45f9ffb640 1 -- 172.21.15.120:0/474326126 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00cd80 con 0x7f45fc06bff0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45f9ffb640 1 -- 172.21.15.120:0/474326126 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ec0054d0 con 0x7f45fc06bff0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1a2ed0 con 0x7f45fc06bff0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f45f9ffb640 1 -- 172.21.15.120:0/474326126 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec010690 con 0x7f45fc06bff0 2024-09-10T23:04:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.218+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1a32e0 con 0x7f45fc06bff0 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f9ffb640 1 -- 172.21.15.120:0/474326126 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f45ec004e90 con 0x7f45fc06bff0 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 Processor -- start 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 -- start start 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 0x5581dbebcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbebf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581dbe2c6f0 con 0x5581dbebfbd0 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5581dbe1bb20 con 0x5581dbeba860 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4605690740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5581dbe1f930 con 0x5581dbebd190 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbebf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36264/0 (socket says 172.21.15.120:36264) 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 -- 172.21.15.120:0/2032525671 learned_addr learned my addr 172.21.15.120:0/2032525671 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4600e5a640 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 0x5581dbebcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 -- 172.21.15.120:0/2032525671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 msgr2=0x5581dbebf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbebf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 -- 172.21.15.120:0/2032525671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 msgr2=0x5581dbebcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 0x5581dbebcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f3fff640 1 -- 172.21.15.120:0/2032525671 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dbde1530 con 0x5581dbebfbd0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f4600e5a640 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 0x5581dbebcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.221+0000 7f45f9ffb640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 0x7f45e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f4600e5a640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 0x7f45e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f45f3fff640 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009900 tx=0x7f45e4031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d29906692e986e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f45f9ffb640 1 -- 172.21.15.120:0/474326126 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f45ec0e5f80 con 0x7f45fc06bff0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f45f27fc640 1 -- 172.21.15.120:0/2032525671 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4040070 con 0x5581dbebfbd0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f45f27fc640 1 -- 172.21.15.120:0/2032525671 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4004030 con 0x5581dbebfbd0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.222+0000 7f45f27fc640 1 -- 172.21.15.120:0/2032525671 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4035040 con 0x5581dbebfbd0 2024-09-10T23:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4600e5a640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 0x7f45e80b77a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005fd0 tx=0x7f45f4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- 172.21.15.120:0/2032525671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 msgr2=0x5581dbec2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009900 tx=0x7f45e4031b10 comp rx=0 tx=0).stop 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- 172.21.15.120:0/2032525671 shutdown_connections 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebfbd0 0x5581dbec2090 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbebf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- 172.21.15.120:0/2032525671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbeba860 0x5581dbebcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- 172.21.15.120:0/2032525671 >> 172.21.15.120:0/2032525671 conn(0x5581dbeb59b0 msgr2=0x5581dbeb6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- 172.21.15.120:0/2032525671 shutdown_connections 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- 172.21.15.120:0/2032525671 wait complete. 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- start start 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 Processor -- start 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebd190 0x5581dbec4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebfbd0 0x5581dbec50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5581dbd53ab0 con 0x5581dbeba860 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5581dbdc1730 con 0x5581dbebd190 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.223+0000 7f4605690740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5581dbec2b50 con 0x5581dbebfbd0 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f4605690740 10 client.0 _ll_register_callbacks cb 0x5581dbe156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36280/0 (socket says 172.21.15.120:36280) 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 -- 172.21.15.120:0/4186686622 learned_addr learned my addr 172.21.15.120:0/4186686622 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f4600e5a640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebfbd0 0x5581dbec50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45f3fff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebd190 0x5581dbec4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 -- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebfbd0 msgr2=0x5581dbec50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebfbd0 0x5581dbec50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 -- 172.21.15.120:0/4186686622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebd190 msgr2=0x5581dbec4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebd190 0x5581dbec4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 -- 172.21.15.120:0/4186686622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dbb7d790 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45fbfff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0e9800 tx=0x7f45ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35b1a1ed6b8af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0ac0c0 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f45ec0ac9d0 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e4004030 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45ec0abee0 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f4605690740 10 client.17787 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581dbbf6620 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.224+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4004030 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f45ec0b5870 con 0x5581dbeba860 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 10 client.17787 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 20 client.17787 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbecf770 0x5581dbed1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 4 client.17787 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581dbe2e640 con 0x5581dbecf770 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f45f3fff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbecf770 0x5581dbed1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f45db7fe640 10 client.17787 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.225+0000 7f45f3fff640 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbecf770 0x5581dbed1b80 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a43047ea2ae0f5e server_cookie=bb87dc03bade72b5 in_seq=0 out_seq=0 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.226+0000 7f45db7fe640 1 -- 172.21.15.120:0/4186686622 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581dbe2e640 con 0x5581dbecf770 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.226+0000 7f45db7fe640 10 client.17787 handle_command_reply: tid=0 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.226+0000 7f4605690740 1 client.17787 shutdown 2024-09-10T23:04:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 msgr2=0x5581dbec3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0e9800 tx=0x7f45ec009310 comp rx=0 tx=0).stop 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 shutdown_connections 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbecf770 0x5581dbed1b80 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebfbd0 0x5581dbec50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebd190 0x5581dbec4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 --2- 172.21.15.120:0/4186686622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec3e70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 >> 172.21.15.120:0/4186686622 conn(0x5581dbeb59b0 msgr2=0x5581dbeb4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f45db7fe640 0 client.17787 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 shutdown_connections 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 1 -- 172.21.15.120:0/4186686622 wait complete. 2024-09-10T23:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.227+0000 7f4605690740 20 client.17787 trim_cache size 0 max 16384 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.242+0000 7f4605690740 1 Processor -- start 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.242+0000 7f4605690740 1 -- start start 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1aa060 con 0x5581dbeba860 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1ac8e0 con 0x5581dbebfbd0 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4605690740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1aa6d0 con 0x5581dbebd190 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4600e5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58554/0 (socket says 172.21.15.120:58554) 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4600e5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47222/0 (socket says 172.21.15.120:47222) 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 -- 172.21.15.120:0/1983252694 learned_addr learned my addr 172.21.15.120:0/1983252694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 -- 172.21.15.120:0/1983252694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 msgr2=0x5581dbeba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 -- 172.21.15.120:0/1983252694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 msgr2=0x5581dbec30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 -- 172.21.15.120:0/1983252694 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dbeb10b0 con 0x5581dbebd190 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f4600e5a640 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f3fff640 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005ab0 tx=0x7f45e402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f75917941a375c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f1ffb640 1 -- 172.21.15.120:0/1983252694 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4040050 con 0x5581dbebd190 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.243+0000 7f45f1ffb640 1 -- 172.21.15.120:0/1983252694 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4005480 con 0x5581dbebd190 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f45f1ffb640 1 -- 172.21.15.120:0/1983252694 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4035040 con 0x5581dbebd190 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- 172.21.15.120:0/1983252694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 msgr2=0x5581dbeb9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005ab0 tx=0x7f45e402a410 comp rx=0 tx=0).stop 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- 172.21.15.120:0/1983252694 shutdown_connections 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbeba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbebd190 0x5581dbeb9cf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 --2- 172.21.15.120:0/1983252694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbeba860 0x5581dbec30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- 172.21.15.120:0/1983252694 >> 172.21.15.120:0/1983252694 conn(0x5581dbeb59b0 msgr2=0x5581dbebffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- 172.21.15.120:0/1983252694 shutdown_connections 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- 172.21.15.120:0/1983252694 wait complete. 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.244+0000 7f4605690740 1 -- start start 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 Processor -- start 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 0x5581dbecbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbecae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1aada0 con 0x5581dbebd190 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1ac210 con 0x5581dbebfbd0 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1ab470 con 0x5581dbeba860 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4605690740 10 client.0 _ll_register_callbacks cb 0x7f45ec0e9750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 0x5581dbecbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4600e5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbecae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36298/0 (socket says 172.21.15.120:36298) 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 0x5581dbecbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58564/0 (socket says 172.21.15.120:58564) 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 -- 172.21.15.120:0/618441841 learned_addr learned my addr 172.21.15.120:0/618441841 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f4600e5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbecae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47224/0 (socket says 172.21.15.120:47224) 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 -- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 msgr2=0x5581dbecbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 0x5581dbecbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 -- 172.21.15.120:0/618441841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 msgr2=0x5581dbecae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbecae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.245+0000 7f45f3fff640 1 -- 172.21.15.120:0/618441841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dbec5450 con 0x5581dbebd190 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45f3fff640 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f45e402a930 tx=0x7f45e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f84aae5dfce70a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e405c070 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e4005480 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f45e40315a0 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45e4040020 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 10 client.17793 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581dbebcfc0 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4005480 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f45ec0051b0 con 0x5581dbebd190 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 10 client.17793 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 10 client.17793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 20 client.17793 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbf3ca70 0x5581dbf3ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 4 client.17793 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581dbe2a0f0 con 0x5581dbf3ca70 2024-09-10T23:04:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.246+0000 7f45fbfff640 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbf3ca70 0x5581dbf3ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.247+0000 7f45db7fe640 10 client.17793 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.247+0000 7f45fbfff640 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbf3ca70 0x5581dbf3ee60 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d94c1484bec000 server_cookie=9569e1435ee30cd1 in_seq=0 out_seq=0 2024-09-10T23:04:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f45db7fe640 1 -- 172.21.15.120:0/618441841 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581dbe2a0f0 con 0x5581dbf3ca70 2024-09-10T23:04:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f45db7fe640 10 client.17793 handle_command_reply: tid=0 2024-09-10T23:04:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f4605690740 1 client.17793 shutdown 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 msgr2=0x5581dbeca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f45e402a930 tx=0x7f45e4009780 comp rx=0 tx=0).stop 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 shutdown_connections 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.486+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5581dbf3ca70 0x5581dbf3ee60 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581dbebfbd0 0x5581dbecae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5581dbebd190 0x5581dbeca7a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 --2- 172.21.15.120:0/618441841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5581dbeba860 0x5581dbecbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 >> 172.21.15.120:0/618441841 conn(0x5581dbeb59b0 msgr2=0x5581dbeb9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f45db7fe640 0 client.17793 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 shutdown_connections 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 1 -- 172.21.15.120:0/618441841 wait complete. 2024-09-10T23:04:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f4605690740 20 client.17793 trim_cache size 0 max 16384 2024-09-10T23:04:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:04:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:04:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:04:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 msgr2=0x7f45e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 0x7f45e80b77a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005fd0 tx=0x7f45f4009600 comp rx=0 tx=0).stop 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 msgr2=0x7f45fc102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.487+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00e9f0 tx=0x7f45ec00edf0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 shutdown_connections 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45e80b52e0 0x7f45e80b77a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45fc1086c0 0x7f45fc101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45fc105b10 0x7f45fc101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 --2- 172.21.15.120:0/474326126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc06bff0 0x7f45fc102cc0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 >> 172.21.15.120:0/474326126 conn(0x7f45fc0fd150 msgr2=0x7f45fc103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 shutdown_connections 2024-09-10T23:04:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:25.488+0000 7f46028bd640 1 -- 172.21.15.120:0/474326126 wait complete. 2024-09-10T23:04:25.632 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:04:25.633 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:04:25.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2028927 15.91 MB/sec execute 560 sec latency 146.390 ms 2024-09-10T23:04:25.907 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:25.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:25 smithi120 ceph-mon[31247]: pgmap v706: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:25 smithi153 ceph-mon[34951]: pgmap v706: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:25 smithi176 ceph-mon[34982]: pgmap v706: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 -- 172.21.15.120:0/208422156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078103b30 msgr2=0x7fc078103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 --2- 172.21.15.120:0/208422156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078103b30 0x7fc078103f10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc074009900 tx=0x7fc07402a410 comp rx=0 tx=0).stop 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 -- 172.21.15.120:0/208422156 shutdown_connections 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 --2- 172.21.15.120:0/208422156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 0x7fc07810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 --2- 172.21.15.120:0/208422156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078104530 0x7fc07810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 --2- 172.21.15.120:0/208422156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078103b30 0x7fc078103f10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 -- 172.21.15.120:0/208422156 >> 172.21.15.120:0/208422156 conn(0x7fc0780fd700 msgr2=0x7fc0780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:26.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 -- 172.21.15.120:0/208422156 shutdown_connections 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.407+0000 7fc07fba1640 1 -- 172.21.15.120:0/208422156 wait complete. 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 Processor -- start 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 -- start start 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078103b30 0x7fc0781992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 0x7fc07819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112d60 con 0x7fc078104530 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112be0 con 0x7fc07810d0e0 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07fba1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc078112ee0 con 0x7fc078103b30 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36320/0 (socket says 172.21.15.120:36320) 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 -- 172.21.15.120:0/1200307791 learned_addr learned my addr 172.21.15.120:0/1200307791 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:26.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d93d640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078103b30 0x7fc0781992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07e13e640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 0x7fc07819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 -- 172.21.15.120:0/1200307791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078103b30 msgr2=0x7fc0781992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078103b30 0x7fc0781992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 -- 172.21.15.120:0/1200307791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 msgr2=0x7fc07819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 0x7fc07819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.408+0000 7fc07d13c640 1 -- 172.21.15.120:0/1200307791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07819f360 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc07d13c640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d950 tx=0x7fc06800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068017030 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068004020 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07819f650 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680047a0 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.409+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a5d10 con 0x7fc078104530 2024-09-10T23:04:26.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.410+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078103f10 con 0x7fc078104530 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.412+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc068009a30 con 0x7fc078104530 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.412+0000 7fc06effd640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 0x7fc0640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.412+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc0680e5610 con 0x7fc078104530 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.412+0000 7fc07d93d640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 0x7fc0640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.413+0000 7fc07d93d640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 0x7fc0640b77a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc074031440 tx=0x7fc074005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:26.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.417+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc0680ac500 con 0x7fc078104530 2024-09-10T23:04:26.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2032005 15.91 MB/sec execute 561 sec latency 29.183 ms 2024-09-10T23:04:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.668+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc07819a3e0 con 0x7fc078104530 2024-09-10T23:04:26.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.669+0000 7fc06effd640 1 -- 172.21.15.120:0/1200307791 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fc0680abe00 con 0x7fc078104530 2024-09-10T23:04:26.671 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:04:26.671 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:04:26.671 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 msgr2=0x7fc0640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 0x7fc0640b77a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc074031440 tx=0x7fc074005b50 comp rx=0 tx=0).stop 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 msgr2=0x7fc0781997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d950 tx=0x7fc06800de10 comp rx=0 tx=0).stop 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 shutdown_connections 2024-09-10T23:04:26.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc0640b52e0 0x7fc0640b77a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc07810d0e0 0x7fc07819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc078104530 0x7fc0781997f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 --2- 172.21.15.120:0/1200307791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc078103b30 0x7fc0781992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 >> 172.21.15.120:0/1200307791 conn(0x7fc0780fd700 msgr2=0x7fc0780fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 shutdown_connections 2024-09-10T23:04:26.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:26.672+0000 7fc07fba1640 1 -- 172.21.15.120:0/1200307791 wait complete. 2024-09-10T23:04:26.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1200307791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:27.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1200307791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:27.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1200307791' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:04:27.341 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:04:27.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2034216 15.90 MB/sec execute 562 sec latency 72.902 ms 2024-09-10T23:04:27.656 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:28.073 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:27 smithi120 ceph-mon[31247]: pgmap v707: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 -- 172.21.15.120:0/3536222628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f767810d120 msgr2=0x7f767810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 --2- 172.21.15.120:0/3536222628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f767810d120 0x7f767810f550 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7670009ca0 tx=0x7f767002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 -- 172.21.15.120:0/3536222628 shutdown_connections 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 --2- 172.21.15.120:0/3536222628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f767810d120 0x7f767810f550 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 --2- 172.21.15.120:0/3536222628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f767810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.182 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 --2- 172.21.15.120:0/3536222628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7678103b70 0x7f7678103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.165+0000 7f767e777640 1 -- 172.21.15.120:0/3536222628 >> 172.21.15.120:0/3536222628 conn(0x7f76780fd700 msgr2=0x7f76780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.166+0000 7f767e777640 1 -- 172.21.15.120:0/3536222628 shutdown_connections 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.166+0000 7f767e777640 1 -- 172.21.15.120:0/3536222628 wait complete. 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.166+0000 7f767e777640 1 Processor -- start 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 -- start start 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f76781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767810d120 0x7f767819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7678112db0 con 0x7f7678103b70 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7678112c30 con 0x7f767810d120 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767e777640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7678112f30 con 0x7f7678104570 2024-09-10T23:04:28.183 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f7677fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f7677fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36338/0 (socket says 172.21.15.120:36338) 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f7677fff640 1 -- 172.21.15.120:0/3971185389 learned_addr learned my addr 172.21.15.120:0/3971185389 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f766ffff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f76781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.167+0000 7f767cd14640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767810d120 0x7f767819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 -- 172.21.15.120:0/3971185389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 msgr2=0x7f76781997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f76781997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 -- 172.21.15.120:0/3971185389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767810d120 msgr2=0x7f767819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767810d120 0x7f767819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 -- 172.21.15.120:0/3971185389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767819f330 con 0x7f7678103b70 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f766ffff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f76781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7677fff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f766800d950 tx=0x7f766800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7675ffb640 1 -- 172.21.15.120:0/3971185389 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f7678103b70 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.168+0000 7f7675ffb640 1 -- 172.21.15.120:0/3971185389 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7668009c30 con 0x7f7678103b70 2024-09-10T23:04:28.184 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.169+0000 7f7675ffb640 1 -- 172.21.15.120:0/3971185389 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766800f620 con 0x7f7678103b70 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.169+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767819f620 con 0x7f7678103b70 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.169+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a5eb0 con 0x7f7678103b70 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.171+0000 7f7680b4c740 1 Processor -- start 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.171+0000 7f7675ffb640 1 -- 172.21.15.120:0/3971185389 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7668010460 con 0x7f7678103b70 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 -- start start 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed8639a10 0x55eed8645460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8647d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eed85afea0 con 0x55eed86459a0 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eed859f2d0 con 0x55eed8639a10 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7680b4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55eed85a30e0 con 0x55eed86390f0 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f766ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8647d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58612/0 (socket says 172.21.15.120:58612) 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 -- 172.21.15.120:0/1155849240 learned_addr learned my addr 172.21.15.120:0/1155849240 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f7677fff640 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed8639a10 0x55eed8645460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 -- 172.21.15.120:0/1155849240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed8639a10 msgr2=0x55eed8645460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed8639a10 0x55eed8645460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 -- 172.21.15.120:0/1155849240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 msgr2=0x55eed8647d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8647d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f767cd14640 1 -- 172.21.15.120:0/1155849240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed82fd650 con 0x55eed86390f0 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.172+0000 7f766ffff640 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8647d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f767cd14640 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7670009ca0 tx=0x7f7670005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37c49bd5babbc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7675ffb640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 0x7f76640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7677fff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 0x7f76640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f766e7fc640 1 -- 172.21.15.120:0/1155849240 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670031db0 con 0x55eed86390f0 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f766e7fc640 1 -- 172.21.15.120:0/1155849240 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7670036030 con 0x55eed86390f0 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f766e7fc640 1 -- 172.21.15.120:0/1155849240 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76700042a0 con 0x55eed86390f0 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7675ffb640 1 -- 172.21.15.120:0/3971185389 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f76680e5710 con 0x7f7678103b70 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7680b4c740 1 -- 172.21.15.120:0/1155849240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 msgr2=0x55eed86394d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7680b4c740 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7670009ca0 tx=0x7f7670005fa0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.173+0000 7f7677fff640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 0x7f76640b7820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f76680e35b0 tx=0x7f766801a750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 -- 172.21.15.120:0/1155849240 shutdown_connections 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8647d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed8639a10 0x55eed8645460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 --2- 172.21.15.120:0/1155849240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed86394d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 -- 172.21.15.120:0/1155849240 >> 172.21.15.120:0/1155849240 conn(0x55eed8636f20 msgr2=0x55eed86380c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 -- 172.21.15.120:0/1155849240 shutdown_connections 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 -- 172.21.15.120:0/1155849240 wait complete. 2024-09-10T23:04:28.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 -- start start 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.174+0000 7f7680b4c740 1 Processor -- start 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed86452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8642d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55eed8400700 con 0x55eed86459a0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55eed853a160 con 0x55eed86390f0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55eed8648440 con 0x55eed8639a10 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f7680b4c740 10 client.0 _ll_register_callbacks cb 0x55eed85539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58626/0 (socket says 172.21.15.120:58626) 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 -- 172.21.15.120:0/235894710 learned_addr learned my addr 172.21.15.120:0/235894710 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f767cd14640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed86452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f767cd14640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8642d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 -- 172.21.15.120:0/235894710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 msgr2=0x55eed86452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed86452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 -- 172.21.15.120:0/235894710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 msgr2=0x55eed8642d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8642d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 -- 172.21.15.120:0/235894710 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed8567c90 con 0x55eed8639a10 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f766ffff640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7660012a10 tx=0x7f7660012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d7a818175b07ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.175+0000 7f76577fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766000de10 con 0x55eed8639a10 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7670036030 con 0x55eed8639a10 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f766001d460 con 0x55eed8639a10 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f766001d660 con 0x55eed8639a10 2024-09-10T23:04:28.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f7680b4c740 10 client.27283 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eed8653e80 con 0x55eed8639a10 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7670036030 con 0x55eed8639a10 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.176+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7660026880 con 0x55eed8639a10 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 10 client.27283 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 20 client.27283 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed8657740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 4 client.27283 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed85b1df0 con 0x55eed8655350 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f767cd14640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed8657740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f767cd14640 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed8657740 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6252d85fd0ad1734 server_cookie=99ff6f74b9e18054 in_seq=0 out_seq=0 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.177+0000 7f76577fe640 10 client.27283 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.178+0000 7f76577fe640 1 -- 172.21.15.120:0/235894710 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eed85b1df0 con 0x55eed8655350 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.178+0000 7f76577fe640 10 client.27283 handle_command_reply: tid=0 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.178+0000 7f7680b4c740 1 client.27283 shutdown 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 msgr2=0x55eed8642710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7660012a10 tx=0x7f7660012ed0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 shutdown_connections 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed8657740 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f76577fe640 0 client.27283 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed86459a0 0x55eed8642d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed8639a10 0x55eed8642710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 --2- 172.21.15.120:0/235894710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed86452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 >> 172.21.15.120:0/235894710 conn(0x55eed8636f20 msgr2=0x55eed863ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 shutdown_connections 2024-09-10T23:04:28.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 1 -- 172.21.15.120:0/235894710 wait complete. 2024-09-10T23:04:28.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.179+0000 7f7680b4c740 20 client.27283 trim_cache size 0 max 16384 2024-09-10T23:04:28.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.194+0000 7f7680b4c740 1 Processor -- start 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.194+0000 7f7680b4c740 1 -- start start 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed8641c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86459a0 0x55eed8637fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aa0b0 con 0x55eed8639a10 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ac930 con 0x55eed86390f0 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f7680b4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aa720 con 0x55eed86459a0 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36366/0 (socket says 172.21.15.120:36366) 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 -- 172.21.15.120:0/3383633905 learned_addr learned my addr 172.21.15.120:0/3383633905 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f767cd14640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed8641c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f767cd14640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86459a0 0x55eed8637fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 -- 172.21.15.120:0/3383633905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86459a0 msgr2=0x55eed8637fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86459a0 0x55eed8637fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 -- 172.21.15.120:0/3383633905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 msgr2=0x55eed8641c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed8641c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 -- 172.21.15.120:0/3383633905 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed8640c10 con 0x55eed8639a10 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f767cd14640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed8641c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.195+0000 7f766ffff640 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7660005e00 tx=0x7f766001d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b7b2435f55dc17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.196+0000 7f766dffb640 1 -- 172.21.15.120:0/3383633905 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766000dc10 con 0x55eed8639a10 2024-09-10T23:04:28.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.196+0000 7f766dffb640 1 -- 172.21.15.120:0/3383633905 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7660022070 con 0x55eed8639a10 2024-09-10T23:04:28.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.196+0000 7f766dffb640 1 -- 172.21.15.120:0/3383633905 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766001dbd0 con 0x55eed8639a10 2024-09-10T23:04:28.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:27 smithi153 ceph-mon[34951]: pgmap v707: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T23:04:28.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- 172.21.15.120:0/3383633905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 msgr2=0x55eed8636760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7660005e00 tx=0x7f766001d5f0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- 172.21.15.120:0/3383633905 shutdown_connections 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86459a0 0x55eed8637fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed8636760 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3383633905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86390f0 0x55eed8641c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- 172.21.15.120:0/3383633905 >> 172.21.15.120:0/3383633905 conn(0x55eed8636f20 msgr2=0x55eed8640f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- 172.21.15.120:0/3383633905 shutdown_connections 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- 172.21.15.120:0/3383633905 wait complete. 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 -- start start 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 Processor -- start 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed864d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.197+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed864dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86459a0 0x55eed863e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aadf0 con 0x55eed8639a10 2024-09-10T23:04:28.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ac260 con 0x55eed86459a0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ab4c0 con 0x55eed86390f0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 10 client.0 _ll_register_callbacks cb 0x7f766000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed864dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f767cd14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed864d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f767cd14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed864d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58642/0 (socket says 172.21.15.120:58642) 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f767cd14640 1 -- 172.21.15.120:0/3577503554 learned_addr learned my addr 172.21.15.120:0/3577503554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f767cd14640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86459a0 0x55eed863e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 -- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 msgr2=0x55eed864d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed864d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 -- 172.21.15.120:0/3577503554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86459a0 msgr2=0x55eed863e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86459a0 0x55eed863e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 -- 172.21.15.120:0/3577503554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed864e470 con 0x55eed8639a10 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f766ffff640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed864dc40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f766001d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f893c8197bbe9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:28.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f76577fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660040060 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f766001db60 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.198+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7660022070 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f766001dd80 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 10 client.17817 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eed863ff60 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76600268a0 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f766003b910 con 0x55eed8639a10 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 10 client.17817 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 20 client.17817 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed86c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 4 client.17817 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7660022070 con 0x55eed8655350 2024-09-10T23:04:28.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f767cd14640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed86c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f76577fe640 10 client.17817 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.199+0000 7f767cd14640 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed86c51e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ccfe3d2f5ae9c43 server_cookie=bbea81dfd241f8b2 in_seq=0 out_seq=0 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.200+0000 7f76577fe640 1 -- 172.21.15.120:0/3577503554 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7660022070 con 0x55eed8655350 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.200+0000 7f76577fe640 10 client.17817 handle_command_reply: tid=0 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.200+0000 7f7680b4c740 1 client.17817 shutdown 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 msgr2=0x55eed864dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed864dc40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f766001d800 comp rx=0 tx=0).stop 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 shutdown_connections 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55eed8655350 0x55eed86c51e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eed86459a0 0x55eed863e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55eed8639a10 0x55eed864dc40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f76577fe640 0 client.17817 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 --2- 172.21.15.120:0/3577503554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55eed86390f0 0x55eed864d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 >> 172.21.15.120:0/3577503554 conn(0x55eed8636f20 msgr2=0x55eed8637e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 shutdown_connections 2024-09-10T23:04:28.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 1 -- 172.21.15.120:0/3577503554 wait complete. 2024-09-10T23:04:28.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f7680b4c740 20 client.17817 trim_cache size 0 max 16384 2024-09-10T23:04:28.208 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 msgr2=0x7f76640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 0x7f76640b7820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f76680e35b0 tx=0x7f766801a750 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 msgr2=0x7f7678199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.201+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f766800d950 tx=0x7f766800de10 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 shutdown_connections 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f76640b5360 0x7f76640b7820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767810d120 0x7f767819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7678104570 0x7f76781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 --2- 172.21.15.120:0/3971185389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7678103b70 0x7f7678199280 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 >> 172.21.15.120:0/3971185389 conn(0x7f76780fd700 msgr2=0x7f767810f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:28.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 shutdown_connections 2024-09-10T23:04:28.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:28.202+0000 7f767e777640 1 -- 172.21.15.120:0/3971185389 wait complete. 2024-09-10T23:04:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:27 smithi176 ceph-mon[34982]: pgmap v707: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T23:04:28.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2036279 15.89 MB/sec execute 563 sec latency 45.940 ms 2024-09-10T23:04:29.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2039354 15.90 MB/sec execute 564 sec latency 17.015 ms 2024-09-10T23:04:29.971 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:04:29.971 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:04:30.124 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:29 smithi176 ceph-mon[34982]: pgmap v708: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-10T23:04:30.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:29 smithi120 ceph-mon[31247]: pgmap v708: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-10T23:04:30.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:29 smithi153 ceph-mon[34951]: pgmap v708: 97 pgs: 97 active+clean; 55 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-10T23:04:30.253 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:30.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2041708 15.88 MB/sec execute 565 sec latency 35.335 ms 2024-09-10T23:04:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 -- 172.21.15.120:0/3121347429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc108020 msgr2=0x7fbbbc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 --2- 172.21.15.120:0/3121347429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc108020 0x7fbbbc10e820 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009ca0 tx=0x7fbbb402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 -- 172.21.15.120:0/3121347429 shutdown_connections 2024-09-10T23:04:30.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 --2- 172.21.15.120:0/3121347429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc108020 0x7fbbbc10e820 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 --2- 172.21.15.120:0/3121347429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 --2- 172.21.15.120:0/3121347429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 -- 172.21.15.120:0/3121347429 >> 172.21.15.120:0/3121347429 conn(0x7fbbbc0fd700 msgr2=0x7fbbbc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 -- 172.21.15.120:0/3121347429 shutdown_connections 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.774+0000 7fbbc4737640 1 -- 172.21.15.120:0/3121347429 wait complete. 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 Processor -- start 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 -- start start 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbbc108020 0x7fbbbc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc111ce0 con 0x7fbbbc1076a0 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc111b60 con 0x7fbbbc108020 2024-09-10T23:04:30.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc4737640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc111e60 con 0x7fbbbc101c90 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc24d3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc24d3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58672/0 (socket says 172.21.15.120:58672) 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc24d3640 1 -- 172.21.15.120:0/1561952963 learned_addr learned my addr 172.21.15.120:0/1561952963 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.775+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 -- 172.21.15.120:0/1561952963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbbc108020 msgr2=0x7fbbbc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbbc108020 0x7fbbbc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 -- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 msgr2=0x7fbbbc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 -- 172.21.15.120:0/1561952963 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc1a53f0 con 0x7fbbbc101c90 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc24d3640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800e9f0 tx=0x7fbbb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbb37fe640 1 -- 172.21.15.120:0/1561952963 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800cd80 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbb37fe640 1 -- 172.21.15.120:0/1561952963 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb80108e0 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbbc1a56e0 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbb37fe640 1 -- 172.21.15.120:0/1561952963 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8018860 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.776+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbbc1a5c20 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.779+0000 7fbbb37fe640 1 -- 172.21.15.120:0/1561952963 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbbb8010430 con 0x7fbbbc101c90 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 Processor -- start 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 -- start start 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676857250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676859b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc24d3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676857250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc2cd4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5556767c1e60 con 0x55567684b800 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5556767b1290 con 0x555676857790 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5556767b50a0 con 0x55567684aee0 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc2cd4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58684/0 (socket says 172.21.15.120:58684) 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/2506999466 learned_addr learned my addr 172.21.15.120:0/2506999466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/2506999466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 msgr2=0x555676859b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.780+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676859b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/2506999466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 msgr2=0x555676857250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676857250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/2506999466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567650f930 con 0x55567684aee0 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbb37fe640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 0x7fbba80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009420 tx=0x7fbbb4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8161418faaabb23c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbc24d3640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 0x7fbba80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbb0ff9640 1 -- 172.21.15.120:0/2506999466 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4031cf0 con 0x55567684aee0 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbb37fe640 1 -- 172.21.15.120:0/1561952963 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbbb80e5f90 con 0x7fbbbc101c90 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbb0ff9640 1 -- 172.21.15.120:0/2506999466 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb403f070 con 0x55567684aee0 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.781+0000 7fbbb0ff9640 1 -- 172.21.15.120:0/2506999466 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4004040 con 0x55567684aee0 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/2506999466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 msgr2=0x55567684b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc24d3640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 0x7fbba80b77a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c6a0 tx=0x7fbbb80e9b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009420 tx=0x7fbbb4005ef0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/2506999466 shutdown_connections 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676859b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676857250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/2506999466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x55567684b2c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/2506999466 >> 172.21.15.120:0/2506999466 conn(0x555676848d10 msgr2=0x555676849eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/2506999466 shutdown_connections 2024-09-10T23:04:30.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.782+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/2506999466 wait complete. 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 -- start start 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 Processor -- start 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 0x555676854d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676855290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555676612890 con 0x55567684aee0 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55567674bd40 con 0x555676857790 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55567685e6a0 con 0x55567684b800 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc6b0c740 10 client.0 _ll_register_callbacks cb 0x5556767afc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc2cd4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc2cd4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36406/0 (socket says 172.21.15.120:36406) 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/3685241541 learned_addr learned my addr 172.21.15.120:0/3685241541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.783+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 0x555676854d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 msgr2=0x555676854d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 0x555676854d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/3685241541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 msgr2=0x555676855290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676855290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 -- 172.21.15.120:0/3685241541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555676867bf0 con 0x55567684aee0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 0x555676854d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbbb402ade0 tx=0x7fbbb402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f989ce03a65a30d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4031650 con 0x55567684aee0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb403f070 con 0x55567684aee0 2024-09-10T23:04:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbbb40317f0 con 0x55567684aee0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbb4004000 con 0x55567684aee0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc6b0c740 10 client.17829 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.784+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55567686a5a0 con 0x55567684aee0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb403f070 con 0x55567684aee0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbbb402fba0 con 0x55567684aee0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 10 client.17829 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 20 client.17829 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55567686b8f0 0x55567686dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 4 client.17829 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556767c3db0 con 0x55567686b8f0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.785+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55567686b8f0 0x55567686dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.786+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55567686b8f0 0x55567686dce0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e0578c55a8985e server_cookie=e0b183f0c9d7eda2 in_seq=0 out_seq=0 2024-09-10T23:04:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.786+0000 7fbba5ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3685241541 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556767c3db0 con 0x55567686b8f0 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbba5ffb640 10 client.17829 handle_command_reply: tid=0 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 client.17829 shutdown 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 msgr2=0x555676854810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbbb402ade0 tx=0x7fbbb402f3a0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 shutdown_connections 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55567686b8f0 0x55567686dce0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676855290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684b800 0x555676854d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3685241541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684aee0 0x555676854810 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 >> 172.21.15.120:0/3685241541 conn(0x555676848d10 msgr2=0x555676847f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbba5ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 shutdown_connections 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3685241541 wait complete. 2024-09-10T23:04:30.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.787+0000 7fbbc6b0c740 20 client.17829 trim_cache size 0 max 16384 2024-09-10T23:04:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 Processor -- start 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 -- start start 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676854e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676851f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1aa070 con 0x55567684b800 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1ac8f0 con 0x555676857790 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1aa6e0 con 0x55567684aee0 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36412/0 (socket says 172.21.15.120:36412) 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/4051360808 learned_addr learned my addr 172.21.15.120:0/4051360808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676854e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676851f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/4051360808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 msgr2=0x555676854e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676854e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/4051360808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 msgr2=0x555676851f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676851f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.808+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/4051360808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555676849790 con 0x55567684b800 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.809+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55567684ae90 tx=0x7fbbac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce14a23621267c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.809+0000 7fbba7fff640 1 -- 172.21.15.120:0/4051360808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac005220 con 0x55567684b800 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.809+0000 7fbba7fff640 1 -- 172.21.15.120:0/4051360808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbac01d030 con 0x55567684b800 2024-09-10T23:04:30.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.809+0000 7fbba7fff640 1 -- 172.21.15.120:0/4051360808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01a600 con 0x55567684b800 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/4051360808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 msgr2=0x555676851a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55567684ae90 tx=0x7fbbac00c330 comp rx=0 tx=0).stop 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/4051360808 shutdown_connections 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x555676851f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x555676851a20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/4051360808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676854e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/4051360808 >> 172.21.15.120:0/4051360808 conn(0x555676848d10 msgr2=0x555676854780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/4051360808 shutdown_connections 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/4051360808 wait complete. 2024-09-10T23:04:30.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 -- start start 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.810+0000 7fbbc6b0c740 1 Processor -- start 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676871330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x55567686fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1aadb0 con 0x55567684b800 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1ac220 con 0x555676857790 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1ab480 con 0x55567684aee0 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc6b0c740 10 client.0 _ll_register_callbacks cb 0x7fbbb80e99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36424/0 (socket says 172.21.15.120:36424) 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/3462447097 learned_addr learned my addr 172.21.15.120:0/3462447097 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:30.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676871330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x55567686fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 msgr2=0x555676871330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676871330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/3462447097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 msgr2=0x55567686fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x55567686fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 -- 172.21.15.120:0/3462447097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555676870410 con 0x55567684b800 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676871330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbbc1cd2640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00ca10 tx=0x7fbbac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da16ff70cf6019dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.811+0000 7fbba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac005510 con 0x55567684b800 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac01d030 con 0x55567684b800 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbbac004730 con 0x55567684b800 2024-09-10T23:04:30.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbac035070 con 0x55567684b800 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 10 client.17835 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556768d9380 con 0x55567684b800 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbac01d030 con 0x55567684b800 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbbac022ac0 con 0x55567684b800 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 10 client.17835 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 20 client.17835 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.812+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556768d9a70 0x5556768dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.813+0000 7fbbc6b0c740 4 client.17835 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.813+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5556767bf860 con 0x5556768d9a70 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.813+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556768d9a70 0x5556768dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.813+0000 7fbba5ffb640 10 client.17835 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:30.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.813+0000 7fbbc2cd4640 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556768d9a70 0x5556768dbe90 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cae4d315be500a8 server_cookie=39c3b3b457880e3a in_seq=0 out_seq=0 2024-09-10T23:04:30.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.814+0000 7fbba5ffb640 1 -- 172.21.15.120:0/3462447097 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5556767bf860 con 0x5556768d9a70 2024-09-10T23:04:30.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.814+0000 7fbba5ffb640 10 client.17835 handle_command_reply: tid=0 2024-09-10T23:04:30.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.814+0000 7fbbc6b0c740 1 client.17835 shutdown 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 msgr2=0x55567686f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00ca10 tx=0x7fbbac005020 comp rx=0 tx=0).stop 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 shutdown_connections 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556768d9a70 0x5556768dbe90 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555676857790 0x55567686fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55567684b800 0x55567686f830 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 --2- 172.21.15.120:0/3462447097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55567684aee0 0x555676871330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 >> 172.21.15.120:0/3462447097 conn(0x555676848d10 msgr2=0x55567684e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbba5ffb640 0 client.17835 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 shutdown_connections 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 1 -- 172.21.15.120:0/3462447097 wait complete. 2024-09-10T23:04:30.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc6b0c740 20 client.17835 trim_cache size 0 max 16384 2024-09-10T23:04:30.821 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:04:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:04:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "840e4494-7018-402f-96de-4461c5098aae", 2024-09-10T23:04:30.821 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:04:30.821 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:04:30.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 msgr2=0x7fbba80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.815+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 0x7fbba80b77a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c6a0 tx=0x7fbbb80e9b00 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 msgr2=0x7fbbbc199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800e9f0 tx=0x7fbbb800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 shutdown_connections 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbba80b52e0 0x7fbba80b77a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbbc108020 0x7fbbbc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbbc1076a0 0x7fbbbc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 --2- 172.21.15.120:0/1561952963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbbc101c90 0x7fbbbc199ff0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 >> 172.21.15.120:0/1561952963 conn(0x7fbbbc0fd700 msgr2=0x7fbbbc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 shutdown_connections 2024-09-10T23:04:30.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:30.816+0000 7fbbc4737640 1 -- 172.21.15.120:0/1561952963 wait complete. 2024-09-10T23:04:31.535 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:04:31.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2044142 15.88 MB/sec execute 566 sec latency 22.716 ms 2024-09-10T23:04:31.792 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:04:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:31 smithi120 ceph-mon[31247]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-09-10T23:04:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:31 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:04:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:31 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:04:32.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:31 smithi153 ceph-mon[34951]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-09-10T23:04:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:31 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:04:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:31 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:04:32.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 -- 172.21.15.120:0/3605402652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5598109170 msgr2=0x7f559810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3605402652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5598109170 0x7f559810f970 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5590009ca0 tx=0x7f559002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 -- 172.21.15.120:0/3605402652 shutdown_connections 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3605402652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5598109170 0x7f559810f970 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3605402652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55981087f0 0x7f5598108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3605402652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598102de0 0x7f55981031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 -- 172.21.15.120:0/3605402652 >> 172.21.15.120:0/3605402652 conn(0x7f55980fe830 msgr2=0x7f5598100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 -- 172.21.15.120:0/3605402652 shutdown_connections 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 -- 172.21.15.120:0/3605402652 wait complete. 2024-09-10T23:04:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.303+0000 7f559fd0c640 1 Processor -- start 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 -- start start 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5598102de0 0x7f5598199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598109170 0x7f559819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112df0 con 0x7f55981087f0 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112c70 con 0x7f5598109170 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559fd0c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112f70 con 0x7f5598102de0 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36444/0 (socket says 172.21.15.120:36444) 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 -- 172.21.15.120:0/3188725400 learned_addr learned my addr 172.21.15.120:0/3188725400 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559daa8640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5598102de0 0x7f5598199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559e2a9640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598109170 0x7f559819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 -- 172.21.15.120:0/3188725400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5598102de0 msgr2=0x7f5598199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5598102de0 0x7f5598199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 -- 172.21.15.120:0/3188725400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598109170 msgr2=0x7f559819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598109170 0x7f559819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 -- 172.21.15.120:0/3188725400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559819f2d0 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.304+0000 7f558effd640 1 -- 172.21.15.120:0/3188725400 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558800cd80 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.305+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559819f5c0 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.305+0000 7f558effd640 1 -- 172.21.15.120:0/3188725400 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55880108e0 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.305+0000 7f558effd640 1 -- 172.21.15.120:0/3188725400 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588018980 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.305+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a5e50 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.307+0000 7f55a20e1740 1 Processor -- start 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.307+0000 7f55a20e1740 1 -- start start 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f558effd640 1 -- 172.21.15.120:0/3188725400 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5588010430 con 0x7f55981087f0 2024-09-10T23:04:32.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff4d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd0feb0e50 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fea0280 con 0x55fd0ff3a9c0 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f55a20e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fea4090 con 0x55fd0ff4ad60 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36446/0 (socket says 172.21.15.120:36446) 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 -- 172.21.15.120:0/1271822947 learned_addr learned my addr 172.21.15.120:0/1271822947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559daa8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559d2a7640 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff4d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 -- 172.21.15.120:0/1271822947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 msgr2=0x55fd0ff4d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff4d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 -- 172.21.15.120:0/1271822947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 msgr2=0x55fd0ff4a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff4a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 -- 172.21.15.120:0/1271822947 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0fbfe650 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559daa8640 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5590005fa0 tx=0x7f5590009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bceebc40c77297c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f5583fff640 1 -- 172.21.15.120:0/1271822947 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590031cf0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f5583fff640 1 -- 172.21.15.120:0/1271822947 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590036030 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.308+0000 7f5583fff640 1 -- 172.21.15.120:0/1271822947 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900042a0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f558effd640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f559daa8640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f558effd640 1 -- 172.21.15.120:0/3188725400 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f55880e5ef0 con 0x7f55981087f0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 -- 172.21.15.120:0/1271822947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 msgr2=0x55fd0ff3a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5590005fa0 tx=0x7f5590009800 comp rx=0 tx=0).stop 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f559daa8640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 0x7f55840b77a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fd0ff3e2e0 tx=0x7f559400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 -- 172.21.15.120:0/1271822947 shutdown_connections 2024-09-10T23:04:32.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff4d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff4a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1271822947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff3a480 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 -- 172.21.15.120:0/1271822947 >> 172.21.15.120:0/1271822947 conn(0x55fd0ff37ed0 msgr2=0x55fd0ff39070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 -- 172.21.15.120:0/1271822947 shutdown_connections 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.309+0000 7f55a20e1740 1 -- 172.21.15.120:0/1271822947 wait complete. 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 -- start start 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 Processor -- start 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff47ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fd01700 con 0x55fd0ff4ad60 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fe3b110 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fd0ff4d830 con 0x55fd0ff3a9c0 2024-09-10T23:04:32.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f55a20e1740 10 client.0 _ll_register_callbacks cb 0x55fd0fe54990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f559daa8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f559d2a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff47ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36460/0 (socket says 172.21.15.120:36460) 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.310+0000 7f559e2a9640 1 -- 172.21.15.120:0/3956789669 learned_addr learned my addr 172.21.15.120:0/3956789669 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 -- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 msgr2=0x55fd0ff47ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff47ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 -- 172.21.15.120:0/3956789669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 msgr2=0x55fd0ff48070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 -- 172.21.15.120:0/3956789669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0fe68c40 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff47ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559e2a9640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f559daa8640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff4a610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5594003c10 tx=0x7f5594004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed36727b6e6c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55817fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559402c060 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5594017030 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590036030 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f559401fcc0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55a20e1740 10 client.27140 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.311+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd0ff591b0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590036030 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f559401c3f0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 10 client.27140 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 20 client.27140 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ff5a670 0x55fd0ff5ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 4 client.27140 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0feb2da0 con 0x55fd0ff5a670 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ff5a670 0x55fd0ff5ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ff5a670 0x55fd0ff5ca90 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aadf5cc7b11721a server_cookie=6605716ac69df92d in_seq=0 out_seq=0 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.312+0000 7f55817fa640 10 client.27140 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.313+0000 7f55817fa640 1 -- 172.21.15.120:0/3956789669 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd0feb2da0 con 0x55fd0ff5a670 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.313+0000 7f55817fa640 10 client.27140 handle_command_reply: tid=0 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.313+0000 7f55a20e1740 1 client.27140 shutdown 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 msgr2=0x55fd0ff4a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff4a610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5594003c10 tx=0x7f5594004730 comp rx=0 tx=0).stop 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 shutdown_connections 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ff5a670 0x55fd0ff5ca90 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff48070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff47ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55817fa640 0 client.27140 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3956789669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff4a610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 >> 172.21.15.120:0/3956789669 conn(0x55fd0ff37ed0 msgr2=0x55fd0ff3da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 shutdown_connections 2024-09-10T23:04:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 1 -- 172.21.15.120:0/3956789669 wait complete. 2024-09-10T23:04:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.314+0000 7f55a20e1740 20 client.27140 trim_cache size 0 max 16384 2024-09-10T23:04:32.329 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:31 smithi176 ceph-mon[34982]: pgmap v709: 97 pgs: 97 active+clean; 43 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-09-10T23:04:32.329 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:31 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:04:32.329 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.329 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:31 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:04:32.329 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:04:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.331+0000 7f55a20e1740 1 Processor -- start 2024-09-10T23:04:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.331+0000 7f55a20e1740 1 -- start start 2024-09-10T23:04:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.331+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff56a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff39000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55a20e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55981aa0b0 con 0x55fd0ff4ad60 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55a20e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ac930 con 0x55fd0ff3a9c0 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55a20e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55981aa720 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36468/0 (socket says 172.21.15.120:36468) 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559daa8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff56a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 -- 172.21.15.120:0/1412789345 learned_addr learned my addr 172.21.15.120:0/1412789345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559d2a7640 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff39000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 -- 172.21.15.120:0/1412789345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a0a0 msgr2=0x55fd0ff56a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff56a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 -- 172.21.15.120:0/1412789345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 msgr2=0x55fd0ff39000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff39000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 -- 172.21.15.120:0/1412789345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0ff38940 con 0x55fd0ff4ad60 2024-09-10T23:04:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559d2a7640 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff39000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f559e2a9640 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5590009a60 tx=0x7f5590003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c118302133e6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55837fe640 1 -- 172.21.15.120:0/1412789345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559004a040 con 0x55fd0ff4ad60 2024-09-10T23:04:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.332+0000 7f55837fe640 1 -- 172.21.15.120:0/1412789345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590036040 con 0x55fd0ff4ad60 2024-09-10T23:04:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55837fe640 1 -- 172.21.15.120:0/1412789345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559002fa80 con 0x55fd0ff4ad60 2024-09-10T23:04:32.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 -- 172.21.15.120:0/1412789345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 msgr2=0x55fd0ff39540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5590009a60 tx=0x7f5590003cb0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 -- 172.21.15.120:0/1412789345 shutdown_connections 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff39540 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff39000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 --2- 172.21.15.120:0/1412789345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff56a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 -- 172.21.15.120:0/1412789345 >> 172.21.15.120:0/1412789345 conn(0x55fd0ff37ed0 msgr2=0x55fd0ff4cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 -- 172.21.15.120:0/1412789345 shutdown_connections 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.333+0000 7f55a20e1740 1 -- 172.21.15.120:0/1412789345 wait complete. 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 -- start start 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 Processor -- start 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff45380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff53030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55981aadf0 con 0x55fd0ff3a9c0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ac260 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ab4c0 con 0x55fd0ff4ad60 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f55a20e1740 10 client.0 _ll_register_callbacks cb 0x7f5594005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f559d2a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f559d2a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36482/0 (socket says 172.21.15.120:36482) 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f559daa8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff45380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f559d2a7640 1 -- 172.21.15.120:0/3530378185 learned_addr learned my addr 172.21.15.120:0/3530378185 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.334+0000 7f559e2a9640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff53030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 -- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 msgr2=0x55fd0ff53030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff53030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 -- 172.21.15.120:0/3530378185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 msgr2=0x55fd0ff52af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 -- 172.21.15.120:0/3530378185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0ff53720 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559e2a9640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff53030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f559daa8640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff45380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f559401c890 tx=0x7f5594005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656b4d8d39b54208 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55817fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594037070 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5594028680 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590036040 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f559401fe60 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55a20e1740 10 client.27146 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.335+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd0ffcbd50 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590036040 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f55940467a0 con 0x55fd0ff3a0a0 2024-09-10T23:04:32.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 10 client.27146 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 20 client.27146 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ffcc430 0x55fd0ffce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 4 client.27146 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd0feae850 con 0x55fd0ffcc430 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ffcc430 0x55fd0ffce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f55817fa640 10 client.27146 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.336+0000 7f559d2a7640 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ffcc430 0x55fd0ffce850 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce94ec536bfe6f07 server_cookie=531008f237380966 in_seq=0 out_seq=0 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55817fa640 1 -- 172.21.15.120:0/3530378185 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x55fd0feae850 con 0x55fd0ffcc430 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55817fa640 10 client.27146 handle_command_reply: tid=0 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 client.27146 shutdown 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 msgr2=0x55fd0ff45380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff45380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f559401c890 tx=0x7f5594005840 comp rx=0 tx=0).stop 2024-09-10T23:04:32.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 shutdown_connections 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fd0ffcc430 0x55fd0ffce850 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd0ff4ad60 0x55fd0ff53030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55817fa640 0 client.27146 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fd0ff3a9c0 0x55fd0ff52af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 --2- 172.21.15.120:0/3530378185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd0ff3a0a0 0x55fd0ff45380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 >> 172.21.15.120:0/3530378185 conn(0x55fd0ff37ed0 msgr2=0x55fd0ff36700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 shutdown_connections 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.337+0000 7f55a20e1740 1 -- 172.21.15.120:0/3530378185 wait complete. 2024-09-10T23:04:32.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f55a20e1740 20 client.27146 trim_cache size 0 max 16384 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "840e4494-7018-402f-96de-4461c5098aae", 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.344 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.346 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:04:32.347 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.348 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.349 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.350 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.351 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.351 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.352 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:04:32.353 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:04:32.354 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:04:32.358 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 msgr2=0x7f55840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 0x7f55840b77a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fd0ff3e2e0 tx=0x7f559400b390 comp rx=0 tx=0).stop 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 msgr2=0x7f5598199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:32.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 shutdown_connections 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5598109170 0x7f559819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55981087f0 0x7f5598199760 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 --2- 172.21.15.120:0/3188725400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5598102de0 0x7f5598199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 >> 172.21.15.120:0/3188725400 conn(0x7f55980fe830 msgr2=0x7f55980ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 shutdown_connections 2024-09-10T23:04:32.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:04:32.338+0000 7f559fd0c640 1 -- 172.21.15.120:0/3188725400 wait complete. 2024-09-10T23:04:32.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2046861 15.88 MB/sec execute 567 sec latency 65.807 ms 2024-09-10T23:04:33.374 INFO:tasks.cephfs.filesystem:scrub status for tag:840e4494-7018-402f-96de-4461c5098aae - {'path': '/', 'tag': '840e4494-7018-402f-96de-4461c5098aae', 'options': 'recursive,force'} 2024-09-10T23:04:33.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2050122 15.88 MB/sec execute 568 sec latency 29.388 ms 2024-09-10T23:04:34.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:33 smithi120 ceph-mon[31247]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T23:04:34.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:33 smithi153 ceph-mon[34951]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T23:04:34.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:33 smithi176 ceph-mon[34982]: pgmap v710: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-09-10T23:04:34.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2053481 15.89 MB/sec execute 569 sec latency 16.325 ms 2024-09-10T23:04:35.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2056691 15.89 MB/sec execute 570 sec latency 19.040 ms 2024-09-10T23:04:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:35 smithi120 ceph-mon[31247]: pgmap v711: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-10T23:04:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:04:36.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:35 smithi153 ceph-mon[34951]: pgmap v711: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-10T23:04:36.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:04:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:35 smithi176 ceph-mon[34982]: pgmap v711: 97 pgs: 97 active+clean; 47 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-10T23:04:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:04:36.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2059818 15.89 MB/sec execute 571 sec latency 13.544 ms 2024-09-10T23:04:37.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2062573 15.88 MB/sec execute 572 sec latency 49.215 ms 2024-09-10T23:04:38.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:37 smithi120 ceph-mon[31247]: pgmap v712: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-09-10T23:04:38.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:37 smithi153 ceph-mon[34951]: pgmap v712: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-09-10T23:04:38.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:37 smithi176 ceph-mon[34982]: pgmap v712: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-09-10T23:04:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 -- 172.21.15.120:0/2204582793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8101860 msgr2=0x7f9fc8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/2204582793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8101860 0x7f9fc8101c60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009930 tx=0x7f9fb802a420 comp rx=0 tx=0).stop 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 -- 172.21.15.120:0/2204582793 shutdown_connections 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/2204582793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc8103370 0x7f9fc8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/2204582793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc81029b0 0x7f9fc8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/2204582793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8101860 0x7f9fc8101c60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 -- 172.21.15.120:0/2204582793 >> 172.21.15.120:0/2204582793 conn(0x7f9fc80fd140 msgr2=0x7f9fc80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 -- 172.21.15.120:0/2204582793 shutdown_connections 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 -- 172.21.15.120:0/2204582793 wait complete. 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.578+0000 7f9fcdc87640 1 Processor -- start 2024-09-10T23:04:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 -- start start 2024-09-10T23:04:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc8101860 0x7f9fc810c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 0x7f9fc810a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc810e5c0 con 0x7f9fc8103370 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc810e440 con 0x7f9fc81029b0 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fcdc87640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc810e740 con 0x7f9fc8101860 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60178/0 (socket says 172.21.15.120:60178) 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 -- 172.21.15.120:0/737822293 learned_addr learned my addr 172.21.15.120:0/737822293 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 0x7f9fc810a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc8101860 0x7f9fc810c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 -- 172.21.15.120:0/737822293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc8101860 msgr2=0x7f9fc810c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc8101860 0x7f9fc810c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 -- 172.21.15.120:0/737822293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 msgr2=0x7f9fc810a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 0x7f9fc810a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 -- 172.21.15.120:0/737822293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc810b6f0 con 0x7f9fc8103370 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 0x7f9fc810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000eb10 tx=0x7f9fc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.579+0000 7f9fc4ff9640 1 -- 172.21.15.120:0/737822293 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0004b20 con 0x7f9fc8103370 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.580+0000 7f9fc4ff9640 1 -- 172.21.15.120:0/737822293 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc0004cc0 con 0x7f9fc8103370 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.580+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc81a2eb0 con 0x7f9fc8103370 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.580+0000 7f9fc4ff9640 1 -- 172.21.15.120:0/737822293 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0010670 con 0x7f9fc8103370 2024-09-10T23:04:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.580+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc81a3320 con 0x7f9fc8103370 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 Processor -- start 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fc4ff9640 1 -- 172.21.15.120:0/737822293 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9fc0005020 con 0x7f9fc8103370 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 -- start start 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55700a4406f0 con 0x55700a4d1190 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55700a42fb20 con 0x55700a4d3bd0 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55700a433930 con 0x55700a4ce860 2024-09-10T23:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60184/0 (socket says 172.21.15.120:60184) 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fc77fe640 1 -- 172.21.15.120:0/1700015916 learned_addr learned my addr 172.21.15.120:0/1700015916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.582+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 -- 172.21.15.120:0/1700015916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 msgr2=0x55700a4d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 -- 172.21.15.120:0/1700015916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 msgr2=0x55700a4d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 -- 172.21.15.120:0/1700015916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700a3f5530 con 0x55700a4d1190 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc4ff9640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 0x7f9fb40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9fb802a940 tx=0x7f9fb8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91abb31378471f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 0x7f9fb40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fc4ff9640 1 -- 172.21.15.120:0/737822293 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fc00e6190 con 0x7f9fc8103370 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fbdffb640 1 -- 172.21.15.120:0/1700015916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8035030 con 0x55700a4d1190 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fbdffb640 1 -- 172.21.15.120:0/1700015916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8040070 con 0x55700a4d1190 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.583+0000 7f9fbdffb640 1 -- 172.21.15.120:0/1700015916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80038e0 con 0x55700a4d1190 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- 172.21.15.120:0/1700015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 msgr2=0x55700a4d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9fb802a940 tx=0x7f9fb8005de0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 0x7f9fb40b77a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00058c0 tx=0x7f9fb0013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- 172.21.15.120:0/1700015916 shutdown_connections 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d1190 0x55700a4d3580 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/1700015916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- 172.21.15.120:0/1700015916 >> 172.21.15.120:0/1700015916 conn(0x55700a4c99b0 msgr2=0x55700a4ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- 172.21.15.120:0/1700015916 shutdown_connections 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- 172.21.15.120:0/1700015916 wait complete. 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 -- start start 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.584+0000 7f9fd0a54740 1 Processor -- start 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55700a367ab0 con 0x55700a4d3bd0 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55700a3d5730 con 0x55700a4d1190 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55700a4d6b50 con 0x55700a4ce860 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 10 client.0 _ll_register_callbacks cb 0x55700a4296b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60198/0 (socket says 172.21.15.120:60198) 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 -- 172.21.15.120:0/2936447738 learned_addr learned my addr 172.21.15.120:0/2936447738 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 -- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 msgr2=0x55700a4d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 -- 172.21.15.120:0/2936447738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 msgr2=0x55700a4d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4d8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 -- 172.21.15.120:0/2936447738 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700a191790 con 0x55700a4d3bd0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00a8f50 tx=0x7f9fc00ac750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bc06df33e0d2b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00a80f0 con 0x55700a4d3bd0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fc00b5a80 con 0x55700a4d3bd0 2024-09-10T23:04:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8040070 con 0x55700a4d3bd0 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fc0005540 con 0x55700a4d3bd0 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 10 client.17865 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.585+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55700a20a620 con 0x55700a4d3bd0 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8040070 con 0x55700a4d3bd0 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9fc00a82d0 con 0x55700a4d3bd0 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 10 client.17865 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 20 client.17865 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a4e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 4 client.17865 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55700a442640 con 0x55700a4e37d0 2024-09-10T23:04:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a4e5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fa6ffd640 10 client.17865 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.586+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a4e5be0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db14dc2700b3097c server_cookie=8426950af16e0947 in_seq=0 out_seq=0 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/2936447738 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55700a442640 con 0x55700a4e37d0 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fa6ffd640 10 client.17865 handle_command_reply: tid=0 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 client.17865 shutdown 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 msgr2=0x55700a4d90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00a8f50 tx=0x7f9fc00ac750 comp rx=0 tx=0).stop 2024-09-10T23:04:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 shutdown_connections 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a4e5be0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4d3bd0 0x55700a4d90c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fa6ffd640 0 client.17865 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/2936447738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4ce860 0x55700a4d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 >> 172.21.15.120:0/2936447738 conn(0x55700a4c99b0 msgr2=0x55700a4c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 shutdown_connections 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.590+0000 7f9fd0a54740 1 -- 172.21.15.120:0/2936447738 wait complete. 2024-09-10T23:04:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.591+0000 7f9fd0a54740 20 client.17865 trim_cache size 0 max 16384 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 Processor -- start 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 -- start start 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d1190 0x55700a4d7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81aa060 con 0x55700a4ce860 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ac8e0 con 0x55700a4d3bd0 2024-09-10T23:04:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81aa6d0 con 0x55700a4d1190 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60206/0 (socket says 172.21.15.120:60206) 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 -- 172.21.15.120:0/3536335601 learned_addr learned my addr 172.21.15.120:0/3536335601 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d1190 0x55700a4d7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 -- 172.21.15.120:0/3536335601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d1190 msgr2=0x55700a4d7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d1190 0x55700a4d7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 -- 172.21.15.120:0/3536335601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 msgr2=0x55700a4cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.605+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fc77fe640 1 -- 172.21.15.120:0/3536335601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700a4d0fc0 con 0x55700a4ce860 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005bc0 tx=0x7f9fb8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9c5f342a23da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fbd7fa640 1 -- 172.21.15.120:0/3536335601 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8035040 con 0x55700a4ce860 2024-09-10T23:04:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fbd7fa640 1 -- 172.21.15.120:0/3536335601 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8040070 con 0x55700a4ce860 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fbd7fa640 1 -- 172.21.15.120:0/3536335601 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb804a040 con 0x55700a4ce860 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fd0a54740 1 -- 172.21.15.120:0/3536335601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 msgr2=0x55700a4d7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005bc0 tx=0x7f9fb8005b50 comp rx=0 tx=0).stop 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fd0a54740 1 -- 172.21.15.120:0/3536335601 shutdown_connections 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.606+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d3bd0 0x55700a4cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d1190 0x55700a4d7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/3536335601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4d7100 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- 172.21.15.120:0/3536335601 >> 172.21.15.120:0/3536335601 conn(0x55700a4c99b0 msgr2=0x55700a4d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- 172.21.15.120:0/3536335601 shutdown_connections 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- 172.21.15.120:0/3536335601 wait complete. 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- start start 2024-09-10T23:04:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 Processor -- start 2024-09-10T23:04:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 0x55700a4de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81aada0 con 0x55700a4ce860 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ac210 con 0x55700a4d1190 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ab470 con 0x55700a4d3bd0 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fd0a54740 10 client.0 _ll_register_callbacks cb 0x7f9fc00a8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.607+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60212/0 (socket says 172.21.15.120:60212) 2024-09-10T23:04:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc77fe640 1 -- 172.21.15.120:0/201189831 learned_addr learned my addr 172.21.15.120:0/201189831 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44264/0 (socket says 172.21.15.120:44264) 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 0x55700a4de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 -- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 msgr2=0x55700a4de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 0x55700a4de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 -- 172.21.15.120:0/201189831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 msgr2=0x55700a4df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 -- 172.21.15.120:0/201189831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700a4deb10 con 0x55700a4d1190 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc7fff640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 0x55700a4de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fc6ffd640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000f790 tx=0x7f9fb0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67d623149802d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:04:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb002f060 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fb001a070 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8040070 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fb002a450 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fd0a54740 10 client.27164 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.608+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55700a4ddb00 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8040070 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9fb001f550 con 0x55700a4d1190 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 10 client.27164 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 20 client.27164 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a552eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 4 client.27164 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:04:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55700a43e0f0 con 0x55700a4e37d0 2024-09-10T23:04:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a552eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fa6ffd640 10 client.27164 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.609+0000 7f9fc77fe640 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a552eb0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a2cf979f4ce832 server_cookie=98536242b3651829 in_seq=0 out_seq=0 2024-09-10T23:04:38.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2064894 15.88 MB/sec execute 573 sec latency 36.552 ms 2024-09-10T23:04:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fa6ffd640 1 -- 172.21.15.120:0/201189831 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55700a43e0f0 con 0x55700a4e37d0 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fa6ffd640 10 client.27164 handle_command_reply: tid=0 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 client.27164 shutdown 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 msgr2=0x55700a4dfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000f790 tx=0x7f9fb0005f90 comp rx=0 tx=0).stop 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 shutdown_connections 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55700a4e37d0 0x55700a552eb0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55700a4d3bd0 0x55700a4de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fa6ffd640 0 client.27164 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55700a4d1190 0x55700a4dfb90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 --2- 172.21.15.120:0/201189831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55700a4ce860 0x55700a4df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 >> 172.21.15.120:0/201189831 conn(0x55700a4c99b0 msgr2=0x55700a4d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 shutdown_connections 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 1 -- 172.21.15.120:0/201189831 wait complete. 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.849+0000 7f9fd0a54740 20 client.27164 trim_cache size 0 max 16384 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:04:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 msgr2=0x7f9fb40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 0x7f9fb40b77a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00058c0 tx=0x7f9fb0013040 comp rx=0 tx=0).stop 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 msgr2=0x7f9fc810ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000eb10 tx=0x7f9fc000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 shutdown_connections 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fb40b52e0 0x7f9fb40b77a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fc8103370 0x7f9fc810ae80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fc81029b0 0x7f9fc810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 --2- 172.21.15.120:0/737822293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fc8101860 0x7f9fc810c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 >> 172.21.15.120:0/737822293 conn(0x7f9fc80fd140 msgr2=0x7f9fc8103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 shutdown_connections 2024-09-10T23:04:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:38.850+0000 7f9fcdc87640 1 -- 172.21.15.120:0/737822293 wait complete. 2024-09-10T23:04:39.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2067771 15.89 MB/sec execute 574 sec latency 126.496 ms 2024-09-10T23:04:40.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:39 smithi120 ceph-mon[31247]: pgmap v713: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-09-10T23:04:40.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:39 smithi153 ceph-mon[34951]: pgmap v713: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-09-10T23:04:40.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:39 smithi176 ceph-mon[34982]: pgmap v713: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-09-10T23:04:40.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2071033 15.89 MB/sec execute 575 sec latency 16.966 ms 2024-09-10T23:04:41.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2074042 15.89 MB/sec execute 576 sec latency 28.293 ms 2024-09-10T23:04:42.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:41 smithi120 ceph-mon[31247]: pgmap v714: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:04:42.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:41 smithi153 ceph-mon[34951]: pgmap v714: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:04:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:41 smithi176 ceph-mon[34982]: pgmap v714: 97 pgs: 97 active+clean; 51 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 474 op/s 2024-09-10T23:04:42.644 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2076508 15.88 MB/sec execute 577 sec latency 65.218 ms 2024-09-10T23:04:43.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2079105 15.88 MB/sec execute 578 sec latency 29.191 ms 2024-09-10T23:04:44.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:43 smithi120 ceph-mon[31247]: pgmap v715: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-09-10T23:04:44.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:43 smithi153 ceph-mon[34951]: pgmap v715: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-09-10T23:04:44.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:43 smithi176 ceph-mon[34982]: pgmap v715: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-09-10T23:04:44.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2082132 15.88 MB/sec execute 579 sec latency 26.287 ms 2024-09-10T23:04:45.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2085240 15.88 MB/sec execute 580 sec latency 16.658 ms 2024-09-10T23:04:46.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:45 smithi120 ceph-mon[31247]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:46.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:45 smithi153 ceph-mon[34951]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:46.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:45 smithi176 ceph-mon[34982]: pgmap v716: 97 pgs: 97 active+clean; 44 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 394 op/s 2024-09-10T23:04:46.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2087657 15.87 MB/sec execute 581 sec latency 58.868 ms 2024-09-10T23:04:47.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:46 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:47.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:46 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:47.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:46 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:04:47.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2089118 15.86 MB/sec execute 582 sec latency 178.286 ms 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 494 op/s 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:48.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 494 op/s 2024-09-10T23:04:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:48.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:48.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: pgmap v717: 97 pgs: 97 active+clean; 48 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 494 op/s 2024-09-10T23:04:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:04:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:04:48.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:04:48.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2091693 15.86 MB/sec execute 583 sec latency 43.199 ms 2024-09-10T23:04:49.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2094013 15.85 MB/sec execute 584 sec latency 17.348 ms 2024-09-10T23:04:50.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:50 smithi176 ceph-mon[34982]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-09-10T23:04:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:50 smithi120 ceph-mon[31247]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-09-10T23:04:50.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:50 smithi153 ceph-mon[34951]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-09-10T23:04:50.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2096633 15.85 MB/sec execute 585 sec latency 19.435 ms 2024-09-10T23:04:51.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2099760 15.85 MB/sec execute 586 sec latency 24.526 ms 2024-09-10T23:04:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 -- 172.21.15.120:0/32204663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c076f30 msgr2=0x7fee6c075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 --2- 172.21.15.120:0/32204663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c076f30 0x7fee6c075580 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-09-10T23:04:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 -- 172.21.15.120:0/32204663 shutdown_connections 2024-09-10T23:04:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 --2- 172.21.15.120:0/32204663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 0x7fee6c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 --2- 172.21.15.120:0/32204663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c075ac0 0x7fee6c075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 --2- 172.21.15.120:0/32204663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c076f30 0x7fee6c075580 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 -- 172.21.15.120:0/32204663 >> 172.21.15.120:0/32204663 conn(0x7fee6c0fe240 msgr2=0x7fee6c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 -- 172.21.15.120:0/32204663 shutdown_connections 2024-09-10T23:04:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.944+0000 7fee73c47640 1 -- 172.21.15.120:0/32204663 wait complete. 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 Processor -- start 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 -- start start 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 0x7fee6c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 0x7fee6c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1107f0 con 0x7fee6c075ac0 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110670 con 0x7fee6c076f30 2024-09-10T23:04:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee73c47640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110970 con 0x7fee6c10a900 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee719e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee719e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36666/0 (socket says 172.21.15.120:36666) 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee719e3640 1 -- 172.21.15.120:0/1888837665 learned_addr learned my addr 172.21.15.120:0/1888837665 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee721e4640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 0x7fee6c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.945+0000 7fee711e2640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 0x7fee6c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 -- 172.21.15.120:0/1888837665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 msgr2=0x7fee6c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 0x7fee6c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 -- 172.21.15.120:0/1888837665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 msgr2=0x7fee6c19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 0x7fee6c19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 -- 172.21.15.120:0/1888837665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c19cac0 con 0x7fee6c075ac0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee711e2640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 0x7fee6c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee719e3640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee62ffd640 1 -- 172.21.15.120:0/1888837665 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6803f030 con 0x7fee6c075ac0 2024-09-10T23:04:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c19cd50 con 0x7fee6c075ac0 2024-09-10T23:04:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee62ffd640 1 -- 172.21.15.120:0/1888837665 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee68003cb0 con 0x7fee6c075ac0 2024-09-10T23:04:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee62ffd640 1 -- 172.21.15.120:0/1888837665 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68035070 con 0x7fee6c075ac0 2024-09-10T23:04:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.946+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1a3480 con 0x7fee6c075ac0 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.949+0000 7fee76a19740 1 Processor -- start 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.949+0000 7fee76a19740 1 -- start start 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.949+0000 7fee76a19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.949+0000 7fee76a19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830af6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee76a19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 0x557830af9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee76a19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557830a636f0 con 0x557830af1860 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee76a19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557830a52b20 con 0x557830af6bd0 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee76a19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557830a56930 con 0x557830af4190 2024-09-10T23:04:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee62ffd640 1 -- 172.21.15.120:0/1888837665 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fee6803bbe0 con 0x7fee6c075ac0 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36670/0 (socket says 172.21.15.120:36670) 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 -- 172.21.15.120:0/3588072678 learned_addr learned my addr 172.21.15.120:0/3588072678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee719e3640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830af6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee711e2640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 0x557830af9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 -- 172.21.15.120:0/3588072678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 msgr2=0x557830af6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830af6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 -- 172.21.15.120:0/3588072678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 msgr2=0x557830af9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 0x557830af9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 -- 172.21.15.120:0/3588072678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557830a18530 con 0x557830af1860 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee711e2640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 0x557830af9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:04:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee721e4640 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fee64009f90 tx=0x7fee640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b540802c665f2ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee57fff640 1 -- 172.21.15.120:0/3588072678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64004d20 con 0x557830af1860 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee57fff640 1 -- 172.21.15.120:0/3588072678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee64004050 con 0x557830af1860 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.950+0000 7fee57fff640 1 -- 172.21.15.120:0/3588072678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64005260 con 0x557830af1860 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee62ffd640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 0x7fee580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee719e3640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 0x7fee580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee62ffd640 1 -- 172.21.15.120:0/1888837665 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fee68106320 con 0x7fee6c075ac0 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee719e3640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 0x7fee580b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee68109fd0 tx=0x7fee68109e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 -- 172.21.15.120:0/3588072678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 msgr2=0x557830af3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fee64009f90 tx=0x7fee640118d0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 -- 172.21.15.120:0/3588072678 shutdown_connections 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af6bd0 0x557830af9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830af6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 --2- 172.21.15.120:0/3588072678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af3c50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 -- 172.21.15.120:0/3588072678 >> 172.21.15.120:0/3588072678 conn(0x557830aec9b0 msgr2=0x557830aed740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 -- 172.21.15.120:0/3588072678 shutdown_connections 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.951+0000 7fee76a19740 1 -- 172.21.15.120:0/3588072678 wait complete. 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 -- start start 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 Processor -- start 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af1860 0x557830afae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830afb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55783098aab0 con 0x557830af6bd0 2024-09-10T23:04:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5578309f8730 con 0x557830af4190 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557830af9bb0 con 0x557830af1860 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee76a19740 10 client.0 _ll_register_callbacks cb 0x557830a4c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36678/0 (socket says 172.21.15.120:36678) 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee721e4640 1 -- 172.21.15.120:0/4077936075 learned_addr learned my addr 172.21.15.120:0/4077936075 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee721e4640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af1860 0x557830afae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.952+0000 7fee711e2640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830afb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 -- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af1860 msgr2=0x557830afae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af1860 0x557830afae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 -- 172.21.15.120:0/4077936075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 msgr2=0x557830afb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830afb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 -- 172.21.15.120:0/4077936075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578307b4790 con 0x557830af6bd0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee721e4640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fee64004840 tx=0x7fee64004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f479b52472e3d6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6402d060 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fee64004c50 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64004050 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee6400f4e0 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 10 client.17883 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55783082d620 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee64004050 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fee64034620 con 0x557830af6bd0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 10 client.17883 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 20 client.17883 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.953+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b063c0 0x557830b087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.954+0000 7fee76a19740 4 client.17883 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.954+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557830a65640 con 0x557830b063c0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.954+0000 7fee711e2640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b063c0 0x557830b087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.954+0000 7fee557fa640 10 client.17883 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.954+0000 7fee711e2640 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b063c0 0x557830b087b0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae0b87a635f846 server_cookie=c8344a5aa12d2b82 in_seq=0 out_seq=0 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee557fa640 1 -- 172.21.15.120:0/4077936075 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557830a65640 con 0x557830b063c0 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee557fa640 10 client.17883 handle_command_reply: tid=0 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 client.17883 shutdown 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 msgr2=0x557830afc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fee64004840 tx=0x7fee64004790 comp rx=0 tx=0).stop 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 shutdown_connections 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b063c0 0x557830b087b0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830afc100 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830afb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee557fa640 0 client.17883 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 --2- 172.21.15.120:0/4077936075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af1860 0x557830afae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 >> 172.21.15.120:0/4077936075 conn(0x557830aec9b0 msgr2=0x557830aebbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 shutdown_connections 2024-09-10T23:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 1 -- 172.21.15.120:0/4077936075 wait complete. 2024-09-10T23:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.956+0000 7fee76a19740 20 client.17883 trim_cache size 0 max 16384 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 Processor -- start 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 -- start start 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830af1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af6bd0 0x557830aef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1a9fd0 con 0x557830af1860 2024-09-10T23:04:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1ac850 con 0x557830af4190 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.971+0000 7fee76a19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1aa640 con 0x557830af6bd0 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36686/0 (socket says 172.21.15.120:36686) 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 -- 172.21.15.120:0/3236353113 learned_addr learned my addr 172.21.15.120:0/3236353113 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee711e2640 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830af1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af6bd0 0x557830aef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 -- 172.21.15.120:0/3236353113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af6bd0 msgr2=0x557830aef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af6bd0 0x557830aef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 -- 172.21.15.120:0/3236353113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 msgr2=0x557830af1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830af1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 -- 172.21.15.120:0/3236353113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557830afac60 con 0x557830af1860 2024-09-10T23:04:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee721e4640 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fee64002230 tx=0x7fee640096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a97433f774ed3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee577fe640 1 -- 172.21.15.120:0/3236353113 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6402d040 con 0x557830af1860 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee577fe640 1 -- 172.21.15.120:0/3236353113 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee64015070 con 0x557830af1860 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.972+0000 7fee577fe640 1 -- 172.21.15.120:0/3236353113 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6400fb70 con 0x557830af1860 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- 172.21.15.120:0/3236353113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 msgr2=0x557830af0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fee64002230 tx=0x7fee640096c0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- 172.21.15.120:0/3236353113 shutdown_connections 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af6bd0 0x557830aef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af4190 0x557830af1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 --2- 172.21.15.120:0/3236353113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af1860 0x557830af0cc0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- 172.21.15.120:0/3236353113 >> 172.21.15.120:0/3236353113 conn(0x557830aec9b0 msgr2=0x557830aed610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- 172.21.15.120:0/3236353113 shutdown_connections 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- 172.21.15.120:0/3236353113 wait complete. 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 -- start start 2024-09-10T23:04:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.973+0000 7fee76a19740 1 Processor -- start 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af1860 0x557830affc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830b00330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830b00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1aad10 con 0x557830af6bd0 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1ac180 con 0x557830af1860 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1ab3e0 con 0x557830af4190 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee76a19740 10 client.0 _ll_register_callbacks cb 0x7fee68109cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830b00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830b00330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee721e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830b00eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36702/0 (socket says 172.21.15.120:36702) 2024-09-10T23:04:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee721e4640 1 -- 172.21.15.120:0/3426479007 learned_addr learned my addr 172.21.15.120:0/3426479007 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee721e4640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af1860 0x557830affc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 -- 172.21.15.120:0/3426479007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af1860 msgr2=0x557830affc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af1860 0x557830affc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 -- 172.21.15.120:0/3426479007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 msgr2=0x557830b00eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830b00eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 -- 172.21.15.120:0/3426479007 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557830b016e0 con 0x557830af4190 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee711e2640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830b00330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x557830b08d90 tx=0x7fee5c0138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f81bcba3f33046 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.974+0000 7fee557fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004030 con 0x557830af4190 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64015070 con 0x557830af4190 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fee5c0041d0 con 0x557830af4190 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee5c0043d0 con 0x557830af4190 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 10 client.27358 fetch_fsmap learned FSMap version 26 2024-09-10T23:04:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557830b02ac0 con 0x557830af4190 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee64015070 con 0x557830af4190 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fee5c0116e0 con 0x557830af4190 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 10 client.27358 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 10 client.27358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 20 client.27358 populate_metadata read hostname 'smithi120' 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b73570 0x557830b75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 4 client.27358 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557830a610f0 con 0x557830b73570 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.975+0000 7fee721e4640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b73570 0x557830b75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.976+0000 7fee557fa640 10 client.27358 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:51.976+0000 7fee721e4640 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b73570 0x557830b75990 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1469fcb722f5087d server_cookie=67d635d521794153 in_seq=0 out_seq=0 2024-09-10T23:04:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee557fa640 1 -- 172.21.15.120:0/3426479007 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557830a610f0 con 0x557830b73570 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee557fa640 10 client.27358 handle_command_reply: tid=0 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 client.27358 shutdown 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 msgr2=0x557830b00330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830b00330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x557830b08d90 tx=0x7fee5c0138f0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 shutdown_connections 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557830b73570 0x557830b75990 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557830af6bd0 0x557830b00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557830af4190 0x557830b00330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 --2- 172.21.15.120:0/3426479007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557830af1860 0x557830affc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 >> 172.21.15.120:0/3426479007 conn(0x557830aec9b0 msgr2=0x557830af0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee557fa640 0 client.27358 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 shutdown_connections 2024-09-10T23:04:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 1 -- 172.21.15.120:0/3426479007 wait complete. 2024-09-10T23:04:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.204+0000 7fee76a19740 20 client.27358 trim_cache size 0 max 16384 2024-09-10T23:04:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:04:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:04:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:04:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 msgr2=0x7fee580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 0x7fee580b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee68109fd0 tx=0x7fee68109e00 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 msgr2=0x7fee6c19af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fee68005940 tx=0x7fee680042f0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 shutdown_connections 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fee580b52e0 0x7fee580b77a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee6c10a900 0x7fee6c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee6c076f30 0x7fee6c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 --2- 172.21.15.120:0/1888837665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee6c075ac0 0x7fee6c19af80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:04:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 >> 172.21.15.120:0/1888837665 conn(0x7fee6c0fe240 msgr2=0x7fee6c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:04:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 shutdown_connections 2024-09-10T23:04:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:04:52.205+0000 7fee73c47640 1 -- 172.21.15.120:0/1888837665 wait complete. 2024-09-10T23:04:52.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:52 smithi176 ceph-mon[34982]: pgmap v719: 97 pgs: 97 active+clean; 50 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-10T23:04:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:52 smithi120 ceph-mon[31247]: pgmap v719: 97 pgs: 97 active+clean; 50 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-10T23:04:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:52 smithi153 ceph-mon[34951]: pgmap v719: 97 pgs: 97 active+clean; 50 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-10T23:04:52.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2100955 15.84 MB/sec execute 587 sec latency 131.582 ms 2024-09-10T23:04:53.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2103640 15.83 MB/sec execute 588 sec latency 28.795 ms 2024-09-10T23:04:54.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:54 smithi176 ceph-mon[34982]: pgmap v720: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-10T23:04:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:54 smithi120 ceph-mon[31247]: pgmap v720: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-10T23:04:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:54 smithi153 ceph-mon[34951]: pgmap v720: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-10T23:04:54.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2106950 15.84 MB/sec execute 589 sec latency 13.915 ms 2024-09-10T23:04:55.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2110317 15.84 MB/sec execute 590 sec latency 26.027 ms 2024-09-10T23:04:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:56 smithi176 ceph-mon[34982]: pgmap v721: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-10T23:04:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:56 smithi120 ceph-mon[31247]: pgmap v721: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-10T23:04:56.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:56 smithi153 ceph-mon[34951]: pgmap v721: 97 pgs: 97 active+clean; 52 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-10T23:04:56.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2113433 15.84 MB/sec execute 591 sec latency 16.142 ms 2024-09-10T23:04:57.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2115878 15.83 MB/sec execute 592 sec latency 43.143 ms 2024-09-10T23:04:58.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:04:58 smithi176 ceph-mon[34982]: pgmap v722: 97 pgs: 97 active+clean; 43 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T23:04:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:04:58 smithi120 ceph-mon[31247]: pgmap v722: 97 pgs: 97 active+clean; 43 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T23:04:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:04:58 smithi153 ceph-mon[34951]: pgmap v722: 97 pgs: 97 active+clean; 43 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T23:04:58.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2117941 15.83 MB/sec execute 593 sec latency 24.821 ms 2024-09-10T23:04:59.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2121108 15.83 MB/sec execute 594 sec latency 17.688 ms 2024-09-10T23:05:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:00 smithi176 ceph-mon[34982]: pgmap v723: 97 pgs: 97 active+clean; 44 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-10T23:05:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:00 smithi120 ceph-mon[31247]: pgmap v723: 97 pgs: 97 active+clean; 44 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-10T23:05:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:00 smithi153 ceph-mon[34951]: pgmap v723: 97 pgs: 97 active+clean; 44 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-09-10T23:05:00.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2124229 15.84 MB/sec execute 595 sec latency 13.359 ms 2024-09-10T23:05:01.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2126686 15.82 MB/sec execute 596 sec latency 12.714 ms 2024-09-10T23:05:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:02 smithi176 ceph-mon[34982]: pgmap v724: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-10T23:05:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:02 smithi120 ceph-mon[31247]: pgmap v724: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-10T23:05:02.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:02 smithi153 ceph-mon[34951]: pgmap v724: 97 pgs: 97 active+clean; 48 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-09-10T23:05:02.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2129210 15.83 MB/sec execute 597 sec latency 14.352 ms 2024-09-10T23:05:03.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:05:03.628 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:03.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2131799 15.82 MB/sec execute 598 sec latency 97.584 ms 2024-09-10T23:05:04.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 -- 172.21.15.120:0/3002821017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0101c90 msgr2=0x7f6dd0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/3002821017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0101c90 0x7f6dd0102070 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009ca0 tx=0x7f6dcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 -- 172.21.15.120:0/3002821017 shutdown_connections 2024-09-10T23:05:04.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/3002821017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd0108020 0x7f6dd010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/3002821017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd01076a0 0x7f6dd0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/3002821017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0101c90 0x7f6dd0102070 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 -- 172.21.15.120:0/3002821017 >> 172.21.15.120:0/3002821017 conn(0x7f6dd00fd700 msgr2=0x7f6dd00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 -- 172.21.15.120:0/3002821017 shutdown_connections 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.134+0000 7f6dd6c95640 1 -- 172.21.15.120:0/3002821017 wait complete. 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 Processor -- start 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 -- start start 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 0x7f6dd019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0111ce0 con 0x7f6dd0108020 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0111b60 con 0x7f6dd01076a0 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd6c95640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0111e60 con 0x7f6dd0101c90 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 0x7f6dd019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd5232640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37932/0 (socket says 172.21.15.120:37932) 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 -- 172.21.15.120:0/2020609811 learned_addr learned my addr 172.21.15.120:0/2020609811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 0x7f6dd019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45374/0 (socket says 172.21.15.120:45374) 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd5232640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34020/0 (socket says 172.21.15.120:34020) 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 -- 172.21.15.120:0/2020609811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 msgr2=0x7f6dd019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 0x7f6dd019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 -- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 msgr2=0x7f6dd019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dc7fff640 1 -- 172.21.15.120:0/2020609811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd01a54f0 con 0x7f6dd01076a0 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.135+0000 7f6dd5232640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e9f0 tx=0x7f6dc000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dc5ffb640 1 -- 172.21.15.120:0/2020609811 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000cd80 con 0x7f6dd01076a0 2024-09-10T23:05:04.147 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd01a57e0 con 0x7f6dd01076a0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dc5ffb640 1 -- 172.21.15.120:0/2020609811 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc00054d0 con 0x7f6dd01076a0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dc5ffb640 1 -- 172.21.15.120:0/2020609811 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0010690 con 0x7f6dd01076a0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.136+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01a5d20 con 0x7f6dd01076a0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dc5ffb640 1 -- 172.21.15.120:0/2020609811 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6dc0004e90 con 0x7f6dd01076a0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 Processor -- start 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 -- start start 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 0x560a7b307860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b316120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560a7b27de50 con 0x560a7b307480 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a7b26d280 con 0x560a7b313d30 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd906a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560a7b271090 con 0x560a7b307da0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd5232640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 0x560a7b307860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45378/0 (socket says 172.21.15.120:45378) 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd5232640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 0x560a7b307860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34030/0 (socket says 172.21.15.120:34030) 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dd4a31640 1 -- 172.21.15.120:0/410030920 learned_addr learned my addr 172.21.15.120:0/410030920 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.139+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b316120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dc5ffb640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 0x7f6db80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 -- 172.21.15.120:0/410030920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 msgr2=0x560a7b316120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b316120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 -- 172.21.15.120:0/410030920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 msgr2=0x560a7b307860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 0x560a7b307860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 -- 172.21.15.120:0/410030920 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7afcb650 con 0x560a7b307da0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd5232640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 0x7f6db80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dc5ffb640 1 -- 172.21.15.120:0/2020609811 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dc00e5ff0 con 0x7f6dd01076a0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc035000 tx=0x7f6dcc0093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8289b22d31fcd973 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.149 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6db6ffd640 1 -- 172.21.15.120:0/410030920 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc031cf0 con 0x560a7b307da0 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6db6ffd640 1 -- 172.21.15.120:0/410030920 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc036030 con 0x560a7b307da0 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.140+0000 7f6db6ffd640 1 -- 172.21.15.120:0/410030920 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc004260 con 0x560a7b307da0 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- 172.21.15.120:0/410030920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 msgr2=0x560a7b3137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd5232640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 0x7f6db80b77a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dd019b0f0 tx=0x7f6dc80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc035000 tx=0x7f6dcc0093f0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- 172.21.15.120:0/410030920 shutdown_connections 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b316120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307da0 0x560a7b3137f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 --2- 172.21.15.120:0/410030920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307480 0x560a7b307860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- 172.21.15.120:0/410030920 >> 172.21.15.120:0/410030920 conn(0x560a7b3052b0 msgr2=0x560a7b306450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- 172.21.15.120:0/410030920 shutdown_connections 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- 172.21.15.120:0/410030920 wait complete. 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 -- start start 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.141+0000 7f6dd906a740 1 Processor -- start 2024-09-10T23:05:04.150 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b313580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b310f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560a7b0ce700 con 0x560a7b307da0 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560a7b208110 con 0x560a7b313d30 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560a7b3167d0 con 0x560a7b307480 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd906a740 10 client.0 _ll_register_callbacks cb 0x560a7b221990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34042/0 (socket says 172.21.15.120:34042) 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 -- 172.21.15.120:0/3947185291 learned_addr learned my addr 172.21.15.120:0/3947185291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b313580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dd5232640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b310f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 -- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 msgr2=0x560a7b313580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.151 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b313580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 -- 172.21.15.120:0/3947185291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 msgr2=0x560a7b310f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b310f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 -- 172.21.15.120:0/3947185291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7b235c40 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0004ad0 tx=0x7f6dc00e9580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b315da26f2f5c929 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.142+0000 7f6daffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00ac0f0 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dc00e80c0 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc036030 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dc00abf10 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 10 client.17901 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a7b322060 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc036030 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dc00aca00 con 0x560a7b307da0 2024-09-10T23:05:04.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 10 client.17901 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 20 client.17901 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b323570 0x560a7b325960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 4 client.17901 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.143+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a7b27fda0 con 0x560a7b323570 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.144+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b323570 0x560a7b325960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.144+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b323570 0x560a7b325960 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e318dadf60bb93 server_cookie=4c4296841e4662fa in_seq=0 out_seq=0 2024-09-10T23:05:04.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.144+0000 7f6daffff640 10 client.17901 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6daffff640 1 -- 172.21.15.120:0/3947185291 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a7b27fda0 con 0x560a7b323570 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6daffff640 10 client.17901 handle_command_reply: tid=0 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 client.17901 shutdown 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 msgr2=0x560a7b3109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0004ad0 tx=0x7f6dc00e9580 comp rx=0 tx=0).stop 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 shutdown_connections 2024-09-10T23:05:04.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b323570 0x560a7b325960 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b310f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b3109d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6daffff640 0 client.17901 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 --2- 172.21.15.120:0/3947185291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b313580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 >> 172.21.15.120:0/3947185291 conn(0x560a7b3052b0 msgr2=0x560a7b30ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 shutdown_connections 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 1 -- 172.21.15.120:0/3947185291 wait complete. 2024-09-10T23:05:04.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.145+0000 7f6dd906a740 20 client.17901 trim_cache size 0 max 16384 2024-09-10T23:05:04.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 Processor -- start 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 -- start start 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b30dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b32b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ac270 con 0x560a7b307da0 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01aeaf0 con 0x560a7b313d30 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd906a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ac8e0 con 0x560a7b307480 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34044/0 (socket says 172.21.15.120:34044) 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1174952088 learned_addr learned my addr 172.21.15.120:0/1174952088 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd5232640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b32b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b30dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1174952088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 msgr2=0x560a7b30dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b30dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.163+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1174952088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 msgr2=0x560a7b32b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b32b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1174952088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7b3112a0 con 0x560a7b307da0 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dd5232640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b32b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00a9a10 tx=0x7f6dc00048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd136e6b1921bcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6db67fc640 1 -- 172.21.15.120:0/1174952088 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0097070 con 0x560a7b307da0 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6db67fc640 1 -- 172.21.15.120:0/1174952088 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc00cf040 con 0x560a7b307da0 2024-09-10T23:05:04.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6db67fc640 1 -- 172.21.15.120:0/1174952088 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00a8df0 con 0x560a7b307da0 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dd906a740 1 -- 172.21.15.120:0/1174952088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 msgr2=0x560a7b30e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.164+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00a9a10 tx=0x7f6dc00048d0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- 172.21.15.120:0/1174952088 shutdown_connections 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b32b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b30e140 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1174952088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b30dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- 172.21.15.120:0/1174952088 >> 172.21.15.120:0/1174952088 conn(0x560a7b3052b0 msgr2=0x560a7b312820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- 172.21.15.120:0/1174952088 shutdown_connections 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- 172.21.15.120:0/1174952088 wait complete. 2024-09-10T23:05:04.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- start start 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 Processor -- start 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b318f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01acfb0 con 0x560a7b307da0 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ae420 con 0x560a7b313d30 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ad680 con 0x560a7b307480 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.165+0000 7f6dd906a740 10 client.0 _ll_register_callbacks cb 0x7f6dc000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd4a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45412/0 (socket says 172.21.15.120:45412) 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34060/0 (socket says 172.21.15.120:34060) 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd4a31640 1 -- 172.21.15.120:0/1562787812 learned_addr learned my addr 172.21.15.120:0/1562787812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:04.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd5232640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b318f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 msgr2=0x560a7b31bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1562787812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 msgr2=0x560a7b318f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b318f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 -- 172.21.15.120:0/1562787812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7b3194e0 con 0x560a7b307da0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd5232640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b318f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dc7fff640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x560a7b300a30 tx=0x7f6dc00ac0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753c9a56e680e2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6daffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0097070 con 0x560a7b307da0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dc00e9140 con 0x560a7b307da0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc00cf040 con 0x560a7b307da0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dc00a8df0 con 0x560a7b307da0 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd906a740 10 client.17907 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:04.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.166+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a7b395600 con 0x560a7b307da0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc00cf040 con 0x560a7b307da0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dc00cfaf0 con 0x560a7b307da0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 10 client.17907 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 20 client.17907 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b395ad0 0x560a7b397ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 4 client.17907 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a7b27b850 con 0x560a7b395ad0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b395ad0 0x560a7b397ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6dd4a31640 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b395ad0 0x560a7b397ec0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11c12f54fd48273 server_cookie=b4fcfea347720c9a in_seq=0 out_seq=0 2024-09-10T23:05:04.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.167+0000 7f6daffff640 10 client.17907 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:04.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.168+0000 7f6daffff640 1 -- 172.21.15.120:0/1562787812 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x560a7b27b850 con 0x560a7b395ad0 2024-09-10T23:05:04.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.168+0000 7f6daffff640 10 client.17907 handle_command_reply: tid=0 2024-09-10T23:05:04.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.168+0000 7f6dd906a740 1 client.17907 shutdown 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 msgr2=0x560a7b318a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x560a7b300a30 tx=0x7f6dc00ac0f0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 shutdown_connections 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560a7b395ad0 0x560a7b397ec0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6daffff640 0 client.17907 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560a7b313d30 0x560a7b318f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560a7b307da0 0x560a7b318a30 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 --2- 172.21.15.120:0/1562787812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560a7b307480 0x560a7b31bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 >> 172.21.15.120:0/1562787812 conn(0x560a7b3052b0 msgr2=0x560a7b303bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 shutdown_connections 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 1 -- 172.21.15.120:0/1562787812 wait complete. 2024-09-10T23:05:04.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.169+0000 7f6dd906a740 20 client.17907 trim_cache size 0 max 16384 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:05:04.175 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:04.186 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 msgr2=0x7f6db80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 0x7f6db80b77a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dd019b0f0 tx=0x7f6dc80046c0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 msgr2=0x7f6dd019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000e9f0 tx=0x7f6dc000edf0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 shutdown_connections 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db80b52e0 0x7f6db80b77a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dd0108020 0x7f6dd019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dd01076a0 0x7f6dd019a650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 --2- 172.21.15.120:0/2020609811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd0101c90 0x7f6dd019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 >> 172.21.15.120:0/2020609811 conn(0x7f6dd00fd700 msgr2=0x7f6dd00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 shutdown_connections 2024-09-10T23:05:04.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:04.170+0000 7f6dd6c95640 1 -- 172.21.15.120:0/2020609811 wait complete. 2024-09-10T23:05:04.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:04 smithi176 ceph-mon[34982]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T23:05:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:04 smithi120 ceph-mon[31247]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T23:05:04.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:04 smithi153 ceph-mon[34951]: pgmap v725: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-09-10T23:05:04.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 2135107 15.83 MB/sec execute 599 sec latency 38.845 ms 2024-09-10T23:05:05.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:05:05.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 -- 172.21.15.120:0/1957883961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e81029c0 msgr2=0x7f16e8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 --2- 172.21.15.120:0/1957883961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e81029c0 0x7f16e8102e40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009930 tx=0x7f16d802a420 comp rx=0 tx=0).stop 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 -- 172.21.15.120:0/1957883961 shutdown_connections 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 --2- 172.21.15.120:0/1957883961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8103380 0x7f16e8109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 --2- 172.21.15.120:0/1957883961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e81029c0 0x7f16e8102e40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 --2- 172.21.15.120:0/1957883961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e8101870 0x7f16e8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 -- 172.21.15.120:0/1957883961 >> 172.21.15.120:0/1957883961 conn(0x7f16e80fd130 msgr2=0x7f16e80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 -- 172.21.15.120:0/1957883961 shutdown_connections 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.297+0000 7f16efa0c640 1 -- 172.21.15.120:0/1957883961 wait complete. 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 Processor -- start 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 -- start start 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8101870 0x7f16e8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e81029c0 0x7f16e81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d4e0 con 0x7f16e8103380 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d360 con 0x7f16e81029c0 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16efa0c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d660 con 0x7f16e8101870 2024-09-10T23:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16edfa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16edfa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34074/0 (socket says 172.21.15.120:34074) 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16edfa9640 1 -- 172.21.15.120:0/3469989225 learned_addr learned my addr 172.21.15.120:0/3469989225 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8101870 0x7f16e8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.298+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e81029c0 0x7f16e81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 -- 172.21.15.120:0/3469989225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8101870 msgr2=0x7f16e8198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8101870 0x7f16e8198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 -- 172.21.15.120:0/3469989225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e81029c0 msgr2=0x7f16e81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e81029c0 0x7f16e81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 -- 172.21.15.120:0/3469989225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e819a8e0 con 0x7f16e8103380 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16edfa9640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f16e000eb10 tx=0x7f16e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16de7fc640 1 -- 172.21.15.120:0/3469989225 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004b20 con 0x7f16e8103380 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e819abd0 con 0x7f16e8103380 2024-09-10T23:05:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16de7fc640 1 -- 172.21.15.120:0/3469989225 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e0004cc0 con 0x7f16e8103380 2024-09-10T23:05:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16de7fc640 1 -- 172.21.15.120:0/3469989225 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0010690 con 0x7f16e8103380 2024-09-10T23:05:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.299+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a12a0 con 0x7f16e8103380 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.302+0000 7f16f27da740 1 Processor -- start 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.302+0000 7f16de7fc640 1 -- 172.21.15.120:0/3469989225 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f16e0005020 con 0x7f16e8103380 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 -- start start 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6cab6f0 con 0x55b7b6d3c190 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6c9ab20 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16f27da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6c9e930 con 0x55b7b6d39860 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34080/0 (socket says 172.21.15.120:34080) 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16ed7a8640 1 -- 172.21.15.120:0/891268354 learned_addr learned my addr 172.21.15.120:0/891268354 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.303+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 -- 172.21.15.120:0/891268354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 msgr2=0x55b7b6d3bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d3bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 -- 172.21.15.120:0/891268354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 msgr2=0x55b7b6d41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 -- 172.21.15.120:0/891268354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b6c60530 con 0x55b7b6d3c190 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16de7fc640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 0x7f16d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b840 tx=0x7f16e400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e553a7ec197157 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 0x7f16d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16de7fc640 1 -- 172.21.15.120:0/3469989225 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f16e00e60d0 con 0x7f16e8103380 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16d37fe640 1 -- 172.21.15.120:0/891268354 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e400d920 con 0x55b7b6d3c190 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16d37fe640 1 -- 172.21.15.120:0/891268354 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4005020 con 0x55b7b6d3c190 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.304+0000 7f16d37fe640 1 -- 172.21.15.120:0/891268354 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4004bc0 con 0x55b7b6d3c190 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- 172.21.15.120:0/891268354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 msgr2=0x55b7b6d3e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b840 tx=0x7f16e400bd00 comp rx=0 tx=0).stop 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 0x7f16d40b77a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006010 tx=0x7f16d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- 172.21.15.120:0/891268354 shutdown_connections 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3c190 0x55b7b6d3e580 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 --2- 172.21.15.120:0/891268354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d3bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- 172.21.15.120:0/891268354 >> 172.21.15.120:0/891268354 conn(0x55b7b6d349b0 msgr2=0x55b7b6d35740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- 172.21.15.120:0/891268354 shutdown_connections 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- 172.21.15.120:0/891268354 wait complete. 2024-09-10T23:05:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 -- start start 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.305+0000 7f16f27da740 1 Processor -- start 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3c190 0x55b7b6d43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d44400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6bd2ab0 con 0x55b7b6d39860 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6c40730 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7b6d41990 con 0x55b7b6d3c190 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16f27da740 10 client.0 _ll_register_callbacks cb 0x55b7b6c946b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d44400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34088/0 (socket says 172.21.15.120:34088) 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16ed7a8640 1 -- 172.21.15.120:0/2690313829 learned_addr learned my addr 172.21.15.120:0/2690313829 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3c190 0x55b7b6d43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 -- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3c190 msgr2=0x55b7b6d43ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3c190 0x55b7b6d43ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 -- 172.21.15.120:0/2690313829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 msgr2=0x55b7b6d451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 -- 172.21.15.120:0/2690313829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b69fc790 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16edfa9640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d44400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16e00049d0 tx=0x7f16e00a8760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8552fa300007851 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.306+0000 7f16d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00ac030 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f16e00b59c0 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e4005020 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16e00c7050 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 10 client.27209 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b7b6a75620 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4005020 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f16e00a8210 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 10 client.27209 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 10 client.27209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 20 client.27209 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6d50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 4 client.27209 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7b6cad640 con 0x55b7b6d4e6b0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.307+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6d50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.308+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6d50aa0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3e6cdd07c0eb74 server_cookie=880a5b109f9a1b5b in_seq=0 out_seq=0 2024-09-10T23:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.308+0000 7f16d0ff9640 10 client.27209 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.308+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2690313829 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7b6cad640 con 0x55b7b6d4e6b0 2024-09-10T23:05:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.308+0000 7f16d0ff9640 10 client.27209 handle_command_reply: tid=0 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 client.27209 shutdown 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 msgr2=0x55b7b6d44400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d44400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16e00049d0 tx=0x7f16e00a8760 comp rx=0 tx=0).stop 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 shutdown_connections 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6d50aa0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d44400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3c190 0x55b7b6d43ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16d0ff9640 0 client.27209 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 --2- 172.21.15.120:0/2690313829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 >> 172.21.15.120:0/2690313829 conn(0x55b7b6d349b0 msgr2=0x55b7b6d33bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 shutdown_connections 2024-09-10T23:05:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 1 -- 172.21.15.120:0/2690313829 wait complete. 2024-09-10T23:05:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.309+0000 7f16f27da740 20 client.27209 trim_cache size 0 max 16384 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 Processor -- start 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 -- start start 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a5c70 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a84f0 con 0x55b7b6d3c190 2024-09-10T23:05:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.332+0000 7f16f27da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a62e0 con 0x55b7b6d39860 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16ecfa7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34094/0 (socket says 172.21.15.120:34094) 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 -- 172.21.15.120:0/597132042 learned_addr learned my addr 172.21.15.120:0/597132042 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16ecfa7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37990/0 (socket says 172.21.15.120:37990) 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 -- 172.21.15.120:0/597132042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 msgr2=0x55b7b6d43ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d43ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 -- 172.21.15.120:0/597132042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 msgr2=0x55b7b6d44400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 -- 172.21.15.120:0/597132042 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b6d302d0 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16edfa9640 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f16e00e63f0 tx=0x7f16e000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec6875d5cab5e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16d2ffd640 1 -- 172.21.15.120:0/597132042 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00cb070 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16d2ffd640 1 -- 172.21.15.120:0/597132042 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e00ac3d0 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.333+0000 7f16d2ffd640 1 -- 172.21.15.120:0/597132042 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00ac8c0 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 -- 172.21.15.120:0/597132042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 msgr2=0x55b7b6d37a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f16e00e63f0 tx=0x7f16e000cee0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 -- 172.21.15.120:0/597132042 shutdown_connections 2024-09-10T23:05:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d37a00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d44400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 --2- 172.21.15.120:0/597132042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d39860 0x55b7b6d43ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 -- 172.21.15.120:0/597132042 >> 172.21.15.120:0/597132042 conn(0x55b7b6d349b0 msgr2=0x55b7b6d35650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 -- 172.21.15.120:0/597132042 shutdown_connections 2024-09-10T23:05:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.334+0000 7f16f27da740 1 -- 172.21.15.120:0/597132042 wait complete. 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 -- start start 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 Processor -- start 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a69b0 con 0x55b7b6d39860 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a7e20 con 0x55b7b6d3c190 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.335+0000 7f16f27da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a7080 con 0x55b7b6d3ebd0 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16f27da740 10 client.0 _ll_register_callbacks cb 0x7f16e00a8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34100/0 (socket says 172.21.15.120:34100) 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 -- 172.21.15.120:0/2274088616 learned_addr learned my addr 172.21.15.120:0/2274088616 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16edfa9640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d48e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 -- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 msgr2=0x55b7b6d49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 -- 172.21.15.120:0/2274088616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 msgr2=0x55b7b6d48e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d48e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 -- 172.21.15.120:0/2274088616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b6d4a290 con 0x55b7b6d39860 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16edfa9640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16ed7a8640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f16e400d710 tx=0x7f16e4005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04daf2db71cb447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4034070 con 0x55b7b6d39860 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f16e400db30 con 0x55b7b6d39860 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.336+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e00ac3d0 con 0x55b7b6d39860 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16e403f070 con 0x55b7b6d39860 2024-09-10T23:05:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 10 client.17925 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b7b6d4b670 con 0x55b7b6d39860 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e00ac3d0 con 0x55b7b6d39860 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f16e401f410 con 0x55b7b6d39860 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 10 client.17925 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 20 client.17925 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6dbd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 4 client.17925 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7b6ca90f0 con 0x55b7b6d4e6b0 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.337+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6dbd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.338+0000 7f16d0ff9640 10 client.17925 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.338+0000 7f16ecfa7640 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6dbd610 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a3767972e1137a server_cookie=880f9ec252c3812f in_seq=0 out_seq=0 2024-09-10T23:05:05.377 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16d0ff9640 1 -- 172.21.15.120:0/2274088616 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7b6ca90f0 con 0x55b7b6d4e6b0 2024-09-10T23:05:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16d0ff9640 10 client.17925 handle_command_reply: tid=0 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 client.17925 shutdown 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 msgr2=0x55b7b6d487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f16e400d710 tx=0x7f16e4005a70 comp rx=0 tx=0).stop 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 shutdown_connections 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b7b6d4e6b0 0x55b7b6dbd610 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7b6d3ebd0 0x55b7b6d49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b7b6d3c190 0x55b7b6d48e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 --2- 172.21.15.120:0/2274088616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b7b6d39860 0x55b7b6d487f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 >> 172.21.15.120:0/2274088616 conn(0x55b7b6d349b0 msgr2=0x55b7b6d43ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16d0ff9640 0 client.17925 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 shutdown_connections 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 1 -- 172.21.15.120:0/2274088616 wait complete. 2024-09-10T23:05:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.593+0000 7f16f27da740 20 client.17925 trim_cache size 0 max 16384 2024-09-10T23:05:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:05:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:05:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:05:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 msgr2=0x7f16d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 0x7f16d40b77a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006010 tx=0x7f16d8009260 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 msgr2=0x7f16e819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f16e000eb10 tx=0x7f16e000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 shutdown_connections 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16d40b52e0 0x7f16d40b77a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e8103380 0x7f16e819a020 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16e81029c0 0x7f16e81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 --2- 172.21.15.120:0/3469989225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e8101870 0x7f16e8198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 >> 172.21.15.120:0/3469989225 conn(0x7f16e80fd130 msgr2=0x7f16e80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 shutdown_connections 2024-09-10T23:05:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:05.594+0000 7f16efa0c640 1 -- 172.21.15.120:0/3469989225 wait complete. 2024-09-10T23:05:05.645 INFO:tasks.workunit.client.0.smithi120.stdout: 1 cleanup 600 sec 2024-09-10T23:05:05.762 INFO:tasks.workunit.client.0.smithi120.stdout: 0 cleanup 600 sec 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Operation Count AvgLat MaxLat 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: ---------------------------------------- 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: NTCreateX 302143 0.747 178.278 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Close 221948 0.002 0.166 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Rename 12790 3.538 131.571 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Unlink 61014 1.298 104.928 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Deltree 8 66.603 143.668 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Mkdir 4 0.006 0.012 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Qpathinfo 273856 0.560 323.110 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Qfileinfo 47996 0.001 0.047 2024-09-10T23:05:05.763 INFO:tasks.workunit.client.0.smithi120.stdout: Qfsinfo 50209 0.144 6.849 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: Sfileinfo 24604 0.014 3.606 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: Find 105874 0.062 83.681 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: WriteX 150698 0.013 103.894 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: ReadX 473522 0.011 33.044 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: LockX 984 0.740 7.253 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: UnlockX 984 0.618 38.487 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: Flush 21177 3.291 187.534 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T23:05:05.764 INFO:tasks.workunit.client.0.smithi120.stdout:Throughput 15.8289 MB/sec 1 clients 1 procs max_latency=323.123 ms 2024-09-10T23:05:05.765 INFO:tasks.workunit.client.0.smithi120.stderr:+ dbench 10 2024-09-10T23:05:05.765 INFO:tasks.workunit.client.0.smithi120.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T23:05:05.765 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T23:05:05.765 INFO:tasks.workunit.client.0.smithi120.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T23:05:05.765 INFO:tasks.workunit.client.0.smithi120.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-10T23:05:05.864 INFO:tasks.workunit.client.0.smithi120.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-10T23:05:05.864 INFO:tasks.workunit.client.0.smithi120.stdout:releasing clients 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 -- 172.21.15.120:0/2868475818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef010d0e0 msgr2=0x7f7ef010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 --2- 172.21.15.120:0/2868475818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef010d0e0 0x7f7ef010f550 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009ca0 tx=0x7f7ee802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 -- 172.21.15.120:0/2868475818 shutdown_connections 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 --2- 172.21.15.120:0/2868475818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef010d0e0 0x7f7ef010f550 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 --2- 172.21.15.120:0/2868475818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef0104530 0x7f7ef010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 --2- 172.21.15.120:0/2868475818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0103b30 0x7f7ef0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.892+0000 7f7ef7558640 1 -- 172.21.15.120:0/2868475818 >> 172.21.15.120:0/2868475818 conn(0x7f7ef00fd700 msgr2=0x7f7ef00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- 172.21.15.120:0/2868475818 shutdown_connections 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- 172.21.15.120:0/2868475818 wait complete. 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 Processor -- start 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- start start 2024-09-10T23:05:05.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 0x7f7ef019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 0x7f7ef01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0112d60 con 0x7f7ef0103b30 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0112be0 con 0x7f7ef0104530 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef7558640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0112ee0 con 0x7f7ef010d0e0 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.893+0000 7f7ef52f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34112/0 (socket says 172.21.15.120:34112) 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 -- 172.21.15.120:0/862720472 learned_addr learned my addr 172.21.15.120:0/862720472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef4af3640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 0x7f7ef019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef5af5640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 0x7f7ef01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 -- 172.21.15.120:0/862720472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 msgr2=0x7f7ef01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 0x7f7ef01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 -- 172.21.15.120:0/862720472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 msgr2=0x7f7ef019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 0x7f7ef019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 -- 172.21.15.120:0/862720472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01a14e0 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef5af5640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 0x7f7ef01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef4af3640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 0x7f7ef019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef52f4640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e9f0 tx=0x7f7eec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00cd80 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec0108e0 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01a17d0 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec018980 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.894+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01a7e90 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.895+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef0103f10 con 0x7f7ef0103b30 2024-09-10T23:05:05.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.897+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7eec010430 con 0x7f7ef0103b30 2024-09-10T23:05:05.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.897+0000 7f7ee67fc640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 0x7f7edc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:05.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.898+0000 7f7ef4af3640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 0x7f7edc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:05.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.898+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7eec0e6450 con 0x7f7ef0103b30 2024-09-10T23:05:05.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.898+0000 7f7ef4af3640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 0x7f7edc0b77a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00ffe00 tx=0x7f7ee0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:05.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:05.900+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7eec0ad340 con 0x7f7ef0103b30 2024-09-10T23:05:06.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.131+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef019c560 con 0x7f7ef0103b30 2024-09-10T23:05:06.132 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:06 smithi120 ceph-mon[31247]: pgmap v726: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-09-10T23:05:06.133 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:05:06.133 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:05:06.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.132+0000 7f7ee67fc640 1 -- 172.21.15.120:0/862720472 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f7ef019c560 con 0x7f7ef0103b30 2024-09-10T23:05:06.133 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:05:06.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 msgr2=0x7f7edc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:06.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 0x7f7edc0b77a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00ffe00 tx=0x7f7ee0005f90 comp rx=0 tx=0).stop 2024-09-10T23:05:06.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 msgr2=0x7f7ef019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e9f0 tx=0x7f7eec00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 shutdown_connections 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7edc0b52e0 0x7f7edc0b77a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ef010d0e0 0x7f7ef01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ef0104530 0x7f7ef019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 --2- 172.21.15.120:0/862720472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ef0103b30 0x7f7ef019b430 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 >> 172.21.15.120:0/862720472 conn(0x7f7ef00fd700 msgr2=0x7f7ef010f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 shutdown_connections 2024-09-10T23:05:06.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:06.134+0000 7f7ef7558640 1 -- 172.21.15.120:0/862720472 wait complete. 2024-09-10T23:05:06.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:06 smithi176 ceph-mon[34982]: pgmap v726: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-09-10T23:05:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:06 smithi153 ceph-mon[34951]: pgmap v726: 97 pgs: 97 active+clean; 48 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-09-10T23:05:06.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:05:06.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 426 150.47 MB/sec warmup 1 sec latency 36.092 ms 2024-09-10T23:05:07.112 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:07.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:07 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/862720472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:07.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:07 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/862720472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:07.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:07 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/862720472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 -- 172.21.15.120:0/285899235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd99810f3e0 msgr2=0x7fd9981117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/285899235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd99810f3e0 0x7fd9981117d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd980009900 tx=0x7fd98002a480 comp rx=0 tx=0).stop 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 -- 172.21.15.120:0/285899235 shutdown_connections 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/285899235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd99810f3e0 0x7fd9981117d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/285899235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd998102150 0x7fd99810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/285899235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd998101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 -- 172.21.15.120:0/285899235 >> 172.21.15.120:0/285899235 conn(0x7fd9980fd700 msgr2=0x7fd9980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 -- 172.21.15.120:0/285899235 shutdown_connections 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.641+0000 7fd99e7eb640 1 -- 172.21.15.120:0/285899235 wait complete. 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.642+0000 7fd99e7eb640 1 Processor -- start 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.642+0000 7fd99e7eb640 1 -- start start 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd99810eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd998102150 0x7fd99810bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 0x7fd99810c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd998114f90 con 0x7fd998102150 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd998114e10 con 0x7fd998101830 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99e7eb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd998115110 con 0x7fd99810f3e0 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd997fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd99810eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd997fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd99810eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38026/0 (socket says 172.21.15.120:38026) 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd997fff640 1 -- 172.21.15.120:0/1716831379 learned_addr learned my addr 172.21.15.120:0/1716831379 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 0x7fd99810c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 -- 172.21.15.120:0/1716831379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 msgr2=0x7fd99810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd99810eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 -- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd998102150 msgr2=0x7fd99810bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd998102150 0x7fd99810bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.643+0000 7fd99cd88640 1 -- 172.21.15.120:0/1716831379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99810cb20 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 0x7fd99810c300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd98002a9a0 tx=0x7fd980003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd9957fa640 1 -- 172.21.15.120:0/1716831379 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98003f070 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9981abcd0 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9981ac200 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd9957fa640 1 -- 172.21.15.120:0/1716831379 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980031b50 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.644+0000 7fd9957fa640 1 -- 172.21.15.120:0/1716831379 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980035030 con 0x7fd99810f3e0 2024-09-10T23:05:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.647+0000 7fd9957fa640 1 -- 172.21.15.120:0/1716831379 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd98004c060 con 0x7fd99810f3e0 2024-09-10T23:05:07.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9957fa640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 Processor -- start 2024-09-10T23:05:07.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 -- start start 2024-09-10T23:05:07.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 0x55c2f32984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2f320eea0 con 0x55c2f32a49a0 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2f31fe2d0 con 0x55c2f32980f0 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c2f32020e0 con 0x55c2f3298a10 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45530/0 (socket says 172.21.15.120:45530) 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd9977fe640 1 -- 172.21.15.120:0/1867876741 learned_addr learned my addr 172.21.15.120:0/1867876741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.649+0000 7fd997fff640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd997fff640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 0x55c2f32984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 -- 172.21.15.120:0/1867876741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 msgr2=0x55c2f32a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 -- 172.21.15.120:0/1867876741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 msgr2=0x55c2f32984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 0x55c2f32984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd99cd88640 1 -- 172.21.15.120:0/1867876741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f2f5c650 con 0x55c2f32a49a0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd997fff640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 0x55c2f32984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.650+0000 7fd9977fe640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.651+0000 7fd9957fa640 1 -- 172.21.15.120:0/1716831379 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd980105ca0 con 0x7fd99810f3e0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.651+0000 7fd99cd88640 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a6d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd9800ca840 tx=0x7fd98009fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef944cc424559b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.651+0000 7fd997fff640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 0x7fd9840b7820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd9900098e0 tx=0x7fd9900117b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.651+0000 7fd98e7fc640 1 -- 172.21.15.120:0/1867876741 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980049020 con 0x55c2f32a49a0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.651+0000 7fd98e7fc640 1 -- 172.21.15.120:0/1867876741 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9800d59c0 con 0x55c2f32a49a0 2024-09-10T23:05:07.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd98e7fc640 1 -- 172.21.15.120:0/1867876741 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980049020 con 0x55c2f32a49a0 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/1867876741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 msgr2=0x55c2f32a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a6d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd9800ca840 tx=0x7fd98009fd60 comp rx=0 tx=0).stop 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/1867876741 shutdown_connections 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a6d90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f3298a10 0x55c2f32a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.652+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/1867876741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32980f0 0x55c2f32984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.653+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/1867876741 >> 172.21.15.120:0/1867876741 conn(0x55c2f3295f20 msgr2=0x55c2f32970c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.653+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/1867876741 shutdown_connections 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.653+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/1867876741 wait complete. 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.653+0000 7fd9a0bc0740 1 -- start start 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 Processor -- start 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 0x55c2f32a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd997fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 0x55c2f32a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c2f305f700 con 0x55c2f32a49a0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 0x55c2f32a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38044/0 (socket says 172.21.15.120:38044) 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9977fe640 1 -- 172.21.15.120:0/3037994390 learned_addr learned my addr 172.21.15.120:0/3037994390 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.654+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c2f3199160 con 0x55c2f3298a10 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c2f320eea0 con 0x55c2f32980f0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd9a0bc0740 10 client.0 _ll_register_callbacks cb 0x55c2f31b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 -- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 msgr2=0x55c2f32a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 -- 172.21.15.120:0/3037994390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 msgr2=0x55c2f32a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 0x55c2f32a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f31c6c90 con 0x55c2f32a49a0 2024-09-10T23:05:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd997fff640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a1bd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55c2f3299040 tx=0x7fd9800c7bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0b1107a4ec1ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd9777fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.655+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980049070 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd9800a2910 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9800d59c0 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9800a2b10 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9a0bc0740 10 client.17940 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f32b2ac0 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.656+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9800d59c0 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd9800d5530 con 0x55c2f32a49a0 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 10 client.17940 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 20 client.17940 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:07.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f32b3e10 0x55c2f32b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 4 client.17940 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.657+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f3210df0 con 0x55c2f32b3e10 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.658+0000 7fd997fff640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f32b3e10 0x55c2f32b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.658+0000 7fd9777fe640 10 client.17940 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.658+0000 7fd997fff640 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f32b3e10 0x55c2f32b6200 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c613aca553a27c server_cookie=9e7bc8e7279b881 in_seq=0 out_seq=0 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9777fe640 1 -- 172.21.15.120:0/3037994390 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2f3210df0 con 0x55c2f32b3e10 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9777fe640 10 client.17940 handle_command_reply: tid=0 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 client.17940 shutdown 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 msgr2=0x55c2f32a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a1bd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55c2f3299040 tx=0x7fd9800c7bc0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 shutdown_connections 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f32b3e10 0x55c2f32b6200 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f32a49a0 0x55c2f32a1bd0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3298a10 0x55c2f32a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3037994390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 >> 172.21.15.120:0/3037994390 conn(0x55c2f3295f20 msgr2=0x55c2f329ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.660+0000 7fd9777fe640 0 client.17940 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.661+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 shutdown_connections 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.661+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3037994390 wait complete. 2024-09-10T23:05:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.661+0000 7fd9a0bc0740 20 client.17940 trim_cache size 0 max 16384 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 Processor -- start 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 -- start start 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f329cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800c380 con 0x55c2f3298a10 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800ec00 con 0x55c2f32a49a0 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.679+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800c9f0 con 0x55c2f32980f0 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f329cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45544/0 (socket says 172.21.15.120:45544) 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f329cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34166/0 (socket says 172.21.15.120:34166) 2024-09-10T23:05:07.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 -- 172.21.15.120:0/2516486431 learned_addr learned my addr 172.21.15.120:0/2516486431 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:07.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd99cd88640 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 -- 172.21.15.120:0/2516486431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 msgr2=0x55c2f32a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 -- 172.21.15.120:0/2516486431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 msgr2=0x55c2f329cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f329cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 -- 172.21.15.120:0/2516486431 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f32a4570 con 0x55c2f32980f0 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd99cd88640 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.680+0000 7fd997fff640 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd990014880 tx=0x7fd990016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a5069e801957e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.681+0000 7fd98dffb640 1 -- 172.21.15.120:0/2516486431 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99002e070 con 0x55c2f32980f0 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.681+0000 7fd98dffb640 1 -- 172.21.15.120:0/2516486431 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd990022400 con 0x55c2f32980f0 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.681+0000 7fd98dffb640 1 -- 172.21.15.120:0/2516486431 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99000f630 con 0x55c2f32980f0 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/2516486431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 msgr2=0x55c2f32b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd990014880 tx=0x7fd990016000 comp rx=0 tx=0).stop 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/2516486431 shutdown_connections 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f329cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/2516486431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32b04f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/2516486431 >> 172.21.15.120:0/2516486431 conn(0x55c2f3295f20 msgr2=0x55c2f32b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/2516486431 shutdown_connections 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/2516486431 wait complete. 2024-09-10T23:05:07.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.682+0000 7fd9a0bc0740 1 -- start start 2024-09-10T23:05:07.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 Processor -- start 2024-09-10T23:05:07.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f32bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800d0c0 con 0x55c2f3298a10 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800e530 con 0x55c2f32a49a0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800d790 con 0x55c2f32980f0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9a0bc0740 10 client.0 _ll_register_callbacks cb 0x7fd988009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd997fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd997fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45546/0 (socket says 172.21.15.120:45546) 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd997fff640 1 -- 172.21.15.120:0/3787183890 learned_addr learned my addr 172.21.15.120:0/3787183890 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd9977fe640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f32bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.683+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 -- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 msgr2=0x55c2f32aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 -- 172.21.15.120:0/3787183890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 msgr2=0x55c2f32bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f32bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 -- 172.21.15.120:0/3787183890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f32abb10 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd99cd88640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32bbf90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd980031b20 tx=0x7fd9801037f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e149a349cddc0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9777fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980049060 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd990022400 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd9800d5530 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9800d5730 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9a0bc0740 10 client.27236 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.684+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f32abb10 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd990022400 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd98003bce0 con 0x55c2f32a49a0 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 10 client.27236 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 10 client.27236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 20 client.27236 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:07.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f3321230 0x55c2f3323650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:07.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 4 client.27236 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:05:07.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f329d220 con 0x55c2f3321230 2024-09-10T23:05:07.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd997fff640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f3321230 0x55c2f3323650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:07.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd9777fe640 10 client.27236 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:07.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.685+0000 7fd997fff640 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f3321230 0x55c2f3323650 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3248bdabc4adb6 server_cookie=89d3ed4dc9b399d2 in_seq=0 out_seq=0 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9777fe640 1 -- 172.21.15.120:0/3787183890 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f329d220 con 0x55c2f3321230 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9777fe640 10 client.27236 handle_command_reply: tid=0 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 client.27236 shutdown 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 msgr2=0x55c2f32bbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32bbf90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd980031b20 tx=0x7fd9801037f0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 shutdown_connections 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c2f3321230 0x55c2f3323650 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f32a49a0 0x55c2f32bbf90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c2f3298a10 0x55c2f32bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 --2- 172.21.15.120:0/3787183890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2f32980f0 0x55c2f32aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9777fe640 0 client.27236 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:07.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.689+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 >> 172.21.15.120:0/3787183890 conn(0x55c2f3295f20 msgr2=0x55c2f3294750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 shutdown_connections 2024-09-10T23:05:07.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd9a0bc0740 1 -- 172.21.15.120:0/3787183890 wait complete. 2024-09-10T23:05:07.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd9a0bc0740 20 client.27236 trim_cache size 0 max 16384 2024-09-10T23:05:07.693 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 msgr2=0x7fd9840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 0x7fd9840b7820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd9900098e0 tx=0x7fd9900117b0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 msgr2=0x7fd99810c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 0x7fd99810c300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd98002a9a0 tx=0x7fd980003b90 comp rx=0 tx=0).stop 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 shutdown_connections 2024-09-10T23:05:07.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd99810f3e0 0x7fd99810c300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd998102150 0x7fd99810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 --2- 172.21.15.120:0/1716831379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd998101830 0x7fd99810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.690+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 >> 172.21.15.120:0/1716831379 conn(0x7fd9980fd700 msgr2=0x7fd998111560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.691+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 shutdown_connections 2024-09-10T23:05:07.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:07.691+0000 7fd99e7eb640 1 -- 172.21.15.120:0/1716831379 wait complete. 2024-09-10T23:05:07.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 901 108.48 MB/sec warmup 2 sec latency 112.510 ms 2024-09-10T23:05:08.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:08 smithi176 ceph-mon[34982]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 478 op/s 2024-09-10T23:05:08.427 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:08 smithi120 ceph-mon[31247]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 478 op/s 2024-09-10T23:05:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:08 smithi153 ceph-mon[34951]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 478 op/s 2024-09-10T23:05:08.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 1425 85.78 MB/sec warmup 3 sec latency 173.070 ms 2024-09-10T23:05:09.458 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:05:09.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:05:09.741 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:09.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 2395 69.97 MB/sec warmup 4 sec latency 63.159 ms 2024-09-10T23:05:10.183 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:10 smithi120 ceph-mon[31247]: pgmap v728: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 -- 172.21.15.120:0/3925180628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f456810d100 msgr2=0x7f456810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 --2- 172.21.15.120:0/3925180628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f456810d100 0x7f456810f560 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 -- 172.21.15.120:0/3925180628 shutdown_connections 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 --2- 172.21.15.120:0/3925180628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f456810d100 0x7f456810f560 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 --2- 172.21.15.120:0/3925180628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45681045c0 0x7f456810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 --2- 172.21.15.120:0/3925180628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4568103ca0 0x7f4568104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 -- 172.21.15.120:0/3925180628 >> 172.21.15.120:0/3925180628 conn(0x7f45680fd700 msgr2=0x7f45680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 -- 172.21.15.120:0/3925180628 shutdown_connections 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.270+0000 7f456dc0e640 1 -- 172.21.15.120:0/3925180628 wait complete. 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 Processor -- start 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 -- start start 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 0x7f456819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4568112d70 con 0x7f4568103ca0 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4568112bf0 con 0x7f45681045c0 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f456dc0e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4568112ef0 con 0x7f456810d100 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.271+0000 7f45677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f455ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f455ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38074/0 (socket says 172.21.15.120:38074) 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f455ffff640 1 -- 172.21.15.120:0/104797812 learned_addr learned my addr 172.21.15.120:0/104797812 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34202/0 (socket says 172.21.15.120:34202) 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f4567fff640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 0x7f456819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 -- 172.21.15.120:0/104797812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 msgr2=0x7f456819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 0x7f456819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 -- 172.21.15.120:0/104797812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 msgr2=0x7f45681997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 -- 172.21.15.120:0/104797812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456819f310 con 0x7f4568103ca0 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f4567fff640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 0x7f456819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f455ffff640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45677fe640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4558009ce0 tx=0x7f455800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45657fa640 1 -- 172.21.15.120:0/104797812 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558004020 con 0x7f4568103ca0 2024-09-10T23:05:10.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45657fa640 1 -- 172.21.15.120:0/104797812 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45580041c0 con 0x7f4568103ca0 2024-09-10T23:05:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f45657fa640 1 -- 172.21.15.120:0/104797812 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558019670 con 0x7f4568103ca0 2024-09-10T23:05:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456819f600 con 0x7f4568103ca0 2024-09-10T23:05:10.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.272+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4568101a00 con 0x7f4568103ca0 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f45657fa640 1 -- 172.21.15.120:0/104797812 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f45580035f0 con 0x7f4568103ca0 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 Processor -- start 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 -- start start 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea10622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bea0fd8e60 con 0x55bea106e790 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bea0fc8290 con 0x55bea1061ee0 2024-09-10T23:05:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.275+0000 7f456ffe3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bea0fcc0a0 con 0x55bea1062800 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34214/0 (socket says 172.21.15.120:34214) 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 -- 172.21.15.120:0/1829090551 learned_addr learned my addr 172.21.15.120:0/1829090551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f45677fe640 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f4567fff640 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea10622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f45657fa640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 0x7f45540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 -- 172.21.15.120:0/1829090551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 msgr2=0x55bea106e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 -- 172.21.15.120:0/1829090551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 msgr2=0x55bea10622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea10622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 -- 172.21.15.120:0/1829090551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea0d26930 con 0x55bea106e790 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f4567fff640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 0x7f45540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455ffff640 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4550011570 tx=0x7f4550011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a658453779a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f45657fa640 1 -- 172.21.15.120:0/104797812 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f45580e9410 con 0x7f4568103ca0 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.276+0000 7f455dffb640 1 -- 172.21.15.120:0/1829090551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550004030 con 0x55bea106e790 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.277+0000 7f455dffb640 1 -- 172.21.15.120:0/1829090551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45500054d0 con 0x55bea106e790 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.277+0000 7f455dffb640 1 -- 172.21.15.120:0/1829090551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550004cc0 con 0x55bea106e790 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f4567fff640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 0x7f45540b7820 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4560004af0 tx=0x7f45600095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- 172.21.15.120:0/1829090551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 msgr2=0x55bea1070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4550011570 tx=0x7f4550011a30 comp rx=0 tx=0).stop 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- 172.21.15.120:0/1829090551 shutdown_connections 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea1070b80 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 --2- 172.21.15.120:0/1829090551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea10622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- 172.21.15.120:0/1829090551 >> 172.21.15.120:0/1829090551 conn(0x55bea105fd10 msgr2=0x55bea1060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- 172.21.15.120:0/1829090551 shutdown_connections 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- 172.21.15.120:0/1829090551 wait complete. 2024-09-10T23:05:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.278+0000 7f456ffe3740 1 -- start start 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 Processor -- start 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea106b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bea0e29890 con 0x55bea106e790 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bea0f62d40 con 0x55bea1061ee0 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bea1071440 con 0x55bea1062800 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f456ffe3740 10 client.0 _ll_register_callbacks cb 0x55bea0fc6c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f4567fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f4567fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34228/0 (socket says 172.21.15.120:34228) 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45590/0 (socket says 172.21.15.120:45590) 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f4567fff640 1 -- 172.21.15.120:0/2369966718 learned_addr learned my addr 172.21.15.120:0/2369966718 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f45677fe640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea106b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 -- 172.21.15.120:0/2369966718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 msgr2=0x55bea106b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea106b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 -- 172.21.15.120:0/2369966718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 msgr2=0x55bea106c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f455ffff640 1 -- 172.21.15.120:0/2369966718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea107a820 con 0x55bea1062800 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.279+0000 7f4567fff640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f455ffff640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4550011540 tx=0x7f455000fda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4749d298621bf7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550027020 con 0x55bea1062800 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4550004bd0 con 0x55bea1062800 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45500054d0 con 0x55bea1062800 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4550009d00 con 0x55bea1062800 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 10 client.27430 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bea107d1d0 con 0x55bea1062800 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45500054d0 con 0x55bea1062800 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f45500236b0 con 0x55bea1062800 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 10 client.27430 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 10 client.27430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.280+0000 7f456ffe3740 20 client.27430 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea107e580 0x55bea1080970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f456ffe3740 4 client.27430 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea0fdadb0 con 0x55bea107e580 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f45677fe640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea107e580 0x55bea1080970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f4546ffd640 10 client.27430 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.281+0000 7f45677fe640 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea107e580 0x55bea1080970 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c3989bd7a3b11c server_cookie=25224ec14a38302d in_seq=0 out_seq=0 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.285+0000 7f4546ffd640 1 -- 172.21.15.120:0/2369966718 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bea0fdadb0 con 0x55bea107e580 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.285+0000 7f4546ffd640 10 client.27430 handle_command_reply: tid=0 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 client.27430 shutdown 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 msgr2=0x55bea106bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4550011540 tx=0x7f455000fda0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 shutdown_connections 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea107e580 0x55bea1080970 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea106e790 0x55bea106c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1062800 0x55bea106bc70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f4546ffd640 0 client.27430 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2369966718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea1061ee0 0x55bea106b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 >> 172.21.15.120:0/2369966718 conn(0x55bea105fd10 msgr2=0x55bea105ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 shutdown_connections 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 1 -- 172.21.15.120:0/2369966718 wait complete. 2024-09-10T23:05:10.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.286+0000 7f456ffe3740 20 client.27430 trim_cache size 0 max 16384 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 Processor -- start 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 -- start start 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1061470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1086690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45681aa150 con 0x55bea1062800 2024-09-10T23:05:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45681ac9d0 con 0x55bea105fd10 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f456ffe3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f45681aa7c0 con 0x55bea1061ee0 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f455ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f4567fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1086690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f455ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45596/0 (socket says 172.21.15.120:45596) 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.304+0000 7f455ffff640 1 -- 172.21.15.120:0/2746802292 learned_addr learned my addr 172.21.15.120:0/2746802292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f45677fe640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1061470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 -- 172.21.15.120:0/2746802292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 msgr2=0x55bea1061470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1061470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 -- 172.21.15.120:0/2746802292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 msgr2=0x55bea1086690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1086690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 -- 172.21.15.120:0/2746802292 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea1069c30 con 0x55bea1061ee0 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f4567fff640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1086690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f45677fe640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1061470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455ffff640 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45500058c0 tx=0x7f4550011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663ba3ad9200c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455d7fa640 1 -- 172.21.15.120:0/2746802292 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550027070 con 0x55bea1061ee0 2024-09-10T23:05:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455d7fa640 1 -- 172.21.15.120:0/2746802292 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4550004040 con 0x55bea1061ee0 2024-09-10T23:05:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.305+0000 7f455d7fa640 1 -- 172.21.15.120:0/2746802292 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550003850 con 0x55bea1061ee0 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 -- 172.21.15.120:0/2746802292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 msgr2=0x55bea1086150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45500058c0 tx=0x7f4550011440 comp rx=0 tx=0).stop 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 -- 172.21.15.120:0/2746802292 shutdown_connections 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1086690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1086150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 --2- 172.21.15.120:0/2746802292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1061470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 -- 172.21.15.120:0/2746802292 >> 172.21.15.120:0/2746802292 conn(0x55bea10670d0 msgr2=0x55bea1060d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 -- 172.21.15.120:0/2746802292 shutdown_connections 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.306+0000 7f456ffe3740 1 -- 172.21.15.120:0/2746802292 wait complete. 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 -- start start 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 Processor -- start 2024-09-10T23:05:10.311 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1077070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45681aae90 con 0x55bea1062800 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f45681ac300 con 0x55bea105fd10 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f45681ab560 con 0x55bea1061ee0 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f456ffe3740 10 client.0 _ll_register_callbacks cb 0x7f4558005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f4567fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f455ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f4567fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34248/0 (socket says 172.21.15.120:34248) 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.307+0000 7f4567fff640 1 -- 172.21.15.120:0/3309395129 learned_addr learned my addr 172.21.15.120:0/3309395129 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f45677fe640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1077070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 -- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 msgr2=0x55bea1072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 -- 172.21.15.120:0/3309395129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 msgr2=0x55bea1077070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:10.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1077070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 -- 172.21.15.120:0/3309395129 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea10ea350 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4567fff640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1ca4cb3e9c8542 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560059060 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4560004690 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4550004040 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4560031e80 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f456ffe3740 10 client.17964 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.308+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bea10eb5b0 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4550004040 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4560053030 con 0x55bea1062800 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 10 client.17964 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 20 client.17964 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:10.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea106e790 0x55bea10edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:10.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 4 client.17964 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:05:10.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bea0fd6860 con 0x55bea106e790 2024-09-10T23:05:10.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f45677fe640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea106e790 0x55bea10edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:10.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f45677fe640 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea106e790 0x55bea10edd80 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64facbf406de805b server_cookie=d0a352b3628b28e3 in_seq=0 out_seq=0 2024-09-10T23:05:10.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.309+0000 7f4546ffd640 10 client.17964 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:10.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f4546ffd640 1 -- 172.21.15.120:0/3309395129 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bea0fd6860 con 0x55bea106e790 2024-09-10T23:05:10.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f4546ffd640 10 client.17964 handle_command_reply: tid=0 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 client.17964 shutdown 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 msgr2=0x55bea1072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 shutdown_connections 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bea106e790 0x55bea10edd80 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bea1062800 0x55bea1072cd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bea1061ee0 0x55bea1072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 --2- 172.21.15.120:0/3309395129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bea105fd10 0x55bea1077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 >> 172.21.15.120:0/3309395129 conn(0x55bea10670d0 msgr2=0x55bea106cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.314+0000 7f4546ffd640 0 client.17964 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 shutdown_connections 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456ffe3740 1 -- 172.21.15.120:0/3309395129 wait complete. 2024-09-10T23:05:10.316 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456ffe3740 20 client.17964 trim_cache size 0 max 16384 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "93126434-a545-4e80-8433-4566deba8ef8", 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:10.317 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 msgr2=0x7f45540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 0x7f45540b7820 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4560004af0 tx=0x7f45600095c0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 msgr2=0x7f4568199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4558009ce0 tx=0x7f455800b690 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 shutdown_connections 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f45540b5360 0x7f45540b7820 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f456810d100 0x7f456819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45681045c0 0x7f45681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 --2- 172.21.15.120:0/104797812 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4568103ca0 0x7f4568199260 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 >> 172.21.15.120:0/104797812 conn(0x7f45680fd700 msgr2=0x7f45680fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 shutdown_connections 2024-09-10T23:05:10.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:10.315+0000 7f456dc0e640 1 -- 172.21.15.120:0/104797812 wait complete. 2024-09-10T23:05:10.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:10 smithi176 ceph-mon[34982]: pgmap v728: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-09-10T23:05:10.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:10 smithi153 ceph-mon[34951]: pgmap v728: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-09-10T23:05:10.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 2828 64.88 MB/sec warmup 5 sec latency 86.841 ms 2024-09-10T23:05:11.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:05:11.404 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:11.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 2897 54.66 MB/sec warmup 6 sec latency 287.436 ms 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 -- 172.21.15.120:0/1992822798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6010d0e0 msgr2=0x7f3d6010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 --2- 172.21.15.120:0/1992822798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6010d0e0 0x7f3d6010f550 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009900 tx=0x7f3d5002a410 comp rx=0 tx=0).stop 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 -- 172.21.15.120:0/1992822798 shutdown_connections 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 --2- 172.21.15.120:0/1992822798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d6010d0e0 0x7f3d6010f550 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 --2- 172.21.15.120:0/1992822798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60104530 0x7f3d6010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 --2- 172.21.15.120:0/1992822798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d60103b30 0x7f3d60103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 -- 172.21.15.120:0/1992822798 >> 172.21.15.120:0/1992822798 conn(0x7f3d600fd700 msgr2=0x7f3d600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 -- 172.21.15.120:0/1992822798 shutdown_connections 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.923+0000 7f3d648f5640 1 -- 172.21.15.120:0/1992822798 wait complete. 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 Processor -- start 2024-09-10T23:05:11.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 -- start start 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 0x7f3d60199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6010d0e0 0x7f3d6019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112d60 con 0x7f3d60104530 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112be0 con 0x7f3d6010d0e0 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d648f5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112ee0 con 0x7f3d60103b30 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d5dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d5dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34276/0 (socket says 172.21.15.120:34276) 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 0x7f3d60199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d5ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6010d0e0 0x7f3d6019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.924+0000 7f3d5dd9b640 1 -- 172.21.15.120:0/2300299371 learned_addr learned my addr 172.21.15.120:0/2300299371 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 -- 172.21.15.120:0/2300299371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 msgr2=0x7f3d60199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 0x7f3d60199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 -- 172.21.15.120:0/2300299371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6010d0e0 msgr2=0x7f3d6019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6010d0e0 0x7f3d6019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 -- 172.21.15.120:0/2300299371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6019f2c0 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 0x7f3d60199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800e9f0 tx=0x7f3d4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d577fe640 1 -- 172.21.15.120:0/2300299371 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800cd80 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6019f5b0 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d577fe640 1 -- 172.21.15.120:0/2300299371 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d480054d0 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.925+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a5c70 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.926+0000 7f3d577fe640 1 -- 172.21.15.120:0/2300299371 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48010690 con 0x7f3d60104530 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 Processor -- start 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 -- start start 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cce923ce50 con 0x55cce92c69c0 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cce922c280 con 0x55cce92d2950 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d66cca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cce9230090 con 0x55cce92c60a0 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d577fe640 1 -- 172.21.15.120:0/2300299371 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3d48010830 con 0x7f3d60104530 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.928+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34278/0 (socket says 172.21.15.120:34278) 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1754782382 learned_addr learned my addr 172.21.15.120:0/1754782382 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1754782382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 msgr2=0x55cce92c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1754782382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 msgr2=0x55cce92d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1754782382 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce8f8a650 con 0x55cce92c69c0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3d580108c0 tx=0x7f3d58010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92ea445c58dc751 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d54ff9640 1 -- 172.21.15.120:0/1754782382 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58004d20 con 0x55cce92c69c0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d54ff9640 1 -- 172.21.15.120:0/1754782382 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x55cce92c69c0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d54ff9640 1 -- 172.21.15.120:0/1754782382 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58005830 con 0x55cce92c69c0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.929+0000 7f3d577fe640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 0x7f3d4c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 0x7f3d4c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d577fe640 1 -- 172.21.15.120:0/2300299371 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d480e5ce0 con 0x7f3d60104530 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 -- 172.21.15.120:0/1754782382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 msgr2=0x55cce92d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3d580108c0 tx=0x7f3d58010d80 comp rx=0 tx=0).stop 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 0x7f3d4c0b7820 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 -- 172.21.15.120:0/1754782382 shutdown_connections 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c69c0 0x55cce92d2410 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1754782382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 -- 172.21.15.120:0/1754782382 >> 172.21.15.120:0/1754782382 conn(0x55cce92c3ed0 msgr2=0x55cce92c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 -- 172.21.15.120:0/1754782382 shutdown_connections 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.930+0000 7f3d66cca740 1 -- 172.21.15.120:0/1754782382 wait complete. 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 -- start start 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 Processor -- start 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cce908d700 con 0x55cce92d2950 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cce91c7110 con 0x55cce92c60a0 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cce92d99e0 con 0x55cce92c69c0 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d66cca740 10 client.0 _ll_register_callbacks cb 0x55cce91e0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:38136/0 (socket says 172.21.15.120:38136) 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1764414091 learned_addr learned my addr 172.21.15.120:0/1764414091 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.931+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 msgr2=0x55cce92cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1764414091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 msgr2=0x55cce92cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1764414091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce91f4c40 con 0x55cce92c60a0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009a00 tx=0x7f3d58009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1b33b4755407d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58029060 con 0x55cce92c60a0 2024-09-10T23:05:11.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d58005770 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d58015070 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d5800ee60 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d66cca740 10 client.27266 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cce92e5360 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.932+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3d58022030 con 0x55cce92c60a0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 10 client.27266 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 10 client.27266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 20 client.27266 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce92e66b0 0x55cce92e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 4 client.27266 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cce923eda0 con 0x55cce92e66b0 2024-09-10T23:05:11.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce92e66b0 0x55cce92e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d45ffb640 10 client.27266 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.933+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce92e66b0 0x55cce92e8aa0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351b7b3a6a01482f server_cookie=398a32c559ee7be9 in_seq=0 out_seq=0 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1764414091 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cce923eda0 con 0x55cce92e66b0 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d45ffb640 10 client.27266 handle_command_reply: tid=0 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 client.27266 shutdown 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 msgr2=0x55cce92d21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009a00 tx=0x7f3d58009ec0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 shutdown_connections 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce92e66b0 0x55cce92e8aa0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d45ffb640 0 client.27266 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1764414091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c60a0 0x55cce92d21f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 >> 172.21.15.120:0/1764414091 conn(0x55cce92c3ed0 msgr2=0x55cce92c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 shutdown_connections 2024-09-10T23:05:11.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 1 -- 172.21.15.120:0/1764414091 wait complete. 2024-09-10T23:05:11.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.936+0000 7f3d66cca740 20 client.27266 trim_cache size 0 max 16384 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 Processor -- start 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 -- start start 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000c380 con 0x55cce92c60a0 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000ec00 con 0x55cce92d2950 2024-09-10T23:05:11.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d66cca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000c9f0 con 0x55cce92c69c0 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34288/0 (socket says 172.21.15.120:34288) 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d5e59c640 1 -- 172.21.15.120:0/294285078 learned_addr learned my addr 172.21.15.120:0/294285078 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.962+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 -- 172.21.15.120:0/294285078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 msgr2=0x55cce92cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 -- 172.21.15.120:0/294285078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 msgr2=0x55cce92ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 -- 172.21.15.120:0/294285078 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce92d2560 con 0x55cce92c60a0 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3d58000f80 tx=0x7f3d58009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7978080808d02f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d47fff640 1 -- 172.21.15.120:0/294285078 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58003970 con 0x55cce92c60a0 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d47fff640 1 -- 172.21.15.120:0/294285078 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x55cce92c60a0 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.963+0000 7f3d47fff640 1 -- 172.21.15.120:0/294285078 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580057c0 con 0x55cce92c60a0 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- 172.21.15.120:0/294285078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 msgr2=0x55cce92d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3d58000f80 tx=0x7f3d58009520 comp rx=0 tx=0).stop 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- 172.21.15.120:0/294285078 shutdown_connections 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92d2950 0x55cce92ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c69c0 0x55cce92cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 --2- 172.21.15.120:0/294285078 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92c60a0 0x55cce92d1380 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- 172.21.15.120:0/294285078 >> 172.21.15.120:0/294285078 conn(0x55cce92c3ed0 msgr2=0x55cce92d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- 172.21.15.120:0/294285078 shutdown_connections 2024-09-10T23:05:11.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- 172.21.15.120:0/294285078 wait complete. 2024-09-10T23:05:11.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.964+0000 7f3d66cca740 1 -- start start 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 Processor -- start 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c69c0 0x55cce92db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000d0c0 con 0x55cce92d2950 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000e530 con 0x55cce92c69c0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4000d790 con 0x55cce92c60a0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d66cca740 10 client.0 _ll_register_callbacks cb 0x7f3d480e9690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45642/0 (socket says 172.21.15.120:45642) 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1386734464 learned_addr learned my addr 172.21.15.120:0/1386734464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c69c0 0x55cce92db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1386734464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c69c0 msgr2=0x55cce92db6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c69c0 0x55cce92db6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1386734464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 msgr2=0x55cce92dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 -- 172.21.15.120:0/1386734464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce92dc1e0 con 0x55cce92c60a0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5ed9d640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d5e59c640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800a010 tx=0x7f3d58005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62fdd27f0017837f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.965+0000 7f3d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58003b70 con 0x55cce92c60a0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d58003d10 con 0x55cce92c60a0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d58015070 con 0x55cce92c60a0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d58049070 con 0x55cce92c60a0 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 10 client.27454 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:11.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cce93540f0 con 0x55cce92c60a0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x55cce92c60a0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3d58022050 con 0x55cce92c60a0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 10 client.27454 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 10 client.27454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 20 client.27454 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce9354730 0x55cce9356b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 4 client.27454 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.966+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cce923a850 con 0x55cce9354730 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.967+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce9354730 0x55cce9356b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.967+0000 7f3d5dd9b640 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce9354730 0x55cce9356b50 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b13c6c7d270dfc6 server_cookie=87a21f9b2b0bca79 in_seq=0 out_seq=0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.967+0000 7f3d45ffb640 10 client.27454 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.968+0000 7f3d45ffb640 1 -- 172.21.15.120:0/1386734464 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x55cce923a850 con 0x55cce9354730 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.968+0000 7f3d45ffb640 10 client.27454 handle_command_reply: tid=0 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.968+0000 7f3d66cca740 1 client.27454 shutdown 2024-09-10T23:05:11.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 msgr2=0x55cce92df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800a010 tx=0x7f3d58005a80 comp rx=0 tx=0).stop 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 shutdown_connections 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cce9354730 0x55cce9356b50 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cce92d2950 0x55cce92dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cce92c69c0 0x55cce92db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 --2- 172.21.15.120:0/1386734464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cce92c60a0 0x55cce92df200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d45ffb640 0 client.27454 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 >> 172.21.15.120:0/1386734464 conn(0x55cce92c3ed0 msgr2=0x55cce92c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 shutdown_connections 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 1 -- 172.21.15.120:0/1386734464 wait complete. 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d66cca740 20 client.27454 trim_cache size 0 max 16384 2024-09-10T23:05:11.974 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "93126434-a545-4e80-8433-4566deba8ef8", 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.975 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:05:11.976 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:05:11.977 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.978 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.979 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.980 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:05:11.981 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.982 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.983 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:11.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 msgr2=0x7f3d4c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 0x7f3d4c0b7820 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d500058d0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 msgr2=0x7f3d60199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800e9f0 tx=0x7f3d4800edf0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 shutdown_connections 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d4c0b5360 0x7f3d4c0b7820 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6010d0e0 0x7f3d6019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d60104530 0x7f3d60199750 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 --2- 172.21.15.120:0/2300299371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d60103b30 0x7f3d60199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.969+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 >> 172.21.15.120:0/2300299371 conn(0x7f3d600fd700 msgr2=0x7f3d600fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.970+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 shutdown_connections 2024-09-10T23:05:11.989 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:11.970+0000 7f3d648f5640 1 -- 172.21.15.120:0/2300299371 wait complete. 2024-09-10T23:05:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:12 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:05:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:12 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:12 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:05:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:12 smithi120 ceph-mon[31247]: pgmap v729: 97 pgs: 97 active+clean; 38 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-10T23:05:12.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:12 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:05:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:12 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:12 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:05:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:12 smithi176 ceph-mon[34982]: pgmap v729: 97 pgs: 97 active+clean; 38 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-10T23:05:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:12 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:05:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:12 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:12 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:05:12.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:12 smithi153 ceph-mon[34951]: pgmap v729: 97 pgs: 97 active+clean; 38 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-10T23:05:12.704 INFO:tasks.cephfs.filesystem:scrub status for tag:93126434-a545-4e80-8433-4566deba8ef8 - {'path': '/', 'tag': '93126434-a545-4e80-8433-4566deba8ef8', 'options': 'recursive,force'} 2024-09-10T23:05:12.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 3488 53.58 MB/sec warmup 7 sec latency 256.420 ms 2024-09-10T23:05:13.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:13 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:13.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:13 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:13.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:13 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:13.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 3802 49.71 MB/sec warmup 8 sec latency 128.669 ms 2024-09-10T23:05:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:14 smithi120 ceph-mon[31247]: pgmap v730: 97 pgs: 97 active+clean; 39 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 431 op/s 2024-09-10T23:05:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:14 smithi153 ceph-mon[34951]: pgmap v730: 97 pgs: 97 active+clean; 39 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 431 op/s 2024-09-10T23:05:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:14 smithi176 ceph-mon[34982]: pgmap v730: 97 pgs: 97 active+clean; 39 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 431 op/s 2024-09-10T23:05:14.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 3978 44.52 MB/sec warmup 9 sec latency 62.422 ms 2024-09-10T23:05:15.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 4199 40.22 MB/sec warmup 10 sec latency 140.934 ms 2024-09-10T23:05:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:16 smithi120 ceph-mon[31247]: pgmap v731: 97 pgs: 97 active+clean; 41 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 410 op/s 2024-09-10T23:05:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:16 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:05:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:16 smithi153 ceph-mon[34951]: pgmap v731: 97 pgs: 97 active+clean; 41 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 410 op/s 2024-09-10T23:05:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:16 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:05:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:16 smithi176 ceph-mon[34982]: pgmap v731: 97 pgs: 97 active+clean; 41 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 410 op/s 2024-09-10T23:05:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:16 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:05:16.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 4600 37.62 MB/sec warmup 11 sec latency 76.514 ms 2024-09-10T23:05:17.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 5290 37.83 MB/sec warmup 12 sec latency 61.253 ms 2024-09-10T23:05:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:18 smithi120 ceph-mon[31247]: pgmap v732: 97 pgs: 97 active+clean; 58 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 533 op/s 2024-09-10T23:05:18.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:18 smithi153 ceph-mon[34951]: pgmap v732: 97 pgs: 97 active+clean; 58 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 533 op/s 2024-09-10T23:05:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:18 smithi176 ceph-mon[34982]: pgmap v732: 97 pgs: 97 active+clean; 58 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 533 op/s 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.686+0000 7fb0177a3640 1 -- 172.21.15.120:0/2184379040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0101029b0 msgr2=0x7fb010102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.686+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2184379040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0101029b0 0x7fb010102e30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb000009930 tx=0x7fb00002a420 comp rx=0 tx=0).stop 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 -- 172.21.15.120:0/2184379040 shutdown_connections 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2184379040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb010109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2184379040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0101029b0 0x7fb010102e30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2184379040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb010101860 0x7fb010101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 -- 172.21.15.120:0/2184379040 >> 172.21.15.120:0/2184379040 conn(0x7fb0100fd140 msgr2=0x7fb0100ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 -- 172.21.15.120:0/2184379040 shutdown_connections 2024-09-10T23:05:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 -- 172.21.15.120:0/2184379040 wait complete. 2024-09-10T23:05:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.687+0000 7fb0177a3640 1 Processor -- start 2024-09-10T23:05:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 -- start start 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0101029b0 0x7fb01019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb0101a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb01010d4b0 con 0x7fb010101860 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb01010d330 con 0x7fb010103370 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0177a3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb01010d630 con 0x7fb0101029b0 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55974/0 (socket says 172.21.15.120:55974) 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 -- 172.21.15.120:0/2155095568 learned_addr learned my addr 172.21.15.120:0/2155095568 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb015d40640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb0101a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb014d3e640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0101029b0 0x7fb01019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 -- 172.21.15.120:0/2155095568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0101029b0 msgr2=0x7fb01019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0101029b0 0x7fb01019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 -- 172.21.15.120:0/2155095568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 msgr2=0x7fb0101a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb0101a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 -- 172.21.15.120:0/2155095568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0101a0990 con 0x7fb010101860 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb015d40640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb0101a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb01553f640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.688+0000 7fb0067fc640 1 -- 172.21.15.120:0/2155095568 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00cd80 con 0x7fb010101860 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.689+0000 7fb0067fc640 1 -- 172.21.15.120:0/2155095568 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c0108e0 con 0x7fb010101860 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.689+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101a0c20 con 0x7fb010101860 2024-09-10T23:05:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.689+0000 7fb0067fc640 1 -- 172.21.15.120:0/2155095568 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c018980 con 0x7fb010101860 2024-09-10T23:05:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.689+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a11c0 con 0x7fb010101860 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb0067fc640 1 -- 172.21.15.120:0/2155095568 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb00c010430 con 0x7fb010101860 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 Processor -- start 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 -- start start 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c8fc70 0x55ab32c92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c925a0 0x55ab32c94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab32bfd6f0 con 0x55ab32c94fe0 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab32becb20 con 0x55ab32c8fc70 2024-09-10T23:05:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.691+0000 7fb01a64b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab32bf0930 con 0x55ab32c925a0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb01553f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c925a0 0x55ab32c94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55984/0 (socket says 172.21.15.120:55984) 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 -- 172.21.15.120:0/2045038655 learned_addr learned my addr 172.21.15.120:0/2045038655 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb015d40640 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c8fc70 0x55ab32c92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 -- 172.21.15.120:0/2045038655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c925a0 msgr2=0x55ab32c94990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c925a0 0x55ab32c94990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 -- 172.21.15.120:0/2045038655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c8fc70 msgr2=0x55ab32c92060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c8fc70 0x55ab32c92060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 -- 172.21.15.120:0/2045038655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab32bb2530 con 0x55ab32c94fe0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb0067fc640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 0x7faffc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb014d3e640 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb00002a940 tx=0x7fb000005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99974fa553811f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb015d40640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 0x7faffc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7fb0067fc640 1 -- 172.21.15.120:0/2155095568 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb00c0e5ef0 con 0x7fb010101860 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7faffb7fe640 1 -- 172.21.15.120:0/2045038655 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000035030 con 0x55ab32c94fe0 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7faffb7fe640 1 -- 172.21.15.120:0/2045038655 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000040070 con 0x55ab32c94fe0 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.692+0000 7faffb7fe640 1 -- 172.21.15.120:0/2045038655 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000003970 con 0x55ab32c94fe0 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 -- 172.21.15.120:0/2045038655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 msgr2=0x55ab32c974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb015d40640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 0x7faffc0b7820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb008009fd0 tx=0x7fb008009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb00002a940 tx=0x7fb000005de0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 -- 172.21.15.120:0/2045038655 shutdown_connections 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c94fe0 0x55ab32c974a0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c925a0 0x55ab32c94990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 --2- 172.21.15.120:0/2045038655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c8fc70 0x55ab32c92060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 -- 172.21.15.120:0/2045038655 >> 172.21.15.120:0/2045038655 conn(0x55ab32c8adc0 msgr2=0x55ab32c8bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 -- 172.21.15.120:0/2045038655 shutdown_connections 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.693+0000 7fb01a64b740 1 -- 172.21.15.120:0/2045038655 wait complete. 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 -- start start 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 Processor -- start 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ab32b24ab0 con 0x55ab32c925a0 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab32b92730 con 0x55ab32c94fe0 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ab32c97f60 con 0x55ab32c8fc70 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01a64b740 10 client.0 _ll_register_callbacks cb 0x55ab32be66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56000/0 (socket says 172.21.15.120:56000) 2024-09-10T23:05:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb014d3e640 1 -- 172.21.15.120:0/599020091 learned_addr learned my addr 172.21.15.120:0/599020091 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb015d40640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.694+0000 7fb01553f640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 -- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 msgr2=0x55ab32c99300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 -- 172.21.15.120:0/599020091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 msgr2=0x55ab32c9a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 -- 172.21.15.120:0/599020091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3294e790 con 0x55ab32c925a0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb015d40640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb014d3e640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb0000092a0 tx=0x7fb00002aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a3790dd4358b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000053060 con 0x55ab32c925a0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0000315a0 con 0x55ab32c925a0 2024-09-10T23:05:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb000040070 con 0x55ab32c925a0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb00004a070 con 0x55ab32c925a0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 10 client.17994 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab329c7620 con 0x55ab32c925a0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000040070 con 0x55ab32c925a0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb000045500 con 0x55ab32c925a0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 10 client.17994 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 20 client.17994 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32ca4730 0x55ab32ca6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 4 client.17994 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.695+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab32bff640 con 0x55ab32ca4730 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.696+0000 7fb01553f640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32ca4730 0x55ab32ca6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.696+0000 7faff8ff9640 10 client.17994 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.696+0000 7fb01553f640 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32ca4730 0x55ab32ca6b20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb26409dcc22a6e5 server_cookie=567504005ee0c8ed in_seq=0 out_seq=0 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7faff8ff9640 1 -- 172.21.15.120:0/599020091 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab32bff640 con 0x55ab32ca4730 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7faff8ff9640 10 client.17994 handle_command_reply: tid=0 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 client.17994 shutdown 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 msgr2=0x55ab32c999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb0000092a0 tx=0x7fb00002aae0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 shutdown_connections 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32ca4730 0x55ab32ca6b20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c999a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7faff8ff9640 0 client.17994 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 --2- 172.21.15.120:0/599020091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 >> 172.21.15.120:0/599020091 conn(0x55ab32c8adc0 msgr2=0x55ab32c85bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.697+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 shutdown_connections 2024-09-10T23:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.698+0000 7fb01a64b740 1 -- 172.21.15.120:0/599020091 wait complete. 2024-09-10T23:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.698+0000 7fb01a64b740 20 client.17994 trim_cache size 0 max 16384 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 Processor -- start 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 -- start start 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c925a0 0x55ab32c99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c94fe0 0x55ab32c98540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a5bb0 con 0x55ab32c8fc70 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a8430 con 0x55ab32c925a0 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01a64b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a6220 con 0x55ab32c94fe0 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56008/0 (socket says 172.21.15.120:56008) 2024-09-10T23:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 -- 172.21.15.120:0/1276758832 learned_addr learned my addr 172.21.15.120:0/1276758832 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb014d3e640 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c925a0 0x55ab32c99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb015d40640 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c94fe0 0x55ab32c98540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 -- 172.21.15.120:0/1276758832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c94fe0 msgr2=0x55ab32c98540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c94fe0 0x55ab32c98540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 -- 172.21.15.120:0/1276758832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c925a0 msgr2=0x55ab32c99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c925a0 0x55ab32c99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 -- 172.21.15.120:0/1276758832 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab32c9a870 con 0x55ab32c8fc70 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.713+0000 7fb01553f640 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c6a0 tx=0x7fb00c0e9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ffdac80aed50cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7faffaffd640 1 -- 172.21.15.120:0/1276758832 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0b5e60 con 0x55ab32c8fc70 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7faffaffd640 1 -- 172.21.15.120:0/1276758832 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c0ac240 con 0x55ab32c8fc70 2024-09-10T23:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7faffaffd640 1 -- 172.21.15.120:0/1276758832 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0b5c10 con 0x55ab32c8fc70 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 -- 172.21.15.120:0/1276758832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 msgr2=0x55ab32c99300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c6a0 tx=0x7fb00c0e9930 comp rx=0 tx=0).stop 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 -- 172.21.15.120:0/1276758832 shutdown_connections 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c94fe0 0x55ab32c98540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c925a0 0x55ab32c99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 --2- 172.21.15.120:0/1276758832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c8fc70 0x55ab32c99300 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.714+0000 7fb01a64b740 1 -- 172.21.15.120:0/1276758832 >> 172.21.15.120:0/1276758832 conn(0x55ab32c8adc0 msgr2=0x55ab32c8f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- 172.21.15.120:0/1276758832 shutdown_connections 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- 172.21.15.120:0/1276758832 wait complete. 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- start start 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 Processor -- start 2024-09-10T23:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32ca0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a68f0 con 0x55ab32c925a0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a7d60 con 0x55ab32c94fe0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101a6fc0 con 0x55ab32c8fc70 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01a64b740 10 client.0 _ll_register_callbacks cb 0x7fb008009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb015d40640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb014d3e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56010/0 (socket says 172.21.15.120:56010) 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb01553f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32ca0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.715+0000 7fb014d3e640 1 -- 172.21.15.120:0/134563652 learned_addr learned my addr 172.21.15.120:0/134563652 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 -- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 msgr2=0x55ab32ca0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32ca0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 -- 172.21.15.120:0/134563652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 msgr2=0x55ab32c9ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 -- 172.21.15.120:0/134563652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab32c923d0 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb01553f640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32ca0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb015d40640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb014d3e640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb0000314e0 tx=0x7fb000004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f291eacfc95a1b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7faff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00005e070 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0000315a0 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb00c0ac240 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb00006a050 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb01a64b740 10 client.18000 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.716+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab32d11610 con 0x55ab32c925a0 2024-09-10T23:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c0ac240 con 0x55ab32c925a0 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb000003ca0 con 0x55ab32c925a0 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 10 client.18000 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 20 client.18000 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32d11de0 0x55ab32d14200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 4 client.18000 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab32bfb0f0 con 0x55ab32d11de0 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01553f640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32d11de0 0x55ab32d14200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7fb01553f640 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32d11de0 0x55ab32d14200 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bb0f6196f3aa60 server_cookie=82128255920dbe2e in_seq=0 out_seq=0 2024-09-10T23:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:18.717+0000 7faff8ff9640 10 client.18000 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:18.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 5962 36.28 MB/sec warmup 13 sec latency 132.360 ms 2024-09-10T23:05:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7faff8ff9640 1 -- 172.21.15.120:0/134563652 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab32bfb0f0 con 0x55ab32d11de0 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7faff8ff9640 10 client.18000 handle_command_reply: tid=0 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 client.18000 shutdown 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 msgr2=0x55ab32c9fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb0000314e0 tx=0x7fb000004480 comp rx=0 tx=0).stop 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 shutdown_connections 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ab32d11de0 0x55ab32d14200 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab32c94fe0 0x55ab32c9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ab32c925a0 0x55ab32c9fa50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 --2- 172.21.15.120:0/134563652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab32c8fc70 0x55ab32ca0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7faff8ff9640 0 client.18000 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 >> 172.21.15.120:0/134563652 conn(0x55ab32c8adc0 msgr2=0x55ab32c98540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 shutdown_connections 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.450+0000 7fb01a64b740 1 -- 172.21.15.120:0/134563652 wait complete. 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb01a64b740 20 client.18000 trim_cache size 0 max 16384 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:05:19.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:05:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:05:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:05:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 msgr2=0x7faffc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 0x7faffc0b7820 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb008009fd0 tx=0x7fb008009500 comp rx=0 tx=0).stop 2024-09-10T23:05:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 msgr2=0x7fb01006b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e9f0 tx=0x7fb00c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 shutdown_connections 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faffc0b5360 0x7faffc0b7820 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb010103370 0x7fb0101a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0101029b0 0x7fb01019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 --2- 172.21.15.120:0/2155095568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb010101860 0x7fb01006b1a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 >> 172.21.15.120:0/2155095568 conn(0x7fb0100fd140 msgr2=0x7fb010105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 shutdown_connections 2024-09-10T23:05:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:19.451+0000 7fb0177a3640 1 -- 172.21.15.120:0/2155095568 wait complete. 2024-09-10T23:05:19.864 INFO:tasks.workunit.client.0.smithi120.stdout: 10 6199 35.11 MB/sec warmup 14 sec latency 417.590 ms 2024-09-10T23:05:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:20 smithi120 ceph-mon[31247]: pgmap v733: 97 pgs: 97 active+clean; 62 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:05:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:20 smithi153 ceph-mon[34951]: pgmap v733: 97 pgs: 97 active+clean; 62 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:05:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:20 smithi176 ceph-mon[34982]: pgmap v733: 97 pgs: 97 active+clean; 62 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:05:20.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 6941 36.81 MB/sec warmup 15 sec latency 48.459 ms 2024-09-10T23:05:21.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 7353 36.76 MB/sec warmup 16 sec latency 67.013 ms 2024-09-10T23:05:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:22 smithi120 ceph-mon[31247]: pgmap v734: 97 pgs: 97 active+clean; 56 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-09-10T23:05:22.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:22 smithi153 ceph-mon[34951]: pgmap v734: 97 pgs: 97 active+clean; 56 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-09-10T23:05:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:22 smithi176 ceph-mon[34982]: pgmap v734: 97 pgs: 97 active+clean; 56 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-09-10T23:05:22.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 7560 34.82 MB/sec warmup 17 sec latency 57.121 ms 2024-09-10T23:05:23.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 7744 32.97 MB/sec warmup 18 sec latency 179.833 ms 2024-09-10T23:05:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:24 smithi120 ceph-mon[31247]: pgmap v735: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 692 op/s 2024-09-10T23:05:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:24 smithi153 ceph-mon[34951]: pgmap v735: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 692 op/s 2024-09-10T23:05:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:24 smithi176 ceph-mon[34982]: pgmap v735: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 692 op/s 2024-09-10T23:05:24.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 8120 31.89 MB/sec warmup 19 sec latency 121.794 ms 2024-09-10T23:05:25.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 8774 32.23 MB/sec warmup 20 sec latency 35.127 ms 2024-09-10T23:05:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:26 smithi120 ceph-mon[31247]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-09-10T23:05:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:26 smithi153 ceph-mon[34951]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-09-10T23:05:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:26 smithi176 ceph-mon[34982]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-09-10T23:05:26.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 9590 31.86 MB/sec warmup 21 sec latency 44.403 ms 2024-09-10T23:05:27.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 10360 33.41 MB/sec warmup 22 sec latency 35.301 ms 2024-09-10T23:05:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:28 smithi120 ceph-mon[31247]: pgmap v737: 97 pgs: 97 active+clean; 52 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 688 op/s 2024-09-10T23:05:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:28 smithi153 ceph-mon[34951]: pgmap v737: 97 pgs: 97 active+clean; 52 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 688 op/s 2024-09-10T23:05:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:28 smithi176 ceph-mon[34982]: pgmap v737: 97 pgs: 97 active+clean; 52 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 688 op/s 2024-09-10T23:05:28.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 10796 33.66 MB/sec warmup 23 sec latency 175.205 ms 2024-09-10T23:05:29.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 11000 32.58 MB/sec warmup 24 sec latency 85.055 ms 2024-09-10T23:05:30.374 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:30 smithi176 ceph-mon[34982]: pgmap v738: 97 pgs: 97 active+clean; 53 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 579 op/s 2024-09-10T23:05:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:30 smithi120 ceph-mon[31247]: pgmap v738: 97 pgs: 97 active+clean; 53 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 579 op/s 2024-09-10T23:05:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:30 smithi153 ceph-mon[34951]: pgmap v738: 97 pgs: 97 active+clean; 53 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 579 op/s 2024-09-10T23:05:30.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 11212 31.36 MB/sec warmup 25 sec latency 83.529 ms 2024-09-10T23:05:31.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 11557 30.51 MB/sec warmup 26 sec latency 92.657 ms 2024-09-10T23:05:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:32 smithi120 ceph-mon[31247]: pgmap v739: 97 pgs: 97 active+clean; 61 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 657 op/s 2024-09-10T23:05:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:32 smithi153 ceph-mon[34951]: pgmap v739: 97 pgs: 97 active+clean; 61 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 657 op/s 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- 172.21.15.120:0/4072896385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 msgr2=0x7f0f40101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 --2- 172.21.15.120:0/4072896385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f40101c70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009930 tx=0x7f0f3002a420 comp rx=0 tx=0).stop 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- 172.21.15.120:0/4072896385 shutdown_connections 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 --2- 172.21.15.120:0/4072896385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f40109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 --2- 172.21.15.120:0/4072896385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 0x7f0f40102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 --2- 172.21.15.120:0/4072896385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f40101c70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- 172.21.15.120:0/4072896385 >> 172.21.15.120:0/4072896385 conn(0x7f0f400fd130 msgr2=0x7f0f400ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- 172.21.15.120:0/4072896385 shutdown_connections 2024-09-10T23:05:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- 172.21.15.120:0/4072896385 wait complete. 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 Processor -- start 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.543+0000 7f0f449fb640 1 -- start start 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 0x7f0f4010a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f4010ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4010e5d0 con 0x7f0f40101870 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4010e450 con 0x7f0f401029c0 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f449fb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4010e750 con 0x7f0f40103380 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50638/0 (socket says 172.21.15.120:50638) 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 -- 172.21.15.120:0/499268065 learned_addr learned my addr 172.21.15.120:0/499268065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f4010ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 0x7f0f4010a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 -- 172.21.15.120:0/499268065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 msgr2=0x7f0f4010ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f4010ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 -- 172.21.15.120:0/499268065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 msgr2=0x7f0f4010a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 0x7f0f4010a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 -- 172.21.15.120:0/499268065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4010b700 con 0x7f0f40101870 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f4010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009a50 tx=0x7f0f300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.544+0000 7f0f2f7fe640 1 -- 172.21.15.120:0/499268065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3003f030 con 0x7f0f40101870 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.545+0000 7f0f2f7fe640 1 -- 172.21.15.120:0/499268065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f30031910 con 0x7f0f40101870 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.545+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f401a2e30 con 0x7f0f40101870 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.545+0000 7f0f2f7fe640 1 -- 172.21.15.120:0/499268065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30035070 con 0x7f0f40101870 2024-09-10T23:05:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.545+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f401a3230 con 0x7f0f40101870 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.547+0000 7f0f2f7fe640 1 -- 172.21.15.120:0/499268065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0f3003ba60 con 0x7f0f40101870 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.547+0000 7f0f477c8740 1 Processor -- start 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.547+0000 7f0f477c8740 1 -- start start 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.547+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.547+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f477c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9b8a6f0 con 0x55d2c9c1b190 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f477c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9b79b20 con 0x55d2c9c18860 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f477c8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9b7d930 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:53830/0 (socket says 172.21.15.120:53830) 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4163123790 learned_addr learned my addr 172.21.15.120:0/4163123790 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4163123790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 msgr2=0x55d2c9c20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4163123790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 msgr2=0x55d2c9c1d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4163123790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c9b3f530 con 0x55d2c9c18860 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f2f7fe640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 0x7f0f280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117e0 tx=0x7f0f38011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c657a4d42f802e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 0x7f0f280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.548+0000 7f0f2cff9640 1 -- 172.21.15.120:0/4163123790 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011310 con 0x55d2c9c18860 2024-09-10T23:05:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f2cff9640 1 -- 172.21.15.120:0/4163123790 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f38003e20 con 0x55d2c9c18860 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f2cff9640 1 -- 172.21.15.120:0/4163123790 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380044f0 con 0x55d2c9c18860 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f2f7fe640 1 -- 172.21.15.120:0/499268065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0f301063e0 con 0x7f0f40101870 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 -- 172.21.15.120:0/4163123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 msgr2=0x55d2c9c1ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f3e59c640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 0x7f0f280b77a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0f30004770 tx=0x7f0f30109ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117e0 tx=0x7f0f38011ca0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 -- 172.21.15.120:0/4163123790 shutdown_connections 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4163123790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c1ac50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.549+0000 7f0f477c8740 1 -- 172.21.15.120:0/4163123790 >> 172.21.15.120:0/4163123790 conn(0x55d2c9c139b0 msgr2=0x55d2c9c14740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- 172.21.15.120:0/4163123790 shutdown_connections 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- 172.21.15.120:0/4163123790 wait complete. 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- start start 2024-09-10T23:05:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 Processor -- start 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c21ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c23130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9ab1ab0 con 0x55d2c9c1b190 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9b1f730 con 0x55d2c9c18860 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9c20bb0 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f477c8740 10 client.0 _ll_register_callbacks cb 0x55d2c9b736b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f3dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c21ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f3dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50662/0 (socket says 172.21.15.120:50662) 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f3dd9b640 1 -- 172.21.15.120:0/888933431 learned_addr learned my addr 172.21.15.120:0/888933431 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.550+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c23130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 -- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 msgr2=0x55d2c9c23130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c23130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 -- 172.21.15.120:0/888933431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 msgr2=0x55d2c9c21ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c21ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 -- 172.21.15.120:0/888933431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c98db790 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012740 tx=0x7f0f34012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94175703c048969b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3400f920 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0f3400fac0 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f38003e20 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f3400fcc0 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f477c8740 10 client.18012 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2c9954620 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.551+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f38003e20 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0f34027420 con 0x55d2c9c1b190 2024-09-10T23:05:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 10 client.18012 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 20 client.18012 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c2d870 0x55d2c9c2fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 4 client.18012 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c9b8c640 con 0x55d2c9c2d870 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c2d870 0x55d2c9c2fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f25ffb640 10 client.18012 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.552+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c2d870 0x55d2c9c2fcb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc7be14be041c68 server_cookie=24e92b00fc689a73 in_seq=0 out_seq=0 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.555+0000 7f0f25ffb640 1 -- 172.21.15.120:0/888933431 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2c9b8c640 con 0x55d2c9c2d870 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.555+0000 7f0f25ffb640 10 client.18012 handle_command_reply: tid=0 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.555+0000 7f0f477c8740 1 client.18012 shutdown 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 msgr2=0x55d2c9c22560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0f34012740 tx=0x7f0f34012c00 comp rx=0 tx=0).stop 2024-09-10T23:05:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 shutdown_connections 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c2d870 0x55d2c9c2fcb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c23130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1b190 0x55d2c9c22560 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f25ffb640 0 client.18012 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 --2- 172.21.15.120:0/888933431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c21ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 >> 172.21.15.120:0/888933431 conn(0x55d2c9c139b0 msgr2=0x55d2c9c12bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 shutdown_connections 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 1 -- 172.21.15.120:0/888933431 wait complete. 2024-09-10T23:05:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.556+0000 7f0f477c8740 20 client.18012 trim_cache size 0 max 16384 2024-09-10T23:05:32.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:32 smithi176 ceph-mon[34982]: pgmap v739: 97 pgs: 97 active+clean; 61 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 657 op/s 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 Processor -- start 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 -- start start 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401aa000 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401ac880 con 0x55d2c9c18860 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f477c8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401aa670 con 0x55d2c9c1b190 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50672/0 (socket says 172.21.15.120:50672) 2024-09-10T23:05:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4261977902 learned_addr learned my addr 172.21.15.120:0/4261977902 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4261977902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1b190 msgr2=0x55d2c9c1dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4261977902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 msgr2=0x55d2c9c23a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c23a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.572+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/4261977902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c9c21c90 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117b0 tx=0x7f0f38009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740ea5a50b0151b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f27fff640 1 -- 172.21.15.120:0/4261977902 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380113f0 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f27fff640 1 -- 172.21.15.120:0/4261977902 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3800fe50 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f27fff640 1 -- 172.21.15.120:0/4261977902 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38005790 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f477c8740 1 -- 172.21.15.120:0/4261977902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 msgr2=0x55d2c9c1e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.573+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117b0 tx=0x7f0f38009500 comp rx=0 tx=0).stop 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- 172.21.15.120:0/4261977902 shutdown_connections 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c1e4f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1b190 0x55d2c9c1dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- 172.21.15.120:0/4261977902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c18860 0x55d2c9c23a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- 172.21.15.120:0/4261977902 >> 172.21.15.120:0/4261977902 conn(0x55d2c9c139b0 msgr2=0x55d2c9c17cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- 172.21.15.120:0/4261977902 shutdown_connections 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- 172.21.15.120:0/4261977902 wait complete. 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- start start 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 Processor -- start 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 0x55d2c9c27550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c28120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401aad40 con 0x55d2c9c18860 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401ac1b0 con 0x55d2c9c1b190 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f401ab410 con 0x55d2c9c1dbd0 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f477c8740 10 client.0 _ll_register_callbacks cb 0x7f0f34009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:50684/0 (socket says 172.21.15.120:50684) 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/1206898227 learned_addr learned my addr 172.21.15.120:0/1206898227 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c28120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.574+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 0x55d2c9c27550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 msgr2=0x55d2c9c28120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c28120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/1206898227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 msgr2=0x55d2c9c27550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 0x55d2c9c27550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 -- 172.21.15.120:0/1206898227 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c9c28950 con 0x55d2c9c18860 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 0x55d2c9c27550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f3ed9d640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0f380051b0 tx=0x7f0f3802f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4986cc71bb967f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380113f0 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0f3800f9a0 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3800fe50 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f38045050 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 10 client.18018 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2c9c29cc0 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3800fe50 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0f38028040 con 0x55d2c9c18860 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 10 client.18018 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 10 client.18018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 20 client.18018 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c9a9d0 0x55d2c9c9cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 4 client.18018 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:05:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.575+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2c9b880f0 con 0x55d2c9c9a9d0 2024-09-10T23:05:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.576+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c9a9d0 0x55d2c9c9cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.576+0000 7f0f25ffb640 10 client.18018 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:32.576+0000 7f0f3dd9b640 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c9a9d0 0x55d2c9c9cdf0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4475c8b98fc43ef1 server_cookie=ccdca4209fc77585 in_seq=0 out_seq=0 2024-09-10T23:05:32.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 12054 30.46 MB/sec warmup 27 sec latency 152.463 ms 2024-09-10T23:05:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f25ffb640 1 -- 172.21.15.120:0/1206898227 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2c9b880f0 con 0x55d2c9c9a9d0 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f25ffb640 10 client.18018 handle_command_reply: tid=0 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 client.18018 shutdown 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 msgr2=0x55d2c9c26eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0f380051b0 tx=0x7f0f3802f040 comp rx=0 tx=0).stop 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 shutdown_connections 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d2c9c9a9d0 0x55d2c9c9cdf0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9c1dbd0 0x55d2c9c28120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d2c9c1b190 0x55d2c9c27550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 --2- 172.21.15.120:0/1206898227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d2c9c18860 0x55d2c9c26eb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 >> 172.21.15.120:0/1206898227 conn(0x55d2c9c139b0 msgr2=0x55d2c9c21e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f25ffb640 0 client.18018 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 shutdown_connections 2024-09-10T23:05:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 1 -- 172.21.15.120:0/1206898227 wait complete. 2024-09-10T23:05:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.084+0000 7f0f477c8740 20 client.18018 trim_cache size 0 max 16384 2024-09-10T23:05:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:05:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:05:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:05:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 msgr2=0x7f0f280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 0x7f0f280b77a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0f30004770 tx=0x7f0f30109ec0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 msgr2=0x7f0f4010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009a50 tx=0x7f0f300042f0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 shutdown_connections 2024-09-10T23:05:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0f280b52e0 0x7f0f280b77a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f40103380 0x7f0f4010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f401029c0 0x7f0f4010a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 --2- 172.21.15.120:0/499268065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f40101870 0x7f0f4010c0f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 >> 172.21.15.120:0/499268065 conn(0x7f0f400fd130 msgr2=0x7f0f40103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 shutdown_connections 2024-09-10T23:05:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:33.085+0000 7f0f449fb640 1 -- 172.21.15.120:0/499268065 wait complete. 2024-09-10T23:05:33.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 12599 30.01 MB/sec warmup 28 sec latency 250.956 ms 2024-09-10T23:05:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:34 smithi120 ceph-mon[31247]: pgmap v740: 97 pgs: 97 active+clean; 62 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2024-09-10T23:05:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:34 smithi153 ceph-mon[34951]: pgmap v740: 97 pgs: 97 active+clean; 62 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2024-09-10T23:05:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:34 smithi176 ceph-mon[34982]: pgmap v740: 97 pgs: 97 active+clean; 62 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2024-09-10T23:05:34.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 13266 29.90 MB/sec warmup 29 sec latency 69.523 ms 2024-09-10T23:05:35.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14016 31.13 MB/sec warmup 30 sec latency 36.133 ms 2024-09-10T23:05:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:36 smithi120 ceph-mon[31247]: pgmap v741: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-09-10T23:05:36.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:36 smithi153 ceph-mon[34951]: pgmap v741: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-09-10T23:05:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:36 smithi176 ceph-mon[34982]: pgmap v741: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-09-10T23:05:36.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14425 31.24 MB/sec warmup 31 sec latency 72.078 ms 2024-09-10T23:05:37.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14664 30.47 MB/sec warmup 32 sec latency 87.684 ms 2024-09-10T23:05:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:38 smithi120 ceph-mon[31247]: pgmap v742: 97 pgs: 97 active+clean; 60 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 546 op/s 2024-09-10T23:05:38.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:38 smithi153 ceph-mon[34951]: pgmap v742: 97 pgs: 97 active+clean; 60 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 546 op/s 2024-09-10T23:05:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:38 smithi176 ceph-mon[34982]: pgmap v742: 97 pgs: 97 active+clean; 60 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 546 op/s 2024-09-10T23:05:38.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 14794 29.59 MB/sec warmup 33 sec latency 253.880 ms 2024-09-10T23:05:39.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 15048 28.92 MB/sec warmup 34 sec latency 304.088 ms 2024-09-10T23:05:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:40 smithi120 ceph-mon[31247]: pgmap v743: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:05:40.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:40 smithi153 ceph-mon[34951]: pgmap v743: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:05:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:40 smithi176 ceph-mon[34982]: pgmap v743: 97 pgs: 97 active+clean; 56 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:05:40.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 15528 28.75 MB/sec warmup 35 sec latency 53.600 ms 2024-09-10T23:05:41.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 16301 28.90 MB/sec warmup 36 sec latency 83.904 ms 2024-09-10T23:05:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:42 smithi120 ceph-mon[31247]: pgmap v744: 97 pgs: 97 active+clean; 163 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-10T23:05:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:42 smithi153 ceph-mon[34951]: pgmap v744: 97 pgs: 97 active+clean; 163 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-10T23:05:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:42 smithi176 ceph-mon[34982]: pgmap v744: 97 pgs: 97 active+clean; 163 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-10T23:05:42.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:05:42.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 17008 29.30 MB/sec warmup 37 sec latency 35.548 ms 2024-09-10T23:05:42.987 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 -- 172.21.15.120:0/501441548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 msgr2=0x7fc18810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 --2- 172.21.15.120:0/501441548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18810f550 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc180009ca0 tx=0x7fc18002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 -- 172.21.15.120:0/501441548 shutdown_connections 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 --2- 172.21.15.120:0/501441548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18810f550 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 --2- 172.21.15.120:0/501441548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188104550 0x7fc18810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 --2- 172.21.15.120:0/501441548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188103b50 0x7fc188103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 -- 172.21.15.120:0/501441548 >> 172.21.15.120:0/501441548 conn(0x7fc1880fd700 msgr2=0x7fc1880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 -- 172.21.15.120:0/501441548 shutdown_connections 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.715+0000 7fc18e573640 1 -- 172.21.15.120:0/501441548 wait complete. 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 Processor -- start 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 -- start start 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 0x7fc188199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188104550 0x7fc1881997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc188112da0 con 0x7fc18810d100 2024-09-10T23:05:43.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc188112c20 con 0x7fc188103b50 2024-09-10T23:05:43.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc18e573640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc188112f20 con 0x7fc188104550 2024-09-10T23:05:43.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc187fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 0x7fc188199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc187fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 0x7fc188199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35486/0 (socket says 172.21.15.120:35486) 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.716+0000 7fc187fff640 1 -- 172.21.15.120:0/3565874291 learned_addr learned my addr 172.21.15.120:0/3565874291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc1877fe640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188104550 0x7fc1881997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 -- 172.21.15.120:0/3565874291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188104550 msgr2=0x7fc1881997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188104550 0x7fc1881997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 -- 172.21.15.120:0/3565874291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 msgr2=0x7fc188199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 0x7fc188199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 -- 172.21.15.120:0/3565874291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18819f340 con 0x7fc18810d100 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.717+0000 7fc18cb10640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18819eb70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc180009bf0 tx=0x7fc1800319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.718+0000 7fc1857fa640 1 -- 172.21.15.120:0/3565874291 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180003b70 con 0x7fc18810d100 2024-09-10T23:05:43.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.718+0000 7fc1857fa640 1 -- 172.21.15.120:0/3565874291 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc180003d10 con 0x7fc18810d100 2024-09-10T23:05:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.718+0000 7fc1857fa640 1 -- 172.21.15.120:0/3565874291 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18002f990 con 0x7fc18810d100 2024-09-10T23:05:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.718+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18819f5d0 con 0x7fc18810d100 2024-09-10T23:05:43.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.719+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1881a5cf0 con 0x7fc18810d100 2024-09-10T23:05:43.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.721+0000 7fc190948740 1 Processor -- start 2024-09-10T23:05:43.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.721+0000 7fc1857fa640 1 -- 172.21.15.120:0/3565874291 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc18002fb30 con 0x7fc18810d100 2024-09-10T23:05:43.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.721+0000 7fc190948740 1 -- start start 2024-09-10T23:05:43.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75427bd30 0x55f75427e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7541e5e50 con 0x55f75426fda0 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f7541d5280 con 0x55f75426f480 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc190948740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f7541d9090 con 0x55f75427bd30 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc1877fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc1877fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35498/0 (socket says 172.21.15.120:35498) 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41684/0 (socket says 172.21.15.120:41684) 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc1877fe640 1 -- 172.21.15.120:0/4179314014 learned_addr learned my addr 172.21.15.120:0/4179314014 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc1877fe640 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75427bd30 0x55f75427e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 -- 172.21.15.120:0/4179314014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75427bd30 msgr2=0x55f75427e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75427bd30 0x55f75427e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 -- 172.21.15.120:0/4179314014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 msgr2=0x55f75426f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc187fff640 1 -- 172.21.15.120:0/4179314014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f753f33650 con 0x55f75426fda0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.722+0000 7fc1877fe640 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc187fff640 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009a50 tx=0x7fc17c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e3bf831ed82ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc1857fa640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 0x7fc1700b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc16e7fc640 1 -- 172.21.15.120:0/4179314014 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c017070 con 0x55f75426fda0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc1877fe640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 0x7fc1700b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc16e7fc640 1 -- 172.21.15.120:0/4179314014 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c013470 con 0x55f75426fda0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc16e7fc640 1 -- 172.21.15.120:0/4179314014 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c012630 con 0x55f75426fda0 2024-09-10T23:05:43.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.723+0000 7fc1857fa640 1 -- 172.21.15.120:0/3565874291 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc180106c70 con 0x7fc18810d100 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- 172.21.15.120:0/4179314014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 msgr2=0x55f75427b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc1877fe640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 0x7fc1700b7860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc178009fd0 tx=0x7fc178005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009a50 tx=0x7fc17c009f10 comp rx=0 tx=0).stop 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- 172.21.15.120:0/4179314014 shutdown_connections 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75427bd30 0x55f75427e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426fda0 0x55f75427b7f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 --2- 172.21.15.120:0/4179314014 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426f480 0x55f75426f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- 172.21.15.120:0/4179314014 >> 172.21.15.120:0/4179314014 conn(0x55f75426d2b0 msgr2=0x55f75426e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- 172.21.15.120:0/4179314014 shutdown_connections 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- 172.21.15.120:0/4179314014 wait complete. 2024-09-10T23:05:43.729 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 -- start start 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.724+0000 7fc190948740 1 Processor -- start 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f75427b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f7542789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f754036700 con 0x55f75427bd30 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f754170110 con 0x55f75426fda0 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f75427e9b0 con 0x55f75426f480 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41688/0 (socket says 172.21.15.120:41688) 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 -- 172.21.15.120:0/2223555554 learned_addr learned my addr 172.21.15.120:0/2223555554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc187fff640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f75427b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc1877fe640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f7542789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc190948740 10 client.0 _ll_register_callbacks cb 0x55f754189990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 -- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 msgr2=0x55f75427b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.730 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f75427b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 -- 172.21.15.120:0/2223555554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 msgr2=0x55f7542789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f7542789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 -- 172.21.15.120:0/2223555554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f75419dc40 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc18cb10640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ca6d0 tx=0x7fc1800ccdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4309bc7adc447645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc1677fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.725+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800cd8c0 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc1800d6560 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c013470 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1800cdb60 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 10 client.18030 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f75428a2a0 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc180036070 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc1800c7b60 con 0x55f75427bd30 2024-09-10T23:05:43.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 10 client.18030 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 20 client.18030 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f75428b840 0x55f75428dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 4 client.18030 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc17c013470 con 0x55f75428b840 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.726+0000 7fc187fff640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f75428b840 0x55f75428dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.727+0000 7fc1677fe640 10 client.18030 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:43.732 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.727+0000 7fc187fff640 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f75428b840 0x55f75428dc90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fd85928ea4409b server_cookie=26ea48fc2312b80f in_seq=0 out_seq=0 2024-09-10T23:05:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc1677fe640 1 -- 172.21.15.120:0/2223555554 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc17c013470 con 0x55f75428b840 2024-09-10T23:05:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc1677fe640 10 client.18030 handle_command_reply: tid=0 2024-09-10T23:05:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 client.18030 shutdown 2024-09-10T23:05:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 msgr2=0x55f754278f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.733 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ca6d0 tx=0x7fc1800ccdb0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 shutdown_connections 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f75428b840 0x55f75428dc90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754278f70 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f7542789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 --2- 172.21.15.120:0/2223555554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f75427b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 >> 172.21.15.120:0/2223555554 conn(0x55f75426d2b0 msgr2=0x55f754272e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc1677fe640 0 client.18030 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.729+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 shutdown_connections 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.730+0000 7fc190948740 1 -- 172.21.15.120:0/2223555554 wait complete. 2024-09-10T23:05:43.734 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.730+0000 7fc190948740 20 client.18030 trim_cache size 0 max 16384 2024-09-10T23:05:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.748+0000 7fc190948740 1 Processor -- start 2024-09-10T23:05:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.748+0000 7fc190948740 1 -- start start 2024-09-10T23:05:43.750 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f754275c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754276140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881aa0a0 con 0x55f75427bd30 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ac920 con 0x55f75426fda0 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc190948740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881aa710 con 0x55f75426f480 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55042/0 (socket says 172.21.15.120:55042) 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 -- 172.21.15.120:0/2764917330 learned_addr learned my addr 172.21.15.120:0/2764917330 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc1877fe640 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f754275c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc18cb10640 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754276140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 -- 172.21.15.120:0/2764917330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 msgr2=0x55f754275c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f754275c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 -- 172.21.15.120:0/2764917330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 msgr2=0x55f754276140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754276140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 -- 172.21.15.120:0/2764917330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f75427b910 con 0x55f75426f480 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.749+0000 7fc187fff640 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc17c008010 tx=0x7fc17c0091c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20b73c65391d54b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc16dffb640 1 -- 172.21.15.120:0/2764917330 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c017040 con 0x55f75426f480 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc16dffb640 1 -- 172.21.15.120:0/2764917330 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c02d030 con 0x55f75426f480 2024-09-10T23:05:43.752 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc16dffb640 1 -- 172.21.15.120:0/2764917330 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c020be0 con 0x55f75426f480 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 -- 172.21.15.120:0/2764917330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 msgr2=0x55f754287c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc17c008010 tx=0x7fc17c0091c0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 -- 172.21.15.120:0/2764917330 shutdown_connections 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75427bd30 0x55f754276140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75426fda0 0x55f754275c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 --2- 172.21.15.120:0/2764917330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426f480 0x55f754287c40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 -- 172.21.15.120:0/2764917330 >> 172.21.15.120:0/2764917330 conn(0x55f75426d2b0 msgr2=0x55f75428d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.750+0000 7fc190948740 1 -- 172.21.15.120:0/2764917330 shutdown_connections 2024-09-10T23:05:43.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.751+0000 7fc190948740 1 -- 172.21.15.120:0/2764917330 wait complete. 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.751+0000 7fc190948740 1 -- start start 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.751+0000 7fc190948740 1 Processor -- start 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 0x55f75427f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc187fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc187fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41716/0 (socket says 172.21.15.120:41716) 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75427bd30 0x55f75427fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc187fff640 1 -- 172.21.15.120:0/904579801 learned_addr learned my addr 172.21.15.120:0/904579801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881aade0 con 0x55f75426f480 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ac250 con 0x55f75427bd30 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc1881ab4b0 con 0x55f75426fda0 2024-09-10T23:05:43.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 0x55f75427f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc190948740 10 client.0 _ll_register_callbacks cb 0x7fc180003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc18cb10640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75427bd30 0x55f75427fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 -- 172.21.15.120:0/904579801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75427bd30 msgr2=0x55f75427fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75427bd30 0x55f75427fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 -- 172.21.15.120:0/904579801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 msgr2=0x55f754284230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7542802a0 con 0x55f75426fda0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc187fff640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1877fe640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 0x55f75427f750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f754268ea0 tx=0x7fc178005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d700e1699c0cfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.752+0000 7fc1677fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17802f060 con 0x55f75426fda0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c02d030 con 0x55f75426fda0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc17801a070 con 0x55f75426fda0 2024-09-10T23:05:43.758 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc17801e410 con 0x55f75426fda0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 10 client.27505 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7542f8520 con 0x55f75426fda0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c02d030 con 0x55f75426fda0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc17801d070 con 0x55f75426fda0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 10 client.27505 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 10 client.27505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 20 client.27505 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f7542f8cd0 0x55f7542fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 4 client.27505 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7541e7da0 con 0x55f7542f8cd0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.753+0000 7fc187fff640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f7542f8cd0 0x55f7542fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.754+0000 7fc187fff640 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f7542f8cd0 0x55f7542fb0f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03bdd91d3b3284a server_cookie=4fe468b1799a705b in_seq=0 out_seq=0 2024-09-10T23:05:43.759 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.754+0000 7fc1677fe640 10 client.27505 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:43.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.755+0000 7fc1677fe640 1 -- 172.21.15.120:0/904579801 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x55f7541e7da0 con 0x55f7542f8cd0 2024-09-10T23:05:43.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.755+0000 7fc1677fe640 10 client.27505 handle_command_reply: tid=0 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.755+0000 7fc190948740 1 client.27505 shutdown 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 msgr2=0x55f75427f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 0x55f75427f750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55f754268ea0 tx=0x7fc178005f90 comp rx=0 tx=0).stop 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 shutdown_connections 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f7542f8cd0 0x55f7542fb0f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f75427bd30 0x55f75427fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f75426fda0 0x55f75427f750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc1677fe640 0 client.27505 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 --2- 172.21.15.120:0/904579801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f75426f480 0x55f754284230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 >> 172.21.15.120:0/904579801 conn(0x55f75426d2b0 msgr2=0x55f75427a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 shutdown_connections 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 1 -- 172.21.15.120:0/904579801 wait complete. 2024-09-10T23:05:43.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc190948740 20 client.27505 trim_cache size 0 max 16384 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.762 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.764 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.765 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.766 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.768 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:05:43.769 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.771 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.772 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:05:43.773 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:43.774 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:43.775 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 msgr2=0x7fc1700b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 0x7fc1700b7860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc178009fd0 tx=0x7fc178005850 comp rx=0 tx=0).stop 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 msgr2=0x7fc18819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.756+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18819eb70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc180009bf0 tx=0x7fc1800319f0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 shutdown_connections 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc1700b53a0 0x7fc1700b7860 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc18810d100 0x7fc18819eb70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc188104550 0x7fc1881997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 --2- 172.21.15.120:0/3565874291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc188103b50 0x7fc188199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 >> 172.21.15.120:0/3565874291 conn(0x7fc1880fd700 msgr2=0x7fc1880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 shutdown_connections 2024-09-10T23:05:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:43.757+0000 7fc18e573640 1 -- 172.21.15.120:0/3565874291 wait complete. 2024-09-10T23:05:43.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 17440 29.33 MB/sec warmup 38 sec latency 277.495 ms 2024-09-10T23:05:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:44 smithi120 ceph-mon[31247]: pgmap v745: 97 pgs: 97 active+clean; 241 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 19 MiB/s wr, 534 op/s 2024-09-10T23:05:44.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:44 smithi153 ceph-mon[34951]: pgmap v745: 97 pgs: 97 active+clean; 241 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 19 MiB/s wr, 534 op/s 2024-09-10T23:05:44.565 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:05:44.566 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:05:44.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:44 smithi176 ceph-mon[34982]: pgmap v745: 97 pgs: 97 active+clean; 241 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 19 MiB/s wr, 534 op/s 2024-09-10T23:05:44.860 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:44.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 17834 29.45 MB/sec warmup 39 sec latency 145.904 ms 2024-09-10T23:05:45.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 -- 172.21.15.120:0/779817801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c108020 msgr2=0x7fbf6c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 --2- 172.21.15.120:0/779817801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c108020 0x7fbf6c10e820 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009ca0 tx=0x7fbf6402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 -- 172.21.15.120:0/779817801 shutdown_connections 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 --2- 172.21.15.120:0/779817801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c108020 0x7fbf6c10e820 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 --2- 172.21.15.120:0/779817801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 0x7fbf6c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 --2- 172.21.15.120:0/779817801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c101c90 0x7fbf6c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 -- 172.21.15.120:0/779817801 >> 172.21.15.120:0/779817801 conn(0x7fbf6c0fd700 msgr2=0x7fbf6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 -- 172.21.15.120:0/779817801 shutdown_connections 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.356+0000 7fbf70def640 1 -- 172.21.15.120:0/779817801 wait complete. 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 Processor -- start 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 -- start start 2024-09-10T23:05:45.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 0x7fbf6c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 0x7fbf6c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c111ca0 con 0x7fbf6c101c90 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c111b20 con 0x7fbf6c108020 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf70def640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c111e20 con 0x7fbf6c1076a0 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf63fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 0x7fbf6c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41740/0 (socket says 172.21.15.120:41740) 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 0x7fbf6c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 -- 172.21.15.120:0/476685717 learned_addr learned my addr 172.21.15.120:0/476685717 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 -- 172.21.15.120:0/476685717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 msgr2=0x7fbf6c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 0x7fbf6c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 -- 172.21.15.120:0/476685717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 msgr2=0x7fbf6c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 0x7fbf6c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6a59c640 1 -- 172.21.15.120:0/476685717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c19f320 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.357+0000 7fbf6ad9d640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 0x7fbf6c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf6a59c640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d950 tx=0x7fbf5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c017030 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c19f610 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c004020 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0047a0 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.358+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1a5cd0 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.359+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c102070 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.360+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbf5c009a30 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.361+0000 7fbf637fe640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 0x7fbf580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.361+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbf5c0e5c00 con 0x7fbf6c101c90 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.361+0000 7fbf63fff640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 0x7fbf580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:45.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.362+0000 7fbf63fff640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 0x7fbf580b77a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c103e00 tx=0x7fbf54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:45.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.364+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbf5c0acaf0 con 0x7fbf6c101c90 2024-09-10T23:05:45.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.590+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf6c1a60b0 con 0x7fbf6c101c90 2024-09-10T23:05:45.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.591+0000 7fbf637fe640 1 -- 172.21.15.120:0/476685717 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fbf5c0ac3f0 con 0x7fbf6c101c90 2024-09-10T23:05:45.592 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:05:45.592 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:05:45.592 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 msgr2=0x7fbf580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 0x7fbf580b77a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c103e00 tx=0x7fbf54005f90 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 msgr2=0x7fbf6c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d950 tx=0x7fbf5c00de10 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 shutdown_connections 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbf580b52e0 0x7fbf580b77a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf6c108020 0x7fbf6c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf6c1076a0 0x7fbf6c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 --2- 172.21.15.120:0/476685717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf6c101c90 0x7fbf6c199270 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 >> 172.21.15.120:0/476685717 conn(0x7fbf6c0fd700 msgr2=0x7fbf6c06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 shutdown_connections 2024-09-10T23:05:45.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:45.594+0000 7fbf70def640 1 -- 172.21.15.120:0/476685717 wait complete. 2024-09-10T23:05:45.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18095 29.09 MB/sec warmup 40 sec latency 61.356 ms 2024-09-10T23:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 -- 172.21.15.120:0/762614277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c81029b0 msgr2=0x7f58c8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/762614277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c81029b0 0x7f58c8102e30 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009930 tx=0x7f58b002a420 comp rx=0 tx=0).stop 2024-09-10T23:05:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 -- 172.21.15.120:0/762614277 shutdown_connections 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/762614277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8103370 0x7f58c8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/762614277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c81029b0 0x7f58c8102e30 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/762614277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c8101860 0x7f58c8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.177+0000 7f58ccdcd640 1 -- 172.21.15.120:0/762614277 >> 172.21.15.120:0/762614277 conn(0x7f58c80fd140 msgr2=0x7f58c80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- 172.21.15.120:0/762614277 shutdown_connections 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- 172.21.15.120:0/762614277 wait complete. 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 Processor -- start 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- start start 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8101860 0x7f58c8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c81029b0 0x7f58c81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58c810d4b0 con 0x7f58c8103370 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58c810d330 con 0x7f58c81029b0 2024-09-10T23:05:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.178+0000 7f58ccdcd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58c810d630 con 0x7f58c8101860 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41756/0 (socket says 172.21.15.120:41756) 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8101860 0x7f58c8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58bffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c81029b0 0x7f58c81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 -- 172.21.15.120:0/4138338639 learned_addr learned my addr 172.21.15.120:0/4138338639 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 -- 172.21.15.120:0/4138338639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8101860 msgr2=0x7f58c8198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8101860 0x7f58c8198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 -- 172.21.15.120:0/4138338639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c81029b0 msgr2=0x7f58c81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c81029b0 0x7f58c81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 -- 172.21.15.120:0/4138338639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c819a8d0 con 0x7f58c8103370 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58c6d9d640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f58c000eb10 tx=0x7f58c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58bf7fe640 1 -- 172.21.15.120:0/4138338639 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0004b20 con 0x7f58c8103370 2024-09-10T23:05:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58bf7fe640 1 -- 172.21.15.120:0/4138338639 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c0004cc0 con 0x7f58c8103370 2024-09-10T23:05:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58bf7fe640 1 -- 172.21.15.120:0/4138338639 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0010670 con 0x7f58c8103370 2024-09-10T23:05:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c819abc0 con 0x7f58c8103370 2024-09-10T23:05:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.179+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c81a1290 con 0x7f58c8103370 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58bf7fe640 1 -- 172.21.15.120:0/4138338639 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f58c0005020 con 0x7f58c8103370 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58cfc78740 1 Processor -- start 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58cfc78740 1 -- start start 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e18f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.182+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58cfc78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e18676f0 con 0x55a1e18f5860 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58cfc78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1e1856b20 con 0x55a1e18fabd0 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58cfc78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a1e185a930 con 0x55a1e18f8190 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bf7fe640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 0x7f58b40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e18f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e18f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41758/0 (socket says 172.21.15.120:41758) 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bffff640 1 -- 172.21.15.120:0/3573456764 learned_addr learned my addr 172.21.15.120:0/3573456764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c6d9d640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 0x7f58b40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bffff640 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 -- 172.21.15.120:0/3573456764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 msgr2=0x55a1e18fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 -- 172.21.15.120:0/3573456764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 msgr2=0x55a1e18f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e18f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 -- 172.21.15.120:0/3573456764 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e181c530 con 0x55a1e18f8190 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bf7fe640 1 -- 172.21.15.120:0/4138338639 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f58c00e60e0 con 0x7f58c8103370 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c659c640 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18fa580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58b800ab40 tx=0x7f58b800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17a7b1efa6b3733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58c6d9d640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 0x7f58b40b7820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f58c0071010 tx=0x7f58c00dcd40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bcff9640 1 -- 172.21.15.120:0/3573456764 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8004020 con 0x55a1e18f8190 2024-09-10T23:05:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bcff9640 1 -- 172.21.15.120:0/3573456764 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58b80041c0 con 0x55a1e18f8190 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.183+0000 7f58bcff9640 1 -- 172.21.15.120:0/3573456764 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8018730 con 0x55a1e18f8190 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 -- 172.21.15.120:0/3573456764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 msgr2=0x55a1e18fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18fa580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58b800ab40 tx=0x7f58b800f3c0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 -- 172.21.15.120:0/3573456764 shutdown_connections 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18fa580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 --2- 172.21.15.120:0/3573456764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e18f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 -- 172.21.15.120:0/3573456764 >> 172.21.15.120:0/3573456764 conn(0x55a1e18f09b0 msgr2=0x55a1e18f1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 -- 172.21.15.120:0/3573456764 shutdown_connections 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.184+0000 7f58cfc78740 1 -- 172.21.15.120:0/3573456764 wait complete. 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 -- start start 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 Processor -- start 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18f5860 0x55a1e18fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a1e178eab0 con 0x55a1e18fabd0 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a1e17fc730 con 0x55a1e18f5860 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a1e18fdbb0 con 0x55a1e18f8190 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58cfc78740 10 client.0 _ll_register_callbacks cb 0x55a1e18506b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41768/0 (socket says 172.21.15.120:41768) 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 -- 172.21.15.120:0/382597801 learned_addr learned my addr 172.21.15.120:0/382597801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58c659c640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18f5860 0x55a1e18fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 -- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 msgr2=0x55a1e18ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 -- 172.21.15.120:0/382597801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18f5860 msgr2=0x55a1e18fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18f5860 0x55a1e18fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.185+0000 7f58bffff640 1 -- 172.21.15.120:0/382597801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e15b8790 con 0x55a1e18fabd0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58bffff640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f58b002a940 tx=0x7f58b00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6597ce660e46ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0035030 con 0x55a1e18fabd0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f58b0004310 con 0x55a1e18fabd0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b80041c0 con 0x55a1e18fabd0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58b00035f0 con 0x55a1e18fabd0 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 10 client.18054 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1e1631620 con 0x55a1e18fabd0 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58b80041c0 con 0x55a1e18fabd0 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f58b0049020 con 0x55a1e18fabd0 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 10 client.18054 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 20 client.18054 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e190a630 0x55a1e190caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 4 client.18054 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e1869640 con 0x55a1e190a630 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.186+0000 7f58c659c640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e190a630 0x55a1e190caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.187+0000 7f58c659c640 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e190a630 0x55a1e190caa0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23a4d591a724edb server_cookie=fbc15b38b7965dda in_seq=0 out_seq=0 2024-09-10T23:05:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.187+0000 7f58adffb640 10 client.18054 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58adffb640 1 -- 172.21.15.120:0/382597801 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1e1869640 con 0x55a1e190a630 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58adffb640 10 client.18054 handle_command_reply: tid=0 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 client.18054 shutdown 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 msgr2=0x55a1e19000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f58b002a940 tx=0x7f58b00314a0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 shutdown_connections 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e190a630 0x55a1e190caa0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18fabd0 0x55a1e19000d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58adffb640 0 client.18054 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 --2- 172.21.15.120:0/382597801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18f5860 0x55a1e18fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 >> 172.21.15.120:0/382597801 conn(0x55a1e18f09b0 msgr2=0x55a1e18efbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 shutdown_connections 2024-09-10T23:05:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 1 -- 172.21.15.120:0/382597801 wait complete. 2024-09-10T23:05:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.191+0000 7f58cfc78740 20 client.18054 trim_cache size 0 max 16384 2024-09-10T23:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 Processor -- start 2024-09-10T23:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 -- start start 2024-09-10T23:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81a7f00 con 0x55a1e18f5860 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81aa780 con 0x55a1e18fabd0 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58cfc78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81a8570 con 0x55a1e18f8190 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.206+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41770/0 (socket says 172.21.15.120:41770) 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 -- 172.21.15.120:0/324417148 learned_addr learned my addr 172.21.15.120:0/324417148 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58c659c640 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 -- 172.21.15.120:0/324417148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 msgr2=0x55a1e18f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 -- 172.21.15.120:0/324417148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 msgr2=0x55a1e18f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 -- 172.21.15.120:0/324417148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e18ff1d0 con 0x55a1e18f5860 2024-09-10T23:05:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58bffff640 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f58b0006010 tx=0x7f58b00317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa09c6b9a78719ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58affff640 1 -- 172.21.15.120:0/324417148 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b005c070 con 0x55a1e18f5860 2024-09-10T23:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58affff640 1 -- 172.21.15.120:0/324417148 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58b003b460 con 0x55a1e18f5860 2024-09-10T23:05:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.207+0000 7f58affff640 1 -- 172.21.15.120:0/324417148 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0035040 con 0x55a1e18f5860 2024-09-10T23:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- 172.21.15.120:0/324417148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 msgr2=0x55a1e19009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f58b0006010 tx=0x7f58b00317c0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- 172.21.15.120:0/324417148 shutdown_connections 2024-09-10T23:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e18f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e18f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 --2- 172.21.15.120:0/324417148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19009a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- 172.21.15.120:0/324417148 >> 172.21.15.120:0/324417148 conn(0x55a1e18f09b0 msgr2=0x55a1e18fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- 172.21.15.120:0/324417148 shutdown_connections 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- 172.21.15.120:0/324417148 wait complete. 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 -- start start 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.208+0000 7f58cfc78740 1 Processor -- start 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e1906a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e19755f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81a8c40 con 0x55a1e18f5860 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81aa0b0 con 0x55a1e18fabd0 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81a9310 con 0x55a1e18f8190 2024-09-10T23:05:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 10 client.0 _ll_register_callbacks cb 0x7f58b0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41784/0 (socket says 172.21.15.120:41784) 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 -- 172.21.15.120:0/132412276 learned_addr learned my addr 172.21.15.120:0/132412276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58c659c640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e1906a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 -- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 msgr2=0x55a1e1906a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e1906a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 -- 172.21.15.120:0/132412276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 msgr2=0x55a1e19755f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e19755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 -- 172.21.15.120:0/132412276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e1975da0 con 0x55a1e18f5860 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58bffff640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f58b005a040 tx=0x7f58b003b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728ce19a342cc242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58adffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b005c070 con 0x55a1e18f5860 2024-09-10T23:05:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f58b0031bd0 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1e18650f0 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58b0068070 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 10 client.18060 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1e19770b0 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.209+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a1e18650f0 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f58b0049070 con 0x55a1e18f5860 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 10 client.18060 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 20 client.18060 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e19779d0 0x55a1e1979df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 4 client.18060 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1e1862520 con 0x55a1e19779d0 2024-09-10T23:05:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58c659c640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e19779d0 0x55a1e1979df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58adffb640 10 client.18060 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.210+0000 7f58c659c640 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e19779d0 0x55a1e1979df0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5412e0b9f6aef1e server_cookie=b81c1ccc9adfc2a7 in_seq=0 out_seq=0 2024-09-10T23:05:46.347 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:46 smithi120 ceph-mon[31247]: pgmap v746: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 20 MiB/s wr, 577 op/s 2024-09-10T23:05:46.347 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/476685717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:46.374 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:05:46.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:46 smithi153 ceph-mon[34951]: pgmap v746: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 20 MiB/s wr, 577 op/s 2024-09-10T23:05:46.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:46 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/476685717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:46 smithi176 ceph-mon[34982]: pgmap v746: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 20 MiB/s wr, 577 op/s 2024-09-10T23:05:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:46 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/476685717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:05:46.801 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:46.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18208 28.43 MB/sec warmup 41 sec latency 345.236 ms 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58adffb640 1 -- 172.21.15.120:0/132412276 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1e1862520 con 0x55a1e19779d0 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58adffb640 10 client.18060 handle_command_reply: tid=0 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 client.18060 shutdown 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 msgr2=0x55a1e19063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f58b005a040 tx=0x7f58b003b7b0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 shutdown_connections 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55a1e19779d0 0x55a1e1979df0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a1e18fabd0 0x55a1e19755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58adffb640 0 client.18060 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a1e18f8190 0x55a1e1906a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 --2- 172.21.15.120:0/132412276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a1e18f5860 0x55a1e19063e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 >> 172.21.15.120:0/132412276 conn(0x55a1e18f09b0 msgr2=0x55a1e18f4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 shutdown_connections 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 1 -- 172.21.15.120:0/132412276 wait complete. 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.994+0000 7f58cfc78740 20 client.18060 trim_cache size 0 max 16384 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:05:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:05:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 msgr2=0x7f58b40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 0x7f58b40b7820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f58c0071010 tx=0x7f58c00dcd40 comp rx=0 tx=0).stop 2024-09-10T23:05:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 msgr2=0x7f58c819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f58c000eb10 tx=0x7f58c000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 shutdown_connections 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f58b40b5360 0x7f58b40b7820 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c8103370 0x7f58c819a010 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58c81029b0 0x7f58c81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 --2- 172.21.15.120:0/4138338639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58c8101860 0x7f58c8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 >> 172.21.15.120:0/4138338639 conn(0x7f58c80fd140 msgr2=0x7f58c806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 shutdown_connections 2024-09-10T23:05:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:05:46.995+0000 7f58ccdcd640 1 -- 172.21.15.120:0/4138338639 wait complete. 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.311+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/2033040348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38101cd0 msgr2=0x7f7b381020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.311+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/2033040348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38101cd0 0x7f7b381020b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402a410 comp rx=0 tx=0).stop 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.311+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/2033040348 shutdown_connections 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.311+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/2033040348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38108060 0x7f7b3810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/2033040348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 0x7f7b38107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/2033040348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38101cd0 0x7f7b381020b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/2033040348 >> 172.21.15.120:0/2033040348 conn(0x7f7b380fd740 msgr2=0x7f7b380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/2033040348 shutdown_connections 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/2033040348 wait complete. 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 Processor -- start 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.312+0000 7f7b3f3b4640 1 -- start start 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38101cd0 0x7f7b3819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 0x7f7b3819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38111d20 con 0x7f7b38108060 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38111ba0 con 0x7f7b381076e0 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38111ea0 con 0x7f7b38101cd0 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41814/0 (socket says 172.21.15.120:41814) 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 -- 172.21.15.120:0/1449190325 learned_addr learned my addr 172.21.15.120:0/1449190325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:47.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d150640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38101cd0 0x7f7b3819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 0x7f7b3819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 -- 172.21.15.120:0/1449190325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38101cd0 msgr2=0x7f7b3819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38101cd0 0x7f7b3819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 -- 172.21.15.120:0/1449190325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 msgr2=0x7f7b3819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 0x7f7b3819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 -- 172.21.15.120:0/1449190325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b381a5490 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3d951640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000eb10 tx=0x7f7b3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b2e7fc640 1 -- 172.21.15.120:0/1449190325 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30004b20 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b2e7fc640 1 -- 172.21.15.120:0/1449190325 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b30004cc0 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b38197ea0 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b2e7fc640 1 -- 172.21.15.120:0/1449190325 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30010670 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.313+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b381983e0 con 0x7f7b38108060 2024-09-10T23:05:47.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.318+0000 7f7b2e7fc640 1 -- 172.21.15.120:0/1449190325 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7b30010810 con 0x7f7b38108060 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.320+0000 7f7b41789740 1 Processor -- start 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b2e7fc640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 0x7f7b240b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b2e7fc640 1 -- 172.21.15.120:0/1449190325 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7b300e61f0 con 0x7f7b38108060 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 -- start start 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea700460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea702d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea66aea0 con 0x55b0ea6f40f0 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea65a2d0 con 0x55b0ea6f4a10 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.321+0000 7f7b41789740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea65e0e0 con 0x55b0ea7009a0 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41820/0 (socket says 172.21.15.120:41820) 2024-09-10T23:05:47.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 -- 172.21.15.120:0/2825537523 learned_addr learned my addr 172.21.15.120:0/2825537523 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 -- 172.21.15.120:0/2825537523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 msgr2=0x55b0ea702d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea702d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 -- 172.21.15.120:0/2825537523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 msgr2=0x55b0ea700460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.322+0000 7f7b3d150640 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea700460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b3d150640 1 -- 172.21.15.120:0/2825537523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ea3b8650 con 0x55b0ea6f40f0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b3d150640 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b34032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bc1a1a41bc238a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 0x7f7b240b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b237fe640 1 -- 172.21.15.120:0/2825537523 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34040070 con 0x55b0ea6f40f0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b237fe640 1 -- 172.21.15.120:0/2825537523 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b34004030 con 0x55b0ea6f40f0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.323+0000 7f7b237fe640 1 -- 172.21.15.120:0/2825537523 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34035040 con 0x55b0ea6f40f0 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 -- 172.21.15.120:0/2825537523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 msgr2=0x55b0ea6f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b34032b20 comp rx=0 tx=0).stop 2024-09-10T23:05:47.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 0x7f7b240b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b0ea6f2e00 tx=0x7f7b28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 -- 172.21.15.120:0/2825537523 shutdown_connections 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea702d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea700460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.324+0000 7f7b41789740 1 --2- 172.21.15.120:0/2825537523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6f44d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.325+0000 7f7b41789740 1 -- 172.21.15.120:0/2825537523 >> 172.21.15.120:0/2825537523 conn(0x55b0ea6f1f20 msgr2=0x55b0ea6f30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.325+0000 7f7b41789740 1 -- 172.21.15.120:0/2825537523 shutdown_connections 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.325+0000 7f7b41789740 1 -- 172.21.15.120:0/2825537523 wait complete. 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.325+0000 7f7b41789740 1 -- start start 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.325+0000 7f7b41789740 1 Processor -- start 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea4bb700 con 0x55b0ea7009a0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea5f5160 con 0x55b0ea6f4a10 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b0ea7035c0 con 0x55b0ea6f40f0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b3d150640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b3d150640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55156/0 (socket says 172.21.15.120:55156) 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b3d150640 1 -- 172.21.15.120:0/801671199 learned_addr learned my addr 172.21.15.120:0/801671199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b41789740 10 client.0 _ll_register_callbacks cb 0x55b0ea60e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.326+0000 7f7b3d951640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 -- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 msgr2=0x55b0ea7002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 -- 172.21.15.120:0/801671199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 msgr2=0x55b0ea6fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 -- 172.21.15.120:0/801671199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ea622c90 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.327+0000 7f7b3d951640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fdd30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7b300049d0 tx=0x7f7b300ac750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a38575a6930b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.329+0000 7f7b20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.329+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b300a80f0 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.329+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7b300b5c80 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.329+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b34004030 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b300ac490 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 10 client.18072 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0ea70ef60 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b300e5040 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7b300a82d0 con 0x55b0ea7009a0 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 10 client.18072 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:47.342 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 20 client.18072 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea710420 0x55b0ea712810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 4 client.18072 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.330+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b34004030 con 0x55b0ea710420 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.331+0000 7f7b3d150640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea710420 0x55b0ea712810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.332+0000 7f7b3d150640 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea710420 0x55b0ea712810 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43ce8bc64b6d41f server_cookie=c7594922d03f2871 in_seq=0 out_seq=0 2024-09-10T23:05:47.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.332+0000 7f7b20ff9640 10 client.18072 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.337+0000 7f7b20ff9640 1 -- 172.21.15.120:0/801671199 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7b34004030 con 0x55b0ea710420 2024-09-10T23:05:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.337+0000 7f7b20ff9640 10 client.18072 handle_command_reply: tid=0 2024-09-10T23:05:47.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.337+0000 7f7b41789740 1 client.18072 shutdown 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.338+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 msgr2=0x55b0ea6fdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.338+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fdd30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7b300049d0 tx=0x7f7b300ac750 comp rx=0 tx=0).stop 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 shutdown_connections 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea710420 0x55b0ea712810 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fdd30 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 --2- 172.21.15.120:0/801671199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 >> 172.21.15.120:0/801671199 conn(0x55b0ea6f1f20 msgr2=0x55b0ea6f7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b20ff9640 0 client.18072 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 shutdown_connections 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.339+0000 7f7b41789740 1 -- 172.21.15.120:0/801671199 wait complete. 2024-09-10T23:05:47.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.340+0000 7f7b41789740 20 client.18072 trim_cache size 0 max 16384 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.361+0000 7f7b41789740 1 Processor -- start 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.361+0000 7f7b41789740 1 -- start start 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.361+0000 7f7b41789740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.361+0000 7f7b41789740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.361+0000 7f7b41789740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b41789740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381ae610 con 0x55b0ea7009a0 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b41789740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381b0e90 con 0x55b0ea6f40f0 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b41789740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381aec80 con 0x55b0ea6f4a10 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41844/0 (socket says 172.21.15.120:41844) 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 -- 172.21.15.120:0/747448080 learned_addr learned my addr 172.21.15.120:0/747448080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d150640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 -- 172.21.15.120:0/747448080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f4a10 msgr2=0x55b0ea6fbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 -- 172.21.15.120:0/747448080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 msgr2=0x55b0ea6fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 -- 172.21.15.120:0/747448080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ea700620 con 0x55b0ea7009a0 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d150640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:47.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b3d951640 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cfb0 tx=0x7f7b300ac880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f899615e5af6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b22ffd640 1 -- 172.21.15.120:0/747448080 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30071070 con 0x55b0ea7009a0 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b22ffd640 1 -- 172.21.15.120:0/747448080 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b300e5040 con 0x55b0ea7009a0 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.362+0000 7f7b22ffd640 1 -- 172.21.15.120:0/747448080 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b300b5f10 con 0x55b0ea7009a0 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 -- 172.21.15.120:0/747448080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 msgr2=0x55b0ea6fc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cfb0 tx=0x7f7b300ac880 comp rx=0 tx=0).stop 2024-09-10T23:05:47.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 -- 172.21.15.120:0/747448080 shutdown_connections 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea7009a0 0x55b0ea6fc4c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea6f4a10 0x55b0ea6fbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 --2- 172.21.15.120:0/747448080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f40f0 0x55b0ea6fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 -- 172.21.15.120:0/747448080 >> 172.21.15.120:0/747448080 conn(0x55b0ea6f1f20 msgr2=0x55b0ea6f89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.363+0000 7f7b41789740 1 -- 172.21.15.120:0/747448080 shutdown_connections 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.364+0000 7f7b41789740 1 -- 172.21.15.120:0/747448080 wait complete. 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.364+0000 7f7b41789740 1 -- start start 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.364+0000 7f7b41789740 1 Processor -- start 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea718660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381af350 con 0x55b0ea6f40f0 2024-09-10T23:05:47.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381b07c0 con 0x55b0ea6f4a10 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381afa20 con 0x55b0ea7009a0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b41789740 10 client.0 _ll_register_callbacks cb 0x7f7b300a8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3c94f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3c94f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35656/0 (socket says 172.21.15.120:35656) 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3c94f640 1 -- 172.21.15.120:0/3820001986 learned_addr learned my addr 172.21.15.120:0/3820001986 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea718660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d150640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 -- 172.21.15.120:0/3820001986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 msgr2=0x55b0ea717ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 -- 172.21.15.120:0/3820001986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 msgr2=0x55b0ea7091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3d951640 1 -- 172.21.15.120:0/3820001986 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ea707e50 con 0x55b0ea7009a0 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.365+0000 7f7b3c94f640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:47.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.366+0000 7f7b3d951640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea718660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b0ea6fe740 tx=0x7f7b300b1460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59c33592d710ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:47.370 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.366+0000 7f7b20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:05:47.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.367+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30071070 con 0x55b0ea7009a0 2024-09-10T23:05:47.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.367+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7b300b5c80 con 0x55b0ea7009a0 2024-09-10T23:05:47.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.367+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b300e5040 con 0x55b0ea7009a0 2024-09-10T23:05:47.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b300ac2d0 con 0x55b0ea7009a0 2024-09-10T23:05:47.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 10 client.27544 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0ea77d4c0 con 0x55b0ea7009a0 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b300ab650 con 0x55b0ea7009a0 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7b300b25f0 con 0x55b0ea7009a0 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 10 client.27544 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 10 client.27544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 20 client.27544 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea77d8d0 0x55b0ea77fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 4 client.27544 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.368+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7b300e5040 con 0x55b0ea77d8d0 2024-09-10T23:05:47.372 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.369+0000 7f7b3d150640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea77d8d0 0x55b0ea77fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:47.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.372+0000 7f7b3d150640 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea77d8d0 0x55b0ea77fcf0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52d19f5b355ce44 server_cookie=a9b3122db01344e9 in_seq=0 out_seq=0 2024-09-10T23:05:47.373 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.372+0000 7f7b20ff9640 10 client.27544 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b20ff9640 1 -- 172.21.15.120:0/3820001986 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7b300e5040 con 0x55b0ea77d8d0 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b20ff9640 10 client.27544 handle_command_reply: tid=0 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 client.27544 shutdown 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 msgr2=0x55b0ea718660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea718660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b0ea6fe740 tx=0x7f7b300b1460 comp rx=0 tx=0).stop 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 shutdown_connections 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b0ea77d8d0 0x55b0ea77fcf0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b0ea7009a0 0x55b0ea718660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0ea6f4a10 0x55b0ea717ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 --2- 172.21.15.120:0/3820001986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0ea6f40f0 0x55b0ea7091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b20ff9640 0 client.27544 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:47.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 >> 172.21.15.120:0/3820001986 conn(0x55b0ea6f1f20 msgr2=0x55b0ea6fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 shutdown_connections 2024-09-10T23:05:47.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 1 -- 172.21.15.120:0/3820001986 wait complete. 2024-09-10T23:05:47.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.378+0000 7f7b41789740 20 client.27544 trim_cache size 0 max 16384 2024-09-10T23:05:47.381 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:05:47.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 msgr2=0x7f7b240b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 0x7f7b240b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b0ea6f2e00 tx=0x7f7b28005f90 comp rx=0 tx=0).stop 2024-09-10T23:05:47.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 msgr2=0x7f7b3819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000eb10 tx=0x7f7b3000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 shutdown_connections 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7b240b5360 0x7f7b240b7820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b38108060 0x7f7b3819ab50 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b381076e0 0x7f7b3819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 --2- 172.21.15.120:0/1449190325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b38101cd0 0x7f7b3819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 >> 172.21.15.120:0/1449190325 conn(0x7f7b380fd740 msgr2=0x7f7b380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 shutdown_connections 2024-09-10T23:05:47.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:47.379+0000 7f7b3f3b4640 1 -- 172.21.15.120:0/1449190325 wait complete. 2024-09-10T23:05:47.865 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18422 27.84 MB/sec warmup 42 sec latency 373.590 ms 2024-09-10T23:05:48.375 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:48 smithi120 ceph-mon[31247]: pgmap v747: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 618 op/s 2024-09-10T23:05:48.375 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:48 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:48 smithi153 ceph-mon[34951]: pgmap v747: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 618 op/s 2024-09-10T23:05:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:48 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:48 smithi176 ceph-mon[34982]: pgmap v747: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 618 op/s 2024-09-10T23:05:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:48 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:05:48.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 18764 27.53 MB/sec warmup 43 sec latency 172.634 ms 2024-09-10T23:05:49.404 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:05:49.404 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:05:49.435 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:49.435 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:49.435 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:05:49.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:49.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:49.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:05:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:05:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:05:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:05:49.717 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:49.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 19204 27.43 MB/sec warmup 44 sec latency 222.735 ms 2024-09-10T23:05:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 -- 172.21.15.120:0/1165424907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720108740 msgr2=0x7ff720108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 --2- 172.21.15.120:0/1165424907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720108740 0x7ff720108b80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff708009900 tx=0x7ff70802a410 comp rx=0 tx=0).stop 2024-09-10T23:05:50.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 -- 172.21.15.120:0/1165424907 shutdown_connections 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 --2- 172.21.15.120:0/1165424907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff72010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 --2- 172.21.15.120:0/1165424907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720108740 0x7ff720108b80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 --2- 172.21.15.120:0/1165424907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720101260 0x7ff720101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 -- 172.21.15.120:0/1165424907 >> 172.21.15.120:0/1165424907 conn(0x7ff720078060 msgr2=0x7ff7200ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 -- 172.21.15.120:0/1165424907 shutdown_connections 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.246+0000 7ff7248c8640 1 -- 172.21.15.120:0/1165424907 wait complete. 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 Processor -- start 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 -- start start 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720108740 0x7ff720102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.265 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff720102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112da0 con 0x7ff720101260 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112c20 con 0x7ff7201090c0 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff7248c8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112f20 con 0x7ff720108740 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41868/0 (socket says 172.21.15.120:41868) 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff71e59c640 1 -- 172.21.15.120:0/29574721 learned_addr learned my addr 172.21.15.120:0/29574721 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff715d9b640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720108740 0x7ff720102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 -- 172.21.15.120:0/29574721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720108740 msgr2=0x7ff720102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720108740 0x7ff720102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 -- 172.21.15.120:0/29574721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 msgr2=0x7ff720102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.247+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff720102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff720102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 -- 172.21.15.120:0/29574721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7201a76d0 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff720102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff71e59c640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d950 tx=0x7ff71000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff717fff640 1 -- 172.21.15.120:0/29574721 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710017030 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201a79c0 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff717fff640 1 -- 172.21.15.120:0/29574721 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710004020 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff717fff640 1 -- 172.21.15.120:0/29574721 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100047a0 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.248+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a7f20 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.250+0000 7ff717fff640 1 -- 172.21.15.120:0/29574721 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff710009a30 con 0x7ff720101260 2024-09-10T23:05:50.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 Processor -- start 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 -- start start 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 0x558b2cf4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b2ceb3e60 con 0x558b2cf3cee0 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b2cea3290 con 0x558b2cf49790 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff726c9d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b2cea70a0 con 0x558b2cf3d800 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.251+0000 7ff71ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff71ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41872/0 (socket says 172.21.15.120:41872) 2024-09-10T23:05:50.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff71ed9d640 1 -- 172.21.15.120:0/2350253546 learned_addr learned my addr 172.21.15.120:0/2350253546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 0x558b2cf4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 -- 172.21.15.120:0/2350253546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 msgr2=0x558b2cf49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 -- 172.21.15.120:0/2350253546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 msgr2=0x558b2cf3d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 -- 172.21.15.120:0/2350253546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2cc01930 con 0x558b2cf49790 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff717fff640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 0x7ff70c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 0x558b2cf4bb80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff708009230 tx=0x7ff708032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82cf05123f71feec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 0x7ff70c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.252+0000 7ff717fff640 1 -- 172.21.15.120:0/29574721 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff7100e5680 con 0x7ff720101260 2024-09-10T23:05:50.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff714d99640 1 -- 172.21.15.120:0/2350253546 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708040070 con 0x558b2cf49790 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff714d99640 1 -- 172.21.15.120:0/2350253546 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff708004030 con 0x558b2cf49790 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff714d99640 1 -- 172.21.15.120:0/2350253546 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708035040 con 0x558b2cf49790 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 -- 172.21.15.120:0/2350253546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 msgr2=0x558b2cf4bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 0x7ff70c0b77a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff718012740 tx=0x7ff718009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 0x558b2cf4bb80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff708009230 tx=0x7ff708032bd0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 -- 172.21.15.120:0/2350253546 shutdown_connections 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf49790 0x558b2cf4bb80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2350253546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3cee0 0x558b2cf3d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 -- 172.21.15.120:0/2350253546 >> 172.21.15.120:0/2350253546 conn(0x558b2cf3ad10 msgr2=0x558b2cf3beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.253+0000 7ff726c9d740 1 -- 172.21.15.120:0/2350253546 shutdown_connections 2024-09-10T23:05:50.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 -- 172.21.15.120:0/2350253546 wait complete. 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 -- start start 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 Processor -- start 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf49790 0x558b2cf471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558b2cd04890 con 0x558b2cf3d800 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558b2ce3dd40 con 0x558b2cf3cee0 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b2cf4c3e0 con 0x558b2cf49790 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff726c9d740 10 client.0 _ll_register_callbacks cb 0x558b2cea1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff71ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf49790 0x558b2cf471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff715d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41876/0 (socket says 172.21.15.120:41876) 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff715d9b640 1 -- 172.21.15.120:0/2558090938 learned_addr learned my addr 172.21.15.120:0/2558090938 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.254+0000 7ff71e59c640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf46760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35716/0 (socket says 172.21.15.120:35716) 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 -- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf49790 msgr2=0x558b2cf471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf49790 0x558b2cf471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 -- 172.21.15.120:0/2558090938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 msgr2=0x558b2cf46760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf46760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 -- 172.21.15.120:0/2558090938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2cf557c0 con 0x558b2cf3d800 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff715d9b640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff708005c40 tx=0x7ff708009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9d74d20c4cf469 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff705ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:50.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708052060 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff7080052d0 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff708004030 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff708035070 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff726c9d740 10 client.18090 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b2cf58170 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.255+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff708004030 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff708004c20 con 0x558b2cf3d800 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 10 client.18090 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 20 client.18090 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:50.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cf5b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 4 client.18090 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b2ceb5db0 con 0x558b2cf594c0 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff71e59c640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cf5b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff705ffb640 10 client.18090 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.256+0000 7ff71e59c640 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cf5b8b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250b69ed8c81d073 server_cookie=aad32b1ff01e93aa in_seq=0 out_seq=0 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff705ffb640 1 -- 172.21.15.120:0/2558090938 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b2ceb5db0 con 0x558b2cf594c0 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff705ffb640 10 client.18090 handle_command_reply: tid=0 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 client.18090 shutdown 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 msgr2=0x558b2cf46ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff708005c40 tx=0x7ff708009260 comp rx=0 tx=0).stop 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 shutdown_connections 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cf5b8b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.274 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff705ffb640 0 client.18090 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf49790 0x558b2cf471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf3d800 0x558b2cf46ca0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 --2- 172.21.15.120:0/2558090938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 >> 172.21.15.120:0/2558090938 conn(0x558b2cf3ad10 msgr2=0x558b2cf39f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 shutdown_connections 2024-09-10T23:05:50.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 1 -- 172.21.15.120:0/2558090938 wait complete. 2024-09-10T23:05:50.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.261+0000 7ff726c9d740 20 client.18090 trim_cache size 0 max 16384 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 Processor -- start 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 -- start start 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf46c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201ae4b0 con 0x558b2cf49790 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201b0d30 con 0x558b2cf3cee0 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff726c9d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201aeb20 con 0x558b2cf3d800 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41880/0 (socket says 172.21.15.120:41880) 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 -- 172.21.15.120:0/691488914 learned_addr learned my addr 172.21.15.120:0/691488914 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff715d9b640 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf46c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 -- 172.21.15.120:0/691488914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 msgr2=0x558b2cf46c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf46c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 -- 172.21.15.120:0/691488914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 msgr2=0x558b2cf484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.281+0000 7ff71ed9d640 1 -- 172.21.15.120:0/691488914 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2cf456c0 con 0x558b2cf49790 2024-09-10T23:05:50.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff718009cf0 tx=0x7ff718015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abc12b977f2b332 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff707fff640 1 -- 172.21.15.120:0/691488914 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718010d70 con 0x558b2cf49790 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff707fff640 1 -- 172.21.15.120:0/691488914 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718003e20 con 0x558b2cf49790 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff707fff640 1 -- 172.21.15.120:0/691488914 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71801c690 con 0x558b2cf49790 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 -- 172.21.15.120:0/691488914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 msgr2=0x558b2cf471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff718009cf0 tx=0x7ff718015000 comp rx=0 tx=0).stop 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 -- 172.21.15.120:0/691488914 shutdown_connections 2024-09-10T23:05:50.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cf471a0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf46c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 --2- 172.21.15.120:0/691488914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.282+0000 7ff726c9d740 1 -- 172.21.15.120:0/691488914 >> 172.21.15.120:0/691488914 conn(0x558b2cf3ad10 msgr2=0x558b2cf47d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- 172.21.15.120:0/691488914 shutdown_connections 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- 172.21.15.120:0/691488914 wait complete. 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- start start 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 Processor -- start 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cfbf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201af1f0 con 0x558b2cf49790 2024-09-10T23:05:50.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201b0660 con 0x558b2cf3cee0 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201af8c0 con 0x558b2cf3d800 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff726c9d740 10 client.0 _ll_register_callbacks cb 0x7ff718009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff71e59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35734/0 (socket says 172.21.15.120:35734) 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff71e59c640 1 -- 172.21.15.120:0/3939332065 learned_addr learned my addr 172.21.15.120:0/3939332065 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff715d9b640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.283+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cfbf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 -- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 msgr2=0x558b2cf4e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf4e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 -- 172.21.15.120:0/3939332065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 msgr2=0x558b2cf4e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:50.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 -- 172.21.15.120:0/3939332065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2cfbf870 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71e59c640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff71ed9d640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cfbf130 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff718009b70 tx=0x7ff718004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591797e8d38f8656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff705ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718003b80 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff718004000 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff718003e20 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff718004200 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff726c9d740 10 client.18096 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.284+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b2cfc6d10 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff718003e20 con 0x558b2cf49790 2024-09-10T23:05:50.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff718031070 con 0x558b2cf49790 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 10 client.18096 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 20 client.18096 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cfc9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 4 client.18096 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b2ceb1860 con 0x558b2cf594c0 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff71e59c640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cfc9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff705ffb640 10 client.18096 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:50.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.285+0000 7ff71e59c640 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cfc9500 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffd0f6c1e0d1875 server_cookie=45cb4272afc31ac9 in_seq=0 out_seq=0 2024-09-10T23:05:50.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff705ffb640 1 -- 172.21.15.120:0/3939332065 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b2ceb1860 con 0x558b2cf594c0 2024-09-10T23:05:50.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff705ffb640 10 client.18096 handle_command_reply: tid=0 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 client.18096 shutdown 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 msgr2=0x558b2cfbf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cfbf130 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff718009b70 tx=0x7ff718004890 comp rx=0 tx=0).stop 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 shutdown_connections 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558b2cf594c0 0x558b2cfc9500 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558b2cf49790 0x558b2cfbf130 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b2cf3d800 0x558b2cf4e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 --2- 172.21.15.120:0/3939332065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b2cf3cee0 0x558b2cf4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 >> 172.21.15.120:0/3939332065 conn(0x558b2cf3ad10 msgr2=0x558b2cf485b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff705ffb640 0 client.18096 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 shutdown_connections 2024-09-10T23:05:50.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 1 -- 172.21.15.120:0/3939332065 wait complete. 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.289+0000 7ff726c9d740 20 client.18096 trim_cache size 0 max 16384 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "010375e9-ff31-4fd9-8442-d91c1ce255e0", 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:05:50.294 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 msgr2=0x7ff70c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 0x7ff70c0b77a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff718012740 tx=0x7ff718009500 comp rx=0 tx=0).stop 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 msgr2=0x7ff720102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d950 tx=0x7ff71000de10 comp rx=0 tx=0).stop 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 shutdown_connections 2024-09-10T23:05:50.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff70c0b52e0 0x7ff70c0b77a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7201090c0 0x7ff720102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff720108740 0x7ff720102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 --2- 172.21.15.120:0/29574721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff720101260 0x7ff720102230 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 >> 172.21.15.120:0/29574721 conn(0x7ff720078060 msgr2=0x7ff7200ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 shutdown_connections 2024-09-10T23:05:50.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:50.290+0000 7ff7248c8640 1 -- 172.21.15.120:0/29574721 wait complete. 2024-09-10T23:05:50.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:50 smithi120 ceph-mon[31247]: pgmap v748: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-09-10T23:05:50.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:50 smithi153 ceph-mon[34951]: pgmap v748: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-09-10T23:05:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:50 smithi176 ceph-mon[34982]: pgmap v748: 97 pgs: 97 active+clean; 252 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 20 MiB/s wr, 352 op/s 2024-09-10T23:05:50.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 19675 27.09 MB/sec warmup 45 sec latency 174.653 ms 2024-09-10T23:05:51.011 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:05:51.330 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:05:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:51 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:05:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:51 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:51 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:05:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:51 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:05:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:51 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:51 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:05:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:51 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:05:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:51 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:51.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:51 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:05:51.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 19993 26.89 MB/sec warmup 46 sec latency 406.348 ms 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 -- 172.21.15.120:0/2592893613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 msgr2=0x7f996010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 --2- 172.21.15.120:0/2592893613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f996010cb30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9950009900 tx=0x7f995002a410 comp rx=0 tx=0).stop 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 -- 172.21.15.120:0/2592893613 shutdown_connections 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 --2- 172.21.15.120:0/2592893613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996010d100 0x7f996010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 --2- 172.21.15.120:0/2592893613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f996010cb30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 --2- 172.21.15.120:0/2592893613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9960103b50 0x7f9960103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.867 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 -- 172.21.15.120:0/2592893613 >> 172.21.15.120:0/2592893613 conn(0x7f99600fd700 msgr2=0x7f99600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 -- 172.21.15.120:0/2592893613 shutdown_connections 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 -- 172.21.15.120:0/2592893613 wait complete. 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.844+0000 7f99681df640 1 Processor -- start 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 -- start start 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 0x7f9960199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f996010d100 0x7f996019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9960112d80 con 0x7f9960104550 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9960112c00 con 0x7f996010d100 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f99681df640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9960112f00 con 0x7f9960103b50 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 0x7f9960199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.868 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41914/0 (socket says 172.21.15.120:41914) 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 -- 172.21.15.120:0/1195397318 learned_addr learned my addr 172.21.15.120:0/1195397318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 0x7f9960199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55250/0 (socket says 172.21.15.120:55250) 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996677c640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f996010d100 0x7f996019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 -- 172.21.15.120:0/1195397318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 msgr2=0x7f9960199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 0x7f9960199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 -- 172.21.15.120:0/1195397318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f996010d100 msgr2=0x7f996019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f996010d100 0x7f996019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 -- 172.21.15.120:0/1195397318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996019f2e0 con 0x7f9960104550 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f996577a640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9950005940 tx=0x7f99500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.845+0000 7f9956ffd640 1 -- 172.21.15.120:0/1195397318 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995003f030 con 0x7f9960104550 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.846+0000 7f9956ffd640 1 -- 172.21.15.120:0/1195397318 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99500044c0 con 0x7f9960104550 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.846+0000 7f9956ffd640 1 -- 172.21.15.120:0/1195397318 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950035070 con 0x7f9960104550 2024-09-10T23:05:51.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.846+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996019f570 con 0x7f9960104550 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.846+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99601a5c90 con 0x7f9960104550 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.848+0000 7f9956ffd640 1 -- 172.21.15.120:0/1195397318 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f995003ba20 con 0x7f9960104550 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.848+0000 7f996a5b4740 1 Processor -- start 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.848+0000 7f996a5b4740 1 -- start start 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556125948e50 con 0x5561259d2da0 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556125938280 con 0x5561259ded30 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996a5b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55612593c090 con 0x5561259d2480 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996677c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41918/0 (socket says 172.21.15.120:41918) 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 -- 172.21.15.120:0/1917437757 learned_addr learned my addr 172.21.15.120:0/1917437757 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:51.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996677c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55254/0 (socket says 172.21.15.120:55254) 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996577a640 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 -- 172.21.15.120:0/1917437757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 msgr2=0x5561259d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 -- 172.21.15.120:0/1917437757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 msgr2=0x5561259e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9965f7b640 1 -- 172.21.15.120:0/1917437757 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556125696650 con 0x5561259d2da0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f996677c640 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.849+0000 7f9956ffd640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 0x7f994c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f996677c640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 0x7f994c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f995c0114f0 tx=0x7f995c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7122cee803f2435b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f9956ffd640 1 -- 172.21.15.120:0/1195397318 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9950106120 con 0x7f9960104550 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f994bfff640 1 -- 172.21.15.120:0/1917437757 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c01e070 con 0x5561259d2da0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f994bfff640 1 -- 172.21.15.120:0/1917437757 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c00fad0 con 0x5561259d2da0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.850+0000 7f994bfff640 1 -- 172.21.15.120:0/1917437757 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c019ab0 con 0x5561259d2da0 2024-09-10T23:05:51.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- 172.21.15.120:0/1917437757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 msgr2=0x5561259de7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f995c0114f0 tx=0x7f995c0119b0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996677c640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 0x7f994c0b7820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9958009f70 tx=0x7f9958005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- 172.21.15.120:0/1917437757 shutdown_connections 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259de7f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1917437757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2480 0x5561259d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- 172.21.15.120:0/1917437757 >> 172.21.15.120:0/1917437757 conn(0x5561259d02b0 msgr2=0x5561259d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- 172.21.15.120:0/1917437757 shutdown_connections 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- 172.21.15.120:0/1917437757 wait complete. 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 -- start start 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.851+0000 7f996a5b4740 1 Processor -- start 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 0x5561259dbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556125799700 con 0x5561259ded30 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5561258d3110 con 0x5561259d2480 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5561259e19b0 con 0x5561259d2da0 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996a5b4740 10 client.0 _ll_register_callbacks cb 0x5561258ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35764/0 (socket says 172.21.15.120:35764) 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 -- 172.21.15.120:0/4032264705 learned_addr learned my addr 172.21.15.120:0/4032264705 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996577a640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996677c640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 0x5561259dbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 -- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 msgr2=0x5561259dba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259dba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 -- 172.21.15.120:0/4032264705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 msgr2=0x5561259dbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 0x5561259dbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f9965f7b640 1 -- 172.21.15.120:0/4032264705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556125900c40 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.852+0000 7f996677c640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 0x5561259dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f9965f7b640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f995c0099d0 tx=0x7f995c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6561df49638e9656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f99497fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c029060 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c00fad0 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f995c00f8b0 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f995c01e070 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f996a5b4740 10 client.27380 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.853+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561259ed2d0 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c00fad0 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f995c0050d0 con 0x5561259d2480 2024-09-10T23:05:51.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 10 client.27380 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 10 client.27380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 20 client.27380 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5561259ee620 0x5561259f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 4 client.27380 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612594ada0 con 0x5561259ee620 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996577a640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5561259ee620 0x5561259f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f99497fa640 10 client.27380 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.854+0000 7f996577a640 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5561259ee620 0x5561259f0a10 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11f50d7f96f4d6f server_cookie=9ff5b36bf7d890d7 in_seq=0 out_seq=0 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f99497fa640 1 -- 172.21.15.120:0/4032264705 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55612594ada0 con 0x5561259ee620 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f99497fa640 10 client.27380 handle_command_reply: tid=0 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 client.27380 shutdown 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 msgr2=0x5561259de5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f995c0099d0 tx=0x7f995c009310 comp rx=0 tx=0).stop 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 shutdown_connections 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5561259ee620 0x5561259f0a10 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259ded30 0x5561259dbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 --2- 172.21.15.120:0/4032264705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259de5e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f99497fa640 0 client.27380 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 >> 172.21.15.120:0/4032264705 conn(0x5561259d02b0 msgr2=0x5561259d5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 shutdown_connections 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 1 -- 172.21.15.120:0/4032264705 wait complete. 2024-09-10T23:05:51.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.857+0000 7f996a5b4740 20 client.27380 trim_cache size 0 max 16384 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.875+0000 7f996a5b4740 1 Processor -- start 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 -- start start 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99601ac270 con 0x5561259d2480 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99601aeaf0 con 0x5561259ded30 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996a5b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99601ac8e0 con 0x5561259d2da0 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41948/0 (socket says 172.21.15.120:41948) 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996577a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.878 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 -- 172.21.15.120:0/1234296988 learned_addr learned my addr 172.21.15.120:0/1234296988 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996677c640 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 -- 172.21.15.120:0/1234296988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 msgr2=0x5561259cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 -- 172.21.15.120:0/1234296988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 msgr2=0x5561259e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 -- 172.21.15.120:0/1234296988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561259db7e0 con 0x5561259d2480 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f996577a640 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.876+0000 7f9965f7b640 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f995c011f30 tx=0x7f995c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498e3e1a0ce69390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f994b7fe640 1 -- 172.21.15.120:0/1234296988 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c029040 con 0x5561259d2480 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f994b7fe640 1 -- 172.21.15.120:0/1234296988 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c00fab0 con 0x5561259d2480 2024-09-10T23:05:51.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f994b7fe640 1 -- 172.21.15.120:0/1234296988 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c019880 con 0x5561259d2480 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 -- 172.21.15.120:0/1234296988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 msgr2=0x5561259eac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f995c011f30 tx=0x7f995c009480 comp rx=0 tx=0).stop 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 -- 172.21.15.120:0/1234296988 shutdown_connections 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259ded30 0x5561259e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259d2da0 0x5561259cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 --2- 172.21.15.120:0/1234296988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2480 0x5561259eac10 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 -- 172.21.15.120:0/1234296988 >> 172.21.15.120:0/1234296988 conn(0x5561259d02b0 msgr2=0x5561259cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.877+0000 7f996a5b4740 1 -- 172.21.15.120:0/1234296988 shutdown_connections 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 -- 172.21.15.120:0/1234296988 wait complete. 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 -- start start 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 Processor -- start 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259ded30 0x5561259e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99601acfb0 con 0x5561259d2da0 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99601ae420 con 0x5561259d2480 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99601ad680 con 0x5561259ded30 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996a5b4740 10 client.0 _ll_register_callbacks cb 0x7f995c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996577a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996577a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41954/0 (socket says 172.21.15.120:41954) 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.878+0000 7f996577a640 1 -- 172.21.15.120:0/3398353733 learned_addr learned my addr 172.21.15.120:0/3398353733 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f9965f7b640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996677c640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259ded30 0x5561259e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 -- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259ded30 msgr2=0x5561259e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259ded30 0x5561259e1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 -- 172.21.15.120:0/3398353733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 msgr2=0x5561259e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 -- 172.21.15.120:0/3398353733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561259e21c0 con 0x5561259d2da0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f9965f7b640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996577a640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9950103c50 tx=0x7f9950109c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cc22d5e4d8fc93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f99497fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950049040 con 0x5561259d2da0 2024-09-10T23:05:51.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f99500c8c30 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c00fab0 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f99500c8130 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996a5b4740 10 client.18111 fetch_fsmap learned FSMap version 26 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556125a5bc10 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.879+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c00fab0 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f99500ef040 con 0x5561259d2da0 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 10 client.18111 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 20 client.18111 populate_metadata read hostname 'smithi120' 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556125a5c370 0x556125a5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 4 client.18111 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:05:51.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556125946850 con 0x556125a5c370 2024-09-10T23:05:51.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f9965f7b640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556125a5c370 0x556125a5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:05:51.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f99497fa640 10 client.18111 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:51.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.880+0000 7f9965f7b640 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556125a5c370 0x556125a5e790 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc76bd8fdd5e8f5 server_cookie=7966535c743e3302 in_seq=0 out_seq=0 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f99497fa640 1 -- 172.21.15.120:0/3398353733 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7102 (crc 0 0 0) 0x556125946850 con 0x556125a5c370 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f99497fa640 10 client.18111 handle_command_reply: tid=0 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 client.18111 shutdown 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 msgr2=0x5561259e71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9950103c50 tx=0x7f9950109c00 comp rx=0 tx=0).stop 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 shutdown_connections 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556125a5c370 0x556125a5e790 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5561259ded30 0x5561259e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561259d2da0 0x5561259e71c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f99497fa640 0 client.18111 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 --2- 172.21.15.120:0/3398353733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561259d2480 0x5561259e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 >> 172.21.15.120:0/3398353733 conn(0x5561259d02b0 msgr2=0x5561259cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 shutdown_connections 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 1 -- 172.21.15.120:0/3398353733 wait complete. 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.883+0000 7f996a5b4740 20 client.18111 trim_cache size 0 max 16384 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:05:51.888 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "010375e9-ff31-4fd9-8442-d91c1ce255e0", 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:05:51.892 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.893 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:05:51.896 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.897 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.900 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.901 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:05:51.902 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 msgr2=0x7f994c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 0x7f994c0b7820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9958009f70 tx=0x7f9958005f90 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 msgr2=0x7f9960199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9950005940 tx=0x7f99500040c0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 shutdown_connections 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f994c0b5360 0x7f994c0b7820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f996010d100 0x7f996019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9960104550 0x7f9960199770 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 --2- 172.21.15.120:0/1195397318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9960103b50 0x7f9960199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 >> 172.21.15.120:0/1195397318 conn(0x7f99600fd700 msgr2=0x7f99600fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 shutdown_connections 2024-09-10T23:05:51.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:05:51.884+0000 7f99681df640 1 -- 172.21.15.120:0/1195397318 wait complete. 2024-09-10T23:05:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:52 smithi120 ceph-mon[31247]: pgmap v749: 97 pgs: 97 active+clean; 244 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 563 op/s 2024-09-10T23:05:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:52 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:52 smithi153 ceph-mon[34951]: pgmap v749: 97 pgs: 97 active+clean; 244 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 563 op/s 2024-09-10T23:05:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:52 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:52 smithi176 ceph-mon[34982]: pgmap v749: 97 pgs: 97 active+clean; 244 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 563 op/s 2024-09-10T23:05:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:52 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:05:52.661 INFO:tasks.cephfs.filesystem:scrub status for tag:010375e9-ff31-4fd9-8442-d91c1ce255e0 - {'path': '/', 'tag': '010375e9-ff31-4fd9-8442-d91c1ce255e0', 'options': 'recursive,force'} 2024-09-10T23:05:52.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 20661 27.21 MB/sec warmup 47 sec latency 57.866 ms 2024-09-10T23:05:53.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 21243 27.71 MB/sec warmup 48 sec latency 43.108 ms 2024-09-10T23:05:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:54 smithi120 ceph-mon[31247]: pgmap v750: 97 pgs: 97 active+clean; 237 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-09-10T23:05:54.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:54 smithi153 ceph-mon[34951]: pgmap v750: 97 pgs: 97 active+clean; 237 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-09-10T23:05:54.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:54 smithi176 ceph-mon[34982]: pgmap v750: 97 pgs: 97 active+clean; 237 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-09-10T23:05:54.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 21515 27.45 MB/sec warmup 49 sec latency 163.045 ms 2024-09-10T23:05:55.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:05:55.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:05:55.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:05:55.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 21750 27.11 MB/sec warmup 50 sec latency 84.878 ms 2024-09-10T23:05:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:56 smithi120 ceph-mon[31247]: pgmap v751: 97 pgs: 97 active+clean; 238 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 628 op/s 2024-09-10T23:05:56.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:56 smithi153 ceph-mon[34951]: pgmap v751: 97 pgs: 97 active+clean; 238 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 628 op/s 2024-09-10T23:05:56.576 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:56 smithi176 ceph-mon[34982]: pgmap v751: 97 pgs: 97 active+clean; 238 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 628 op/s 2024-09-10T23:05:56.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 22018 26.69 MB/sec warmup 51 sec latency 74.074 ms 2024-09-10T23:05:57.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 22427 26.44 MB/sec warmup 52 sec latency 85.561 ms 2024-09-10T23:05:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:05:58 smithi120 ceph-mon[31247]: pgmap v752: 97 pgs: 97 active+clean; 242 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:05:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:05:58 smithi153 ceph-mon[34951]: pgmap v752: 97 pgs: 97 active+clean; 242 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:05:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:05:58 smithi176 ceph-mon[34982]: pgmap v752: 97 pgs: 97 active+clean; 242 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:05:58.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 23190 26.61 MB/sec warmup 53 sec latency 42.061 ms 2024-09-10T23:05:59.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 23527 26.39 MB/sec warmup 54 sec latency 251.931 ms 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- 172.21.15.120:0/2835691654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554101860 msgr2=0x7fb554101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/2835691654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554101860 0x7fb554101c60 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb548009930 tx=0x7fb54802a420 comp rx=0 tx=0).stop 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- 172.21.15.120:0/2835691654 shutdown_connections 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/2835691654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb554103370 0x7fb554109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/2835691654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5541029b0 0x7fb554102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/2835691654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554101860 0x7fb554101c60 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- 172.21.15.120:0/2835691654 >> 172.21.15.120:0/2835691654 conn(0x7fb5540fd120 msgr2=0x7fb5540ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- 172.21.15.120:0/2835691654 shutdown_connections 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- 172.21.15.120:0/2835691654 wait complete. 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 Processor -- start 2024-09-10T23:06:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.096+0000 7fb55ab7d640 1 -- start start 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 0x7fb554197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5541029b0 0x7fb5541982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410d4b0 con 0x7fb554103370 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410d330 con 0x7fb554101860 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410d630 con 0x7fb5541029b0 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55911a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55911a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55512/0 (socket says 172.21.15.120:55512) 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55911a640 1 -- 172.21.15.120:0/3387009989 learned_addr learned my addr 172.21.15.120:0/3387009989 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 0x7fb554197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb553fff640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5541029b0 0x7fb5541982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 -- 172.21.15.120:0/3387009989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5541029b0 msgr2=0x7fb5541982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5541029b0 0x7fb5541982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 -- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 msgr2=0x7fb554198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 -- 172.21.15.120:0/3387009989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5541997a0 con 0x7fb554101860 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55911a640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb558919640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 0x7fb554197c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb548009230 tx=0x7fb5480042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb551ffb640 1 -- 172.21.15.120:0/3387009989 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54803f030 con 0x7fb554101860 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.097+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb554199a30 con 0x7fb554101860 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.098+0000 7fb551ffb640 1 -- 172.21.15.120:0/3387009989 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb548031910 con 0x7fb554101860 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.098+0000 7fb551ffb640 1 -- 172.21.15.120:0/3387009989 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548035070 con 0x7fb554101860 2024-09-10T23:06:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.098+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55406b550 con 0x7fb554101860 2024-09-10T23:06:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 Processor -- start 2024-09-10T23:06:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 -- start start 2024-09-10T23:06:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 0x5608fe54dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb551ffb640 1 -- 172.21.15.120:0/3387009989 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb54803b9c0 con 0x7fb554101860 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 0x5608fe553090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608fe4bd6f0 con 0x5608fe54e190 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608fe4acb20 con 0x5608fe550bd0 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55da24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5608fe4b0930 con 0x5608fe54b860 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55524/0 (socket says 172.21.15.120:55524) 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb558919640 1 -- 172.21.15.120:0/3127636271 learned_addr learned my addr 172.21.15.120:0/3127636271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb55911a640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 0x5608fe54dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.100+0000 7fb553fff640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 0x5608fe553090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 -- 172.21.15.120:0/3127636271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 msgr2=0x5608fe54dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 0x5608fe54dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 -- 172.21.15.120:0/3127636271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 msgr2=0x5608fe550580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 -- 172.21.15.120:0/3127636271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fe472530 con 0x5608fe550bd0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb55911a640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 0x5608fe54dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb558919640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb553fff640 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 0x5608fe553090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb540009c20 tx=0x7fb540011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9f78e374f8d11b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb551ffb640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 0x7fb5440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb53effd640 1 -- 172.21.15.120:0/3127636271 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540004050 con 0x5608fe550bd0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb53effd640 1 -- 172.21.15.120:0/3127636271 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5400054d0 con 0x5608fe550bd0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb53effd640 1 -- 172.21.15.120:0/3127636271 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540004ba0 con 0x5608fe550bd0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb55911a640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 0x7fb5440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.101+0000 7fb551ffb640 1 -- 172.21.15.120:0/3387009989 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb5481062c0 con 0x7fb554101860 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55911a640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 0x7fb5440b77a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb5541989e0 tx=0x7fb54c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- 172.21.15.120:0/3127636271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 msgr2=0x5608fe553090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 0x5608fe553090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb540009c20 tx=0x7fb540011810 comp rx=0 tx=0).stop 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- 172.21.15.120:0/3127636271 shutdown_connections 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe550bd0 0x5608fe553090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe550580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 --2- 172.21.15.120:0/3127636271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe54b860 0x5608fe54dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- 172.21.15.120:0/3127636271 >> 172.21.15.120:0/3127636271 conn(0x5608fe5469b0 msgr2=0x5608fe547740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- 172.21.15.120:0/3127636271 shutdown_connections 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- 172.21.15.120:0/3127636271 wait complete. 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 -- start start 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.102+0000 7fb55da24740 1 Processor -- start 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe555530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe556100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608fe3e4ab0 con 0x5608fe54b860 2024-09-10T23:06:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5608fe452730 con 0x5608fe54e190 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5608fe553bb0 con 0x5608fe550bd0 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 10 client.0 _ll_register_callbacks cb 0x5608fe4a66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55528/0 (socket says 172.21.15.120:55528) 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 -- 172.21.15.120:0/2604864778 learned_addr learned my addr 172.21.15.120:0/2604864778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb553fff640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe555530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55911a640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe556100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 -- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 msgr2=0x5608fe556100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe556100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 -- 172.21.15.120:0/2604864778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 msgr2=0x5608fe555530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe555530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 -- 172.21.15.120:0/2604864778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fe20e790 con 0x5608fe54b860 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb553fff640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe555530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb558919640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ccaf0 tx=0x7fb548005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a5b18709c896a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb533fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548049070 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5400054d0 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.103+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb5480d5db0 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5480c91b0 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 10 client.18123 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608fe287620 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5400054d0 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb5480b5020 con 0x5608fe54b860 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 10 client.18123 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 10 client.18123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 20 client.18123 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe560760 0x5608fe562be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 4 client.18123 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608fe4bf640 con 0x5608fe560760 2024-09-10T23:06:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.104+0000 7fb553fff640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe560760 0x5608fe562be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.105+0000 7fb533fff640 10 client.18123 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.105+0000 7fb553fff640 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe560760 0x5608fe562be0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbe410d144c3ac2 server_cookie=fc647241db84a21b in_seq=0 out_seq=0 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb533fff640 1 -- 172.21.15.120:0/2604864778 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608fe4bf640 con 0x5608fe560760 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb533fff640 10 client.18123 handle_command_reply: tid=0 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 client.18123 shutdown 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 msgr2=0x5608fe554e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ccaf0 tx=0x7fb548005e00 comp rx=0 tx=0).stop 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 shutdown_connections 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe560760 0x5608fe562be0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe556100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb533fff640 0 client.18123 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe555530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 --2- 172.21.15.120:0/2604864778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe554e90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 >> 172.21.15.120:0/2604864778 conn(0x5608fe5469b0 msgr2=0x5608fe545bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 shutdown_connections 2024-09-10T23:06:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 1 -- 172.21.15.120:0/2604864778 wait complete. 2024-09-10T23:06:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.106+0000 7fb55da24740 20 client.18123 trim_cache size 0 max 16384 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 Processor -- start 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 -- start start 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe550fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe5514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a5c30 con 0x5608fe54b860 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a84b0 con 0x5608fe54e190 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55da24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a62a0 con 0x5608fe550bd0 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55532/0 (socket says 172.21.15.120:55532) 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 -- 172.21.15.120:0/594228469 learned_addr learned my addr 172.21.15.120:0/594228469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb55911a640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe5514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb553fff640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe550fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 -- 172.21.15.120:0/594228469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 msgr2=0x5608fe5514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe5514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 -- 172.21.15.120:0/594228469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 msgr2=0x5608fe550fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe550fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 -- 172.21.15.120:0/594228469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fe556470 con 0x5608fe54b860 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb553fff640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe550fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.123+0000 7fb558919640 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb5480cc860 tx=0x7fb548106670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b43734c0587992 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb53e7fc640 1 -- 172.21.15.120:0/594228469 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5480f3070 con 0x5608fe54b860 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb53e7fc640 1 -- 172.21.15.120:0/594228469 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5480d1590 con 0x5608fe54b860 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb53e7fc640 1 -- 172.21.15.120:0/594228469 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548049070 con 0x5608fe54b860 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 -- 172.21.15.120:0/594228469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 msgr2=0x5608fe553fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb5480cc860 tx=0x7fb548106670 comp rx=0 tx=0).stop 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 -- 172.21.15.120:0/594228469 shutdown_connections 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe5514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54e190 0x5608fe550fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 --2- 172.21.15.120:0/594228469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54b860 0x5608fe553fb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 -- 172.21.15.120:0/594228469 >> 172.21.15.120:0/594228469 conn(0x5608fe5469b0 msgr2=0x5608fe554a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 -- 172.21.15.120:0/594228469 shutdown_connections 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.124+0000 7fb55da24740 1 -- 172.21.15.120:0/594228469 wait complete. 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 -- start start 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 Processor -- start 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 0x5608fe55c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe55b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a6970 con 0x5608fe54e190 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a7de0 con 0x5608fe54b860 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541a7040 con 0x5608fe550bd0 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55da24740 10 client.0 _ll_register_callbacks cb 0x7fb548009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb553fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb553fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55548/0 (socket says 172.21.15.120:55548) 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb553fff640 1 -- 172.21.15.120:0/472775174 learned_addr learned my addr 172.21.15.120:0/472775174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb55911a640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe55b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.125+0000 7fb558919640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 0x5608fe55c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 -- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 msgr2=0x5608fe55b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe55b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 -- 172.21.15.120:0/472775174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 msgr2=0x5608fe55c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 0x5608fe55c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 -- 172.21.15.120:0/472775174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fe55bac0 con 0x5608fe54e190 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb558919640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 0x5608fe55c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb553fff640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb540004290 tx=0x7fb540004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4089f0f4be2e0cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540011d10 con 0x5608fe54e190 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb540005020 con 0x5608fe54e190 2024-09-10T23:06:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5480d1590 con 0x5608fe54e190 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb540005220 con 0x5608fe54e190 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 10 client.18129 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608fe55a8e0 con 0x5608fe54e190 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5480d1590 con 0x5608fe54e190 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb54002d070 con 0x5608fe54e190 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 10 client.18129 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 10 client.18129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 20 client.18129 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe5cdb20 0x5608fe5cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 4 client.18129 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.126+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608fe4bb0f0 con 0x5608fe5cdb20 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.127+0000 7fb558919640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe5cdb20 0x5608fe5cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.127+0000 7fb558919640 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe5cdb20 0x5608fe5cff40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21523444b282cec7 server_cookie=5cda09ceea0d28ad in_seq=0 out_seq=0 2024-09-10T23:06:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.127+0000 7fb533fff640 10 client.18129 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:00 smithi120 ceph-mon[31247]: pgmap v753: 97 pgs: 97 active+clean; 247 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 645 op/s 2024-09-10T23:06:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:00 smithi153 ceph-mon[34951]: pgmap v753: 97 pgs: 97 active+clean; 247 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 645 op/s 2024-09-10T23:06:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:00 smithi176 ceph-mon[34982]: pgmap v753: 97 pgs: 97 active+clean; 247 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 645 op/s 2024-09-10T23:06:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb533fff640 1 -- 172.21.15.120:0/472775174 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608fe4bb0f0 con 0x5608fe5cdb20 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb533fff640 10 client.18129 handle_command_reply: tid=0 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 client.18129 shutdown 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 msgr2=0x5608fe55cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb540004290 tx=0x7fb540004070 comp rx=0 tx=0).stop 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 shutdown_connections 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5608fe5cdb20 0x5608fe5cff40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5608fe550bd0 0x5608fe55b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb533fff640 0 client.18129 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608fe54e190 0x5608fe55cb90 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 --2- 172.21.15.120:0/472775174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608fe54b860 0x5608fe55c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 >> 172.21.15.120:0/472775174 conn(0x5608fe5469b0 msgr2=0x5608fe553fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 shutdown_connections 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 1 -- 172.21.15.120:0/472775174 wait complete. 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.748+0000 7fb55da24740 20 client.18129 trim_cache size 0 max 16384 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:06:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 msgr2=0x7fb5440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 0x7fb5440b77a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb5541989e0 tx=0x7fb54c010040 comp rx=0 tx=0).stop 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 msgr2=0x7fb554197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 0x7fb554197c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb548009230 tx=0x7fb5480042f0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 shutdown_connections 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb5440b52e0 0x7fb5440b77a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb554103370 0x7fb554198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5541029b0 0x7fb5541982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 --2- 172.21.15.120:0/3387009989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb554101860 0x7fb554197c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 >> 172.21.15.120:0/3387009989 conn(0x7fb5540fd120 msgr2=0x7fb554103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 shutdown_connections 2024-09-10T23:06:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:00.749+0000 7fb55ab7d640 1 -- 172.21.15.120:0/3387009989 wait complete. 2024-09-10T23:06:00.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 23919 26.38 MB/sec warmup 55 sec latency 321.791 ms 2024-09-10T23:06:01.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 24627 26.91 MB/sec warmup 56 sec latency 33.978 ms 2024-09-10T23:06:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:02 smithi120 ceph-mon[31247]: pgmap v754: 97 pgs: 97 active+clean; 256 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-10T23:06:02.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:02 smithi153 ceph-mon[34951]: pgmap v754: 97 pgs: 97 active+clean; 256 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-10T23:06:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:02 smithi176 ceph-mon[34982]: pgmap v754: 97 pgs: 97 active+clean; 256 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-09-10T23:06:02.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 25019 26.94 MB/sec warmup 57 sec latency 62.510 ms 2024-09-10T23:06:03.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 25280 26.67 MB/sec warmup 58 sec latency 97.192 ms 2024-09-10T23:06:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:04 smithi120 ceph-mon[31247]: pgmap v755: 97 pgs: 97 active+clean; 250 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 636 op/s 2024-09-10T23:06:04.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:04 smithi153 ceph-mon[34951]: pgmap v755: 97 pgs: 97 active+clean; 250 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 636 op/s 2024-09-10T23:06:04.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:04 smithi176 ceph-mon[34982]: pgmap v755: 97 pgs: 97 active+clean; 250 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 636 op/s 2024-09-10T23:06:04.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 25449 26.28 MB/sec warmup 59 sec latency 196.625 ms 2024-09-10T23:06:05.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 25767 26.04 MB/sec warmup 60 sec latency 117.665 ms 2024-09-10T23:06:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:06 smithi120 ceph-mon[31247]: pgmap v756: 97 pgs: 97 active+clean; 244 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-09-10T23:06:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:06 smithi153 ceph-mon[34951]: pgmap v756: 97 pgs: 97 active+clean; 244 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-09-10T23:06:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:06 smithi176 ceph-mon[34982]: pgmap v756: 97 pgs: 97 active+clean; 244 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-09-10T23:06:06.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 26253 25.93 MB/sec warmup 61 sec latency 51.306 ms 2024-09-10T23:06:07.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 26898 25.98 MB/sec warmup 62 sec latency 54.225 ms 2024-09-10T23:06:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:08 smithi120 ceph-mon[31247]: pgmap v757: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-10T23:06:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:08 smithi153 ceph-mon[34951]: pgmap v757: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-10T23:06:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:08 smithi176 ceph-mon[34982]: pgmap v757: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-10T23:06:08.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 27610 26.22 MB/sec warmup 63 sec latency 44.181 ms 2024-09-10T23:06:09.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28077 26.35 MB/sec warmup 64 sec latency 174.639 ms 2024-09-10T23:06:10.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:10 smithi120 ceph-mon[31247]: pgmap v758: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-09-10T23:06:10.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:10 smithi153 ceph-mon[34951]: pgmap v758: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-09-10T23:06:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:10 smithi176 ceph-mon[34982]: pgmap v758: 97 pgs: 97 active+clean; 239 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-09-10T23:06:10.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28469 26.42 MB/sec warmup 65 sec latency 84.586 ms 2024-09-10T23:06:11.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 28769 26.30 MB/sec warmup 66 sec latency 79.122 ms 2024-09-10T23:06:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:12 smithi120 ceph-mon[31247]: pgmap v759: 97 pgs: 97 active+clean; 257 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T23:06:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:12 smithi153 ceph-mon[34951]: pgmap v759: 97 pgs: 97 active+clean; 257 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T23:06:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:12 smithi176 ceph-mon[34982]: pgmap v759: 97 pgs: 97 active+clean; 257 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T23:06:12.866 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29021 26.01 MB/sec warmup 67 sec latency 119.655 ms 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 -- 172.21.15.120:0/400709077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 msgr2=0x7f894c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 --2- 172.21.15.120:0/400709077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c109cf0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f893c009cd0 tx=0x7f893c02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 -- 172.21.15.120:0/400709077 shutdown_connections 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 --2- 172.21.15.120:0/400709077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c109cf0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 --2- 172.21.15.120:0/400709077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c1029b0 0x7f894c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 --2- 172.21.15.120:0/400709077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c101860 0x7f894c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 -- 172.21.15.120:0/400709077 >> 172.21.15.120:0/400709077 conn(0x7f894c0fd140 msgr2=0x7f894c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 -- 172.21.15.120:0/400709077 shutdown_connections 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 -- 172.21.15.120:0/400709077 wait complete. 2024-09-10T23:06:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.843+0000 7f8953333640 1 Processor -- start 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 -- start start 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c101860 0x7f894c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 0x7f894c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10d4d0 con 0x7f894c103370 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10d350 con 0x7f894c101860 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f8953333640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10d650 con 0x7f894c1029b0 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89508ce640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 0x7f894c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44784/0 (socket says 172.21.15.120:44784) 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 -- 172.21.15.120:0/3649492663 learned_addr learned my addr 172.21.15.120:0/3649492663 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89510cf640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c101860 0x7f894c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 -- 172.21.15.120:0/3649492663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 msgr2=0x7f894c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 0x7f894c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 -- 172.21.15.120:0/3649492663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c101860 msgr2=0x7f894c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c101860 0x7f894c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 -- 172.21.15.120:0/3649492663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c19a870 con 0x7f894c103370 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89508ce640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 0x7f894c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f89518d0640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f893c033040 tx=0x7f893c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f894a7fc640 1 -- 172.21.15.120:0/3649492663 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c004240 con 0x7f894c103370 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.844+0000 7f894a7fc640 1 -- 172.21.15.120:0/3649492663 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893c0305a0 con 0x7f894c103370 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.845+0000 7f894a7fc640 1 -- 172.21.15.120:0/3649492663 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0466e0 con 0x7f894c103370 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.845+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c19ab00 con 0x7f894c103370 2024-09-10T23:06:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.845+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c1a1230 con 0x7f894c103370 2024-09-10T23:06:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 Processor -- start 2024-09-10T23:06:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f894a7fc640 1 -- 172.21.15.120:0/3649492663 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f893c04e070 con 0x7f894c103370 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 -- start start 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 0x55f0c9520c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9526090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0c94906f0 con 0x55f0c951e860 2024-09-10T23:06:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0c947fb20 con 0x55f0c9523bd0 2024-09-10T23:06:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.847+0000 7f89561da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9483930 con 0x55f0c9521190 2024-09-10T23:06:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 0x55f0c9520c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34164/0 (socket says 172.21.15.120:34164) 2024-09-10T23:06:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 -- 172.21.15.120:0/779107449 learned_addr learned my addr 172.21.15.120:0/779107449 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89508ce640 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9526090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 -- 172.21.15.120:0/779107449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 msgr2=0x55f0c9526090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9526090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 -- 172.21.15.120:0/779107449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 msgr2=0x55f0c9520c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 0x55f0c9520c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 -- 172.21.15.120:0/779107449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c9445530 con 0x55f0c9521190 2024-09-10T23:06:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89508ce640 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 0x55f0c9520c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f894a7fc640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 0x7f89340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89510cf640 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8944009d20 tx=0x7f8944011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcfc0c359b327c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89508ce640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 0x7f89340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f894a7fc640 1 -- 172.21.15.120:0/3649492663 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f893c02e030 con 0x7f894c103370 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89337fe640 1 -- 172.21.15.120:0/779107449 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894401e070 con 0x55f0c9521190 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89337fe640 1 -- 172.21.15.120:0/779107449 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8944019ae0 con 0x55f0c9521190 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.848+0000 7f89337fe640 1 -- 172.21.15.120:0/779107449 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894400ea10 con 0x55f0c9521190 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- 172.21.15.120:0/779107449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 msgr2=0x55f0c9523580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89508ce640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 0x7f89340b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f894000bfd0 tx=0x7f894000bb20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8944009d20 tx=0x7f8944011810 comp rx=0 tx=0).stop 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- 172.21.15.120:0/779107449 shutdown_connections 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9526090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9521190 0x55f0c9523580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 --2- 172.21.15.120:0/779107449 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c951e860 0x55f0c9520c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- 172.21.15.120:0/779107449 >> 172.21.15.120:0/779107449 conn(0x55f0c95199b0 msgr2=0x55f0c951a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- 172.21.15.120:0/779107449 shutdown_connections 2024-09-10T23:06:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- 172.21.15.120:0/779107449 wait complete. 2024-09-10T23:06:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.849+0000 7f89561da740 1 -- start start 2024-09-10T23:06:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 Processor -- start 2024-09-10T23:06:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c9528560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9529130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0c93b7ab0 con 0x55f0c9521190 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9425730 con 0x55f0c9523bd0 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9526bb0 con 0x55f0c951e860 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89561da740 10 client.0 _ll_register_callbacks cb 0x55f0c94796b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c9528560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c9528560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44790/0 (socket says 172.21.15.120:44790) 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34174/0 (socket says 172.21.15.120:34174) 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89508ce640 1 -- 172.21.15.120:0/2632272761 learned_addr learned my addr 172.21.15.120:0/2632272761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89518d0640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9529130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 -- 172.21.15.120:0/2632272761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 msgr2=0x55f0c9529130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9529130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 -- 172.21.15.120:0/2632272761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 msgr2=0x55f0c9528560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c9528560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.850+0000 7f89510cf640 1 -- 172.21.15.120:0/2632272761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c91e1790 con 0x55f0c951e860 2024-09-10T23:06:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89510cf640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8944005890 tx=0x7f8944004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d732261ade024208 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894402f060 con 0x55f0c951e860 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8944019ae0 con 0x55f0c951e860 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8944019430 con 0x55f0c951e860 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89440098f0 con 0x55f0c951e860 2024-09-10T23:06:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 10 client.27610 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0c925a620 con 0x55f0c951e860 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8944019ae0 con 0x55f0c951e860 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8944003710 con 0x55f0c951e860 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 10 client.27610 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 10 client.27610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 20 client.27610 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 4 client.27610 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.851+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c9492640 con 0x55f0c95334b0 2024-09-10T23:06:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.852+0000 7f89508ce640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.852+0000 7f89508ce640 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95358f0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f1d86ff7ae97b server_cookie=9dfa829ddb430943 in_seq=0 out_seq=0 2024-09-10T23:06:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.852+0000 7f8930ff9640 10 client.27610 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.853+0000 7f8930ff9640 1 -- 172.21.15.120:0/2632272761 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0c9492640 con 0x55f0c95334b0 2024-09-10T23:06:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.853+0000 7f8930ff9640 10 client.27610 handle_command_reply: tid=0 2024-09-10T23:06:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.853+0000 7f89561da740 1 client.27610 shutdown 2024-09-10T23:06:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.853+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 msgr2=0x55f0c9527ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.853+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8944005890 tx=0x7f8944004270 comp rx=0 tx=0).stop 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 shutdown_connections 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95358f0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f8930ff9640 0 client.27610 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c9529130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c9528560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 --2- 172.21.15.120:0/2632272761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c9527ec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 >> 172.21.15.120:0/2632272761 conn(0x55f0c95199b0 msgr2=0x55f0c9518bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 shutdown_connections 2024-09-10T23:06:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 1 -- 172.21.15.120:0/2632272761 wait complete. 2024-09-10T23:06:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.854+0000 7f89561da740 20 client.27610 trim_cache size 0 max 16384 2024-09-10T23:06:13.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29396 25.80 MB/sec warmup 68 sec latency 75.320 ms 2024-09-10T23:06:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.869+0000 7f89561da740 1 Processor -- start 2024-09-10T23:06:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.869+0000 7f89561da740 1 -- start start 2024-09-10T23:06:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.869+0000 7f89561da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c951e860 0x55f0c9529110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.869+0000 7f89561da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89561da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9523bd0 0x55f0c9526fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89561da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a5ad0 con 0x55f0c9521190 2024-09-10T23:06:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89561da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a8350 con 0x55f0c951e860 2024-09-10T23:06:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89561da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a6140 con 0x55f0c9523bd0 2024-09-10T23:06:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44806/0 (socket says 172.21.15.120:44806) 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 -- 172.21.15.120:0/267493725 learned_addr learned my addr 172.21.15.120:0/267493725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89518d0640 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9523bd0 0x55f0c9526fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89510cf640 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c951e860 0x55f0c9529110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 -- 172.21.15.120:0/267493725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9523bd0 msgr2=0x55f0c9526fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9523bd0 0x55f0c9526fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 -- 172.21.15.120:0/267493725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c951e860 msgr2=0x55f0c9529110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c951e860 0x55f0c9529110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 -- 172.21.15.120:0/267493725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c95289c0 con 0x55f0c9521190 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f89508ce640 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f894000bec0 tx=0x7f89400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c39465af7904c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f8932ffd640 1 -- 172.21.15.120:0/267493725 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400229a0 con 0x55f0c9521190 2024-09-10T23:06:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f8932ffd640 1 -- 172.21.15.120:0/267493725 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8940005020 con 0x55f0c9521190 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.870+0000 7f8932ffd640 1 -- 172.21.15.120:0/267493725 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940004890 con 0x55f0c9521190 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- 172.21.15.120:0/267493725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 msgr2=0x55f0c951e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f894000bec0 tx=0x7f89400046c0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- 172.21.15.120:0/267493725 shutdown_connections 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c9523bd0 0x55f0c9526fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c951e2a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 --2- 172.21.15.120:0/267493725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c951e860 0x55f0c9529110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- 172.21.15.120:0/267493725 >> 172.21.15.120:0/267493725 conn(0x55f0c95199b0 msgr2=0x55f0c951db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- 172.21.15.120:0/267493725 shutdown_connections 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- 172.21.15.120:0/267493725 wait complete. 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 -- start start 2024-09-10T23:06:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.871+0000 7f89561da740 1 Processor -- start 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c952cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c952e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a6810 con 0x55f0c9521190 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a7c80 con 0x55f0c9523bd0 2024-09-10T23:06:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a6ee0 con 0x55f0c951e860 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89561da740 10 client.0 _ll_register_callbacks cb 0x7f894401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c952cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89510cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c952cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34196/0 (socket says 172.21.15.120:34196) 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89510cf640 1 -- 172.21.15.120:0/3763602481 learned_addr learned my addr 172.21.15.120:0/3763602481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44814/0 (socket says 172.21.15.120:44814) 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89518d0640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c952e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 -- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 msgr2=0x55f0c952cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c952cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 -- 172.21.15.120:0/3763602481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 msgr2=0x55f0c952e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c952e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 -- 172.21.15.120:0/3763602481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c952e970 con 0x55f0c9521190 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f89508ce640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8940024040 tx=0x7f894000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2199630934405fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.872+0000 7f8930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940005290 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8940005430 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8940005020 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89400229a0 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 10 client.18147 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0c952fce0 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8940005020 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f894002b070 con 0x55f0c9521190 2024-09-10T23:06:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 10 client.18147 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 10 client.18147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 20 client.18147 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 4 client.18147 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0c948e0f0 con 0x55f0c95334b0 2024-09-10T23:06:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.873+0000 7f89510cf640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.874+0000 7f8930ff9640 10 client.18147 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:13.874+0000 7f89510cf640 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95a2890 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b0d2c020f67995 server_cookie=a1ee5055c201ac81 in_seq=0 out_seq=0 2024-09-10T23:06:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:14 smithi120 ceph-mon[31247]: pgmap v760: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-10T23:06:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:14 smithi153 ceph-mon[34951]: pgmap v760: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-10T23:06:14.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:14 smithi176 ceph-mon[34982]: pgmap v760: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-09-10T23:06:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f8930ff9640 1 -- 172.21.15.120:0/3763602481 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0c948e0f0 con 0x55f0c95334b0 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f8930ff9640 10 client.18147 handle_command_reply: tid=0 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 client.18147 shutdown 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 msgr2=0x55f0c952d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8940024040 tx=0x7f894000eb70 comp rx=0 tx=0).stop 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 shutdown_connections 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f0c95334b0 0x55f0c95a2890 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f0c9523bd0 0x55f0c952e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0c9521190 0x55f0c952d680 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 --2- 172.21.15.120:0/3763602481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f0c951e860 0x55f0c952cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 >> 172.21.15.120:0/3763602481 conn(0x55f0c95199b0 msgr2=0x55f0c95183d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f8930ff9640 0 client.18147 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 shutdown_connections 2024-09-10T23:06:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.592+0000 7f89561da740 1 -- 172.21.15.120:0/3763602481 wait complete. 2024-09-10T23:06:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f89561da740 20 client.18147 trim_cache size 0 max 16384 2024-09-10T23:06:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:06:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:06:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:06:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 msgr2=0x7f89340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 0x7f89340b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f894000bfd0 tx=0x7f894000bb20 comp rx=0 tx=0).stop 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 msgr2=0x7f894c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f893c033040 tx=0x7f893c003b80 comp rx=0 tx=0).stop 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 shutdown_connections 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89340b52e0 0x7f89340b77a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f894c103370 0x7f894c199fb0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f894c1029b0 0x7f894c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 --2- 172.21.15.120:0/3649492663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f894c101860 0x7f894c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 >> 172.21.15.120:0/3649492663 conn(0x7f894c0fd140 msgr2=0x7f894c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 shutdown_connections 2024-09-10T23:06:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:14.593+0000 7f8953333640 1 -- 172.21.15.120:0/3649492663 wait complete. 2024-09-10T23:06:14.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29536 25.61 MB/sec warmup 69 sec latency 328.449 ms 2024-09-10T23:06:15.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 29956 25.37 MB/sec warmup 70 sec latency 161.314 ms 2024-09-10T23:06:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:16 smithi120 ceph-mon[31247]: pgmap v761: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 418 op/s 2024-09-10T23:06:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:16 smithi153 ceph-mon[34951]: pgmap v761: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 418 op/s 2024-09-10T23:06:16.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:16 smithi176 ceph-mon[34982]: pgmap v761: 97 pgs: 97 active+clean; 258 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 418 op/s 2024-09-10T23:06:16.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 30538 25.41 MB/sec warmup 71 sec latency 74.922 ms 2024-09-10T23:06:17.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 31216 25.68 MB/sec warmup 72 sec latency 83.229 ms 2024-09-10T23:06:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:18 smithi120 ceph-mon[31247]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-09-10T23:06:18.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:18 smithi153 ceph-mon[34951]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-09-10T23:06:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:18 smithi176 ceph-mon[34982]: pgmap v762: 97 pgs: 97 active+clean; 246 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-09-10T23:06:18.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 31763 25.87 MB/sec warmup 73 sec latency 75.615 ms 2024-09-10T23:06:19.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 32091 25.87 MB/sec warmup 74 sec latency 142.018 ms 2024-09-10T23:06:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:20 smithi120 ceph-mon[31247]: pgmap v763: 97 pgs: 97 active+clean; 242 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-09-10T23:06:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:20 smithi153 ceph-mon[34951]: pgmap v763: 97 pgs: 97 active+clean; 242 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-09-10T23:06:20.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:20 smithi176 ceph-mon[34982]: pgmap v763: 97 pgs: 97 active+clean; 242 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 498 op/s 2024-09-10T23:06:20.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 32378 25.70 MB/sec warmup 75 sec latency 154.176 ms 2024-09-10T23:06:21.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 32705 25.51 MB/sec warmup 76 sec latency 69.611 ms 2024-09-10T23:06:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:22 smithi120 ceph-mon[31247]: pgmap v764: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-10T23:06:22.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:22 smithi153 ceph-mon[34951]: pgmap v764: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-10T23:06:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:22 smithi176 ceph-mon[34982]: pgmap v764: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-10T23:06:22.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:06:22.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 33159 25.44 MB/sec warmup 77 sec latency 79.251 ms 2024-09-10T23:06:22.931 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:23.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1119406371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 msgr2=0x7f67ec103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1119406371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec103f10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009900 tx=0x7f67e802a410 comp rx=0 tx=0).stop 2024-09-10T23:06:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1119406371 shutdown_connections 2024-09-10T23:06:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1119406371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec10d0e0 0x7f67ec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1119406371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec104530 0x7f67ec10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1119406371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec103f10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1119406371 >> 172.21.15.120:0/1119406371 conn(0x7f67ec0fd700 msgr2=0x7f67ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1119406371 shutdown_connections 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.436+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1119406371 wait complete. 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 Processor -- start 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 -- start start 2024-09-10T23:06:23.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 0x7f67ec102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec112d60 con 0x7f67ec103b30 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec112be0 con 0x7f67ec104530 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f3b6e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec112ee0 con 0x7f67ec10d0e0 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49464/0 (socket says 172.21.15.120:49464) 2024-09-10T23:06:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f190a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 -- 172.21.15.120:0/1774208049 learned_addr learned my addr 172.21.15.120:0/1774208049 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f210b640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 0x7f67ec102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.459 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 -- 172.21.15.120:0/1774208049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 msgr2=0x7f67ec102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 0x7f67ec102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 -- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 msgr2=0x7f67ec1019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec1019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f1109640 1 -- 172.21.15.120:0/1774208049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec1a76d0 con 0x7f67ec104530 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f210b640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 0x7f67ec102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:23.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.437+0000 7f67f190a640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67f1109640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9f0 tx=0x7f67e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67deffd640 1 -- 172.21.15.120:0/1774208049 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e000cd80 con 0x7f67ec104530 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec1a79c0 con 0x7f67ec104530 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67deffd640 1 -- 172.21.15.120:0/1774208049 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e00054d0 con 0x7f67ec104530 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67deffd640 1 -- 172.21.15.120:0/1774208049 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010690 con 0x7f67ec104530 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.438+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec1a7eb0 con 0x7f67ec104530 2024-09-10T23:06:23.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.440+0000 7f67f5f43740 1 Processor -- start 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.440+0000 7f67f5f43740 1 -- start start 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c57bf1e50 con 0x556c57c7b480 2024-09-10T23:06:23.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556c57be1280 con 0x556c57c87d30 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f5f43740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556c57be5090 con 0x556c57c7bda0 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f210b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f210b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47720/0 (socket says 172.21.15.120:47720) 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f210b640 1 -- 172.21.15.120:0/477302327 learned_addr learned my addr 172.21.15.120:0/477302327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:23.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 -- 172.21.15.120:0/477302327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 msgr2=0x556c57c8a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c8a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 -- 172.21.15.120:0/477302327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 msgr2=0x556c57c7b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 -- 172.21.15.120:0/477302327 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5793f650 con 0x556c57c7bda0 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f210b640 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:23.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67deffd640 1 -- 172.21.15.120:0/1774208049 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f67e0004e90 con 0x7f67ec104530 2024-09-10T23:06:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.441+0000 7f67f190a640 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c877f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009230 tx=0x7f67e8005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ea4e503dcede5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67d7fff640 1 -- 172.21.15.120:0/477302327 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8040070 con 0x556c57c7bda0 2024-09-10T23:06:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67d7fff640 1 -- 172.21.15.120:0/477302327 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e8031860 con 0x556c57c7bda0 2024-09-10T23:06:23.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67d7fff640 1 -- 172.21.15.120:0/477302327 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8035040 con 0x556c57c7bda0 2024-09-10T23:06:23.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67deffd640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 0x7f67d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67f210b640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 0x7f67d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.442+0000 7f67deffd640 1 -- 172.21.15.120:0/1774208049 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f67e00e5fc0 con 0x7f67ec104530 2024-09-10T23:06:23.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 -- 172.21.15.120:0/477302327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 msgr2=0x556c57c877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f210b640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 0x7f67d80b77a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005c40 tx=0x7f67e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c877f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009230 tx=0x7f67e8005d90 comp rx=0 tx=0).stop 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 -- 172.21.15.120:0/477302327 shutdown_connections 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c8a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c877f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 --2- 172.21.15.120:0/477302327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c7b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 -- 172.21.15.120:0/477302327 >> 172.21.15.120:0/477302327 conn(0x556c57c792b0 msgr2=0x556c57c7a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 -- 172.21.15.120:0/477302327 shutdown_connections 2024-09-10T23:06:23.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.443+0000 7f67f5f43740 1 -- 172.21.15.120:0/477302327 wait complete. 2024-09-10T23:06:23.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 -- start start 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 Processor -- start 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c84ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556c57a42700 con 0x556c57c7bda0 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556c57b7c110 con 0x556c57c7b480 2024-09-10T23:06:23.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f1109640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c84ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f210b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49472/0 (socket says 172.21.15.120:49472) 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 -- 172.21.15.120:0/2241204513 learned_addr learned my addr 172.21.15.120:0/2241204513 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.444+0000 7f67f5f43740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556c57c8a950 con 0x556c57c87d30 2024-09-10T23:06:23.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f5f43740 10 client.0 _ll_register_callbacks cb 0x556c57b95990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 -- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 msgr2=0x556c57c85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c85140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 -- 172.21.15.120:0/2241204513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 msgr2=0x556c57c84ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c84ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 -- 172.21.15.120:0/2241204513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c57ba9c40 con 0x556c57c7b480 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f1109640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c84ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:23.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f190a640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67e8031a90 tx=0x7f67e8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa01c05f89f535ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:06:23.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8040050 con 0x556c57c7b480 2024-09-10T23:06:23.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f67e8003800 con 0x556c57c7b480 2024-09-10T23:06:23.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.445+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e8031860 con 0x556c57c7b480 2024-09-10T23:06:23.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67e8057050 con 0x556c57c7b480 2024-09-10T23:06:23.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 10 client.27428 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:23.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c57c962b0 con 0x556c57c7b480 2024-09-10T23:06:23.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e8031860 con 0x556c57c7b480 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f67e803c8d0 con 0x556c57c7b480 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 10 client.27428 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 20 client.27428 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57c97600 0x556c57c999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 4 client.27428 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c57bf3da0 con 0x556c57c97600 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.446+0000 7f67f1109640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57c97600 0x556c57c999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.447+0000 7f67f1109640 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57c97600 0x556c57c999f0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a3072741ccb599 server_cookie=1b53d7824d8470e8 in_seq=0 out_seq=0 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.447+0000 7f67d57fa640 10 client.27428 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.449+0000 7f67d57fa640 1 -- 172.21.15.120:0/2241204513 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c57bf3da0 con 0x556c57c97600 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.449+0000 7f67d57fa640 10 client.27428 handle_command_reply: tid=0 2024-09-10T23:06:23.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.449+0000 7f67f5f43740 1 client.27428 shutdown 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 msgr2=0x556c57c87670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67e8031a90 tx=0x7f67e8009260 comp rx=0 tx=0).stop 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 shutdown_connections 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57c97600 0x556c57c999f0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c84ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2241204513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c87670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67d57fa640 0 client.27428 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:23.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 >> 172.21.15.120:0/2241204513 conn(0x556c57c792b0 msgr2=0x556c57c7ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 shutdown_connections 2024-09-10T23:06:23.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 1 -- 172.21.15.120:0/2241204513 wait complete. 2024-09-10T23:06:23.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.450+0000 7f67f5f43740 20 client.27428 trim_cache size 0 max 16384 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 Processor -- start 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 -- start start 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c83340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c83880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1ae460 con 0x556c57c7b480 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b0ce0 con 0x556c57c87d30 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f5f43740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1aead0 con 0x556c57c7bda0 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f190a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f190a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47742/0 (socket says 172.21.15.120:47742) 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f190a640 1 -- 172.21.15.120:0/2160919361 learned_addr learned my addr 172.21.15.120:0/2160919361 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f210b640 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c83880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.476+0000 7f67f1109640 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c83340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 -- 172.21.15.120:0/2160919361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 msgr2=0x556c57c83340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c83340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 -- 172.21.15.120:0/2160919361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 msgr2=0x556c57c83880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c83880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 -- 172.21.15.120:0/2160919361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c57c854b0 con 0x556c57c7b480 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67f190a640 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f67e802a930 tx=0x7f67e8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c18574af3ee1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67d77fe640 1 -- 172.21.15.120:0/2160919361 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8040050 con 0x556c57c7b480 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67d77fe640 1 -- 172.21.15.120:0/2160919361 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e8031860 con 0x556c57c7b480 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.477+0000 7f67d77fe640 1 -- 172.21.15.120:0/2160919361 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8035040 con 0x556c57c7b480 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- 172.21.15.120:0/2160919361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 msgr2=0x556c57c93b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f67e802a930 tx=0x7f67e8005d90 comp rx=0 tx=0).stop 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- 172.21.15.120:0/2160919361 shutdown_connections 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c87d30 0x556c57c83880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c7bda0 0x556c57c83340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2160919361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7b480 0x556c57c93b80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- 172.21.15.120:0/2160919361 >> 172.21.15.120:0/2160919361 conn(0x556c57c792b0 msgr2=0x556c57c7fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- 172.21.15.120:0/2160919361 shutdown_connections 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- 172.21.15.120:0/2160919361 wait complete. 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 -- start start 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.478+0000 7f67f5f43740 1 Processor -- start 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c8fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c8aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1af1a0 con 0x556c57c7bda0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b0610 con 0x556c57c7b480 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1af870 con 0x556c57c87d30 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f5f43740 10 client.0 _ll_register_callbacks cb 0x7f67e4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47756/0 (socket says 172.21.15.120:47756) 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 -- 172.21.15.120:0/2942053453 learned_addr learned my addr 172.21.15.120:0/2942053453 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f210b640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c8aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f190a640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c8fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 -- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 msgr2=0x556c57c8aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c8aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 -- 172.21.15.120:0/2942053453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 msgr2=0x556c57c8fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c8fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f1109640 1 -- 172.21.15.120:0/2942053453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c57c8b190 con 0x556c57c7bda0 2024-09-10T23:06:23.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.479+0000 7f67f210b640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c8aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67f1109640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9c0 tx=0x7f67e000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b4c411a898660e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e00ac360 con 0x556c57c7bda0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f67e00a8ac0 con 0x556c57c7bda0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e8031860 con 0x556c57c7bda0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67e0005680 con 0x556c57c7bda0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67f5f43740 10 client.18165 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c57d04ac0 con 0x556c57c7bda0 2024-09-10T23:06:23.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e8031860 con 0x556c57c7bda0 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.480+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f67e00b5950 con 0x556c57c7bda0 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 10 client.18165 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 20 client.18165 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57d051f0 0x556c57d07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 4 client.18165 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c57bef850 con 0x556c57d051f0 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f190a640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57d051f0 0x556c57d07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67f190a640 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57d051f0 0x556c57d07610 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0cb70513553650 server_cookie=4228ca6fff631ed6 in_seq=0 out_seq=0 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.481+0000 7f67d57fa640 10 client.18165 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:23.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67d57fa640 1 -- 172.21.15.120:0/2942053453 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x556c57bef850 con 0x556c57d051f0 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67d57fa640 10 client.18165 handle_command_reply: tid=0 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 client.18165 shutdown 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 msgr2=0x556c57c90170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9c0 tx=0x7f67e000cf20 comp rx=0 tx=0).stop 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 shutdown_connections 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556c57d051f0 0x556c57d07610 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556c57c87d30 0x556c57c8aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556c57c7bda0 0x556c57c90170 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67d57fa640 0 client.18165 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 --2- 172.21.15.120:0/2942053453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556c57c7b480 0x556c57c8fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 >> 172.21.15.120:0/2942053453 conn(0x556c57c792b0 msgr2=0x556c57c832a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 shutdown_connections 2024-09-10T23:06:23.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 1 -- 172.21.15.120:0/2942053453 wait complete. 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.484+0000 7f67f5f43740 20 client.18165 trim_cache size 0 max 16384 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:06:23.490 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.491 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.492 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.493 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.494 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.495 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.496 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.497 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.498 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:06:23.499 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.500 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:06:23.501 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:23.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 msgr2=0x7f67d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 0x7f67d80b77a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005c40 tx=0x7f67e4009500 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 msgr2=0x7f67ec101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9f0 tx=0x7f67e000edf0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 shutdown_connections 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f67d80b52e0 0x7f67d80b77a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67ec10d0e0 0x7f67ec102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67ec104530 0x7f67ec101ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 --2- 172.21.15.120:0/1774208049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f67ec103b30 0x7f67ec1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 >> 172.21.15.120:0/1774208049 conn(0x7f67ec0fd700 msgr2=0x7f67ec0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 shutdown_connections 2024-09-10T23:06:23.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:23.485+0000 7f67f3b6e640 1 -- 172.21.15.120:0/1774208049 wait complete. 2024-09-10T23:06:23.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 33759 25.38 MB/sec warmup 78 sec latency 51.149 ms 2024-09-10T23:06:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:24 smithi120 ceph-mon[31247]: pgmap v765: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 576 op/s 2024-09-10T23:06:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:24 smithi153 ceph-mon[34951]: pgmap v765: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 576 op/s 2024-09-10T23:06:24.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:06:24.500 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:06:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:24 smithi176 ceph-mon[34982]: pgmap v765: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 576 op/s 2024-09-10T23:06:24.857 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:24.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 34227 25.45 MB/sec warmup 79 sec latency 227.192 ms 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- 172.21.15.120:0/839218145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 msgr2=0x7f653010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 --2- 172.21.15.120:0/839218145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f653010f560 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6528009ca0 tx=0x7f652802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- 172.21.15.120:0/839218145 shutdown_connections 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 --2- 172.21.15.120:0/839218145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f653010f560 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 --2- 172.21.15.120:0/839218145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65301045c0 0x7f653010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 --2- 172.21.15.120:0/839218145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6530103ca0 0x7f6530104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- 172.21.15.120:0/839218145 >> 172.21.15.120:0/839218145 conn(0x7f65300fd700 msgr2=0x7f65300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:25.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- 172.21.15.120:0/839218145 shutdown_connections 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- 172.21.15.120:0/839218145 wait complete. 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 Processor -- start 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.407+0000 7f6536338640 1 -- start start 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 0x7f6530101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 0x7f6530101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6530112d70 con 0x7f653010d100 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6530112bf0 con 0x7f6530103ca0 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6536338640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6530112ef0 con 0x7f65301045c0 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6527fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 0x7f6530101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:25.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47778/0 (socket says 172.21.15.120:47778) 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f6527fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 0x7f6530101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57172/0 (socket says 172.21.15.120:57172) 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 -- 172.21.15.120:0/2194242808 learned_addr learned my addr 172.21.15.120:0/2194242808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f652ffff640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 0x7f6530101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 -- 172.21.15.120:0/2194242808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 msgr2=0x7f6530101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 0x7f6530101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 -- 172.21.15.120:0/2194242808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 msgr2=0x7f6530101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 0x7f6530101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 -- 172.21.15.120:0/2194242808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301a7680 con 0x7f653010d100 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f652ffff640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 0x7f6530101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.408+0000 7f65348d5640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6528005cc0 tx=0x7f6528031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.409+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528003b70 con 0x7f653010d100 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.409+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65301a7910 con 0x7f653010d100 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.409+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65280318f0 con 0x7f653010d100 2024-09-10T23:06:25.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.409+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652802f950 con 0x7f653010d100 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.409+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a7ea0 con 0x7f653010d100 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.410+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6530104080 con 0x7f653010d100 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.411+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6528004b40 con 0x7f653010d100 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.412+0000 7f652dffb640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 0x7f651c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.413+0000 7f652ffff640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 0x7f651c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.413+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6528106df0 con 0x7f653010d100 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.413+0000 7f652ffff640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 0x7f651c0b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fc0 tx=0x7f6520005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:25.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.417+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65280cdce0 con 0x7f653010d100 2024-09-10T23:06:25.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.668+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65301a81f0 con 0x7f653010d100 2024-09-10T23:06:25.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.669+0000 7f652dffb640 1 -- 172.21.15.120:0/2194242808 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f65280cd5e0 con 0x7f653010d100 2024-09-10T23:06:25.671 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:06:25.672 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:06:25.672 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 msgr2=0x7f651c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 0x7f651c0b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6520009fc0 tx=0x7f6520005f90 comp rx=0 tx=0).stop 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 msgr2=0x7f65301023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6528005cc0 tx=0x7f6528031700 comp rx=0 tx=0).stop 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 shutdown_connections 2024-09-10T23:06:25.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f651c0b52e0 0x7f651c0b77a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f653010d100 0x7f65301023d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65301045c0 0x7f6530101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 --2- 172.21.15.120:0/2194242808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6530103ca0 0x7f6530101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 >> 172.21.15.120:0/2194242808 conn(0x7f65300fd700 msgr2=0x7f653010f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 shutdown_connections 2024-09-10T23:06:25.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:25.672+0000 7f6536338640 1 -- 172.21.15.120:0/2194242808 wait complete. 2024-09-10T23:06:25.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 34795 25.59 MB/sec warmup 80 sec latency 132.396 ms 2024-09-10T23:06:26.331 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:26 smithi120 ceph-mon[31247]: pgmap v766: 97 pgs: 97 active+clean; 239 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-09-10T23:06:26.331 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2194242808' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:26.361 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:06:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:26 smithi153 ceph-mon[34951]: pgmap v766: 97 pgs: 97 active+clean; 239 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-09-10T23:06:26.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2194242808' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:26 smithi176 ceph-mon[34982]: pgmap v766: 97 pgs: 97 active+clean; 239 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-09-10T23:06:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2194242808' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:06:26.658 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:26.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 35298 25.70 MB/sec warmup 81 sec latency 55.455 ms 2024-09-10T23:06:27.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 -- 172.21.15.120:0/3833859507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc101076a0 msgr2=0x7fdc10107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 --2- 172.21.15.120:0/3833859507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc101076a0 0x7fdc10107ae0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc0402a410 comp rx=0 tx=0).stop 2024-09-10T23:06:27.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 -- 172.21.15.120:0/3833859507 shutdown_connections 2024-09-10T23:06:27.187 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 --2- 172.21.15.120:0/3833859507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc10108020 0x7fdc1010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 --2- 172.21.15.120:0/3833859507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc101076a0 0x7fdc10107ae0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 --2- 172.21.15.120:0/3833859507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc10102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 -- 172.21.15.120:0/3833859507 >> 172.21.15.120:0/3833859507 conn(0x7fdc100fd700 msgr2=0x7fdc100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 -- 172.21.15.120:0/3833859507 shutdown_connections 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.151+0000 7fdc1673a640 1 -- 172.21.15.120:0/3833859507 wait complete. 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.152+0000 7fdc1673a640 1 Processor -- start 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.152+0000 7fdc1673a640 1 -- start start 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc101981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc101076a0 0x7fdc101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10111ca0 con 0x7fdc10108020 2024-09-10T23:06:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10111b20 con 0x7fdc10101c90 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc1673a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10111e20 con 0x7fdc101076a0 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc14cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc101981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc14cd7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47792/0 (socket says 172.21.15.120:47792) 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc14cd7640 1 -- 172.21.15.120:0/2532446899 learned_addr learned my addr 172.21.15.120:0/2532446899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc101076a0 0x7fdc101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0ffff640 1 -- 172.21.15.120:0/2532446899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc101076a0 msgr2=0x7fdc101986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc101076a0 0x7fdc101986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0ffff640 1 -- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 msgr2=0x7fdc1019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.153+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc0ffff640 1 -- 172.21.15.120:0/2532446899 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1019e260 con 0x7fdc10101c90 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc101981b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9f0 tx=0x7fdc0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc0d7fa640 1 -- 172.21.15.120:0/2532446899 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000ccc0 con 0x7fdc10101c90 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1019e550 con 0x7fdc10101c90 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc0d7fa640 1 -- 172.21.15.120:0/2532446899 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc000108e0 con 0x7fdc10101c90 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc0d7fa640 1 -- 172.21.15.120:0/2532446899 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00018880 con 0x7fdc10101c90 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.154+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc101a4de0 con 0x7fdc10101c90 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.156+0000 7fdc18b0f740 1 Processor -- start 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.156+0000 7fdc18b0f740 1 -- start start 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fc88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a11f3eea0 con 0x562a11fd4d80 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a11f2e2d0 con 0x562a11fc8df0 2024-09-10T23:06:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a11f320e0 con 0x562a11fc84d0 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47806/0 (socket says 172.21.15.120:47806) 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/1296321543 learned_addr learned my addr 172.21.15.120:0/1296321543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc14cd7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fc88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0d7fa640 1 -- 172.21.15.120:0/2532446899 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdc00010430 con 0x7fdc10101c90 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/1296321543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 msgr2=0x562a11fc88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fc88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/1296321543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 msgr2=0x562a11fd4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/1296321543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a11c8c650 con 0x562a11fd4d80 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc04032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71f4d6c33ae5404 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdbf67fc640 1 -- 172.21.15.120:0/1296321543 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04040070 con 0x562a11fd4d80 2024-09-10T23:06:27.191 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdbf67fc640 1 -- 172.21.15.120:0/1296321543 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc04004030 con 0x562a11fd4d80 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.157+0000 7fdbf67fc640 1 -- 172.21.15.120:0/1296321543 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04035040 con 0x562a11fd4d80 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc0d7fa640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 0x7fdbf80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 0x7fdbf80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc0d7fa640 1 -- 172.21.15.120:0/2532446899 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc000e5ec0 con 0x7fdc10101c90 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1296321543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 msgr2=0x562a11fd7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc04032b20 comp rx=0 tx=0).stop 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 0x7fdbf80b77a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdc080049d0 tx=0x7fdc08009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1296321543 shutdown_connections 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd7170 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1296321543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fc88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1296321543 >> 172.21.15.120:0/1296321543 conn(0x562a11fc6300 msgr2=0x562a11fc74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1296321543 shutdown_connections 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.158+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1296321543 wait complete. 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 -- start start 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 Processor -- start 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562a11d8f700 con 0x562a11fd4d80 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562a11ec9160 con 0x562a11fc8df0 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a11fd79a0 con 0x562a11fc84d0 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.159+0000 7fdc18b0f740 10 client.0 _ll_register_callbacks cb 0x562a11ee29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57212/0 (socket says 172.21.15.120:57212) 2024-09-10T23:06:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 -- 172.21.15.120:0/1263586061 learned_addr learned my addr 172.21.15.120:0/1263586061 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 -- 172.21.15.120:0/1263586061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 msgr2=0x562a11fd1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 -- 172.21.15.120:0/1263586061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 msgr2=0x562a11fd2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 -- 172.21.15.120:0/1263586061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a11ef6c90 con 0x562a11fc84d0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc000a99d0 tx=0x7fdc000abed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c42e58da6fb650 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdbef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000a6e60 con 0x562a11fc84d0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc000abbc0 con 0x562a11fc84d0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc04004030 con 0x562a11fc84d0 2024-09-10T23:06:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.160+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc000b1190 con 0x562a11fc84d0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 10 client.27652 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a11fe33f0 con 0x562a11fc84d0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc04004030 con 0x562a11fc84d0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdc000b2320 con 0x562a11fc84d0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 10 client.27652 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 20 client.27652 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a11fe4950 0x562a11fe6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 4 client.27652 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a11f40df0 con 0x562a11fe4950 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.161+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a11fe4950 0x562a11fe6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.162+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a11fe4950 0x562a11fe6d80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222a85c522bf204a server_cookie=e613ba1919f2545d in_seq=0 out_seq=0 2024-09-10T23:06:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.162+0000 7fdbef7fe640 10 client.27652 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.162+0000 7fdbef7fe640 1 -- 172.21.15.120:0/1263586061 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a11f40df0 con 0x562a11fe4950 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.162+0000 7fdbef7fe640 10 client.27652 handle_command_reply: tid=0 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.162+0000 7fdc18b0f740 1 client.27652 shutdown 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 msgr2=0x562a11fd4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc000a99d0 tx=0x7fdc000abed0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 shutdown_connections 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a11fe4950 0x562a11fe6d80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fd4d80 0x562a11fd2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc8df0 0x562a11fd1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/1263586061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fc84d0 0x562a11fd4790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdbef7fe640 0 client.27652 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 >> 172.21.15.120:0/1263586061 conn(0x562a11fc6300 msgr2=0x562a11fcbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.196 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 shutdown_connections 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 1 -- 172.21.15.120:0/1263586061 wait complete. 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.163+0000 7fdc18b0f740 20 client.27652 trim_cache size 0 max 16384 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.176+0000 7fdc18b0f740 1 Processor -- start 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 -- start start 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fc5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fcf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101ae4e0 con 0x562a11fc8df0 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101b0d40 con 0x562a11fc84d0 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101aeb30 con 0x562a11fd4d80 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47828/0 (socket says 172.21.15.120:47828) 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/449738869 learned_addr learned my addr 172.21.15.120:0/449738869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fcf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fc5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/449738869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 msgr2=0x562a11fcf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fcf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/449738869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 msgr2=0x562a11fc5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fc5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/449738869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a11fd43c0 con 0x562a11fc8df0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fc5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.177+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc040316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39424055f41358e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdbf5ffb640 1 -- 172.21.15.120:0/449738869 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04035040 con 0x562a11fc8df0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdbf5ffb640 1 -- 172.21.15.120:0/449738869 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc04005030 con 0x562a11fc8df0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdbf5ffb640 1 -- 172.21.15.120:0/449738869 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04040040 con 0x562a11fc8df0 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 -- 172.21.15.120:0/449738869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 msgr2=0x562a11fcedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc040316e0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 -- 172.21.15.120:0/449738869 shutdown_connections 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fcf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fcedf0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/449738869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fc5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 -- 172.21.15.120:0/449738869 >> 172.21.15.120:0/449738869 conn(0x562a11fc6300 msgr2=0x562a11fd3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 -- 172.21.15.120:0/449738869 shutdown_connections 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.178+0000 7fdc18b0f740 1 -- 172.21.15.120:0/449738869 wait complete. 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 -- start start 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 Processor -- start 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fd7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101af200 con 0x562a11fc8df0 2024-09-10T23:06:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101b0670 con 0x562a11fc84d0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdc101af8d0 con 0x562a11fd4d80 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.179+0000 7fdc18b0f740 10 client.0 _ll_register_callbacks cb 0x7fdc00010750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47840/0 (socket says 172.21.15.120:47840) 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/984238188 learned_addr learned my addr 172.21.15.120:0/984238188 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc14cd7640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fd7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 msgr2=0x562a11fd7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fd7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/984238188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 msgr2=0x562a11fdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 -- 172.21.15.120:0/984238188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a11fd8330 con 0x562a11fc8df0 2024-09-10T23:06:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc0f7fe640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc0402a930 tx=0x7fdc04009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b0dd87687ec1ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdbef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0405a070 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdc04031d10 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc04005030 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc04066050 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc18b0f740 10 client.18189 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a12052250 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.180+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc04005030 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdc04004870 con 0x562a11fc8df0 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 10 client.18189 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 20 client.18189 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a12052520 0x562a12054910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 4 client.18189 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:06:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a11f3c8a0 con 0x562a12052520 2024-09-10T23:06:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a12052520 0x562a12054910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdbef7fe640 10 client.18189 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.181+0000 7fdc0ffff640 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a12052520 0x562a12054910 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc1e0d2c618be35 server_cookie=f16bb44fe66ad8e6 in_seq=0 out_seq=0 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdbef7fe640 1 -- 172.21.15.120:0/984238188 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a11f3c8a0 con 0x562a12052520 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdbef7fe640 10 client.18189 handle_command_reply: tid=0 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 client.18189 shutdown 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 msgr2=0x562a11fdd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdc0402a930 tx=0x7fdc04009520 comp rx=0 tx=0).stop 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 shutdown_connections 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562a12052520 0x562a12054910 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a11fd4d80 0x562a11fd7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562a11fc8df0 0x562a11fdd360 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 --2- 172.21.15.120:0/984238188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562a11fc84d0 0x562a11fdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 >> 172.21.15.120:0/984238188 conn(0x562a11fc6300 msgr2=0x562a11fdfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdbef7fe640 0 client.18189 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 shutdown_connections 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 1 -- 172.21.15.120:0/984238188 wait complete. 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc18b0f740 20 client.18189 trim_cache size 0 max 16384 2024-09-10T23:06:27.204 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:06:27.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 msgr2=0x7fdbf80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 0x7fdbf80b77a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdc080049d0 tx=0x7fdc08009500 comp rx=0 tx=0).stop 2024-09-10T23:06:27.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 msgr2=0x7fdc101981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc101981b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9f0 tx=0x7fdc0000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 shutdown_connections 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdbf80b52e0 0x7fdbf80b77a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc10108020 0x7fdc1019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc101076a0 0x7fdc101986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 --2- 172.21.15.120:0/2532446899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc10101c90 0x7fdc101981b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 >> 172.21.15.120:0/2532446899 conn(0x7fdc100fd700 msgr2=0x7fdc100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 shutdown_connections 2024-09-10T23:06:27.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:27.189+0000 7fdc1673a640 1 -- 172.21.15.120:0/2532446899 wait complete. 2024-09-10T23:06:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 -- 172.21.15.120:0/651803390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc1044a0 msgr2=0x7fbebc10ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 --2- 172.21.15.120:0/651803390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc1044a0 0x7fbebc10ae20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009c10 tx=0x7fbeb402a6e0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 -- 172.21.15.120:0/651803390 shutdown_connections 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 --2- 172.21.15.120:0/651803390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc1044a0 0x7fbebc10ae20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 --2- 172.21.15.120:0/651803390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc103ae0 0x7fbebc103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 --2- 172.21.15.120:0/651803390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 -- 172.21.15.120:0/651803390 >> 172.21.15.120:0/651803390 conn(0x7fbebc0fe250 msgr2=0x7fbebc100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 -- 172.21.15.120:0/651803390 shutdown_connections 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.690+0000 7fbec1c33640 1 -- 172.21.15.120:0/651803390 wait complete. 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 Processor -- start 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 -- start start 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc103ae0 0x7fbebc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 0x7fbebc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10e5e0 con 0x7fbebc102990 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10e460 con 0x7fbebc1044a0 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbec1c33640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10e760 con 0x7fbebc103ae0 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47868/0 (socket says 172.21.15.120:47868) 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.691+0000 7fbebb7fe640 1 -- 172.21.15.120:0/2045932760 learned_addr learned my addr 172.21.15.120:0/2045932760 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebaffd640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc103ae0 0x7fbebc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 0x7fbebc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 -- 172.21.15.120:0/2045932760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc103ae0 msgr2=0x7fbebc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc103ae0 0x7fbebc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 -- 172.21.15.120:0/2045932760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 msgr2=0x7fbebc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 0x7fbebc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 -- 172.21.15.120:0/2045932760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc19a840 con 0x7fbebc102990 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 0x7fbebc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e9c0 tx=0x7fbeac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbeb8ff9640 1 -- 172.21.15.120:0/2045932760 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00cde0 con 0x7fbebc102990 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc19ab30 con 0x7fbebc102990 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1a1200 con 0x7fbebc102990 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbeb8ff9640 1 -- 172.21.15.120:0/2045932760 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeac0035f0 con 0x7fbebc102990 2024-09-10T23:06:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.692+0000 7fbeb8ff9640 1 -- 172.21.15.120:0/2045932760 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac010690 con 0x7fbebc102990 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbeb8ff9640 1 -- 172.21.15.120:0/2045932760 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbeac003aa0 con 0x7fbebc102990 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 Processor -- start 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 -- start start 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe7090990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe70934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556fe6ff96f0 con 0x556fe708bc70 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556fe6fe8b20 con 0x556fe7090fe0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.695+0000 7fbec4a07740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556fe6fec930 con 0x556fe708e5a0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe7090990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47884/0 (socket says 172.21.15.120:47884) 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbeb8ff9640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 0x7fbea40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 -- 172.21.15.120:0/2304679955 learned_addr learned my addr 172.21.15.120:0/2304679955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebaffd640 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe70934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 0x7fbea40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbeb8ff9640 1 -- 172.21.15.120:0/2045932760 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbeac0e5f40 con 0x7fbebc102990 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 -- 172.21.15.120:0/2304679955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 msgr2=0x556fe7090990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe7090990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 -- 172.21.15.120:0/2304679955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 msgr2=0x556fe70934a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe70934a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 -- 172.21.15.120:0/2304679955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe6fae530 con 0x556fe708bc70 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe7090990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4005cc0 tx=0x7fbeb4030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711612367bb4e37f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbea1ffb640 1 -- 172.21.15.120:0/2304679955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4036070 con 0x556fe708bc70 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbea1ffb640 1 -- 172.21.15.120:0/2304679955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb4037040 con 0x556fe708bc70 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.696+0000 7fbea1ffb640 1 -- 172.21.15.120:0/2304679955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4031640 con 0x556fe708bc70 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 -- 172.21.15.120:0/2304679955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 msgr2=0x556fe708e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 0x7fbea40b77a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4047000 tx=0x7fbeb4048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4005cc0 tx=0x7fbeb4030d90 comp rx=0 tx=0).stop 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 -- 172.21.15.120:0/2304679955 shutdown_connections 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe70934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe7090990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2304679955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708e060 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 -- 172.21.15.120:0/2304679955 >> 172.21.15.120:0/2304679955 conn(0x556fe70829b0 msgr2=0x556fe7083740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 -- 172.21.15.120:0/2304679955 shutdown_connections 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.697+0000 7fbec4a07740 1 -- 172.21.15.120:0/2304679955 wait complete. 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 -- start start 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 Processor -- start 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 0x556fe7095980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 0x556fe7096550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556fe6f20ab0 con 0x556fe7090fe0 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556fe6f8e730 con 0x556fe708e5a0 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556fe7093fc0 con 0x556fe708bc70 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbec4a07740 10 client.0 _ll_register_callbacks cb 0x556fe6fe26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 0x556fe7096550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 0x556fe7095980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57274/0 (socket says 172.21.15.120:57274) 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 -- 172.21.15.120:0/32602706 learned_addr learned my addr 172.21.15.120:0/32602706 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 -- 172.21.15.120:0/32602706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 msgr2=0x556fe7095980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 0x556fe7095980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 -- 172.21.15.120:0/32602706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 msgr2=0x556fe7096550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.698+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 0x556fe7096550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbebb7fe640 1 -- 172.21.15.120:0/32602706 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe6d4a790 con 0x556fe708bc70 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbebaffd640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 0x556fe7095980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbebbfff640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 0x556fe7096550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0e3a70 tx=0x7fbeac0b3a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e796ce9c1d554cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0ace30 con 0x556fe708bc70 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbeac0e80d0 con 0x556fe708bc70 2024-09-10T23:06:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb4037040 con 0x556fe708bc70 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeac0ac730 con 0x556fe708bc70 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 10 client.27670 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fe6dc3620 con 0x556fe708bc70 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb4037040 con 0x556fe708bc70 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbeac0a8110 con 0x556fe708bc70 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 10 client.27670 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 10 client.27670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 20 client.27670 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe70a0be0 0x556fe70a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 4 client.27670 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.699+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe6ffb640 con 0x556fe70a0be0 2024-09-10T23:06:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.700+0000 7fbebaffd640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe70a0be0 0x556fe70a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.700+0000 7fbebaffd640 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe70a0be0 0x556fe70a3010 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f8d95d792a5be1 server_cookie=b0ff03b3a89eaf21 in_seq=0 out_seq=0 2024-09-10T23:06:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.700+0000 7fbe9affd640 10 client.27670 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.702+0000 7fbe9affd640 1 -- 172.21.15.120:0/32602706 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fe6ffb640 con 0x556fe70a0be0 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.702+0000 7fbe9affd640 10 client.27670 handle_command_reply: tid=0 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.702+0000 7fbec4a07740 1 client.27670 shutdown 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 msgr2=0x556fe70952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0e3a70 tx=0x7fbeac0b3a00 comp rx=0 tx=0).stop 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 shutdown_connections 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe70a0be0 0x556fe70a3010 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe7090fe0 0x556fe7096550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708e5a0 0x556fe7095980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 --2- 172.21.15.120:0/32602706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708bc70 0x556fe70952e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbe9affd640 0 client.27670 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 >> 172.21.15.120:0/32602706 conn(0x556fe70829b0 msgr2=0x556fe7081bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 shutdown_connections 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 1 -- 172.21.15.120:0/32602706 wait complete. 2024-09-10T23:06:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.703+0000 7fbec4a07740 20 client.27670 trim_cache size 0 max 16384 2024-09-10T23:06:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 Processor -- start 2024-09-10T23:06:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 -- start start 2024-09-10T23:06:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 0x556fe7096e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe7090fe0 0x556fe70913c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800c390 con 0x556fe708e5a0 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800ec10 con 0x556fe708bc70 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbec4a07740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800ca00 con 0x556fe7090fe0 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47912/0 (socket says 172.21.15.120:47912) 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 -- 172.21.15.120:0/2944440697 learned_addr learned my addr 172.21.15.120:0/2944440697 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 0x556fe7096e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebbfff640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe7090fe0 0x556fe70913c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 -- 172.21.15.120:0/2944440697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe7090fe0 msgr2=0x556fe70913c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe7090fe0 0x556fe70913c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 -- 172.21.15.120:0/2944440697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 msgr2=0x556fe7096e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 0x556fe7096e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 -- 172.21.15.120:0/2944440697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe708e3d0 con 0x556fe708e5a0 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 0x556fe7096e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbebaffd640 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005fd0 tx=0x7fbeb000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f192cac2f2342f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.719+0000 7fbea17fa640 1 -- 172.21.15.120:0/2944440697 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004020 con 0x556fe708e5a0 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbea17fa640 1 -- 172.21.15.120:0/2944440697 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb0010dc0 con 0x556fe708e5a0 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbea17fa640 1 -- 172.21.15.120:0/2944440697 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00054b0 con 0x556fe708e5a0 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 -- 172.21.15.120:0/2944440697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 msgr2=0x556fe7083370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005fd0 tx=0x7fbeb000b410 comp rx=0 tx=0).stop 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 -- 172.21.15.120:0/2944440697 shutdown_connections 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe7090fe0 0x556fe70913c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708e5a0 0x556fe7083370 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 --2- 172.21.15.120:0/2944440697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe708bc70 0x556fe7096e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 -- 172.21.15.120:0/2944440697 >> 172.21.15.120:0/2944440697 conn(0x556fe70829b0 msgr2=0x556fe7094e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 -- 172.21.15.120:0/2944440697 shutdown_connections 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.720+0000 7fbec4a07740 1 -- 172.21.15.120:0/2944440697 wait complete. 2024-09-10T23:06:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 -- start start 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 Processor -- start 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe708a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe708aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800d0d0 con 0x556fe708bc70 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800e540 con 0x556fe7090fe0 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbea800d7a0 con 0x556fe708e5a0 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbec4a07740 10 client.0 _ll_register_callbacks cb 0x7fbeac0abea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47914/0 (socket says 172.21.15.120:47914) 2024-09-10T23:06:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 -- 172.21.15.120:0/15507565 learned_addr learned my addr 172.21.15.120:0/15507565 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebbfff640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe708aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebaffd640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe708a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 -- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 msgr2=0x556fe708a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe708a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 -- 172.21.15.120:0/15507565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 msgr2=0x556fe708aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe708aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.721+0000 7fbebb7fe640 1 -- 172.21.15.120:0/15507565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe7110360 con 0x556fe708bc70 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbebb7fe640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0a8cb0 tx=0x7fbeac0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4559459cbbb579 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0ac080 con 0x556fe708bc70 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbeac0abb90 con 0x556fe708bc70 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb0010dc0 con 0x556fe708bc70 2024-09-10T23:06:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeac0ace30 con 0x556fe708bc70 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 10 client.18207 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fe7111670 con 0x556fe708bc70 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb0010dc0 con 0x556fe708bc70 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbeac0711c0 con 0x556fe708bc70 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 10 client.18207 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 10 client.18207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 20 client.18207 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe7111eb0 0x556fe71142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 4 client.18207 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fe6ff70f0 con 0x556fe7111eb0 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.722+0000 7fbebaffd640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe7111eb0 0x556fe71142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.723+0000 7fbe9affd640 10 client.18207 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:27.723+0000 7fbebaffd640 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe7111eb0 0x556fe71142d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def535c26d6786e9 server_cookie=5c932eed13907eb6 in_seq=0 out_seq=0 2024-09-10T23:06:27.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 35644 25.68 MB/sec warmup 82 sec latency 112.820 ms 2024-09-10T23:06:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbe9affd640 1 -- 172.21.15.120:0/15507565 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fe6ff70f0 con 0x556fe7111eb0 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbe9affd640 10 client.18207 handle_command_reply: tid=0 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 client.18207 shutdown 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 msgr2=0x556fe708a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0a8cb0 tx=0x7fbeac0039d0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 shutdown_connections 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556fe7111eb0 0x556fe71142d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556fe7090fe0 0x556fe708aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fe708e5a0 0x556fe708a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbe9affd640 0 client.18207 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 --2- 172.21.15.120:0/15507565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556fe708bc70 0x556fe708a040 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 >> 172.21.15.120:0/15507565 conn(0x556fe70829b0 msgr2=0x556fe70913c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 shutdown_connections 2024-09-10T23:06:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 1 -- 172.21.15.120:0/15507565 wait complete. 2024-09-10T23:06:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.345+0000 7fbec4a07740 20 client.18207 trim_cache size 0 max 16384 2024-09-10T23:06:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:06:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:06:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:06:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 msgr2=0x7fbea40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 0x7fbea40b77a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4047000 tx=0x7fbeb4048000 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 msgr2=0x7fbebc198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e9c0 tx=0x7fbeac00ee80 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 shutdown_connections 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbea40b52e0 0x7fbea40b77a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbebc1044a0 0x7fbebc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbebc103ae0 0x7fbebc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 --2- 172.21.15.120:0/2045932760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbebc102990 0x7fbebc198d00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 >> 172.21.15.120:0/2045932760 conn(0x7fbebc0fe250 msgr2=0x7fbebc106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 shutdown_connections 2024-09-10T23:06:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:28.346+0000 7fbec1c33640 1 -- 172.21.15.120:0/2045932760 wait complete. 2024-09-10T23:06:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:28 smithi120 ceph-mon[31247]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.9 MiB/s wr, 622 op/s 2024-09-10T23:06:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:28 smithi153 ceph-mon[34951]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.9 MiB/s wr, 622 op/s 2024-09-10T23:06:28.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:28 smithi176 ceph-mon[34982]: pgmap v767: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.9 MiB/s wr, 622 op/s 2024-09-10T23:06:28.867 INFO:tasks.workunit.client.0.smithi120.stdout: 10 35822 25.45 MB/sec warmup 83 sec latency 279.452 ms 2024-09-10T23:06:28.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:06:28.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:06:29.222 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:29.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36095 25.29 MB/sec warmup 84 sec latency 149.869 ms 2024-09-10T23:06:30.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- 172.21.15.120:0/3687092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 msgr2=0x7fe69810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 --2- 172.21.15.120:0/3687092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69810e820 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe690009ca0 tx=0x7fe69002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- 172.21.15.120:0/3687092080 shutdown_connections 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 --2- 172.21.15.120:0/3687092080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69810e820 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 --2- 172.21.15.120:0/3687092080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 0x7fe698107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 --2- 172.21.15.120:0/3687092080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 0x7fe698102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- 172.21.15.120:0/3687092080 >> 172.21.15.120:0/3687092080 conn(0x7fe6980fd700 msgr2=0x7fe6980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- 172.21.15.120:0/3687092080 shutdown_connections 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- 172.21.15.120:0/3687092080 wait complete. 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 Processor -- start 2024-09-10T23:06:30.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.761+0000 7fe69e89e640 1 -- start start 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 0x7fe698199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 0x7fe698199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111ca0 con 0x7fe698108020 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111b20 con 0x7fe6981076a0 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69e89e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111e20 con 0x7fe698101c90 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47926/0 (socket says 172.21.15.120:47926) 2024-09-10T23:06:30.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 -- 172.21.15.120:0/4182411500 learned_addr learned my addr 172.21.15.120:0/4182411500 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe697fff640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 0x7fe698199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe6977fe640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 0x7fe698199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 -- 172.21.15.120:0/4182411500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 msgr2=0x7fe698199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 0x7fe698199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 -- 172.21.15.120:0/4182411500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 msgr2=0x7fe698199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 0x7fe698199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 -- 172.21.15.120:0/4182411500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69819f2d0 con 0x7fe698108020 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.762+0000 7fe69ce3b640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe690005cc0 tx=0x7fe690031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.763+0000 7fe6957fa640 1 -- 172.21.15.120:0/4182411500 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690003b70 con 0x7fe698108020 2024-09-10T23:06:30.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.763+0000 7fe6957fa640 1 -- 172.21.15.120:0/4182411500 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6900318f0 con 0x7fe698108020 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.763+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69819f560 con 0x7fe698108020 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.763+0000 7fe6957fa640 1 -- 172.21.15.120:0/4182411500 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002f950 con 0x7fe698108020 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.763+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6981a5c80 con 0x7fe698108020 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.765+0000 7fe6957fa640 1 -- 172.21.15.120:0/4182411500 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe69002fb50 con 0x7fe698108020 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 Processor -- start 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 -- start start 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e68790 0x555be7e6ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555be7dd2e60 con 0x555be7e5bee0 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555be7dc2290 con 0x555be7e68790 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555be7dc60a0 con 0x555be7e5c800 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47940/0 (socket says 172.21.15.120:47940) 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6977fe640 1 -- 172.21.15.120:0/2008886089 learned_addr learned my addr 172.21.15.120:0/2008886089 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe697fff640 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e68790 0x555be7e6ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.766+0000 7fe6957fa640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 0x7fe6800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe697fff640 1 -- 172.21.15.120:0/2008886089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e68790 msgr2=0x555be7e6ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe697fff640 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e68790 0x555be7e6ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe697fff640 1 -- 172.21.15.120:0/2008886089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 msgr2=0x555be7e5c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe697fff640 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe697fff640 1 -- 172.21.15.120:0/2008886089 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be7b20930 con 0x555be7e5c800 2024-09-10T23:06:30.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.767+0000 7fe6957fa640 1 -- 172.21.15.120:0/4182411500 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe690106c20 con 0x7fe698108020 2024-09-10T23:06:30.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.768+0000 7fe69ce3b640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 0x7fe6800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.768+0000 7fe697fff640 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e68250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe688009c50 tx=0x7fe688011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bd3a9f51b50835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.769+0000 7fe69ce3b640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 0x7fe6800b7820 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6900c99e0 tx=0x7fe6900047b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.769+0000 7fe67e7fc640 1 -- 172.21.15.120:0/2008886089 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004050 con 0x555be7e5c800 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.769+0000 7fe67e7fc640 1 -- 172.21.15.120:0/2008886089 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6880054d0 con 0x555be7e5c800 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.769+0000 7fe67e7fc640 1 -- 172.21.15.120:0/2008886089 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004d50 con 0x555be7e5c800 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.770+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2008886089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 msgr2=0x555be7e68250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.770+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e68250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe688009c50 tx=0x7fe688011810 comp rx=0 tx=0).stop 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.770+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2008886089 shutdown_connections 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.770+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e68790 0x555be7e6ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e68250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2008886089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2008886089 >> 172.21.15.120:0/2008886089 conn(0x555be7e59d10 msgr2=0x555be7e5aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2008886089 shutdown_connections 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2008886089 wait complete. 2024-09-10T23:06:30.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 -- start start 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.771+0000 7fe6a0c73740 1 Processor -- start 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47942/0 (socket says 172.21.15.120:47942) 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555be7c23890 con 0x555be7e5c800 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 -- 172.21.15.120:0/2668737931 learned_addr learned my addr 172.21.15.120:0/2668737931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555be7d5cd40 con 0x555be7e5bee0 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555be7e6b440 con 0x555be7e68790 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6a0c73740 10 client.0 _ll_register_callbacks cb 0x555be7dc0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:30.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 -- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 msgr2=0x555be7e66280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e66280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 -- 172.21.15.120:0/2668737931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 msgr2=0x555be7e65800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e65800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be7e748d0 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe697fff640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe6880058c0 tx=0x7fe688005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4850fdbcd9421ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.772+0000 7fe6777fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68803a070 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe688003a00 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6880054d0 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe688046050 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 10 client.18219 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555be7e77240 con 0x555be7e5c800 2024-09-10T23:06:30.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6880046c0 con 0x555be7e5c800 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe688036450 con 0x555be7e5c800 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 10 client.18219 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 20 client.18219 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7e785f0 0x555be7e7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 4 client.18219 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe6880054d0 con 0x555be7e785f0 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.773+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7e785f0 0x555be7e7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.774+0000 7fe6777fe640 10 client.18219 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.774+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7e785f0 0x555be7e7a9e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e4341eb8a4b236 server_cookie=c474b0187fdaca96 in_seq=0 out_seq=0 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6777fe640 1 -- 172.21.15.120:0/2668737931 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe6880054d0 con 0x555be7e785f0 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6777fe640 10 client.18219 handle_command_reply: tid=0 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 client.18219 shutdown 2024-09-10T23:06:30.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 msgr2=0x555be7e65d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe6880058c0 tx=0x7fe688005d30 comp rx=0 tx=0).stop 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 shutdown_connections 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7e785f0 0x555be7e7a9e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e66280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6777fe640 0 client.18219 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5c800 0x555be7e65d40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2668737931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e65800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.776+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 >> 172.21.15.120:0/2668737931 conn(0x555be7e59d10 msgr2=0x555be7e58f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.777+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 shutdown_connections 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.777+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2668737931 wait complete. 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.777+0000 7fe6a0c73740 20 client.18219 trim_cache size 0 max 16384 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.804+0000 7fe6a0c73740 1 Processor -- start 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.804+0000 7fe6a0c73740 1 -- start start 2024-09-10T23:06:30.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.804+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.804+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5c800 0x555be7e65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.804+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe698075140 con 0x555be7e5bee0 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981a8520 con 0x555be7e5c800 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981a6310 con 0x555be7e68790 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47952/0 (socket says 172.21.15.120:47952) 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 -- 172.21.15.120:0/4167289495 learned_addr learned my addr 172.21.15.120:0/4167289495 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe697fff640 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5c800 0x555be7e65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 -- 172.21.15.120:0/4167289495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 msgr2=0x555be7e62a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 -- 172.21.15.120:0/4167289495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5c800 msgr2=0x555be7e65e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5c800 0x555be7e65e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 -- 172.21.15.120:0/4167289495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be7e5a790 con 0x555be7e5bee0 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.805+0000 7fe6977fe640 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe68c010780 tx=0x7fe68c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4a0b71fb466a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.806+0000 7fe67dffb640 1 -- 172.21.15.120:0/4167289495 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c012ce0 con 0x555be7e5bee0 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.806+0000 7fe67dffb640 1 -- 172.21.15.120:0/4167289495 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68c012e80 con 0x555be7e5bee0 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.806+0000 7fe67dffb640 1 -- 172.21.15.120:0/4167289495 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0035f0 con 0x555be7e5bee0 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 -- 172.21.15.120:0/4167289495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 msgr2=0x555be7e5ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe68c010780 tx=0x7fe68c009b60 comp rx=0 tx=0).stop 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 -- 172.21.15.120:0/4167289495 shutdown_connections 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e68790 0x555be7e62a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5c800 0x555be7e65e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/4167289495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e5bee0 0x555be7e5ffe0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 -- 172.21.15.120:0/4167289495 >> 172.21.15.120:0/4167289495 conn(0x555be7e59d10 msgr2=0x555be7e65770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 -- 172.21.15.120:0/4167289495 shutdown_connections 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.807+0000 7fe6a0c73740 1 -- 172.21.15.120:0/4167289495 wait complete. 2024-09-10T23:06:30.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 -- start start 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 Processor -- start 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e68790 0x555be7ede180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981a69e0 con 0x555be7e68790 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981a7e50 con 0x555be7e5bee0 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981a70b0 con 0x555be7e5c800 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.808+0000 7fe6a0c73740 10 client.0 _ll_register_callbacks cb 0x7fe68c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57340/0 (socket says 172.21.15.120:57340) 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 -- 172.21.15.120:0/2658141871 learned_addr learned my addr 172.21.15.120:0/2658141871 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e68790 0x555be7ede180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 -- 172.21.15.120:0/2658141871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 msgr2=0x555be7e6cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e6cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 -- 172.21.15.120:0/2658141871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e68790 msgr2=0x555be7ede180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e68790 0x555be7ede180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 -- 172.21.15.120:0/2658141871 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be7e59b40 con 0x555be7e5c800 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe697fff640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe688011850 tx=0x7fe688037fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c7c6861c58487a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.809+0000 7fe6777fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68804e020 con 0x555be7e5c800 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe688036450 con 0x555be7e5c800 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555be7dd4db0 con 0x555be7e5c800 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe68803a040 con 0x555be7e5c800 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6a0c73740 10 client.27691 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:30.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.810+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555be7ee5af0 con 0x555be7e5c800 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe688009a70 con 0x555be7e5c800 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe68803f4d0 con 0x555be7e5c800 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 10 client.27691 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 20 client.27691 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7ee61e0 0x555be7ee8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 4 client.27691 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555be7dd4db0 con 0x555be7ee61e0 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.811+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7ee61e0 0x555be7ee8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.812+0000 7fe6777fe640 10 client.27691 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.812+0000 7fe6977fe640 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7ee61e0 0x555be7ee8600 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59308b2d9f422762 server_cookie=36c50734a7a4efcd in_seq=0 out_seq=0 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6777fe640 1 -- 172.21.15.120:0/2658141871 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555be7dd4db0 con 0x555be7ee61e0 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6777fe640 10 client.27691 handle_command_reply: tid=0 2024-09-10T23:06:30.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 client.27691 shutdown 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 msgr2=0x555be7e6d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe688011850 tx=0x7fe688037fc0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 shutdown_connections 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555be7ee61e0 0x555be7ee8600 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:06:30.440 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "ff0a0b63-8f51-43d9-943f-ad59d73316ad", 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555be7e68790 0x555be7ede180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6777fe640 0 client.27691 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555be7e5c800 0x555be7e6d530 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 --2- 172.21.15.120:0/2658141871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be7e5bee0 0x555be7e6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 >> 172.21.15.120:0/2658141871 conn(0x555be7e59d10 msgr2=0x555be7e65f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 shutdown_connections 2024-09-10T23:06:30.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 1 -- 172.21.15.120:0/2658141871 wait complete. 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.826+0000 7fe6a0c73740 20 client.27691 trim_cache size 0 max 16384 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 msgr2=0x7fe6800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 0x7fe6800b7820 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6900c99e0 tx=0x7fe6900047b0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 msgr2=0x7fe69819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe690005cc0 tx=0x7fe690031700 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 shutdown_connections 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe6800b5360 0x7fe6800b7820 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe698108020 0x7fe69819eb00 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6981076a0 0x7fe698199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 --2- 172.21.15.120:0/4182411500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe698101c90 0x7fe698199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:30.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 >> 172.21.15.120:0/4182411500 conn(0x7fe6980fd700 msgr2=0x7fe69806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:30.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 shutdown_connections 2024-09-10T23:06:30.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:29.827+0000 7fe69e89e640 1 -- 172.21.15.120:0/4182411500 wait complete. 2024-09-10T23:06:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:30 smithi120 ceph-mon[31247]: pgmap v768: 97 pgs: 97 active+clean; 262 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 369 op/s 2024-09-10T23:06:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:30 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:06:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:30 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:30 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:06:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:30 smithi153 ceph-mon[34951]: pgmap v768: 97 pgs: 97 active+clean; 262 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 369 op/s 2024-09-10T23:06:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:30 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:06:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:30 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:30 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:06:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:30 smithi176 ceph-mon[34982]: pgmap v768: 97 pgs: 97 active+clean; 262 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 369 op/s 2024-09-10T23:06:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:30 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:06:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:30 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:30.574 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:30 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:06:30.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36194 25.04 MB/sec warmup 85 sec latency 219.911 ms 2024-09-10T23:06:31.132 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:06:31.393 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:06:31.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36271 24.77 MB/sec warmup 86 sec latency 272.728 ms 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- 172.21.15.120:0/1640380423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c108020 msgr2=0x7fe55c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 --2- 172.21.15.120:0/1640380423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c108020 0x7fe55c10e820 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ca0 tx=0x7fe55402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- 172.21.15.120:0/1640380423 shutdown_connections 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 --2- 172.21.15.120:0/1640380423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c108020 0x7fe55c10e820 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 --2- 172.21.15.120:0/1640380423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c1076a0 0x7fe55c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 --2- 172.21.15.120:0/1640380423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c101c90 0x7fe55c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- 172.21.15.120:0/1640380423 >> 172.21.15.120:0/1640380423 conn(0x7fe55c0fd700 msgr2=0x7fe55c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- 172.21.15.120:0/1640380423 shutdown_connections 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- 172.21.15.120:0/1640380423 wait complete. 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 Processor -- start 2024-09-10T23:06:31.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.937+0000 7fe561780640 1 -- start start 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 0x7fe55c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c108020 0x7fe55c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c111ce0 con 0x7fe55c1076a0 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c111b60 con 0x7fe55c101c90 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe561780640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c111e60 con 0x7fe55c108020 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47978/0 (socket says 172.21.15.120:47978) 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 -- 172.21.15.120:0/617905289 learned_addr learned my addr 172.21.15.120:0/617905289 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c108020 0x7fe55c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe55affd640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 0x7fe55c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 -- 172.21.15.120:0/617905289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c108020 msgr2=0x7fe55c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c108020 0x7fe55c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 -- 172.21.15.120:0/617905289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 msgr2=0x7fe55c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 0x7fe55c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 -- 172.21.15.120:0/617905289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19f2f0 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe55affd640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 0x7fe55c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe5527fc640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe54400e9f0 tx=0x7fe54400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.938+0000 7fe558ff9640 1 -- 172.21.15.120:0/617905289 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400cd80 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.939+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19f5e0 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.939+0000 7fe558ff9640 1 -- 172.21.15.120:0/617905289 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5440108e0 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.939+0000 7fe558ff9640 1 -- 172.21.15.120:0/617905289 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544018980 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.939+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a5e70 con 0x7fe55c1076a0 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 Processor -- start 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 -- start start 2024-09-10T23:06:31.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe558ff9640 1 -- 172.21.15.120:0/617905289 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe544010430 con 0x7fe55c1076a0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47f6d60 0x558fb47f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb475ce50 con 0x558fb47e69c0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558fb474c280 con 0x558fb47e60a0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.941+0000 7fe563b55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558fb4750090 con 0x558fb47f6d60 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47994/0 (socket says 172.21.15.120:47994) 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55affd640 1 -- 172.21.15.120:0/3291336951 learned_addr learned my addr 172.21.15.120:0/3291336951 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe5527fc640 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47f6d60 0x558fb47f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 -- 172.21.15.120:0/3291336951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47f6d60 msgr2=0x558fb47f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47f6d60 0x558fb47f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 -- 172.21.15.120:0/3291336951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 msgr2=0x558fb47f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 -- 172.21.15.120:0/3291336951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb44aa650 con 0x558fb47e60a0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55affd640 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.942+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47e6480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe554005e30 tx=0x7fe554005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f336276507a83c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe5517fa640 1 -- 172.21.15.120:0/3291336951 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554031cf0 con 0x558fb47e60a0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe558ff9640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 0x7fe5480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe5517fa640 1 -- 172.21.15.120:0/3291336951 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe554036030 con 0x558fb47e60a0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe5517fa640 1 -- 172.21.15.120:0/3291336951 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004040 con 0x558fb47e60a0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe55affd640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 0x7fe5480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe558ff9640 1 -- 172.21.15.120:0/617905289 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe5440e5f40 con 0x7fe55c1076a0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe55affd640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 0x7fe5480b77a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x558fb47ea2e0 tx=0x7fe54c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe563b55740 1 -- 172.21.15.120:0/3291336951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 msgr2=0x558fb47e6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.943+0000 7fe563b55740 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47e6480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe554005e30 tx=0x7fe554005ef0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 -- 172.21.15.120:0/3291336951 shutdown_connections 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47f6d60 0x558fb47f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e69c0 0x558fb47f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- 172.21.15.120:0/3291336951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47e6480 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 -- 172.21.15.120:0/3291336951 >> 172.21.15.120:0/3291336951 conn(0x558fb47e3ed0 msgr2=0x558fb47e5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 -- 172.21.15.120:0/3291336951 shutdown_connections 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 -- 172.21.15.120:0/3291336951 wait complete. 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 -- start start 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 Processor -- start 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.944+0000 7fe563b55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe563b55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558fb45ad700 con 0x558fb47f6d60 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe563b55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558fb46e7110 con 0x558fb47e60a0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe563b55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558fb47f9830 con 0x558fb47e69c0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe563b55740 10 client.0 _ll_register_callbacks cb 0x558fb4700990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49720/0 (socket says 172.21.15.120:49720) 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 -- 172.21.15.120:0/2149877554 learned_addr learned my addr 172.21.15.120:0/2149877554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe5527fc640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 -- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 msgr2=0x558fb47f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 -- 172.21.15.120:0/2149877554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 msgr2=0x558fb47f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 -- 172.21.15.120:0/2149877554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb4714c40 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe55affd640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0058c0 tx=0x7fe54c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361b9017beb4a9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.945+0000 7fe5427fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c02c060 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe54c01d640 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe554036030 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe54c009d00 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 10 client.27500 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558fb48051c0 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe554036030 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe54c026030 con 0x558fb47e60a0 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 10 client.27500 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:31.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 20 client.27500 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4806620 0x558fb4808a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 4 client.27500 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.946+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fb475eda0 con 0x558fb4806620 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.947+0000 7fe5527fc640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4806620 0x558fb4808a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.947+0000 7fe5427fc640 10 client.27500 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.947+0000 7fe5527fc640 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4806620 0x558fb4808a50 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a721ca709ac000 server_cookie=9ed7c1cfd28de49a in_seq=0 out_seq=0 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe5427fc640 1 -- 172.21.15.120:0/2149877554 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558fb475eda0 con 0x558fb4806620 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe5427fc640 10 client.27500 handle_command_reply: tid=0 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 client.27500 shutdown 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 msgr2=0x558fb47f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0058c0 tx=0x7fe54c004730 comp rx=0 tx=0).stop 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 shutdown_connections 2024-09-10T23:06:31.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4806620 0x558fb4808a50 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 --2- 172.21.15.120:0/2149877554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47f6620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe5427fc640 0 client.27500 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 >> 172.21.15.120:0/2149877554 conn(0x558fb47e3ed0 msgr2=0x558fb47e9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 shutdown_connections 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 1 -- 172.21.15.120:0/2149877554 wait complete. 2024-09-10T23:06:31.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.952+0000 7fe563b55740 20 client.27500 trim_cache size 0 max 16384 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 Processor -- start 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 -- start start 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb480e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47f6d60 0x558fb480e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1a9fc0 con 0x558fb47e60a0 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ac840 con 0x558fb47f6d60 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe563b55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1aa630 con 0x558fb47e69c0 2024-09-10T23:06:31.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48000/0 (socket says 172.21.15.120:48000) 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 -- 172.21.15.120:0/1914136541 learned_addr learned my addr 172.21.15.120:0/1914136541 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47f6d60 0x558fb480e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe5527fc640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb480e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 -- 172.21.15.120:0/1914136541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 msgr2=0x558fb480e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb480e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 -- 172.21.15.120:0/1914136541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47f6d60 msgr2=0x558fb480e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.970+0000 7fe55affd640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47f6d60 0x558fb480e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe55affd640 1 -- 172.21.15.120:0/1914136541 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fb47e3710 con 0x558fb47e60a0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe5527fc640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb480e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe55affd640 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009550 tx=0x7fe54c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9882302334467cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe550ff9640 1 -- 172.21.15.120:0/1914136541 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037070 con 0x558fb47e60a0 2024-09-10T23:06:31.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe550ff9640 1 -- 172.21.15.120:0/1914136541 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c005020 con 0x558fb47e60a0 2024-09-10T23:06:31.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.971+0000 7fe550ff9640 1 -- 172.21.15.120:0/1914136541 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004c10 con 0x558fb47e60a0 2024-09-10T23:06:31.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- 172.21.15.120:0/1914136541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 msgr2=0x558fb4802a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.976 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009550 tx=0x7fe54c004680 comp rx=0 tx=0).stop 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- 172.21.15.120:0/1914136541 shutdown_connections 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47f6d60 0x558fb480e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb480e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- 172.21.15.120:0/1914136541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47e60a0 0x558fb4802a90 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- 172.21.15.120:0/1914136541 >> 172.21.15.120:0/1914136541 conn(0x558fb47e3ed0 msgr2=0x558fb47e4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- 172.21.15.120:0/1914136541 shutdown_connections 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- 172.21.15.120:0/1914136541 wait complete. 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- start start 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 Processor -- start 2024-09-10T23:06:31.977 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47fef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1aad00 con 0x558fb47f6d60 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ac170 con 0x558fb47e60a0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.972+0000 7fe563b55740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ab3d0 con 0x558fb47e69c0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe563b55740 10 client.0 _ll_register_callbacks cb 0x7fe54c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5527fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48012/0 (socket says 172.21.15.120:48012) 2024-09-10T23:06:31.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5527fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47ff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57410/0 (socket says 172.21.15.120:57410) 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55b7fe640 1 -- 172.21.15.120:0/449075168 learned_addr learned my addr 172.21.15.120:0/449075168 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 -- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 msgr2=0x558fb47ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 -- 172.21.15.120:0/449075168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 msgr2=0x558fb47fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 -- 172.21.15.120:0/449075168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fb47fc510 con 0x558fb47e60a0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55b7fe640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe55affd640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47fef80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0058c0 tx=0x7fe54c01e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90579c068e611d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5427fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c02c050 con 0x558fb47e60a0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54c005020 con 0x558fb47e60a0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe54c0048b0 con 0x558fb47e60a0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe54c037040 con 0x558fb47e60a0 2024-09-10T23:06:31.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe563b55740 10 client.27506 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.973+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558fb4874280 con 0x558fb47e60a0 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c005020 con 0x558fb47e60a0 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe54c026040 con 0x558fb47e60a0 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 10 client.27506 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 20 client.27506 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4874880 0x558fb4876c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 4 client.27506 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fb475a850 con 0x558fb4874880 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe5527fc640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4874880 0x558fb4876c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe5427fc640 10 client.27506 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:31.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.974+0000 7fe5527fc640 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4874880 0x558fb4876c70 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3210dfd29b6a2100 server_cookie=c13a6f2bc1cabbc4 in_seq=0 out_seq=0 2024-09-10T23:06:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe5427fc640 1 -- 172.21.15.120:0/449075168 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x558fb475a850 con 0x558fb4874880 2024-09-10T23:06:31.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe5427fc640 10 client.27506 handle_command_reply: tid=0 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 client.27506 shutdown 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 msgr2=0x558fb47fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47fef80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0058c0 tx=0x7fe54c01e640 comp rx=0 tx=0).stop 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 shutdown_connections 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558fb4874880 0x558fb4876c70 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558fb47f6d60 0x558fb47fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558fb47e69c0 0x558fb47ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 --2- 172.21.15.120:0/449075168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fb47e60a0 0x558fb47fef80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe5427fc640 0 client.27506 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 >> 172.21.15.120:0/449075168 conn(0x558fb47e3ed0 msgr2=0x558fb47e5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 shutdown_connections 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 1 -- 172.21.15.120:0/449075168 wait complete. 2024-09-10T23:06:31.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.979+0000 7fe563b55740 20 client.27506 trim_cache size 0 max 16384 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "ff0a0b63-8f51-43d9-943f-ad59d73316ad", 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:06:31.983 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.984 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.985 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:06:31.986 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.988 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:06:31.989 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.991 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.992 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:06:31.993 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.994 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.995 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:06:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 msgr2=0x7fe5480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 0x7fe5480b77a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x558fb47ea2e0 tx=0x7fe54c005f90 comp rx=0 tx=0).stop 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 msgr2=0x7fe55c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:31.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe54400e9f0 tx=0x7fe54400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 shutdown_connections 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5480b52e0 0x7fe5480b77a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe55c108020 0x7fe55c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe55c1076a0 0x7fe55c199780 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 --2- 172.21.15.120:0/617905289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe55c101c90 0x7fe55c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 >> 172.21.15.120:0/617905289 conn(0x7fe55c0fd700 msgr2=0x7fe55c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 shutdown_connections 2024-09-10T23:06:31.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:06:31.980+0000 7fe561780640 1 -- 172.21.15.120:0/617905289 wait complete. 2024-09-10T23:06:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:32 smithi120 ceph-mon[31247]: pgmap v769: 97 pgs: 97 active+clean; 248 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 551 op/s 2024-09-10T23:06:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:32 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:32 smithi153 ceph-mon[34951]: pgmap v769: 97 pgs: 97 active+clean; 248 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 551 op/s 2024-09-10T23:06:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:32 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:32 smithi176 ceph-mon[34982]: pgmap v769: 97 pgs: 97 active+clean; 248 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 551 op/s 2024-09-10T23:06:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:32 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:06:32.700 INFO:tasks.cephfs.filesystem:scrub status for tag:ff0a0b63-8f51-43d9-943f-ad59d73316ad - {'path': '/', 'tag': 'ff0a0b63-8f51-43d9-943f-ad59d73316ad', 'options': 'recursive,force'} 2024-09-10T23:06:32.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 36677 24.69 MB/sec warmup 87 sec latency 117.453 ms 2024-09-10T23:06:33.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 37236 24.63 MB/sec warmup 88 sec latency 81.527 ms 2024-09-10T23:06:34.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:34 smithi120 ceph-mon[31247]: pgmap v770: 97 pgs: 97 active+clean; 233 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 615 op/s 2024-09-10T23:06:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:34 smithi153 ceph-mon[34951]: pgmap v770: 97 pgs: 97 active+clean; 233 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 615 op/s 2024-09-10T23:06:34.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:34 smithi176 ceph-mon[34982]: pgmap v770: 97 pgs: 97 active+clean; 233 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 615 op/s 2024-09-10T23:06:34.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 37759 24.74 MB/sec warmup 89 sec latency 103.954 ms 2024-09-10T23:06:35.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:06:35.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:06:35.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:06:35.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 38313 24.88 MB/sec warmup 90 sec latency 166.619 ms 2024-09-10T23:06:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:36 smithi120 ceph-mon[31247]: pgmap v771: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 630 op/s 2024-09-10T23:06:36.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:36 smithi153 ceph-mon[34951]: pgmap v771: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 630 op/s 2024-09-10T23:06:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:36 smithi176 ceph-mon[34982]: pgmap v771: 97 pgs: 97 active+clean; 238 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 630 op/s 2024-09-10T23:06:36.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 38789 24.93 MB/sec warmup 91 sec latency 108.838 ms 2024-09-10T23:06:37.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 39189 24.96 MB/sec warmup 92 sec latency 51.037 ms 2024-09-10T23:06:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:38 smithi120 ceph-mon[31247]: pgmap v772: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.7 MiB/s wr, 772 op/s 2024-09-10T23:06:38.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:38 smithi153 ceph-mon[34951]: pgmap v772: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.7 MiB/s wr, 772 op/s 2024-09-10T23:06:38.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:38 smithi176 ceph-mon[34982]: pgmap v772: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.7 MiB/s wr, 772 op/s 2024-09-10T23:06:38.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 39510 24.85 MB/sec warmup 93 sec latency 93.447 ms 2024-09-10T23:06:39.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 39846 24.73 MB/sec warmup 94 sec latency 67.294 ms 2024-09-10T23:06:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:40 smithi120 ceph-mon[31247]: pgmap v773: 97 pgs: 97 active+clean; 255 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 704 op/s 2024-09-10T23:06:40.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:40 smithi153 ceph-mon[34951]: pgmap v773: 97 pgs: 97 active+clean; 255 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 704 op/s 2024-09-10T23:06:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:40 smithi176 ceph-mon[34982]: pgmap v773: 97 pgs: 97 active+clean; 255 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 704 op/s 2024-09-10T23:06:40.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 40114 24.61 MB/sec warmup 95 sec latency 291.935 ms 2024-09-10T23:06:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 -- 172.21.15.120:0/399902637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6981033c0 msgr2=0x7fa698109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/399902637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6981033c0 0x7fa698109d40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa690009930 tx=0x7fa69002a420 comp rx=0 tx=0).stop 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 -- 172.21.15.120:0/399902637 shutdown_connections 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/399902637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6981033c0 0x7fa698109d40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/399902637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa698102a00 0x7fa698102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/399902637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 0x7fa698101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 -- 172.21.15.120:0/399902637 >> 172.21.15.120:0/399902637 conn(0x7fa6980fd150 msgr2=0x7fa6980ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 -- 172.21.15.120:0/399902637 shutdown_connections 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.437+0000 7fa69cf2b640 1 -- 172.21.15.120:0/399902637 wait complete. 2024-09-10T23:06:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 Processor -- start 2024-09-10T23:06:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 -- start start 2024-09-10T23:06:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 0x7fa69806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6981033c0 0x7fa6981a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810d520 con 0x7fa698102a00 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810d3a0 con 0x7fa6981018b0 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa69cf2b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810d6a0 con 0x7fa6981033c0 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa68dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa68dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59828/0 (socket says 172.21.15.120:59828) 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa68dd9b640 1 -- 172.21.15.120:0/2788623282 learned_addr learned my addr 172.21.15.120:0/2788623282 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.438+0000 7fa696d9d640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6981033c0 0x7fa6981a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa69659c640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 0x7fa69806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 -- 172.21.15.120:0/2788623282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6981033c0 msgr2=0x7fa6981a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6981033c0 0x7fa6981a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 -- 172.21.15.120:0/2788623282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 msgr2=0x7fa69806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 0x7fa69806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 -- 172.21.15.120:0/2788623282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6981a09f0 con 0x7fa698102a00 2024-09-10T23:06:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e9f0 tx=0x7fa68000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68ffff640 1 -- 172.21.15.120:0/2788623282 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68000cd80 con 0x7fa698102a00 2024-09-10T23:06:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68ffff640 1 -- 172.21.15.120:0/2788623282 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6800054d0 con 0x7fa698102a00 2024-09-10T23:06:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6981a0ce0 con 0x7fa698102a00 2024-09-10T23:06:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa68ffff640 1 -- 172.21.15.120:0/2788623282 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680010690 con 0x7fa698102a00 2024-09-10T23:06:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.439+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a1180 con 0x7fa698102a00 2024-09-10T23:06:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.441+0000 7fa69fdd5740 1 Processor -- start 2024-09-10T23:06:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 -- start start 2024-09-10T23:06:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6981090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5625c68eb6f0 con 0x5625c6979860 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5625c68dab20 con 0x5625c697ebd0 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625c68de930 con 0x5625c697c190 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa68ffff640 1 -- 172.21.15.120:0/2788623282 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa680010830 con 0x7fa698102a00 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48930/0 (socket says 172.21.15.120:48930) 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59830/0 (socket says 172.21.15.120:59830) 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69659c640 1 -- 172.21.15.120:0/2557036403 learned_addr learned my addr 172.21.15.120:0/2557036403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6981090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 -- 172.21.15.120:0/2557036403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 msgr2=0x5625c697e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 -- 172.21.15.120:0/2557036403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 msgr2=0x5625c6981090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6981090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa696d9d640 1 -- 172.21.15.120:0/2557036403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c68a0530 con 0x5625c6979860 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa69659c640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.442+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6981090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa696d9d640 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa690009900 tx=0x7fa690005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b97babe7e6dd14c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa68cd99640 1 -- 172.21.15.120:0/2557036403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690040070 con 0x5625c6979860 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa68cd99640 1 -- 172.21.15.120:0/2557036403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa690004030 con 0x5625c6979860 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa68cd99640 1 -- 172.21.15.120:0/2557036403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690035040 con 0x5625c6979860 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa68ffff640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 0x7fa6840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa69659c640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 0x7fa6840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.443+0000 7fa68ffff640 1 -- 172.21.15.120:0/2788623282 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa6800e5f50 con 0x7fa698102a00 2024-09-10T23:06:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69659c640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 0x7fa6840b77a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa688003de0 tx=0x7fa688009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 -- 172.21.15.120:0/2557036403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 msgr2=0x5625c697bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa690009900 tx=0x7fa690005c00 comp rx=0 tx=0).stop 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 -- 172.21.15.120:0/2557036403 shutdown_connections 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6981090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c697e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/2557036403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c697bc50 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 -- 172.21.15.120:0/2557036403 >> 172.21.15.120:0/2557036403 conn(0x5625c69749b0 msgr2=0x5625c6975740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 -- 172.21.15.120:0/2557036403 shutdown_connections 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.444+0000 7fa69fdd5740 1 -- 172.21.15.120:0/2557036403 wait complete. 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 -- start start 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 Processor -- start 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 0x5625c6982f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5625c6812ab0 con 0x5625c697ebd0 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5625c6880730 con 0x5625c697c190 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625c6981b50 con 0x5625c6979860 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69fdd5740 10 client.0 _ll_register_callbacks cb 0x5625c68d46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa68dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 0x5625c6982f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59840/0 (socket says 172.21.15.120:59840) 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa69659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 0x5625c6982f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48932/0 (socket says 172.21.15.120:48932) 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa696d9d640 1 -- 172.21.15.120:0/1330606412 learned_addr learned my addr 172.21.15.120:0/1330606412 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.445+0000 7fa68dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59606/0 (socket says 172.21.15.120:59606) 2024-09-10T23:06:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 -- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 msgr2=0x5625c6982f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 0x5625c6982f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 -- 172.21.15.120:0/1330606412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 msgr2=0x5625c69835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 -- 172.21.15.120:0/1330606412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c663c790 con 0x5625c697ebd0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa690004470 tx=0x7fa6900044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faafc61869319fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69004c020 con 0x5625c697ebd0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa6900315a0 con 0x5625c697ebd0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa690004030 con 0x5625c697ebd0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa690040070 con 0x5625c697ebd0 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa69fdd5740 10 client.18255 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625c66b5620 con 0x5625c697ebd0 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.446+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa690004030 con 0x5625c697ebd0 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa6900035f0 con 0x5625c697ebd0 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 10 client.18255 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 10 client.18255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 20 client.18255 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c698e3b0 0x5625c69907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 4 client.18255 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c68ed640 con 0x5625c698e3b0 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69659c640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c698e3b0 0x5625c69907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa67dffb640 10 client.18255 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.447+0000 7fa69659c640 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c698e3b0 0x5625c69907a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511daed0230a51ed server_cookie=b662ba923e05ae1f in_seq=0 out_seq=0 2024-09-10T23:06:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa67dffb640 1 -- 172.21.15.120:0/1330606412 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625c68ed640 con 0x5625c698e3b0 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa67dffb640 10 client.18255 handle_command_reply: tid=0 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 client.18255 shutdown 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 msgr2=0x5625c6984180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa690004470 tx=0x7fa6900044a0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 shutdown_connections 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c698e3b0 0x5625c69907a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c6984180 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697c190 0x5625c69835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1330606412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c6979860 0x5625c6982f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.449+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 >> 172.21.15.120:0/1330606412 conn(0x5625c69749b0 msgr2=0x5625c6973bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.450+0000 7fa67dffb640 0 client.18255 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.450+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 shutdown_connections 2024-09-10T23:06:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.450+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1330606412 wait complete. 2024-09-10T23:06:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.450+0000 7fa69fdd5740 20 client.18255 trim_cache size 0 max 16384 2024-09-10T23:06:41.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 Processor -- start 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 -- start start 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c698aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6995030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a7e60 con 0x5625c6979860 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981aa6e0 con 0x5625c697ebd0 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a84d0 con 0x5625c697c190 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59842/0 (socket says 172.21.15.120:59842) 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 -- 172.21.15.120:0/4076851271 learned_addr learned my addr 172.21.15.120:0/4076851271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c698aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa696d9d640 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6995030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 -- 172.21.15.120:0/4076851271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 msgr2=0x5625c698aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c698aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 -- 172.21.15.120:0/4076851271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 msgr2=0x5625c6995030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6995030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.466+0000 7fa69659c640 1 -- 172.21.15.120:0/4076851271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c697e8f0 con 0x5625c6979860 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa69659c640 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa688016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48be3e54e0e4d4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa67ffff640 1 -- 172.21.15.120:0/4076851271 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688011b60 con 0x5625c6979860 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa67ffff640 1 -- 172.21.15.120:0/4076851271 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa688011d00 con 0x5625c6979860 2024-09-10T23:06:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa67ffff640 1 -- 172.21.15.120:0/4076851271 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688032420 con 0x5625c6979860 2024-09-10T23:06:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa69fdd5740 1 -- 172.21.15.120:0/4076851271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 msgr2=0x5625c6977f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.467+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa688016000 comp rx=0 tx=0).stop 2024-09-10T23:06:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- 172.21.15.120:0/4076851271 shutdown_connections 2024-09-10T23:06:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c697ebd0 0x5625c6995030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c698aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/4076851271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c6979860 0x5625c6977f00 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- 172.21.15.120:0/4076851271 >> 172.21.15.120:0/4076851271 conn(0x5625c69749b0 msgr2=0x5625c69767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- 172.21.15.120:0/4076851271 shutdown_connections 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- 172.21.15.120:0/4076851271 wait complete. 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- start start 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 Processor -- start 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c6979860 0x5625c69897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c6989d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a8ba0 con 0x5625c697ebd0 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981aa010 con 0x5625c6979860 2024-09-10T23:06:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a9270 con 0x5625c697c190 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69fdd5740 10 client.0 _ll_register_callbacks cb 0x7fa688009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59854/0 (socket says 172.21.15.120:59854) 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa696d9d640 1 -- 172.21.15.120:0/1002198496 learned_addr learned my addr 172.21.15.120:0/1002198496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa68dd9b640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c6989d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.468+0000 7fa69659c640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c6979860 0x5625c69897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 -- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 msgr2=0x5625c6989d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c6989d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 -- 172.21.15.120:0/1002198496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c6979860 msgr2=0x5625c69897d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c6979860 0x5625c69897d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 -- 172.21.15.120:0/1002198496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c698a5a0 con 0x5625c697ebd0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa696d9d640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa690006010 tx=0x7fa690005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae24bb34226c78d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690061060 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa6900315a0 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa688011d00 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa69004c030 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa69fdd5740 10 client.18261 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625c69fa850 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.469+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa688011d00 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa690004d70 con 0x5625c697ebd0 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 10 client.18261 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 10 client.18261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 20 client.18261 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c69fb1d0 0x5625c69fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 4 client.18261 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:06:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625c68e90f0 con 0x5625c69fb1d0 2024-09-10T23:06:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69659c640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c69fb1d0 0x5625c69fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa67dffb640 10 client.18261 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:41.470+0000 7fa69659c640 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c69fb1d0 0x5625c69fd5f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fae4cd6d9b6aa7c server_cookie=a728af4a237948b in_seq=0 out_seq=0 2024-09-10T23:06:41.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 40511 24.48 MB/sec warmup 96 sec latency 291.442 ms 2024-09-10T23:06:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.090+0000 7fa67dffb640 1 -- 172.21.15.120:0/1002198496 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625c68e90f0 con 0x5625c69fb1d0 2024-09-10T23:06:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa67dffb640 10 client.18261 handle_command_reply: tid=0 2024-09-10T23:06:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 client.18261 shutdown 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 msgr2=0x5625c69f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa690006010 tx=0x7fa690005770 comp rx=0 tx=0).stop 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 shutdown_connections 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625c69fb1d0 0x5625c69fd5f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625c697ebd0 0x5625c69f9230 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625c697c190 0x5625c6989d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa67dffb640 0 client.18261 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 --2- 172.21.15.120:0/1002198496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625c6979860 0x5625c69897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 >> 172.21.15.120:0/1002198496 conn(0x5625c69749b0 msgr2=0x5625c6977000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 shutdown_connections 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 1 -- 172.21.15.120:0/1002198496 wait complete. 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.091+0000 7fa69fdd5740 20 client.18261 trim_cache size 0 max 16384 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:06:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:06:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 msgr2=0x7fa6840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 0x7fa6840b77a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa688003de0 tx=0x7fa688009760 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 msgr2=0x7fa69819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e9f0 tx=0x7fa68000edf0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 shutdown_connections 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa6840b52e0 0x7fa6840b77a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6981033c0 0x7fa6981a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa698102a00 0x7fa69819db40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 --2- 172.21.15.120:0/2788623282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6981018b0 0x7fa69806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 >> 172.21.15.120:0/2788623282 conn(0x7fa6980fd150 msgr2=0x7fa698103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 shutdown_connections 2024-09-10T23:06:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:42.092+0000 7fa69cf2b640 1 -- 172.21.15.120:0/2788623282 wait complete. 2024-09-10T23:06:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:42 smithi120 ceph-mon[31247]: pgmap v774: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-09-10T23:06:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:42 smithi153 ceph-mon[34951]: pgmap v774: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-09-10T23:06:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:42 smithi176 ceph-mon[34982]: pgmap v774: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-09-10T23:06:42.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 41033 24.49 MB/sec warmup 97 sec latency 343.187 ms 2024-09-10T23:06:43.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 41770 24.72 MB/sec warmup 98 sec latency 35.645 ms 2024-09-10T23:06:44.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:44 smithi153 ceph-mon[34951]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-10T23:06:44.509 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:44 smithi120 ceph-mon[31247]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-10T23:06:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:44 smithi176 ceph-mon[34982]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-09-10T23:06:44.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 42332 24.85 MB/sec warmup 99 sec latency 63.034 ms 2024-09-10T23:06:45.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 42659 24.84 MB/sec warmup 100 sec latency 160.995 ms 2024-09-10T23:06:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:46 smithi176 ceph-mon[34982]: pgmap v776: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-10T23:06:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:46 smithi120 ceph-mon[31247]: pgmap v776: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-10T23:06:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:46 smithi153 ceph-mon[34951]: pgmap v776: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-10T23:06:46.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 42965 24.76 MB/sec warmup 101 sec latency 117.081 ms 2024-09-10T23:06:47.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 43237 24.60 MB/sec warmup 102 sec latency 110.282 ms 2024-09-10T23:06:48.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:48 smithi176 ceph-mon[34982]: pgmap v777: 97 pgs: 97 active+clean; 244 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-09-10T23:06:48.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:48 smithi120 ceph-mon[31247]: pgmap v777: 97 pgs: 97 active+clean; 244 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-09-10T23:06:48.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:48 smithi153 ceph-mon[34951]: pgmap v777: 97 pgs: 97 active+clean; 244 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-09-10T23:06:48.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 43604 24.52 MB/sec warmup 103 sec latency 77.080 ms 2024-09-10T23:06:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:49.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:06:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:49.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:06:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:06:49.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:06:49.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:06:49.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:06:49.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 44167 24.48 MB/sec warmup 104 sec latency 57.355 ms 2024-09-10T23:06:50.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:50 smithi176 ceph-mon[34982]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:06:50.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:50 smithi120 ceph-mon[31247]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:06:50.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:50 smithi153 ceph-mon[34951]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:06:50.868 INFO:tasks.workunit.client.0.smithi120.stdout: 10 44536 24.38 MB/sec warmup 105 sec latency 236.163 ms 2024-09-10T23:06:51.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 45230 24.59 MB/sec warmup 106 sec latency 79.426 ms 2024-09-10T23:06:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:52 smithi176 ceph-mon[34982]: pgmap v779: 97 pgs: 97 active+clean; 243 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-09-10T23:06:52.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:52 smithi120 ceph-mon[31247]: pgmap v779: 97 pgs: 97 active+clean; 243 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-09-10T23:06:52.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:52 smithi153 ceph-mon[34951]: pgmap v779: 97 pgs: 97 active+clean; 243 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-09-10T23:06:52.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 45851 24.77 MB/sec warmup 107 sec latency 65.063 ms 2024-09-10T23:06:53.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 46241 24.80 MB/sec warmup 108 sec latency 55.323 ms 2024-09-10T23:06:54.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:54 smithi176 ceph-mon[34982]: pgmap v780: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 609 op/s 2024-09-10T23:06:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:54 smithi120 ceph-mon[31247]: pgmap v780: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 609 op/s 2024-09-10T23:06:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:54 smithi153 ceph-mon[34951]: pgmap v780: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 609 op/s 2024-09-10T23:06:54.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 46542 24.70 MB/sec warmup 109 sec latency 94.388 ms 2024-09-10T23:06:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 -- 172.21.15.120:0/801891410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958101860 msgr2=0x7f9958101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 --2- 172.21.15.120:0/801891410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958101860 0x7f9958101c60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9948009cd0 tx=0x7f994802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 -- 172.21.15.120:0/801891410 shutdown_connections 2024-09-10T23:06:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 --2- 172.21.15.120:0/801891410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9958103370 0x7f9958109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 --2- 172.21.15.120:0/801891410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99581029b0 0x7f9958102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 --2- 172.21.15.120:0/801891410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958101860 0x7f9958101c60 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 -- 172.21.15.120:0/801891410 >> 172.21.15.120:0/801891410 conn(0x7f99580fd140 msgr2=0x7f99580ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 -- 172.21.15.120:0/801891410 shutdown_connections 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 -- 172.21.15.120:0/801891410 wait complete. 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.183+0000 7f995e047640 1 Processor -- start 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 -- start start 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99581029b0 0x7f9958199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f995810d4d0 con 0x7f9958103370 2024-09-10T23:06:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f995810d350 con 0x7f9958101860 2024-09-10T23:06:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f995e047640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f995810d650 con 0x7f99581029b0 2024-09-10T23:06:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f9957fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50642/0 (socket says 172.21.15.120:50642) 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 -- 172.21.15.120:0/1470698931 learned_addr learned my addr 172.21.15.120:0/1470698931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f9957fff640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51474/0 (socket says 172.21.15.120:51474) 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f9956ffd640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99581029b0 0x7f9958199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 -- 172.21.15.120:0/1470698931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99581029b0 msgr2=0x7f9958199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99581029b0 0x7f9958199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 -- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 msgr2=0x7f995819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 -- 172.21.15.120:0/1470698931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995819a910 con 0x7f9958101860 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f9957fff640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.184+0000 7f99577fe640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9948006010 tx=0x7f9948003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.185+0000 7f9954ff9640 1 -- 172.21.15.120:0/1470698931 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948004240 con 0x7f9958101860 2024-09-10T23:06:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.185+0000 7f9954ff9640 1 -- 172.21.15.120:0/1470698931 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9948003e10 con 0x7f9958101860 2024-09-10T23:06:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.185+0000 7f9954ff9640 1 -- 172.21.15.120:0/1470698931 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948030ed0 con 0x7f9958101860 2024-09-10T23:06:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.185+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995819aba0 con 0x7f9958101860 2024-09-10T23:06:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.185+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99581a10f0 con 0x7f9958101860 2024-09-10T23:06:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 Processor -- start 2024-09-10T23:06:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 -- start start 2024-09-10T23:06:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9954ff9640 1 -- 172.21.15.120:0/1470698931 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9948030070 con 0x7f9958101860 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab5dc70 0x555fcab60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab605a0 0x555fcab62990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555fcaacb6f0 con 0x555fcab62fe0 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555fcaabab20 con 0x555fcab605a0 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9960e15740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555fcaabe930 con 0x555fcab5dc70 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.187+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51486/0 (socket says 172.21.15.120:51486) 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f99577fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab605a0 0x555fcab62990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9957fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab5dc70 0x555fcab60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 -- 172.21.15.120:0/3046279935 learned_addr learned my addr 172.21.15.120:0/3046279935 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 -- 172.21.15.120:0/3046279935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab5dc70 msgr2=0x555fcab60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab5dc70 0x555fcab60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 -- 172.21.15.120:0/3046279935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab605a0 msgr2=0x555fcab62990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab605a0 0x555fcab62990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 -- 172.21.15.120:0/3046279935 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fcaa80530 con 0x555fcab62fe0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9956ffd640 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f994c009cf0 tx=0x7f994c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac27620cea91bc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f993dffb640 1 -- 172.21.15.120:0/3046279935 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c004230 con 0x555fcab62fe0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9954ff9640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 0x7f99400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f993dffb640 1 -- 172.21.15.120:0/3046279935 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c0043d0 con 0x555fcab62fe0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f993dffb640 1 -- 172.21.15.120:0/3046279935 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c019880 con 0x555fcab62fe0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9957fff640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 0x7f99400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.188+0000 7f9954ff9640 1 -- 172.21.15.120:0/1470698931 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f994802e030 con 0x7f9958101860 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- 172.21.15.120:0/3046279935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 msgr2=0x555fcab654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9957fff640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 0x7f99400b77a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9958199b50 tx=0x7f995000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f994c009cf0 tx=0x7f994c011810 comp rx=0 tx=0).stop 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- 172.21.15.120:0/3046279935 shutdown_connections 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab654a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab605a0 0x555fcab62990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 --2- 172.21.15.120:0/3046279935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab5dc70 0x555fcab60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- 172.21.15.120:0/3046279935 >> 172.21.15.120:0/3046279935 conn(0x555fcab549b0 msgr2=0x555fcab55740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- 172.21.15.120:0/3046279935 shutdown_connections 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- 172.21.15.120:0/3046279935 wait complete. 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.189+0000 7f9960e15740 1 -- start start 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 Processor -- start 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab62fe0 0x555fcab684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555fca9f2ab0 con 0x555fcab5dc70 2024-09-10T23:06:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555fcaa60730 con 0x555fcab62fe0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555fcab65fc0 con 0x555fcab605a0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 10 client.0 _ll_register_callbacks cb 0x555fcaab46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51492/0 (socket says 172.21.15.120:51492) 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 -- 172.21.15.120:0/2926083615 learned_addr learned my addr 172.21.15.120:0/2926083615 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9957fff640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab62fe0 0x555fcab684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9956ffd640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 -- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 msgr2=0x555fcab67920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab67920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 -- 172.21.15.120:0/2926083615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab62fe0 msgr2=0x555fcab684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab62fe0 0x555fcab684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 -- 172.21.15.120:0/2926083615 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fca81c790 con 0x555fcab5dc70 2024-09-10T23:06:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f99577fe640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f99480ccd40 tx=0x7f99480305a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbf7a6081257cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9936ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480b2e60 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9948030630 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.190+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c0043d0 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f99480cc7b0 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 10 client.18273 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555fca895620 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c0043d0 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9948108b70 con 0x555fcab5dc70 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 10 client.18273 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 20 client.18273 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcab72ab0 0x555fcab74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 4 client.18273 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fcaacd640 con 0x555fcab72ab0 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.191+0000 7f9956ffd640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcab72ab0 0x555fcab74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.192+0000 7f9936ffd640 10 client.18273 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.192+0000 7f9956ffd640 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcab72ab0 0x555fcab74f40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59967a13e6304c6 server_cookie=d7c4bc280b2bd35 in_seq=0 out_seq=0 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9936ffd640 1 -- 172.21.15.120:0/2926083615 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fcaacd640 con 0x555fcab72ab0 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9936ffd640 10 client.18273 handle_command_reply: tid=0 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 client.18273 shutdown 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 msgr2=0x555fcab67280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f99480ccd40 tx=0x7f99480305a0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 shutdown_connections 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcab72ab0 0x555fcab74f40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab62fe0 0x555fcab684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab67920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 --2- 172.21.15.120:0/2926083615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab5dc70 0x555fcab67280 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 >> 172.21.15.120:0/2926083615 conn(0x555fcab549b0 msgr2=0x555fcab53bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9936ffd640 0 client.18273 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 shutdown_connections 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 1 -- 172.21.15.120:0/2926083615 wait complete. 2024-09-10T23:06:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.197+0000 7f9960e15740 20 client.18273 trim_cache size 0 max 16384 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.213+0000 7f9960e15740 1 Processor -- start 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.213+0000 7f9960e15740 1 -- start start 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab66e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 0x555fcab63900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a5c50 con 0x555fcab605a0 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a84d0 con 0x555fcab5dc70 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9960e15740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a62c0 con 0x555fcab62fe0 2024-09-10T23:06:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51494/0 (socket says 172.21.15.120:51494) 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 -- 172.21.15.120:0/842769888 learned_addr learned my addr 172.21.15.120:0/842769888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f99577fe640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab66e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9957fff640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 0x555fcab63900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 -- 172.21.15.120:0/842769888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 msgr2=0x555fcab63900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 0x555fcab63900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 -- 172.21.15.120:0/842769888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 msgr2=0x555fcab66e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab66e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 -- 172.21.15.120:0/842769888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fcab68860 con 0x555fcab605a0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9957fff640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 0x555fcab63900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f9956ffd640 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f994c009cf0 tx=0x7f994c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c8c203bd528551 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.214+0000 7f993d7fa640 1 -- 172.21.15.120:0/842769888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c003a00 con 0x555fcab605a0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f993d7fa640 1 -- 172.21.15.120:0/842769888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c003ba0 con 0x555fcab605a0 2024-09-10T23:06:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f993d7fa640 1 -- 172.21.15.120:0/842769888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c019660 con 0x555fcab605a0 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 -- 172.21.15.120:0/842769888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 msgr2=0x555fcab633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f994c009cf0 tx=0x7f994c011810 comp rx=0 tx=0).stop 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 -- 172.21.15.120:0/842769888 shutdown_connections 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab62fe0 0x555fcab63900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab605a0 0x555fcab633c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 --2- 172.21.15.120:0/842769888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab66e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 -- 172.21.15.120:0/842769888 >> 172.21.15.120:0/842769888 conn(0x555fcab549b0 msgr2=0x555fcab4faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 -- 172.21.15.120:0/842769888 shutdown_connections 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.215+0000 7f9960e15740 1 -- 172.21.15.120:0/842769888 wait complete. 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 -- start start 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 Processor -- start 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab6ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a6990 con 0x555fcab62fe0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a7e00 con 0x555fcab5dc70 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99581a7060 con 0x555fcab605a0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9960e15740 10 client.0 _ll_register_callbacks cb 0x7f994810c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51504/0 (socket says 172.21.15.120:51504) 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 -- 172.21.15.120:0/4156761436 learned_addr learned my addr 172.21.15.120:0/4156761436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f99577fe640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9956ffd640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab6ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 -- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 msgr2=0x555fcab6ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab6ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 -- 172.21.15.120:0/4156761436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 msgr2=0x555fcab6c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab6c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.216+0000 7f9957fff640 1 -- 172.21.15.120:0/4156761436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fcab6de90 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9957fff640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f995001e590 tx=0x7f99500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3f052c75e106c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995002d060 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9950028420 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c003ba0 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9950028620 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 10 client.18279 fetch_fsmap learned FSMap version 26 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555fcab6f2a0 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c003ba0 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f995001d8d0 con 0x555fcab62fe0 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 10 client.18279 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:06:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 10 client.18279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 20 client.18279 populate_metadata read hostname 'smithi120' 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcabe0140 0x555fcabe2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 4 client.18279 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fcaac90f0 con 0x555fcabe0140 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.217+0000 7f99577fe640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcabe0140 0x555fcabe2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.218+0000 7f9936ffd640 10 client.18279 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.218+0000 7f99577fe640 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcabe0140 0x555fcabe2560 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899ecb8f8a139eb0 server_cookie=83d8cf030e9cb2a7 in_seq=0 out_seq=0 2024-09-10T23:06:55.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 46637 24.49 MB/sec warmup 110 sec latency 581.643 ms 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9936ffd640 1 -- 172.21.15.120:0/4156761436 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fcaac90f0 con 0x555fcabe0140 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9936ffd640 10 client.18279 handle_command_reply: tid=0 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 client.18279 shutdown 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 msgr2=0x555fcab6d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f995001e590 tx=0x7f99500046c0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 shutdown_connections 2024-09-10T23:06:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x555fcabe0140 0x555fcabe2560 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555fcab62fe0 0x555fcab6d690 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9936ffd640 0 client.18279 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555fcab605a0 0x555fcab6ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 --2- 172.21.15.120:0/4156761436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555fcab5dc70 0x555fcab6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 >> 172.21.15.120:0/4156761436 conn(0x555fcab549b0 msgr2=0x555fcab66e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 shutdown_connections 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 1 -- 172.21.15.120:0/4156761436 wait complete. 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.929+0000 7f9960e15740 20 client.18279 trim_cache size 0 max 16384 2024-09-10T23:06:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:06:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:06:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:06:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 msgr2=0x7f99400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 0x7f99400b77a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9958199b50 tx=0x7f995000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 msgr2=0x7f9958198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9948006010 tx=0x7f9948003c20 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 shutdown_connections 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99400b52e0 0x7f99400b77a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9958103370 0x7f995819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99581029b0 0x7f9958199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 --2- 172.21.15.120:0/1470698931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9958101860 0x7f9958198dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.930+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 >> 172.21.15.120:0/1470698931 conn(0x7f99580fd140 msgr2=0x7f99580feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.931+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 shutdown_connections 2024-09-10T23:06:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:06:55.931+0000 7f995e047640 1 -- 172.21.15.120:0/1470698931 wait complete. 2024-09-10T23:06:56.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:56 smithi176 ceph-mon[34982]: pgmap v781: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-10T23:06:56.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:56 smithi120 ceph-mon[31247]: pgmap v781: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-10T23:06:56.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:56 smithi153 ceph-mon[34951]: pgmap v781: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-10T23:06:56.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 46896 24.36 MB/sec warmup 111 sec latency 609.719 ms 2024-09-10T23:06:57.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 47257 24.31 MB/sec warmup 112 sec latency 138.849 ms 2024-09-10T23:06:58.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:06:58 smithi176 ceph-mon[34982]: pgmap v782: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2024-09-10T23:06:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:06:58 smithi120 ceph-mon[31247]: pgmap v782: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2024-09-10T23:06:58.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:06:58 smithi153 ceph-mon[34951]: pgmap v782: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2024-09-10T23:06:58.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 47907 24.28 MB/sec warmup 113 sec latency 40.198 ms 2024-09-10T23:06:59.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 48528 24.41 MB/sec warmup 114 sec latency 34.222 ms 2024-09-10T23:07:00.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:00 smithi176 ceph-mon[34982]: pgmap v783: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-09-10T23:07:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:00 smithi120 ceph-mon[31247]: pgmap v783: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-09-10T23:07:00.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:00 smithi153 ceph-mon[34951]: pgmap v783: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-09-10T23:07:00.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 49109 24.57 MB/sec warmup 115 sec latency 148.478 ms 2024-09-10T23:07:01.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 49561 24.59 MB/sec warmup 116 sec latency 66.829 ms 2024-09-10T23:07:02.498 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:02 smithi120 ceph-mon[31247]: pgmap v784: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 613 op/s 2024-09-10T23:07:02.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:02 smithi153 ceph-mon[34951]: pgmap v784: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 613 op/s 2024-09-10T23:07:02.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:02 smithi176 ceph-mon[34982]: pgmap v784: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 613 op/s 2024-09-10T23:07:02.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:07:02.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 49896 24.55 MB/sec warmup 117 sec latency 62.533 ms 2024-09-10T23:07:03.006 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:03.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/2210600353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f18108020 msgr2=0x7f7f1810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/2210600353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f18108020 0x7f7f1810e820 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009ca0 tx=0x7f7f1002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/2210600353 shutdown_connections 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/2210600353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f18108020 0x7f7f1810e820 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/2210600353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f181076a0 0x7f7f18107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/2210600353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18101c90 0x7f7f18102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/2210600353 >> 172.21.15.120:0/2210600353 conn(0x7f7f180fd700 msgr2=0x7f7f180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/2210600353 shutdown_connections 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.522+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/2210600353 wait complete. 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 Processor -- start 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 -- start start 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18101c90 0x7f7f181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18108020 0x7f7f1819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112de0 con 0x7f7f181076a0 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112c60 con 0x7f7f18101c90 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1ebc1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18112f60 con 0x7f7f18108020 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47514/0 (socket says 172.21.15.120:47514) 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 -- 172.21.15.120:0/1420261175 learned_addr learned my addr 172.21.15.120:0/1420261175 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18101c90 0x7f7f181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18108020 0x7f7f1819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 -- 172.21.15.120:0/1420261175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18108020 msgr2=0x7f7f1819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18108020 0x7f7f1819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 -- 172.21.15.120:0/1420261175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18101c90 msgr2=0x7f7f181992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.523+0000 7f7f17fff640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18101c90 0x7f7f181992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f17fff640 1 -- 172.21.15.120:0/1420261175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1819f380 con 0x7f7f181076a0 2024-09-10T23:07:03.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f17fff640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e9f0 tx=0x7f7f0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f15ffb640 1 -- 172.21.15.120:0/1420261175 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0400cd80 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f15ffb640 1 -- 172.21.15.120:0/1420261175 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f040108e0 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1819f670 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f15ffb640 1 -- 172.21.15.120:0/1420261175 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04018980 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.524+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a5d30 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f15ffb640 1 -- 172.21.15.120:0/1420261175 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7f04010430 con 0x7f7f181076a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 Processor -- start 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 -- start start 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e3e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 0x564641e4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564641db4e50 con 0x564641e3e0a0 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564641da4280 con 0x564641e4a950 2024-09-10T23:07:03.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f20f96740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564641da8090 con 0x564641e3e9c0 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f1d15e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e3e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f15ffb640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 0x7f7f080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.527+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40780/0 (socket says 172.21.15.120:40780) 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 -- 172.21.15.120:0/2071352376 learned_addr learned my addr 172.21.15.120:0/2071352376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f17fff640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 0x564641e4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 0x7f7f080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 -- 172.21.15.120:0/2071352376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 msgr2=0x564641e4cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 0x564641e4cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 -- 172.21.15.120:0/2071352376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 msgr2=0x564641e3e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e3e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 -- 172.21.15.120:0/2071352376 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564641b02650 con 0x564641e3e9c0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e3e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f17fff640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 0x564641e4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f15ffb640 1 -- 172.21.15.120:0/1420261175 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f040e5e20 con 0x7f7f181076a0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009cf0 tx=0x7f7f0c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad092901b4abcd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 0x7f7f080b77a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009a60 tx=0x7f7f1002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f02ffd640 1 -- 172.21.15.120:0/2071352376 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c004230 con 0x564641e3e9c0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f02ffd640 1 -- 172.21.15.120:0/2071352376 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c0043d0 con 0x564641e3e9c0 2024-09-10T23:07:03.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.528+0000 7f7f02ffd640 1 -- 172.21.15.120:0/2071352376 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c019880 con 0x564641e3e9c0 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- 172.21.15.120:0/2071352376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 msgr2=0x564641e4a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009cf0 tx=0x7f7f0c011810 comp rx=0 tx=0).stop 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- 172.21.15.120:0/2071352376 shutdown_connections 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e4a950 0x564641e4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e4a410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 --2- 172.21.15.120:0/2071352376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e3e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- 172.21.15.120:0/2071352376 >> 172.21.15.120:0/2071352376 conn(0x564641e3bed0 msgr2=0x564641e3d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- 172.21.15.120:0/2071352376 shutdown_connections 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- 172.21.15.120:0/2071352376 wait complete. 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.529+0000 7f7f20f96740 1 -- start start 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 Processor -- start 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564641c05700 con 0x564641e4a950 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564641d3f110 con 0x564641e3e9c0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564641e4d5d0 con 0x564641e3e0a0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f20f96740 10 client.0 _ll_register_callbacks cb 0x564641d58990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1d15e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40782/0 (socket says 172.21.15.120:40782) 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 -- 172.21.15.120:0/290488118 learned_addr learned my addr 172.21.15.120:0/290488118 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f17fff640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 -- 172.21.15.120:0/290488118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 msgr2=0x564641e47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e47630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.530+0000 7f7f1c95d640 1 -- 172.21.15.120:0/290488118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 msgr2=0x564641e47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e47bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f1c95d640 1 -- 172.21.15.120:0/290488118 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564641d6cc40 con 0x564641e3e0a0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009560 tx=0x7f7f0c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ede43d1e8f5db15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7ef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c027070 con 0x564641e3e0a0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0c0043d0 con 0x564641e3e0a0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f0c005240 con 0x564641e3e0a0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f0c033050 con 0x564641e3e0a0 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f20f96740 10 client.27757 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:03.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564641e58ed0 con 0x564641e3e0a0 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.531+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c0043d0 con 0x564641e3e0a0 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7f0c004b70 con 0x564641e3e0a0 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 10 client.27757 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 10 client.27757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 20 client.27757 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641e5a280 0x564641e5c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 4 client.27757 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564641db6da0 con 0x564641e5a280 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f17fff640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641e5a280 0x564641e5c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7f17fff640 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641e5a280 0x564641e5c670 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680923b83fed2e25 server_cookie=3f2527c9441a60d1 in_seq=0 out_seq=0 2024-09-10T23:07:03.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.532+0000 7f7ef7fff640 10 client.27757 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:03.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.535+0000 7f7ef7fff640 1 -- 172.21.15.120:0/290488118 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564641db6da0 con 0x564641e5a280 2024-09-10T23:07:03.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.535+0000 7f7ef7fff640 10 client.27757 handle_command_reply: tid=0 2024-09-10T23:07:03.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.535+0000 7f7f20f96740 1 client.27757 shutdown 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.535+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 msgr2=0x564641e4a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.535+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009560 tx=0x7f7f0c0038a0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 shutdown_connections 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641e5a280 0x564641e5c670 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7ef7fff640 0 client.27757 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:03.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 --2- 172.21.15.120:0/290488118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e0a0 0x564641e4a1e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 >> 172.21.15.120:0/290488118 conn(0x564641e3bed0 msgr2=0x564641e41a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 shutdown_connections 2024-09-10T23:07:03.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 1 -- 172.21.15.120:0/290488118 wait complete. 2024-09-10T23:07:03.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.536+0000 7f7f20f96740 20 client.27757 trim_cache size 0 max 16384 2024-09-10T23:07:03.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 Processor -- start 2024-09-10T23:07:03.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 -- start start 2024-09-10T23:07:03.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e4a950 0x564641e44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181aa100 con 0x564641e3e0a0 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181ac980 con 0x564641e3e9c0 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f20f96740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181aa770 con 0x564641e4a950 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.554+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47542/0 (socket says 172.21.15.120:47542) 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 -- 172.21.15.120:0/3090584621 learned_addr learned my addr 172.21.15.120:0/3090584621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:03.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f17fff640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e4a950 0x564641e44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 -- 172.21.15.120:0/3090584621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e4a950 msgr2=0x564641e44840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e4a950 0x564641e44840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 -- 172.21.15.120:0/3090584621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 msgr2=0x564641e4c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e4c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 -- 172.21.15.120:0/3090584621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564641e5cc30 con 0x564641e3e0a0 2024-09-10T23:07:03.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f17fff640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e4c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:03.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c019490 tx=0x7f7f0c0096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142a5d1e01e59b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f027fc640 1 -- 172.21.15.120:0/3090584621 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c027070 con 0x564641e3e0a0 2024-09-10T23:07:03.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f027fc640 1 -- 172.21.15.120:0/3090584621 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c009910 con 0x564641e3e0a0 2024-09-10T23:07:03.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.555+0000 7f7f027fc640 1 -- 172.21.15.120:0/3090584621 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003e00 con 0x564641e3e0a0 2024-09-10T23:07:03.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- 172.21.15.120:0/3090584621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 msgr2=0x564641e4bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c019490 tx=0x7f7f0c0096b0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.567 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- 172.21.15.120:0/3090584621 shutdown_connections 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e4a950 0x564641e44840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e9c0 0x564641e4c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 --2- 172.21.15.120:0/3090584621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e3e0a0 0x564641e4bd40 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- 172.21.15.120:0/3090584621 >> 172.21.15.120:0/3090584621 conn(0x564641e3bed0 msgr2=0x564641e3af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- 172.21.15.120:0/3090584621 shutdown_connections 2024-09-10T23:07:03.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- 172.21.15.120:0/3090584621 wait complete. 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.556+0000 7f7f20f96740 1 -- start start 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 Processor -- start 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e0a0 0x564641e52b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e53100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181aae40 con 0x564641e4a950 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181ac2b0 con 0x564641e3e0a0 2024-09-10T23:07:03.569 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181ab510 con 0x564641e3e9c0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47558/0 (socket says 172.21.15.120:47558) 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 -- 172.21.15.120:0/1644256144 learned_addr learned my addr 172.21.15.120:0/1644256144 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f17fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e53100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1c95d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e0a0 0x564641e52b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f20f96740 10 client.0 _ll_register_callbacks cb 0x7f7f0c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 -- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 msgr2=0x564641e53100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e53100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 -- 172.21.15.120:0/1644256144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e0a0 msgr2=0x564641e52b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e0a0 0x564641e52b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.557+0000 7f7f1d15e640 1 -- 172.21.15.120:0/1644256144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564641e3c950 con 0x564641e4a950 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f1d15e640 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009ca0 tx=0x7f7f100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaf8f994d3bce5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:03.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1005c060 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7f10040820 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0c009910 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f10003e40 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 10 client.18297 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564641ec7a50 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c009910 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7f1002f660 con 0x564641e4a950 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 10 client.18297 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 20 client.18297 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641ec8230 0x564641eca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 4 client.18297 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.558+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564641db2850 con 0x564641ec8230 2024-09-10T23:07:03.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.559+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641ec8230 0x564641eca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.559+0000 7f7ef7fff640 10 client.18297 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:03.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.559+0000 7f7f1c95d640 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641ec8230 0x564641eca650 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584782ae9c8c7317 server_cookie=4eb2c2e039664c12 in_seq=0 out_seq=0 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.563+0000 7f7ef7fff640 1 -- 172.21.15.120:0/1644256144 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x564641db2850 con 0x564641ec8230 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.563+0000 7f7ef7fff640 10 client.18297 handle_command_reply: tid=0 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.563+0000 7f7f20f96740 1 client.18297 shutdown 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.563+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 msgr2=0x564641e4d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.563+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009ca0 tx=0x7f7f100095c0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 shutdown_connections 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564641ec8230 0x564641eca650 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564641e4a950 0x564641e4d8d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7ef7fff640 0 client.18297 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564641e3e9c0 0x564641e53100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 --2- 172.21.15.120:0/1644256144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564641e3e0a0 0x564641e52b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 >> 172.21.15.120:0/1644256144 conn(0x564641e3bed0 msgr2=0x564641e48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 shutdown_connections 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 1 -- 172.21.15.120:0/1644256144 wait complete. 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.564+0000 7f7f20f96740 20 client.18297 trim_cache size 0 max 16384 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:03.574 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:07:03.575 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.576 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.577 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:07:03.578 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.579 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.580 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.582 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:07:03.583 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.585 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:07:03.586 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:07:03.587 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.588 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:07:03.589 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:03.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 msgr2=0x7f7f080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 0x7f7f080b77a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009a60 tx=0x7f7f1002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 msgr2=0x7f7f18199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e9f0 tx=0x7f7f0400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 shutdown_connections 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7f080b52e0 0x7f7f080b77a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7f18108020 0x7f7f1819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f181076a0 0x7f7f18199810 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 --2- 172.21.15.120:0/1420261175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f18101c90 0x7f7f181992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 >> 172.21.15.120:0/1420261175 conn(0x7f7f180fd700 msgr2=0x7f7f180fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 shutdown_connections 2024-09-10T23:07:03.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:03.565+0000 7f7f1ebc1640 1 -- 172.21.15.120:0/1420261175 wait complete. 2024-09-10T23:07:03.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 50162 24.45 MB/sec warmup 118 sec latency 82.156 ms 2024-09-10T23:07:04.382 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:07:04.382 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:07:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:04 smithi120 ceph-mon[31247]: pgmap v785: 97 pgs: 97 active+clean; 232 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-09-10T23:07:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:04 smithi176 ceph-mon[34982]: pgmap v785: 97 pgs: 97 active+clean; 232 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-09-10T23:07:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:04.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:04 smithi153 ceph-mon[34951]: pgmap v785: 97 pgs: 97 active+clean; 232 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-09-10T23:07:04.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 50493 24.34 MB/sec warmup 119 sec latency 83.686 ms 2024-09-10T23:07:05.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 -- 172.21.15.120:0/3997094064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1011c0a0 msgr2=0x7feb1011e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:05.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 --2- 172.21.15.120:0/3997094064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1011c0a0 0x7feb1011e490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802a4d0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 -- 172.21.15.120:0/3997094064 shutdown_connections 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 --2- 172.21.15.120:0/3997094064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1011c0a0 0x7feb1011e490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 --2- 172.21.15.120:0/3997094064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1010a530 0x7feb1010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.547+0000 7feb16787640 1 --2- 172.21.15.120:0/3997094064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a150 0x7feb101111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.548+0000 7feb16787640 1 -- 172.21.15.120:0/3997094064 >> 172.21.15.120:0/3997094064 conn(0x7feb1006dfc0 msgr2=0x7feb1006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.548+0000 7feb16787640 1 -- 172.21.15.120:0/3997094064 shutdown_connections 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.548+0000 7feb16787640 1 -- 172.21.15.120:0/3997094064 wait complete. 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.548+0000 7feb16787640 1 Processor -- start 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 -- start start 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1010a150 0x7feb101194f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 0x7feb10119a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 0x7feb101128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:05.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb10121cb0 con 0x7feb1011c0a0 2024-09-10T23:07:05.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb10121b30 con 0x7feb1010a150 2024-09-10T23:07:05.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb16787640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feb10121e30 con 0x7feb1010a530 2024-09-10T23:07:05.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.549+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 0x7feb10119a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:05.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.550+0000 7feb0f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 0x7feb10119a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40798/0 (socket says 172.21.15.120:40798) 2024-09-10T23:07:05.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.550+0000 7feb0f7fe640 1 -- 172.21.15.120:0/4258223017 learned_addr learned my addr 172.21.15.120:0/4258223017 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 0x7feb101128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 -- 172.21.15.120:0/4258223017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 msgr2=0x7feb10119a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 0x7feb10119a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 -- 172.21.15.120:0/4258223017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1010a150 msgr2=0x7feb101194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1010a150 0x7feb101194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.552+0000 7feb14d24640 1 -- 172.21.15.120:0/4258223017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10113180 con 0x7feb1011c0a0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb14d24640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 0x7feb101128c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feb0802a9f0 tx=0x7feb08003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0803f070 con 0x7feb1011c0a0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08031d60 con 0x7feb1011c0a0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08035030 con 0x7feb1011c0a0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb16787640 1 -- 172.21.15.120:0/4258223017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb10113410 con 0x7feb1011c0a0 2024-09-10T23:07:05.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.553+0000 7feb16787640 1 -- 172.21.15.120:0/4258223017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101183b0 con 0x7feb1011c0a0 2024-09-10T23:07:05.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.557+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb10074940 con 0x7feb1011c0a0 2024-09-10T23:07:05.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.559+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7feb0803b5f0 con 0x7feb1011c0a0 2024-09-10T23:07:05.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.560+0000 7feb0d7fa640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 0x7feafc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:05.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.561+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7feb08104710 con 0x7feb1011c0a0 2024-09-10T23:07:05.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.561+0000 7feb0ffff640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 0x7feafc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:05.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.572+0000 7feb0ffff640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 0x7feafc0b7820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feaf80096f0 tx=0x7feaf8006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:05.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.573+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7feb080cd9c0 con 0x7feb1011c0a0 2024-09-10T23:07:05.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.824+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb10118f10 con 0x7feb1011c0a0 2024-09-10T23:07:05.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.824+0000 7feb0d7fa640 1 -- 172.21.15.120:0/4258223017 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7feb080cd2c0 con 0x7feb1011c0a0 2024-09-10T23:07:05.827 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:07:05.827 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:07:05.827 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 msgr2=0x7feafc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 0x7feafc0b7820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feaf80096f0 tx=0x7feaf8006c40 comp rx=0 tx=0).stop 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 msgr2=0x7feb101128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 0x7feb101128c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feb0802a9f0 tx=0x7feb08003cd0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 shutdown_connections 2024-09-10T23:07:05.829 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7feafc0b5360 0x7feafc0b7820 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1011c0a0 0x7feb101128c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb1010a530 0x7feb10119a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 --2- 172.21.15.120:0/4258223017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb1010a150 0x7feb101194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 >> 172.21.15.120:0/4258223017 conn(0x7feb1006dfc0 msgr2=0x7feb1011c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 shutdown_connections 2024-09-10T23:07:05.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:05.827+0000 7feb06ffd640 1 -- 172.21.15.120:0/4258223017 wait complete. 2024-09-10T23:07:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:06 smithi120 ceph-mon[31247]: pgmap v786: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-09-10T23:07:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/4258223017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:06.549 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:07:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:06 smithi176 ceph-mon[34982]: pgmap v786: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-09-10T23:07:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/4258223017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:06 smithi153 ceph-mon[34951]: pgmap v786: 97 pgs: 97 active+clean; 238 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-09-10T23:07:06.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/4258223017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:06.816 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:06.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 51423 19.93 MB/sec execute 1 sec latency 223.481 ms 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 -- 172.21.15.120:0/276561060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac1045c0 msgr2=0x7f53ac10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 --2- 172.21.15.120:0/276561060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac1045c0 0x7f53ac10cab0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009900 tx=0x7f53a002a410 comp rx=0 tx=0).stop 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 -- 172.21.15.120:0/276561060 shutdown_connections 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 --2- 172.21.15.120:0/276561060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac10d100 0x7f53ac10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 --2- 172.21.15.120:0/276561060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac1045c0 0x7f53ac10cab0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 --2- 172.21.15.120:0/276561060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac103ca0 0x7f53ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 -- 172.21.15.120:0/276561060 >> 172.21.15.120:0/276561060 conn(0x7f53ac0fd700 msgr2=0x7f53ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.641 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 -- 172.21.15.120:0/276561060 shutdown_connections 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.628+0000 7f53b13b0640 1 -- 172.21.15.120:0/276561060 wait complete. 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 Processor -- start 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 -- start start 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 0x7f53ac199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 0x7f53ac19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac112d70 con 0x7f53ac103ca0 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac112bf0 con 0x7f53ac1045c0 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53b13b0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac112ef0 con 0x7f53ac10d100 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47592/0 (socket says 172.21.15.120:47592) 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 0x7f53ac19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53aaffd640 1 -- 172.21.15.120:0/3299053236 learned_addr learned my addr 172.21.15.120:0/3299053236 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.629+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 0x7f53ac199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 -- 172.21.15.120:0/3299053236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 msgr2=0x7f53ac19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 0x7f53ac19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 -- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 msgr2=0x7f53ac199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 -- 172.21.15.120:0/3299053236 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac19f3d0 con 0x7f53ac1045c0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 0x7f53ac19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aaffd640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 0x7f53ac199860 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53a002a930 tx=0x7f53a0031970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f5393fff640 1 -- 172.21.15.120:0/3299053236 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a003f070 con 0x7f53ac1045c0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f5393fff640 1 -- 172.21.15.120:0/3299053236 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a0003aa0 con 0x7f53ac1045c0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac19f660 con 0x7f53ac1045c0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f5393fff640 1 -- 172.21.15.120:0/3299053236 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0035070 con 0x7f53ac1045c0 2024-09-10T23:07:07.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.630+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac1a5d80 con 0x7f53ac1045c0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f5393fff640 1 -- 172.21.15.120:0/3299053236 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f53a004b060 con 0x7f53ac1045c0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 Processor -- start 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 -- start start 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422aa10 0x55741423a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5574141a0ea0 con 0x55741423adb0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5574141902d0 con 0x55741422aa10 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53b3785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5574141940e0 con 0x55741422a0f0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40822/0 (socket says 172.21.15.120:40822) 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.633+0000 7f53ab7fe640 1 -- 172.21.15.120:0/2447438802 learned_addr learned my addr 172.21.15.120:0/2447438802 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53aaffd640 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422aa10 0x55741423a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f5393fff640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 0x7f53940c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 -- 172.21.15.120:0/2447438802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422aa10 msgr2=0x55741423a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422aa10 0x55741423a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 -- 172.21.15.120:0/2447438802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 msgr2=0x55741423d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 -- 172.21.15.120:0/2447438802 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557413eee650 con 0x55741422a0f0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 0x7f53940c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f5393fff640 1 -- 172.21.15.120:0/3299053236 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53a0105780 con 0x7f53ac1045c0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53a40119c0 tx=0x7f53a4011e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d9da126d92e306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53917fa640 1 -- 172.21.15.120:0/2447438802 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4017050 con 0x55741422a0f0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53917fa640 1 -- 172.21.15.120:0/2447438802 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a401cd40 con 0x55741422a0f0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.634+0000 7f53917fa640 1 -- 172.21.15.120:0/2447438802 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4010940 con 0x55741422a0f0 2024-09-10T23:07:07.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- 172.21.15.120:0/2447438802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 msgr2=0x55741422a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53a40119c0 tx=0x7f53a4011e80 comp rx=0 tx=0).stop 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 0x7f53940c8830 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f53a4023000 tx=0x7f53a4024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- 172.21.15.120:0/2447438802 shutdown_connections 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422aa10 0x55741423a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 --2- 172.21.15.120:0/2447438802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422a0f0 0x55741422a4d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- 172.21.15.120:0/2447438802 >> 172.21.15.120:0/2447438802 conn(0x557414227f20 msgr2=0x5574142290c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- 172.21.15.120:0/2447438802 shutdown_connections 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- 172.21.15.120:0/2447438802 wait complete. 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 -- start start 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.635+0000 7f53b3785740 1 Processor -- start 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x557414237ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 0x557414238110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557413ff1700 con 0x55741422a0f0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55741412b160 con 0x55741423adb0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55741423da60 con 0x55741422aa10 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53b3785740 10 client.0 _ll_register_callbacks cb 0x5574141449e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aa7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x557414237ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 0x557414238110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47612/0 (socket says 172.21.15.120:47612) 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 -- 172.21.15.120:0/57177805 learned_addr learned my addr 172.21.15.120:0/57177805 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 -- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 msgr2=0x557414237ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x557414237ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 -- 172.21.15.120:0/57177805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 msgr2=0x557414238110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 0x557414238110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 -- 172.21.15.120:0/57177805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557414158c90 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 0x557414238110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x557414237ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f53aaffd640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f539c012d20 tx=0x7f539c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef5bb3451d4ae25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.636+0000 7f538eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c005020 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f539c0051c0 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a401cd40 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f539c0053c0 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 10 client.18312 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557414249440 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f539c005560 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f539c01c520 con 0x55741422a0f0 2024-09-10T23:07:07.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 10 client.18312 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 20 client.18312 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55741424a900 0x55741424cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 4 client.18312 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.637+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f53a401cd40 con 0x55741424a900 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.638+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55741424a900 0x55741424cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.638+0000 7f538eff5640 10 client.18312 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:07.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.638+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55741424a900 0x55741424cd40 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919634fef28c2f97 server_cookie=db7c098d6ac19d07 in_seq=0 out_seq=0 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f538eff5640 1 -- 172.21.15.120:0/57177805 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f53a401cd40 con 0x55741424a900 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f538eff5640 10 client.18312 handle_command_reply: tid=0 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 client.18312 shutdown 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 msgr2=0x55741423a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f539c012d20 tx=0x7f539c00f790 comp rx=0 tx=0).stop 2024-09-10T23:07:07.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 shutdown_connections 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55741424a900 0x55741424cd40 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741423adb0 0x557414238110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x557414237ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 --2- 172.21.15.120:0/57177805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422a0f0 0x55741423a690 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f538eff5640 0 client.18312 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 >> 172.21.15.120:0/57177805 conn(0x557414227f20 msgr2=0x55741422dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 shutdown_connections 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 1 -- 172.21.15.120:0/57177805 wait complete. 2024-09-10T23:07:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.641+0000 7f53b3785740 20 client.18312 trim_cache size 0 max 16384 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 Processor -- start 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 -- start start 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x557414246cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x5574142230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac070c60 con 0x55741423adb0 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac0734e0 con 0x55741422a0f0 2024-09-10T23:07:07.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53b3785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac0712d0 con 0x55741422aa10 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53aa7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x5574142230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47624/0 (socket says 172.21.15.120:47624) 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x557414246cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.657+0000 7f53ab7fe640 1 -- 172.21.15.120:0/2653855686 learned_addr learned my addr 172.21.15.120:0/2653855686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 -- 172.21.15.120:0/2653855686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 msgr2=0x5574142230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x5574142230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 -- 172.21.15.120:0/2653855686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 msgr2=0x55741423bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 -- 172.21.15.120:0/2653855686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55741422a840 con 0x55741422a0f0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f53aaffd640 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x557414246cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f539c01c9c0 tx=0x7f539c009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2141a63c363fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f5390ff9640 1 -- 172.21.15.120:0/2653855686 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c005560 con 0x55741422a0f0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f5390ff9640 1 -- 172.21.15.120:0/2653855686 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f539c02b020 con 0x55741422a0f0 2024-09-10T23:07:07.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.658+0000 7f5390ff9640 1 -- 172.21.15.120:0/2653855686 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c01ce20 con 0x55741422a0f0 2024-09-10T23:07:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- 172.21.15.120:0/2653855686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 msgr2=0x557414246cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x557414246cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f539c01c9c0 tx=0x7f539c009e80 comp rx=0 tx=0).stop 2024-09-10T23:07:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- 172.21.15.120:0/2653855686 shutdown_connections 2024-09-10T23:07:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741423adb0 0x55741423bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741422aa10 0x5574142230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 --2- 172.21.15.120:0/2653855686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x557414246cf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- 172.21.15.120:0/2653855686 >> 172.21.15.120:0/2653855686 conn(0x557414227f20 msgr2=0x557414229050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- 172.21.15.120:0/2653855686 shutdown_connections 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- 172.21.15.120:0/2653855686 wait complete. 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 -- start start 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.659+0000 7f53b3785740 1 Processor -- start 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x5574142353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741423adb0 0x557414243280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac0719a0 con 0x55741422aa10 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac072e10 con 0x55741422a0f0 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac072070 con 0x55741423adb0 2024-09-10T23:07:07.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53b3785740 10 client.0 _ll_register_callbacks cb 0x7f539c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aaffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x5574142353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47630/0 (socket says 172.21.15.120:47630) 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 -- 172.21.15.120:0/741034550 learned_addr learned my addr 172.21.15.120:0/741034550 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53ab7fe640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741423adb0 0x557414243280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 -- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741423adb0 msgr2=0x557414243280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741423adb0 0x557414243280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 -- 172.21.15.120:0/741034550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 msgr2=0x5574142353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x5574142353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 -- 172.21.15.120:0/741034550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557414243920 con 0x55741422aa10 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aaffd640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x5574142353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f53aa7fc640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f53a0103620 tx=0x7f53a0103180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95059aa2f1ef610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f538eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:07.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.660+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0048040 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53a00cc1d0 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f539c02b020 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53a00cb6e0 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 10 client.18318 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574142bc040 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f539c02b020 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f53a00b7170 con 0x55741422aa10 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 10 client.18318 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 20 client.18318 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574142bc6f0 0x5574142beb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 4 client.18318 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574141a2df0 con 0x5574142bc6f0 2024-09-10T23:07:07.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.661+0000 7f53aaffd640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574142bc6f0 0x5574142beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.662+0000 7f53aaffd640 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574142bc6f0 0x5574142beb10 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e47a7942b78319c server_cookie=d3ce3b1ac1df508e in_seq=0 out_seq=0 2024-09-10T23:07:07.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.662+0000 7f538eff5640 10 client.18318 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.666+0000 7f538eff5640 1 -- 172.21.15.120:0/741034550 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574141a2df0 con 0x5574142bc6f0 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.666+0000 7f538eff5640 10 client.18318 handle_command_reply: tid=0 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 client.18318 shutdown 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 msgr2=0x557414242d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f53a0103620 tx=0x7f53a0103180 comp rx=0 tx=0).stop 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 shutdown_connections 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5574142bc6f0 0x5574142beb10 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55741423adb0 0x557414243280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55741422aa10 0x557414242d40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 --2- 172.21.15.120:0/741034550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55741422a0f0 0x5574142353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 >> 172.21.15.120:0/741034550 conn(0x557414227f20 msgr2=0x55741423bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f538eff5640 0 client.18318 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 shutdown_connections 2024-09-10T23:07:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 1 -- 172.21.15.120:0/741034550 wait complete. 2024-09-10T23:07:07.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b3785740 20 client.18318 trim_cache size 0 max 16384 2024-09-10T23:07:07.670 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 msgr2=0x7f53940c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 0x7f53940c8830 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f53a4023000 tx=0x7f53a4024000 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 msgr2=0x7f53ac199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.667+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 0x7f53ac199860 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f53a002a930 tx=0x7f53a0031970 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 shutdown_connections 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53940c6370 0x7f53940c8830 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53ac10d100 0x7f53ac19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53ac1045c0 0x7f53ac199860 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 --2- 172.21.15.120:0/3299053236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53ac103ca0 0x7f53ac199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 >> 172.21.15.120:0/3299053236 conn(0x7f53ac0fd700 msgr2=0x7f53ac0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 shutdown_connections 2024-09-10T23:07:07.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:07.668+0000 7f53b13b0640 1 -- 172.21.15.120:0/3299053236 wait complete. 2024-09-10T23:07:07.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 51986 27.34 MB/sec execute 2 sec latency 94.490 ms 2024-09-10T23:07:08.422 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:08 smithi120 ceph-mon[31247]: pgmap v787: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 599 op/s 2024-09-10T23:07:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:08 smithi176 ceph-mon[34982]: pgmap v787: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 599 op/s 2024-09-10T23:07:08.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:08 smithi153 ceph-mon[34951]: pgmap v787: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.9 MiB/s wr, 599 op/s 2024-09-10T23:07:08.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 52644 33.84 MB/sec execute 3 sec latency 47.948 ms 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1979356279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc102810 msgr2=0x7f9fcc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1979356279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc102810 0x7f9fcc102c90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009930 tx=0x7f9fb802a420 comp rx=0 tx=0).stop 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1979356279 shutdown_connections 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1979356279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc103380 0x7f9fcc109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1979356279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc102810 0x7f9fcc102c90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1979356279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1979356279 >> 172.21.15.120:0/1979356279 conn(0x7f9fcc0fc880 msgr2=0x7f9fcc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1979356279 shutdown_connections 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.025+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1979356279 wait complete. 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 Processor -- start 2024-09-10T23:07:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 -- start start 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc102810 0x7f9fcc19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc103380 0x7f9fcc1a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10d4d0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10d350 con 0x7f9fcc102810 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd2e5a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10d650 con 0x7f9fcc103380 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47654/0 (socket says 172.21.15.120:47654) 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/1429668171 learned_addr learned my addr 172.21.15.120:0/1429668171 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc103380 0x7f9fcc1a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc102810 0x7f9fcc19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/1429668171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc103380 msgr2=0x7f9fcc1a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc103380 0x7f9fcc1a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/1429668171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc102810 msgr2=0x7f9fcc19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc102810 0x7f9fcc19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/1429668171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc1a09b0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.026+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.027+0000 7f9fc9ffb640 1 -- 172.21.15.120:0/1429668171 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ccc0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.027+0000 7f9fc9ffb640 1 -- 172.21.15.120:0/1429668171 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc000ce60 con 0x7f9fcc101ed0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.027+0000 7f9fc9ffb640 1 -- 172.21.15.120:0/1429668171 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0010430 con 0x7f9fcc101ed0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.027+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc1a0ca0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.027+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1a11a0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.029+0000 7f9fd5c24740 1 Processor -- start 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fc9ffb640 1 -- 172.21.15.120:0/1429668171 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9fc00046c0 con 0x7f9fcc101ed0 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 -- start start 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c78e02f6f0 con 0x55c78e0cb480 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c78e01eb20 con 0x55c78e0bb800 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c78e022930 con 0x55c78e0be500 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd13f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd13f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40868/0 (socket says 172.21.15.120:40868) 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47658/0 (socket says 172.21.15.120:47658) 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd13f7640 1 -- 172.21.15.120:0/1258697020 learned_addr learned my addr 172.21.15.120:0/1258697020 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1258697020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 msgr2=0x55c78e0bb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1258697020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 msgr2=0x55c78e0bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1258697020 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78dfe4530 con 0x55c78e0cb480 2024-09-10T23:07:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009900 tx=0x7f9fb8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baab01227705c823 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.030+0000 7f9fc9ffb640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 0x7f9fbc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fb6ffd640 1 -- 172.21.15.120:0/1258697020 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb803f070 con 0x55c78e0cb480 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 0x7f9fbc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fb6ffd640 1 -- 172.21.15.120:0/1258697020 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8004040 con 0x55c78e0cb480 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fb6ffd640 1 -- 172.21.15.120:0/1258697020 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8035070 con 0x55c78e0cb480 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fc9ffb640 1 -- 172.21.15.120:0/1429668171 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fc00e5f80 con 0x7f9fcc101ed0 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 0x7f9fbc0b77a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4012840 tx=0x7f9fc4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1258697020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 msgr2=0x55c78e0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.031+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009900 tx=0x7f9fb8032b20 comp rx=0 tx=0).stop 2024-09-10T23:07:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1258697020 shutdown_connections 2024-09-10T23:07:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cd8b0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1258697020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0bb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1258697020 >> 172.21.15.120:0/1258697020 conn(0x55c78e0bcdc0 msgr2=0x55c78e0bdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1258697020 shutdown_connections 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1258697020 wait complete. 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 -- start start 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.032+0000 7f9fd5c24740 1 Processor -- start 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 0x55c78e0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 0x55c78e0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 0x55c78e0cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47668/0 (socket says 172.21.15.120:47668) 2024-09-10T23:07:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0cb480 0x55c78e0d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/1043737607 learned_addr learned my addr 172.21.15.120:0/1043737607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c78df56ab0 con 0x55c78e0bb800 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c78dfc4730 con 0x55c78e0cb480 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c78e0ce370 con 0x55c78e0be500 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd5c24740 10 client.0 _ll_register_callbacks cb 0x55c78e0186b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.033+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0cb480 0x55c78e0d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1043737607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0cb480 msgr2=0x55c78e0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0cb480 0x55c78e0d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1043737607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 msgr2=0x55c78e0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 0x55c78e0cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78dd80790 con 0x55c78e0be500 2024-09-10T23:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0cfd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb80044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5a1662b7818b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fabfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb803f070 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fb80035f0 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8004040 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fb8035040 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fd5c24740 10 client.27796 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.034+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c78ddf9620 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8004040 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9fb8004990 con 0x55c78e0be500 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 10 client.27796 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 10 client.27796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 20 client.27796 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e0dcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 4 client.27796 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c78e031640 con 0x55c78e0dab70 2024-09-10T23:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.035+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e0dcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.036+0000 7f9fabfff640 10 client.27796 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.036+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e0dcf60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3a72a545914c01 server_cookie=fd74394e3cef924f in_seq=0 out_seq=0 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fabfff640 1 -- 172.21.15.120:0/1043737607 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c78e031640 con 0x55c78e0dab70 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fabfff640 10 client.27796 handle_command_reply: tid=0 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fd5c24740 1 client.27796 shutdown 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 msgr2=0x55c78e0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0cfd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb80044a0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.038+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 shutdown_connections 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e0dcf60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0cb480 0x55c78e0d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0be500 0x55c78e0cfd90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fabfff640 0 client.27796 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/1043737607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0bb800 0x55c78e0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 >> 172.21.15.120:0/1043737607 conn(0x55c78e0bcdc0 msgr2=0x55c78e0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 shutdown_connections 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 1 -- 172.21.15.120:0/1043737607 wait complete. 2024-09-10T23:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.039+0000 7f9fd5c24740 20 client.27796 trim_cache size 0 max 16384 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 Processor -- start 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 -- start start 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0bb800 0x55c78e0bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0be500 0x55c78e0bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a5bc0 con 0x55c78e0cb480 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a8440 con 0x55c78e0be500 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.054+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a6230 con 0x55c78e0bb800 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47684/0 (socket says 172.21.15.120:47684) 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 -- 172.21.15.120:0/147537814 learned_addr learned my addr 172.21.15.120:0/147537814 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0bb800 0x55c78e0bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0be500 0x55c78e0bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 -- 172.21.15.120:0/147537814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0bb800 msgr2=0x55c78e0bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0bb800 0x55c78e0bad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 -- 172.21.15.120:0/147537814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0be500 msgr2=0x55c78e0bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0be500 0x55c78e0bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 -- 172.21.15.120:0/147537814 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78e0ce740 con 0x55c78e0cb480 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009f60 tx=0x7f9fc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3504419080c89ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fb67fc640 1 -- 172.21.15.120:0/147537814 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4004b20 con 0x55c78e0cb480 2024-09-10T23:07:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fb67fc640 1 -- 172.21.15.120:0/147537814 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc400fdf0 con 0x55c78e0cb480 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.055+0000 7f9fb67fc640 1 -- 172.21.15.120:0/147537814 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4004500 con 0x55c78e0cb480 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- 172.21.15.120:0/147537814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 msgr2=0x55c78e0cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009f60 tx=0x7f9fc4017000 comp rx=0 tx=0).stop 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- 172.21.15.120:0/147537814 shutdown_connections 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0cb480 0x55c78e0cf6f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0be500 0x55c78e0bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/147537814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0bb800 0x55c78e0bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- 172.21.15.120:0/147537814 >> 172.21.15.120:0/147537814 conn(0x55c78e0bcdc0 msgr2=0x55c78e0bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- 172.21.15.120:0/147537814 shutdown_connections 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- 172.21.15.120:0/147537814 wait complete. 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 -- start start 2024-09-10T23:07:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.056+0000 7f9fd5c24740 1 Processor -- start 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0cb480 0x55c78e145f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a6900 con 0x55c78e0be500 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a7d70 con 0x55c78e0bb800 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a6fd0 con 0x55c78e0cb480 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd5c24740 10 client.0 _ll_register_callbacks cb 0x7f9fc00e99c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd0bf6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42654/0 (socket says 172.21.15.120:42654) 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47698/0 (socket says 172.21.15.120:47698) 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd0bf6640 1 -- 172.21.15.120:0/2967837520 learned_addr learned my addr 172.21.15.120:0/2967837520 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd13f7640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0cb480 0x55c78e145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 -- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0cb480 msgr2=0x55c78e145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0cb480 0x55c78e145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 -- 172.21.15.120:0/2967837520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 msgr2=0x55c78e0b9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 -- 172.21.15.120:0/2967837520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78e1466f0 con 0x55c78e0be500 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fcbfff640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8004f20 tx=0x7f9fb8032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba7c200f6c7548e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.057+0000 7f9fabfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb803f070 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9fb8004ad0 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc400fdf0 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fb8054460 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 10 client.18336 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c78e1479e0 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc400fdf0 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9fb8044460 con 0x55c78e0be500 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 10 client.18336 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 20 client.18336 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e14a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 4 client.18336 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.058+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c78e02d0f0 con 0x55c78e0dab70 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.059+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.059+0000 7f9fd0bf6640 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e14a420 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3937f306ab899304 server_cookie=1a1c58f88db93dc4 in_seq=0 out_seq=0 2024-09-10T23:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.059+0000 7f9fabfff640 10 client.18336 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:09.455 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:07:09.456 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:07:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fabfff640 1 -- 172.21.15.120:0/2967837520 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c78e02d0f0 con 0x55c78e0dab70 2024-09-10T23:07:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fabfff640 10 client.18336 handle_command_reply: tid=0 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 client.18336 shutdown 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 msgr2=0x55c78e1439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8004f20 tx=0x7f9fb8032a50 comp rx=0 tx=0).stop 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 shutdown_connections 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c78e0dab70 0x55c78e14a420 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c78e0cb480 0x55c78e145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c78e0be500 0x55c78e1439c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 --2- 172.21.15.120:0/2967837520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c78e0bb800 0x55c78e0b9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fabfff640 0 client.18336 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 >> 172.21.15.120:0/2967837520 conn(0x55c78e0bcdc0 msgr2=0x55c78e0baca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 shutdown_connections 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 1 -- 172.21.15.120:0/2967837520 wait complete. 2024-09-10T23:07:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.647+0000 7f9fd5c24740 20 client.18336 trim_cache size 0 max 16384 2024-09-10T23:07:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:07:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:07:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:07:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 msgr2=0x7f9fbc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 0x7f9fbc0b77a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4012840 tx=0x7f9fc4009500 comp rx=0 tx=0).stop 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 msgr2=0x7f9fcc101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000edf0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 shutdown_connections 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9fbc0b52e0 0x7f9fbc0b77a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9fcc103380 0x7f9fcc1a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9fcc102810 0x7f9fcc19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 --2- 172.21.15.120:0/1429668171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9fcc101ed0 0x7f9fcc101500 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 >> 172.21.15.120:0/1429668171 conn(0x7f9fcc0fc880 msgr2=0x7f9fcc105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 shutdown_connections 2024-09-10T23:07:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:09.648+0000 7f9fd2e5a640 1 -- 172.21.15.120:0/1429668171 wait complete. 2024-09-10T23:07:09.760 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:09.869 INFO:tasks.workunit.client.0.smithi120.stdout: 10 52936 28.91 MB/sec execute 4 sec latency 318.923 ms 2024-09-10T23:07:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 -- 172.21.15.120:0/3750958524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470104570 msgr2=0x7f547010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 --2- 172.21.15.120:0/3750958524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470104570 0x7f547010cb50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5458009900 tx=0x7f545802a410 comp rx=0 tx=0).stop 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 -- 172.21.15.120:0/3750958524 shutdown_connections 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 --2- 172.21.15.120:0/3750958524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f547010d120 0x7f547010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 --2- 172.21.15.120:0/3750958524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470104570 0x7f547010cb50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 --2- 172.21.15.120:0/3750958524 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5470103b70 0x7f5470103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 -- 172.21.15.120:0/3750958524 >> 172.21.15.120:0/3750958524 conn(0x7f54700fd700 msgr2=0x7f54700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 -- 172.21.15.120:0/3750958524 shutdown_connections 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 -- 172.21.15.120:0/3750958524 wait complete. 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.261+0000 7f5475274640 1 Processor -- start 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 -- start start 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5470104570 0x7f5470101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f547010d120 0x7f5470102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5470112d60 con 0x7f5470103b70 2024-09-10T23:07:10.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5470112be0 con 0x7f5470104570 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f5475274640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5470112ee0 con 0x7f547010d120 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47714/0 (socket says 172.21.15.120:47714) 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 -- 172.21.15.120:0/1718868048 learned_addr learned my addr 172.21.15.120:0/1718868048 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546f7fe640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f547010d120 0x7f5470102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f54667fc640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5470104570 0x7f5470101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 -- 172.21.15.120:0/1718868048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f547010d120 msgr2=0x7f5470102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f547010d120 0x7f5470102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 -- 172.21.15.120:0/1718868048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5470104570 msgr2=0x7f5470101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5470104570 0x7f5470101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.262+0000 7f546effd640 1 -- 172.21.15.120:0/1718868048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54701a7670 con 0x7f5470103b70 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f546effd640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f546cff9640 1 -- 172.21.15.120:0/1718868048 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546000cd80 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54701a7960 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f546cff9640 1 -- 172.21.15.120:0/1718868048 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54600054d0 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f546cff9640 1 -- 172.21.15.120:0/1718868048 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460010690 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.263+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701a7e70 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 Processor -- start 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 -- start start 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 0x56168c20f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 0x56168c21db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56168c185e60 con 0x56168c20f800 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56168c175290 con 0x56168c21b790 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f5477649740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56168c1790a0 con 0x56168c20eee0 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f546cff9640 1 -- 172.21.15.120:0/1718868048 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5460004e90 con 0x7f5470103b70 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f546effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.282 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 0x56168c20f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.266+0000 7f546effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47722/0 (socket says 172.21.15.120:47722) 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546effd640 1 -- 172.21.15.120:0/2522474517 learned_addr learned my addr 172.21.15.120:0/2522474517 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f54667fc640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 0x56168c21db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 -- 172.21.15.120:0/2522474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 msgr2=0x56168c21db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 0x56168c21db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 -- 172.21.15.120:0/2522474517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 msgr2=0x56168c21b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 -- 172.21.15.120:0/2522474517 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168bed3930 con 0x56168c20eee0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546effd640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f54667fc640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 0x56168c21db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546f7fe640 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 0x56168c20f2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5468009f30 tx=0x7f5468010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ce02730ccf5160 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f54657fa640 1 -- 172.21.15.120:0/2522474517 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468016070 con 0x56168c20eee0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f54657fa640 1 -- 172.21.15.120:0/2522474517 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5468004d20 con 0x56168c20eee0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f54657fa640 1 -- 172.21.15.120:0/2522474517 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468012720 con 0x56168c20eee0 2024-09-10T23:07:10.283 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.267+0000 7f546cff9640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 0x7f545c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f546effd640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 0x7f545c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f546cff9640 1 -- 172.21.15.120:0/1718868048 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f54600e5fb0 con 0x7f5470103b70 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 -- 172.21.15.120:0/2522474517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 msgr2=0x56168c20f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f546effd640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 0x7f545c0b77a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54600ac4a0 tx=0x7f54600e9a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 0x56168c20f2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5468009f30 tx=0x7f5468010800 comp rx=0 tx=0).stop 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 -- 172.21.15.120:0/2522474517 shutdown_connections 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c21b790 0x56168c21db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20f800 0x56168c21b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 --2- 172.21.15.120:0/2522474517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20eee0 0x56168c20f2c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 -- 172.21.15.120:0/2522474517 >> 172.21.15.120:0/2522474517 conn(0x56168c20cd10 msgr2=0x56168c20deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.268+0000 7f5477649740 1 -- 172.21.15.120:0/2522474517 shutdown_connections 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 -- 172.21.15.120:0/2522474517 wait complete. 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 -- start start 2024-09-10T23:07:10.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 Processor -- start 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 0x56168c218c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c21b790 0x56168c219180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56168bfd6890 con 0x56168c20eee0 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56168c10fd40 con 0x56168c20f800 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56168c21e440 con 0x56168c21b790 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f5477649740 10 client.0 _ll_register_callbacks cb 0x56168c173c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47736/0 (socket says 172.21.15.120:47736) 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f546f7fe640 1 -- 172.21.15.120:0/3415556855 learned_addr learned my addr 172.21.15.120:0/3415556855 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c21b790 0x56168c219180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.269+0000 7f54667fc640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 0x56168c218c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 -- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c21b790 msgr2=0x56168c219180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c21b790 0x56168c219180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 -- 172.21.15.120:0/3415556855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 msgr2=0x56168c218c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 0x56168c218c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 -- 172.21.15.120:0/3415556855 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168c2277f0 con 0x56168c20eee0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54667fc640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 0x56168c218c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54680059e0 tx=0x7f5468005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0675c3b42160f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54567fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468016070 con 0x56168c20eee0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5468005700 con 0x56168c20eee0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5468004d20 con 0x56168c20eee0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5468003970 con 0x56168c20eee0 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f5477649740 10 client.18348 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:10.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56168c22a1a0 con 0x56168c20eee0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.270+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5468004d20 con 0x56168c20eee0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f546800f040 con 0x56168c20eee0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 10 client.18348 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 20 client.18348 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c22d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 4 client.18348 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56168c187db0 con 0x56168c22b600 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f54667fc640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c22d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f54667fc640 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c22d9f0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d70d0571d3a142e server_cookie=955ca67e15fcff2c in_seq=0 out_seq=0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.271+0000 7f54567fc640 10 client.18348 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.272+0000 7f54567fc640 1 -- 172.21.15.120:0/3415556855 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56168c187db0 con 0x56168c22b600 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.272+0000 7f54567fc640 10 client.18348 handle_command_reply: tid=0 2024-09-10T23:07:10.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.272+0000 7f5477649740 1 client.18348 shutdown 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 msgr2=0x56168c218700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f54680059e0 tx=0x7f5468005ae0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 shutdown_connections 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c22d9f0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f54567fc640 0 client.18348 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c21b790 0x56168c219180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20f800 0x56168c218c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 --2- 172.21.15.120:0/3415556855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c20eee0 0x56168c218700 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 >> 172.21.15.120:0/3415556855 conn(0x56168c20cd10 msgr2=0x56168c20bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 shutdown_connections 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 1 -- 172.21.15.120:0/3415556855 wait complete. 2024-09-10T23:07:10.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.273+0000 7f5477649740 20 client.18348 trim_cache size 0 max 16384 2024-09-10T23:07:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 Processor -- start 2024-09-10T23:07:10.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 -- start start 2024-09-10T23:07:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c2127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c215910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.295 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.292+0000 7f5477649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54701ae4b0 con 0x56168c21b790 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f5477649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54701b0d30 con 0x56168c20eee0 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f5477649740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54701aeb20 con 0x56168c20f800 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47740/0 (socket says 172.21.15.120:47740) 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 -- 172.21.15.120:0/1258577551 learned_addr learned my addr 172.21.15.120:0/1258577551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f54667fc640 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c215910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c2127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 -- 172.21.15.120:0/1258577551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 msgr2=0x56168c215910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c215910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 -- 172.21.15.120:0/1258577551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 msgr2=0x56168c2127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c2127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.296 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 -- 172.21.15.120:0/1258577551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168c20cb40 con 0x56168c21b790 2024-09-10T23:07:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f546f7fe640 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f54680096c0 tx=0x7f5468010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1edc50b7aa3365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f5464ff9640 1 -- 172.21.15.120:0/1258577551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468016070 con 0x56168c21b790 2024-09-10T23:07:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f5464ff9640 1 -- 172.21.15.120:0/1258577551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546801f030 con 0x56168c21b790 2024-09-10T23:07:10.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.293+0000 7f5464ff9640 1 -- 172.21.15.120:0/1258577551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468005760 con 0x56168c21b790 2024-09-10T23:07:10.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 -- 172.21.15.120:0/1258577551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 msgr2=0x56168c215e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f54680096c0 tx=0x7f5468010800 comp rx=0 tx=0).stop 2024-09-10T23:07:10.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 -- 172.21.15.120:0/1258577551 shutdown_connections 2024-09-10T23:07:10.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c215e50 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.299 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c215910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 --2- 172.21.15.120:0/1258577551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c2127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 -- 172.21.15.120:0/1258577551 >> 172.21.15.120:0/1258577551 conn(0x56168c20cd10 msgr2=0x56168c212090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 -- 172.21.15.120:0/1258577551 shutdown_connections 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.294+0000 7f5477649740 1 -- 172.21.15.120:0/1258577551 wait complete. 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 -- start start 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 Processor -- start 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c220220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c2207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54701af1f0 con 0x56168c21b790 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54701b0660 con 0x56168c20eee0 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f54701af8c0 con 0x56168c20f800 2024-09-10T23:07:10.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f5477649740 10 client.0 _ll_register_callbacks cb 0x7f54600e9960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47744/0 (socket says 172.21.15.120:47744) 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f546f7fe640 1 -- 172.21.15.120:0/3048525308 learned_addr learned my addr 172.21.15.120:0/3048525308 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.295+0000 7f54667fc640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c2207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c220220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 -- 172.21.15.120:0/3048525308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 msgr2=0x56168c220220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c220220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 -- 172.21.15.120:0/3048525308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 msgr2=0x56168c2914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 -- 172.21.15.120:0/3048525308 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168c20d790 con 0x56168c20f800 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54667fc640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c2207c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5096849c5191c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:10.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54567fc640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458035040 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5458005040 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546801f030 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5458005240 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f5477649740 10 client.27820 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.296+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56168c298f30 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546801f030 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f54580035f0 con 0x56168c20f800 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 10 client.27820 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 10 client.27820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 20 client.27820 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c29b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:10.302 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 4 client.27820 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:07:10.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56168c183860 con 0x56168c22b600 2024-09-10T23:07:10.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.297+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c29b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:10.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.298+0000 7f54567fc640 10 client.27820 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:10.303 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.298+0000 7f546f7fe640 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c29b6d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2fdd0ea3f6ff17 server_cookie=e86fdc4aee13cf3d in_seq=0 out_seq=0 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f54567fc640 1 -- 172.21.15.120:0/3048525308 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56168c183860 con 0x56168c22b600 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f54567fc640 10 client.27820 handle_command_reply: tid=0 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 client.27820 shutdown 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 msgr2=0x56168c2207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c2207c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458005e70 comp rx=0 tx=0).stop 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 shutdown_connections 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56168c22b600 0x56168c29b6d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56168c21b790 0x56168c2914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56168c20f800 0x56168c2207c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f54567fc640 0 client.27820 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 --2- 172.21.15.120:0/3048525308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56168c20eee0 0x56168c220220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 >> 172.21.15.120:0/3048525308 conn(0x56168c20cd10 msgr2=0x56168c2128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 shutdown_connections 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 1 -- 172.21.15.120:0/3048525308 wait complete. 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.303+0000 7f5477649740 20 client.27820 trim_cache size 0 max 16384 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "8888eb7b-edf6-49c6-ab37-3c5f565cc85f", 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:07:10.306 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 msgr2=0x7f545c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 0x7f545c0b77a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54600ac4a0 tx=0x7f54600e9a90 comp rx=0 tx=0).stop 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 msgr2=0x7f54701019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000edf0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 shutdown_connections 2024-09-10T23:07:10.307 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f545c0b52e0 0x7f545c0b77a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f547010d120 0x7f5470102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5470104570 0x7f5470101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 --2- 172.21.15.120:0/1718868048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5470103b70 0x7f54701019f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 >> 172.21.15.120:0/1718868048 conn(0x7f54700fd700 msgr2=0x7f54700fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 shutdown_connections 2024-09-10T23:07:10.308 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:10.304+0000 7f5475274640 1 -- 172.21.15.120:0/1718868048 wait complete. 2024-09-10T23:07:10.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:10 smithi120 ceph-mon[31247]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 462 op/s 2024-09-10T23:07:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:10 smithi176 ceph-mon[34982]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 462 op/s 2024-09-10T23:07:10.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:10 smithi153 ceph-mon[34951]: pgmap v788: 97 pgs: 97 active+clean; 246 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 462 op/s 2024-09-10T23:07:10.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 53195 26.37 MB/sec execute 5 sec latency 200.077 ms 2024-09-10T23:07:11.123 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:07:11.435 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:11.560 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:11 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:07:11.560 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:11 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:11.560 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:11 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:07:11.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:11 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:07:11.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:11 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:11.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:11 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:07:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:11 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:07:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:11 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:11.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:11 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:07:11.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 53210 22.09 MB/sec execute 6 sec latency 548.442 ms 2024-09-10T23:07:12.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.000+0000 7f2c42e07640 1 -- 172.21.15.120:0/2266301795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c108fc0 msgr2=0x7f2c3c10f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.000+0000 7f2c42e07640 1 --2- 172.21.15.120:0/2266301795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c10f7c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009900 tx=0x7f2c2c02a410 comp rx=0 tx=0).stop 2024-09-10T23:07:12.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.001+0000 7f2c42e07640 1 -- 172.21.15.120:0/2266301795 shutdown_connections 2024-09-10T23:07:12.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.001+0000 7f2c42e07640 1 --2- 172.21.15.120:0/2266301795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c10f7c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.001+0000 7f2c42e07640 1 --2- 172.21.15.120:0/2266301795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108640 0x7f2c3c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.001+0000 7f2c42e07640 1 --2- 172.21.15.120:0/2266301795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c102c30 0x7f2c3c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.001+0000 7f2c42e07640 1 -- 172.21.15.120:0/2266301795 >> 172.21.15.120:0/2266301795 conn(0x7f2c3c0fe6a0 msgr2=0x7f2c3c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.002+0000 7f2c42e07640 1 -- 172.21.15.120:0/2266301795 shutdown_connections 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.002+0000 7f2c42e07640 1 -- 172.21.15.120:0/2266301795 wait complete. 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.003+0000 7f2c42e07640 1 Processor -- start 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.003+0000 7f2c42e07640 1 -- start start 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.003+0000 7f2c42e07640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c102c30 0x7f2c3c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c42e07640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c42e07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c42e07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06d430 con 0x7f2c3c108fc0 2024-09-10T23:07:12.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c42e07640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06d2b0 con 0x7f2c3c108640 2024-09-10T23:07:12.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c42e07640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06d5b0 con 0x7f2c3c102c30 2024-09-10T23:07:12.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42728/0 (socket says 172.21.15.120:42728) 2024-09-10T23:07:12.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 -- 172.21.15.120:0/1530983802 learned_addr learned my addr 172.21.15.120:0/1530983802 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:12.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c413a4640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 -- 172.21.15.120:0/1530983802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c102c30 msgr2=0x7f2c3c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c102c30 0x7f2c3c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 -- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108fc0 msgr2=0x7f2c3c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.004+0000 7f2c3bfff640 1 -- 172.21.15.120:0/1530983802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c0749a0 con 0x7f2c3c108640 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.005+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9f0 tx=0x7f2c3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.005+0000 7f2c39ffb640 1 -- 172.21.15.120:0/1530983802 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000cd80 con 0x7f2c3c108640 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.006+0000 7f2c39ffb640 1 -- 172.21.15.120:0/1530983802 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c300108e0 con 0x7f2c3c108640 2024-09-10T23:07:12.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.006+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c074c90 con 0x7f2c3c108640 2024-09-10T23:07:12.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.006+0000 7f2c39ffb640 1 -- 172.21.15.120:0/1530983802 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30018980 con 0x7f2c3c108640 2024-09-10T23:07:12.010 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.006+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1bd190 con 0x7f2c3c108640 2024-09-10T23:07:12.014 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.010+0000 7f2c39ffb640 1 -- 172.21.15.120:0/1530983802 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2c30010430 con 0x7f2c3c108640 2024-09-10T23:07:12.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.013+0000 7f2c451dc740 1 Processor -- start 2024-09-10T23:07:12.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 -- start start 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ce9348e50 con 0x564ce93def00 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ce9338280 con 0x564ce93d2650 2024-09-10T23:07:12.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c451dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ce933c090 con 0x564ce93d2f70 2024-09-10T23:07:12.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c40ba3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c40ba3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40988/0 (socket says 172.21.15.120:40988) 2024-09-10T23:07:12.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c40ba3640 1 -- 172.21.15.120:0/4210720147 learned_addr learned my addr 172.21.15.120:0/4210720147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.014+0000 7f2c413a4640 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 -- 172.21.15.120:0/4210720147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 msgr2=0x564ce93d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 -- 172.21.15.120:0/4210720147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 msgr2=0x564ce93e12f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93e12f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 -- 172.21.15.120:0/4210720147 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce9096650 con 0x564ce93d2f70 2024-09-10T23:07:12.018 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.015+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009da0 tx=0x7f2c3400b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77424cd9667f45f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.016+0000 7f2c39ffb640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 0x7f2c240b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.017+0000 7f2c413a4640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 0x7f2c240b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.017+0000 7f2c39ffb640 1 -- 172.21.15.120:0/1530983802 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2c300e5f40 con 0x7f2c3c108640 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.017+0000 7f2c22ffd640 1 -- 172.21.15.120:0/4210720147 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400bd40 con 0x564ce93d2f70 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.017+0000 7f2c22ffd640 1 -- 172.21.15.120:0/4210720147 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34017070 con 0x564ce93d2f70 2024-09-10T23:07:12.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.018+0000 7f2c451dc740 1 -- 172.21.15.120:0/4210720147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 msgr2=0x564ce93de9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.018+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009da0 tx=0x7f2c3400b420 comp rx=0 tx=0).stop 2024-09-10T23:07:12.020 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.018+0000 7f2c413a4640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 0x7f2c240b79f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c031440 tx=0x7f2c2c004300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.019+0000 7f2c451dc740 1 -- 172.21.15.120:0/4210720147 shutdown_connections 2024-09-10T23:07:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.019+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.019+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93de9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.019+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4210720147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.021 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.019+0000 7f2c451dc740 1 -- 172.21.15.120:0/4210720147 >> 172.21.15.120:0/4210720147 conn(0x564ce93d0480 msgr2=0x564ce93d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.022 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.020+0000 7f2c451dc740 1 -- 172.21.15.120:0/4210720147 shutdown_connections 2024-09-10T23:07:12.023 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.021+0000 7f2c451dc740 1 -- 172.21.15.120:0/4210720147 wait complete. 2024-09-10T23:07:12.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.022+0000 7f2c451dc740 1 -- start start 2024-09-10T23:07:12.024 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.022+0000 7f2c451dc740 1 Processor -- start 2024-09-10T23:07:12.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 0x564ce93dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564ce9199700 con 0x564ce93d2f70 2024-09-10T23:07:12.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564ce92d3110 con 0x564ce93d2650 2024-09-10T23:07:12.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564ce93e1bd0 con 0x564ce93def00 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c451dc740 10 client.0 _ll_register_callbacks cb 0x564ce92ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c413a4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c413a4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41000/0 (socket says 172.21.15.120:41000) 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c413a4640 1 -- 172.21.15.120:0/27858964 learned_addr learned my addr 172.21.15.120:0/27858964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.023+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 0x564ce93dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c413a4640 1 -- 172.21.15.120:0/27858964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 msgr2=0x564ce93de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c413a4640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c413a4640 1 -- 172.21.15.120:0/27858964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 msgr2=0x564ce93dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c413a4640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 0x564ce93dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c413a4640 1 -- 172.21.15.120:0/27858964 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce9300c40 con 0x564ce93def00 2024-09-10T23:07:12.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.024+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 0x564ce93dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:12.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.026+0000 7f2c413a4640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c033000 tx=0x7f2c2c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0e305ec8de1c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.029 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.026+0000 7f2c1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.026+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c04c020 con 0x564ce93def00 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.026+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34017070 con 0x564ce93def00 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.026+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2c2c0048c0 con 0x564ce93def00 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.027+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c2c035050 con 0x564ce93def00 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.027+0000 7f2c451dc740 10 client.27829 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:12.030 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.027+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ce93ed5d0 con 0x564ce93def00 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34017070 con 0x564ce93def00 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2c2c0324b0 con 0x564ce93def00 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 10 client.27829 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 10 client.27829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 20 client.27829 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce93eee40 0x564ce93f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 4 client.27829 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:12.031 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.028+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce934ada0 con 0x564ce93eee40 2024-09-10T23:07:12.032 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.030+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce93eee40 0x564ce93f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.031+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce93eee40 0x564ce93f1230 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9884479e46e5374e server_cookie=4889a178656aa756 in_seq=0 out_seq=0 2024-09-10T23:07:12.033 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.031+0000 7f2c1bfff640 10 client.27829 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:12.035 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.034+0000 7f2c1bfff640 1 -- 172.21.15.120:0/27858964 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ce934ada0 con 0x564ce93eee40 2024-09-10T23:07:12.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.034+0000 7f2c1bfff640 10 client.27829 handle_command_reply: tid=0 2024-09-10T23:07:12.036 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.034+0000 7f2c451dc740 1 client.27829 shutdown 2024-09-10T23:07:12.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 msgr2=0x564ce93dc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c033000 tx=0x7f2c2c02a410 comp rx=0 tx=0).stop 2024-09-10T23:07:12.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 shutdown_connections 2024-09-10T23:07:12.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce93eee40 0x564ce93f1230 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93dc260 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2f70 0x564ce93dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 --2- 172.21.15.120:0/27858964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 >> 172.21.15.120:0/27858964 conn(0x564ce93d0480 msgr2=0x564ce93d5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c1bfff640 0 client.27829 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 shutdown_connections 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 1 -- 172.21.15.120:0/27858964 wait complete. 2024-09-10T23:07:12.038 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.035+0000 7f2c451dc740 20 client.27829 trim_cache size 0 max 16384 2024-09-10T23:07:12.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 Processor -- start 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 -- start start 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800c4e0 con 0x564ce93def00 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800ed60 con 0x564ce93d2650 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c451dc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800cb50 con 0x564ce93d2f70 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c413a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c40ba3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c413a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47784/0 (socket says 172.21.15.120:47784) 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c413a4640 1 -- 172.21.15.120:0/476549391 learned_addr learned my addr 172.21.15.120:0/476549391 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:12.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c40ba3640 1 -- 172.21.15.120:0/476549391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 msgr2=0x564ce93d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.060+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c40ba3640 1 -- 172.21.15.120:0/476549391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 msgr2=0x564ce93d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c40ba3640 1 -- 172.21.15.120:0/476549391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce93cbd30 con 0x564ce93d2650 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93cfe10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009d70 tx=0x7f2c34004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5389051bc33a2a61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c227fc640 1 -- 172.21.15.120:0/476549391 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34003c10 con 0x564ce93d2650 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c227fc640 1 -- 172.21.15.120:0/476549391 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34017040 con 0x564ce93d2650 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.061+0000 7f2c227fc640 1 -- 172.21.15.120:0/476549391 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34004420 con 0x564ce93d2650 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- 172.21.15.120:0/476549391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 msgr2=0x564ce93cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93cfe10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009d70 tx=0x7f2c34004af0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- 172.21.15.120:0/476549391 shutdown_connections 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93def00 0x564ce93d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93d2f70 0x564ce93d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 --2- 172.21.15.120:0/476549391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2650 0x564ce93cfe10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- 172.21.15.120:0/476549391 >> 172.21.15.120:0/476549391 conn(0x564ce93d0480 msgr2=0x564ce93cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- 172.21.15.120:0/476549391 shutdown_connections 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- 172.21.15.120:0/476549391 wait complete. 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 -- start start 2024-09-10T23:07:12.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.062+0000 7f2c451dc740 1 Processor -- start 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 0x564ce93e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800d220 con 0x564ce93d2650 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800e690 con 0x564ce93d2f70 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:42780/0 (socket says 172.21.15.120:42780) 2024-09-10T23:07:12.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c3bfff640 1 -- 172.21.15.120:0/4015118981 learned_addr learned my addr 172.21.15.120:0/4015118981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2800d8f0 con 0x564ce93def00 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.063+0000 7f2c451dc740 10 client.0 _ll_register_callbacks cb 0x7f2c2c03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 0x564ce93e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c413a4640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 -- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 msgr2=0x564ce93e1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93e1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 -- 172.21.15.120:0/4015118981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 msgr2=0x564ce93e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 0x564ce93e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 -- 172.21.15.120:0/4015118981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce93e7a50 con 0x564ce93d2f70 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c413a4640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93e1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 0x564ce93e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c3bfff640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c300a8d00 tx=0x7f2c300abd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d9d413b2800079 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:12.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300ac240 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2c300ab400 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.064+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34017040 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c300ace30 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 10 client.27632 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ce945be90 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34017040 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f2c300ab620 con 0x564ce93d2f70 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 10 client.27632 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 10 client.27632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 20 client.27632 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce945c5f0 0x564ce945ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 4 client.27632 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:07:12.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.065+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ce9346850 con 0x564ce945c5f0 2024-09-10T23:07:12.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.066+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce945c5f0 0x564ce945ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:12.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.067+0000 7f2c40ba3640 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce945c5f0 0x564ce945ea10 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862620f3c9e47608 server_cookie=8a88fea725eb6aeb in_seq=0 out_seq=0 2024-09-10T23:07:12.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.067+0000 7f2c1bfff640 10 client.27632 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.073+0000 7f2c1bfff640 1 -- 172.21.15.120:0/4015118981 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x564ce9346850 con 0x564ce945c5f0 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.073+0000 7f2c1bfff640 10 client.27632 handle_command_reply: tid=0 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.073+0000 7f2c451dc740 1 client.27632 shutdown 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 msgr2=0x564ce93e7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c300a8d00 tx=0x7f2c300abd60 comp rx=0 tx=0).stop 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 shutdown_connections 2024-09-10T23:07:12.076 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564ce945c5f0 0x564ce945ea10 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564ce93def00 0x564ce93e1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ce93d2f70 0x564ce93e7490 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 --2- 172.21.15.120:0/4015118981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564ce93d2650 0x564ce93e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 >> 172.21.15.120:0/4015118981 conn(0x564ce93d0480 msgr2=0x564ce93d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c1bfff640 0 client.27632 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 shutdown_connections 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 1 -- 172.21.15.120:0/4015118981 wait complete. 2024-09-10T23:07:12.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.074+0000 7f2c451dc740 20 client.27632 trim_cache size 0 max 16384 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "8888eb7b-edf6-49c6-ab37-3c5f565cc85f", 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.078 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.079 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.080 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.081 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.082 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.083 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.084 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.085 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.086 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.087 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.088 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.089 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:07:12.090 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.092 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:07:12.093 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 msgr2=0x7f2c240b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 0x7f2c240b79f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c031440 tx=0x7f2c2c004300 comp rx=0 tx=0).stop 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 msgr2=0x7f2c3c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:12.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9f0 tx=0x7f2c3000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 shutdown_connections 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2c240b5530 0x7f2c240b79f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3c108fc0 0x7f2c3c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c3c108640 0x7f2c3c06ee80 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9f0 tx=0x7f2c3000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 --2- 172.21.15.120:0/1530983802 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c3c102c30 0x7f2c3c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 >> 172.21.15.120:0/1530983802 conn(0x7f2c3c0fe6a0 msgr2=0x7f2c3c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 shutdown_connections 2024-09-10T23:07:12.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:12.075+0000 7f2c42e07640 1 -- 172.21.15.120:0/1530983802 wait complete. 2024-09-10T23:07:12.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:12 smithi120 ceph-mon[31247]: pgmap v789: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 603 op/s 2024-09-10T23:07:12.545 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:12 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:12 smithi176 ceph-mon[34982]: pgmap v789: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 603 op/s 2024-09-10T23:07:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:12 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:12 smithi153 ceph-mon[34951]: pgmap v789: 97 pgs: 97 active+clean; 244 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 603 op/s 2024-09-10T23:07:12.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:12 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:12.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 53469 21.37 MB/sec execute 7 sec latency 440.024 ms 2024-09-10T23:07:12.921 INFO:tasks.cephfs.filesystem:scrub status for tag:8888eb7b-edf6-49c6-ab37-3c5f565cc85f - {'path': '/', 'tag': '8888eb7b-edf6-49c6-ab37-3c5f565cc85f', 'options': 'recursive,force'} 2024-09-10T23:07:13.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 53761 20.28 MB/sec execute 8 sec latency 58.190 ms 2024-09-10T23:07:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:14 smithi176 ceph-mon[34982]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-09-10T23:07:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:14 smithi120 ceph-mon[31247]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-09-10T23:07:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:14 smithi153 ceph-mon[34951]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-09-10T23:07:14.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 54103 19.66 MB/sec execute 9 sec latency 67.971 ms 2024-09-10T23:07:15.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:07:15.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:07:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:07:15.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 54564 19.51 MB/sec execute 10 sec latency 127.480 ms 2024-09-10T23:07:16.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:16 smithi176 ceph-mon[34982]: pgmap v791: 97 pgs: 97 active+clean; 237 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 559 op/s 2024-09-10T23:07:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:16 smithi120 ceph-mon[31247]: pgmap v791: 97 pgs: 97 active+clean; 237 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 559 op/s 2024-09-10T23:07:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:16 smithi153 ceph-mon[34951]: pgmap v791: 97 pgs: 97 active+clean; 237 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 559 op/s 2024-09-10T23:07:16.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55038 19.33 MB/sec execute 11 sec latency 220.632 ms 2024-09-10T23:07:17.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 55601 20.66 MB/sec execute 12 sec latency 70.010 ms 2024-09-10T23:07:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:18 smithi176 ceph-mon[34982]: pgmap v792: 97 pgs: 97 active+clean; 245 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-10T23:07:18.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:18 smithi120 ceph-mon[31247]: pgmap v792: 97 pgs: 97 active+clean; 245 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-10T23:07:18.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:18 smithi153 ceph-mon[34951]: pgmap v792: 97 pgs: 97 active+clean; 245 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 708 op/s 2024-09-10T23:07:18.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 56239 22.73 MB/sec execute 13 sec latency 82.225 ms 2024-09-10T23:07:19.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 56696 22.96 MB/sec execute 14 sec latency 60.968 ms 2024-09-10T23:07:20.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:20 smithi176 ceph-mon[34982]: pgmap v793: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 656 op/s 2024-09-10T23:07:20.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:20 smithi120 ceph-mon[31247]: pgmap v793: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 656 op/s 2024-09-10T23:07:20.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:20 smithi153 ceph-mon[34951]: pgmap v793: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 656 op/s 2024-09-10T23:07:20.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 56962 22.64 MB/sec execute 15 sec latency 133.407 ms 2024-09-10T23:07:21.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57192 21.80 MB/sec execute 16 sec latency 221.926 ms 2024-09-10T23:07:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:22 smithi176 ceph-mon[34982]: pgmap v794: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-10T23:07:22.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:22 smithi120 ceph-mon[31247]: pgmap v794: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-10T23:07:22.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:22 smithi153 ceph-mon[34951]: pgmap v794: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-10T23:07:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 -- 172.21.15.120:0/2350170487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 msgr2=0x7fe88410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 --2- 172.21.15.120:0/2350170487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe88410ab40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009be0 tx=0x7fe87c02a6c0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 -- 172.21.15.120:0/2350170487 shutdown_connections 2024-09-10T23:07:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 --2- 172.21.15.120:0/2350170487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe88410ab40 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 --2- 172.21.15.120:0/2350170487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 0x7fe884107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.743+0000 7fe888d4b640 1 --2- 172.21.15.120:0/2350170487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 0x7fe884105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- 172.21.15.120:0/2350170487 >> 172.21.15.120:0/2350170487 conn(0x7fe8840fd150 msgr2=0x7fe8840ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- 172.21.15.120:0/2350170487 shutdown_connections 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- 172.21.15.120:0/2350170487 wait complete. 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 Processor -- start 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- start start 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 0x7fe884198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 0x7fe884199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410e5d0 con 0x7fe884108680 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410e450 con 0x7fe884105ad0 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.744+0000 7fe888d4b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe88410e750 con 0x7fe88406bfb0 2024-09-10T23:07:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37046/0 (socket says 172.21.15.120:37046) 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 -- 172.21.15.120:0/997331431 learned_addr learned my addr 172.21.15.120:0/997331431 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe87bfff640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 0x7fe884199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe88259c640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 0x7fe884198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 -- 172.21.15.120:0/997331431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 msgr2=0x7fe884198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 0x7fe884198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 -- 172.21.15.120:0/997331431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 msgr2=0x7fe884199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 0x7fe884199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 -- 172.21.15.120:0/997331431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88419a840 con 0x7fe884108680 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe882d9d640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034040 tx=0x7fe87c030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe87b7fe640 1 -- 172.21.15.120:0/997331431 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c041020 con 0x7fe884108680 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe87b7fe640 1 -- 172.21.15.120:0/997331431 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87c0318f0 con 0x7fe884108680 2024-09-10T23:07:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe87b7fe640 1 -- 172.21.15.120:0/997331431 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c036040 con 0x7fe884108680 2024-09-10T23:07:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88419aad0 con 0x7fe884108680 2024-09-10T23:07:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.745+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841a1200 con 0x7fe884108680 2024-09-10T23:07:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 Processor -- start 2024-09-10T23:07:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87b7fe640 1 -- 172.21.15.120:0/997331431 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe87c004670 con 0x7fe884108680 2024-09-10T23:07:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 -- start start 2024-09-10T23:07:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dff2f1e6f0 con 0x55dff2fb35a0 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dff2f0db20 con 0x55dff2fb5fe0 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dff2f11930 con 0x55dff2fb0c70 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37062/0 (socket says 172.21.15.120:37062) 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:51076/0 (socket says 172.21.15.120:51076) 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe88259c640 1 -- 172.21.15.120:0/4026698231 learned_addr learned my addr 172.21.15.120:0/4026698231 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 -- 172.21.15.120:0/4026698231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 msgr2=0x55dff2fb84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 -- 172.21.15.120:0/4026698231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 msgr2=0x55dff2fb5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87bfff640 1 -- 172.21.15.120:0/4026698231 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff2ed3530 con 0x55dff2fb0c70 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.748+0000 7fe87b7fe640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 0x7fe8700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe87bfff640 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380140c554828656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe88259c640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 0x7fe8700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe87b7fe640 1 -- 172.21.15.120:0/997331431 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe87c106b60 con 0x7fe884108680 2024-09-10T23:07:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe878ff9640 1 -- 172.21.15.120:0/4026698231 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004020 con 0x55dff2fb0c70 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe878ff9640 1 -- 172.21.15.120:0/4026698231 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x55dff2fb0c70 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.749+0000 7fe878ff9640 1 -- 172.21.15.120:0/4026698231 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004b80 con 0x55dff2fb0c70 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- 172.21.15.120:0/4026698231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 msgr2=0x55dff2fb3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88259c640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 0x7fe8700b7820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe874009ee0 tx=0x7fe874005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c0097b0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- 172.21.15.120:0/4026698231 shutdown_connections 2024-09-10T23:07:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/4026698231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fb3060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- 172.21.15.120:0/4026698231 >> 172.21.15.120:0/4026698231 conn(0x55dff2fa79b0 msgr2=0x55dff2fa8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- 172.21.15.120:0/4026698231 shutdown_connections 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- 172.21.15.120:0/4026698231 wait complete. 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- start start 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 Processor -- start 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fbb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dff2e45ab0 con 0x55dff2fb35a0 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dff2eb3730 con 0x55dff2fb5fe0 2024-09-10T23:07:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dff2fb8fc0 con 0x55dff2fb0c70 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88bbf0740 10 client.0 _ll_register_callbacks cb 0x55dff2f076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:51092/0 (socket says 172.21.15.120:51092) 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37076/0 (socket says 172.21.15.120:37076) 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.750+0000 7fe88259c640 1 -- 172.21.15.120:0/2210044082 learned_addr learned my addr 172.21.15.120:0/2210044082 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe882d9d640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fbb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 -- 172.21.15.120:0/2210044082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 msgr2=0x55dff2fbb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fbb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 -- 172.21.15.120:0/2210044082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 msgr2=0x55dff2fba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 -- 172.21.15.120:0/2210044082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff2c6f790 con 0x55dff2fb0c70 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe882d9d640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe87bfff640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88259c640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe87401e9d0 tx=0x7fe874004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e39f68445e71e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402e060 con 0x55dff2fb0c70 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe874018030 con 0x55dff2fb0c70 2024-09-10T23:07:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c0054d0 con 0x55dff2fb0c70 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe874029420 con 0x55dff2fb0c70 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88bbf0740 10 client.27844 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dff2ce8620 con 0x55dff2fb0c70 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.751+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x55dff2fb0c70 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe87401c4c0 con 0x55dff2fb0c70 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 10 client.27844 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 10 client.27844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 20 client.27844 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff2fc5bf0 0x55dff2fc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 4 client.27844 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff2f20640 con 0x55dff2fc5bf0 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe87bfff640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff2fc5bf0 0x55dff2fc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe87bfff640 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff2fc5bf0 0x55dff2fc8020 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590dea055cc7d809 server_cookie=5dc9767333dabeb0 in_seq=0 out_seq=0 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.752+0000 7fe869ffb640 10 client.27844 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe869ffb640 1 -- 172.21.15.120:0/2210044082 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dff2f20640 con 0x55dff2fc5bf0 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe869ffb640 10 client.27844 handle_command_reply: tid=0 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 client.27844 shutdown 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 msgr2=0x55dff2fba300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe87401e9d0 tx=0x7fe874004730 comp rx=0 tx=0).stop 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 shutdown_connections 2024-09-10T23:07:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff2fc5bf0 0x55dff2fc8020 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb5fe0 0x55dff2fbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe869ffb640 0 client.27844 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/2210044082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb0c70 0x55dff2fba300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 >> 172.21.15.120:0/2210044082 conn(0x55dff2fa79b0 msgr2=0x55dff2fa6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 shutdown_connections 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 1 -- 172.21.15.120:0/2210044082 wait complete. 2024-09-10T23:07:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.754+0000 7fe88bbf0740 20 client.27844 trim_cache size 0 max 16384 2024-09-10T23:07:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.770+0000 7fe88bbf0740 1 Processor -- start 2024-09-10T23:07:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.770+0000 7fe88bbf0740 1 -- start start 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fa8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a5bb0 con 0x55dff2fb35a0 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a8430 con 0x55dff2fb0c70 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a6220 con 0x55dff2fb5fe0 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37088/0 (socket says 172.21.15.120:37088) 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 -- 172.21.15.120:0/763643422 learned_addr learned my addr 172.21.15.120:0/763643422 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe882d9d640 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe88259c640 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fa8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 -- 172.21.15.120:0/763643422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb5fe0 msgr2=0x55dff2fb6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 -- 172.21.15.120:0/763643422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 msgr2=0x55dff2fa8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fa8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 -- 172.21.15.120:0/763643422 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff2fbb8e0 con 0x55dff2fb35a0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe87bfff640 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd608e7066f24ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe86bfff640 1 -- 172.21.15.120:0/763643422 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0043f0 con 0x55dff2fb35a0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe86bfff640 1 -- 172.21.15.120:0/763643422 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x55dff2fb35a0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.771+0000 7fe86bfff640 1 -- 172.21.15.120:0/763643422 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00f790 con 0x55dff2fb35a0 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- 172.21.15.120:0/763643422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 msgr2=0x55dff2fb63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c0046c0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- 172.21.15.120:0/763643422 shutdown_connections 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb5fe0 0x55dff2fb6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb35a0 0x55dff2fb63c0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/763643422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fa8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- 172.21.15.120:0/763643422 >> 172.21.15.120:0/763643422 conn(0x55dff2fa79b0 msgr2=0x55dff2fabd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- 172.21.15.120:0/763643422 shutdown_connections 2024-09-10T23:07:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- 172.21.15.120:0/763643422 wait complete. 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.772+0000 7fe88bbf0740 1 -- start start 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 Processor -- start 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fc1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb35a0 0x55dff2fc0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a68f0 con 0x55dff2fb5fe0 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a7d60 con 0x55dff2fb0c70 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841a6fc0 con 0x55dff2fb35a0 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88bbf0740 10 client.0 _ll_register_callbacks cb 0x7fe874009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37102/0 (socket says 172.21.15.120:37102) 2024-09-10T23:07:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 -- 172.21.15.120:0/3030565614 learned_addr learned my addr 172.21.15.120:0/3030565614 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe87bfff640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb35a0 0x55dff2fc0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe88259c640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fc1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 -- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb35a0 msgr2=0x55dff2fc0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb35a0 0x55dff2fc0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 -- 172.21.15.120:0/3030565614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 msgr2=0x55dff2fc1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fc1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.773+0000 7fe882d9d640 1 -- 172.21.15.120:0/3030565614 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff2fbf750 con 0x55dff2fb5fe0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe882d9d640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe87c1046e0 tx=0x7fe87c0ccca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38dc3f8d2468ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0b4040 con 0x55dff2fb5fe0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe87c0f40c0 con 0x55dff2fb5fe0 2024-09-10T23:07:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c0054d0 con 0x55dff2fb5fe0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe87c0cce60 con 0x55dff2fb5fe0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 10 client.18384 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dff3032d10 con 0x55dff2fb5fe0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x55dff2fb5fe0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe87c031440 con 0x55dff2fb5fe0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 10 client.18384 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 20 client.18384 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff3033670 0x55dff3035a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 4 client.18384 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dff2f1c0f0 con 0x55dff3033670 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.774+0000 7fe88259c640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff3033670 0x55dff3035a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.775+0000 7fe88259c640 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff3033670 0x55dff3035a90 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e20bdf596ac3c14 server_cookie=a038891a3e82a10d in_seq=0 out_seq=0 2024-09-10T23:07:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:22.775+0000 7fe869ffb640 10 client.18384 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:22.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57481 21.16 MB/sec execute 17 sec latency 101.792 ms 2024-09-10T23:07:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe869ffb640 1 -- 172.21.15.120:0/3030565614 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dff2f1c0f0 con 0x55dff3033670 2024-09-10T23:07:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe869ffb640 10 client.18384 handle_command_reply: tid=0 2024-09-10T23:07:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 client.18384 shutdown 2024-09-10T23:07:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 msgr2=0x55dff2fc12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe87c1046e0 tx=0x7fe87c0ccca0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 shutdown_connections 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dff3033670 0x55dff3035a90 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dff2fb5fe0 0x55dff2fc12b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dff2fb35a0 0x55dff2fc0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe869ffb640 0 client.18384 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 --2- 172.21.15.120:0/3030565614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dff2fb0c70 0x55dff2fc1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 >> 172.21.15.120:0/3030565614 conn(0x55dff2fa79b0 msgr2=0x55dff2fabd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 shutdown_connections 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 1 -- 172.21.15.120:0/3030565614 wait complete. 2024-09-10T23:07:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.344+0000 7fe88bbf0740 20 client.18384 trim_cache size 0 max 16384 2024-09-10T23:07:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:07:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:07:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:07:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 msgr2=0x7fe8700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 0x7fe8700b7820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe874009ee0 tx=0x7fe874005f90 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 msgr2=0x7fe884199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034040 tx=0x7fe87c030d70 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 shutdown_connections 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe8700b5360 0x7fe8700b7820 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe884108680 0x7fe884199f80 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe884105ad0 0x7fe884199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 --2- 172.21.15.120:0/997331431 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe88406bfb0 0x7fe884198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 >> 172.21.15.120:0/997331431 conn(0x7fe8840fd150 msgr2=0x7fe884103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 shutdown_connections 2024-09-10T23:07:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:23.345+0000 7fe888d4b640 1 -- 172.21.15.120:0/997331431 wait complete. 2024-09-10T23:07:23.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 57719 20.50 MB/sec execute 18 sec latency 231.681 ms 2024-09-10T23:07:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:24 smithi176 ceph-mon[34982]: pgmap v795: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-09-10T23:07:24.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:24 smithi120 ceph-mon[31247]: pgmap v795: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-09-10T23:07:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:24 smithi153 ceph-mon[34951]: pgmap v795: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-09-10T23:07:24.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 58329 20.57 MB/sec execute 19 sec latency 101.086 ms 2024-09-10T23:07:25.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 58955 21.34 MB/sec execute 20 sec latency 50.693 ms 2024-09-10T23:07:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:26 smithi176 ceph-mon[34982]: pgmap v796: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 425 op/s 2024-09-10T23:07:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:26 smithi120 ceph-mon[31247]: pgmap v796: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 425 op/s 2024-09-10T23:07:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:26 smithi153 ceph-mon[34951]: pgmap v796: 97 pgs: 97 active+clean; 246 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 425 op/s 2024-09-10T23:07:26.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 59469 21.89 MB/sec execute 21 sec latency 212.433 ms 2024-09-10T23:07:27.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 60052 22.69 MB/sec execute 22 sec latency 81.480 ms 2024-09-10T23:07:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:28 smithi176 ceph-mon[34982]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.9 MiB/s wr, 663 op/s 2024-09-10T23:07:28.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:28 smithi120 ceph-mon[31247]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.9 MiB/s wr, 663 op/s 2024-09-10T23:07:28.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:28 smithi153 ceph-mon[34951]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.9 MiB/s wr, 663 op/s 2024-09-10T23:07:28.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 60431 23.00 MB/sec execute 23 sec latency 54.484 ms 2024-09-10T23:07:29.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 60715 22.49 MB/sec execute 24 sec latency 59.555 ms 2024-09-10T23:07:30.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:30 smithi176 ceph-mon[34982]: pgmap v798: 97 pgs: 97 active+clean; 239 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:07:30.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:30 smithi120 ceph-mon[31247]: pgmap v798: 97 pgs: 97 active+clean; 239 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:07:30.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:30 smithi153 ceph-mon[34951]: pgmap v798: 97 pgs: 97 active+clean; 239 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:07:30.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 61042 22.08 MB/sec execute 25 sec latency 110.577 ms 2024-09-10T23:07:31.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 61286 21.67 MB/sec execute 26 sec latency 212.629 ms 2024-09-10T23:07:32.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:32 smithi176 ceph-mon[34982]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-10T23:07:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:32 smithi120 ceph-mon[31247]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-10T23:07:32.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:32 smithi153 ceph-mon[34951]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-10T23:07:32.870 INFO:tasks.workunit.client.0.smithi120.stdout: 10 61795 21.48 MB/sec execute 27 sec latency 203.170 ms 2024-09-10T23:07:33.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 62346 21.65 MB/sec execute 28 sec latency 71.943 ms 2024-09-10T23:07:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:34 smithi176 ceph-mon[34982]: pgmap v800: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 600 op/s 2024-09-10T23:07:34.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:34 smithi120 ceph-mon[31247]: pgmap v800: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 600 op/s 2024-09-10T23:07:34.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:34 smithi153 ceph-mon[34951]: pgmap v800: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 600 op/s 2024-09-10T23:07:34.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 63050 22.43 MB/sec execute 29 sec latency 57.487 ms 2024-09-10T23:07:35.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 63667 23.18 MB/sec execute 30 sec latency 41.026 ms 2024-09-10T23:07:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 -- 172.21.15.120:0/1927953844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50105ac0 msgr2=0x7f4c50107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 --2- 172.21.15.120:0/1927953844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50105ac0 0x7f4c50107f30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009900 tx=0x7f4c4002a410 comp rx=0 tx=0).stop 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 -- 172.21.15.120:0/1927953844 shutdown_connections 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 --2- 172.21.15.120:0/1927953844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50108670 0x7f4c5010ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 --2- 172.21.15.120:0/1927953844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50105ac0 0x7f4c50107f30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 --2- 172.21.15.120:0/1927953844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.437+0000 7f4c58552640 1 -- 172.21.15.120:0/1927953844 >> 172.21.15.120:0/1927953844 conn(0x7f4c500fd140 msgr2=0x7f4c500ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- 172.21.15.120:0/1927953844 shutdown_connections 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- 172.21.15.120:0/1927953844 wait complete. 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 Processor -- start 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- start start 2024-09-10T23:07:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50105ac0 0x7f4c50199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5010e610 con 0x7f4c50108670 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5010e490 con 0x7f4c5006b470 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c58552640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5010e790 con 0x7f4c50105ac0 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c56aef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c56aef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39610/0 (socket says 172.21.15.120:39610) 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.438+0000 7f4c56aef640 1 -- 172.21.15.120:0/1968640377 learned_addr learned my addr 172.21.15.120:0/1968640377 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50105ac0 0x7f4c50199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 -- 172.21.15.120:0/1968640377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50105ac0 msgr2=0x7f4c50199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50105ac0 0x7f4c50199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 -- 172.21.15.120:0/1968640377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 msgr2=0x7f4c50198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 -- 172.21.15.120:0/1968640377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5019a880 con 0x7f4c50108670 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c56aef640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800eb10 tx=0x7f4c4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c477fe640 1 -- 172.21.15.120:0/1968640377 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48004b20 con 0x7f4c50108670 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c477fe640 1 -- 172.21.15.120:0/1968640377 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c48004cc0 con 0x7f4c50108670 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5019ab70 con 0x7f4c50108670 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c477fe640 1 -- 172.21.15.120:0/1968640377 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010670 con 0x7f4c50108670 2024-09-10T23:07:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.439+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c501a1240 con 0x7f4c50108670 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.441+0000 7f4c477fe640 1 -- 172.21.15.120:0/1968640377 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4c48005020 con 0x7f4c50108670 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 Processor -- start 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 -- start start 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca584fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58503090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ca5846d6f0 con 0x55ca584fe190 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ca5845cb20 con 0x55ca584fb860 2024-09-10T23:07:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ca58460930 con 0x55ca58500bd0 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39626/0 (socket says 172.21.15.120:39626) 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c562ee640 1 -- 172.21.15.120:0/36060436 learned_addr learned my addr 172.21.15.120:0/36060436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.442+0000 7f4c55aed640 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58503090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c477fe640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 0x7f4c3c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c55aed640 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca584fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 -- 172.21.15.120:0/36060436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 msgr2=0x55ca58503090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58503090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 -- 172.21.15.120:0/36060436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 msgr2=0x55ca584fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca584fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c56aef640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 0x7f4c3c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 -- 172.21.15.120:0/36060436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca58422530 con 0x55ca584fe190 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c477fe640 1 -- 172.21.15.120:0/1968640377 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c480e6240 con 0x7f4c50108670 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c56aef640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 0x7f4c3c0b77e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c480a9000 tx=0x7f4c4800cee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c562ee640 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d50 tx=0x7f4c4c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1548dc54efcf870e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c44bfe640 1 -- 172.21.15.120:0/36060436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c012450 con 0x55ca584fe190 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c44bfe640 1 -- 172.21.15.120:0/36060436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c012aa0 con 0x55ca584fe190 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.443+0000 7f4c44bfe640 1 -- 172.21.15.120:0/36060436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00fb00 con 0x55ca584fe190 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/36060436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 msgr2=0x55ca58500580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d50 tx=0x7f4c4c0117b0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/36060436 shutdown_connections 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58503090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58500580 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/36060436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca584fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/36060436 >> 172.21.15.120:0/36060436 conn(0x55ca584f69b0 msgr2=0x55ca584f7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/36060436 shutdown_connections 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.444+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/36060436 wait complete. 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 -- start start 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 Processor -- start 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca58504ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58506130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ca58394ab0 con 0x55ca584fe190 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ca58402730 con 0x55ca584fb860 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ca58503bb0 con 0x55ca58500bd0 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c5b3fd740 10 client.0 _ll_register_callbacks cb 0x55ca584566b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c55aed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58506130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39642/0 (socket says 172.21.15.120:39642) 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.445+0000 7f4c562ee640 1 -- 172.21.15.120:0/3830678899 learned_addr learned my addr 172.21.15.120:0/3830678899 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c55aed640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca58504ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 -- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 msgr2=0x55ca58506130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58506130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 -- 172.21.15.120:0/3830678899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 msgr2=0x55ca58504ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca58504ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 -- 172.21.15.120:0/3830678899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca581be790 con 0x55ca584fe190 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c562ee640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d20 tx=0x7f4c4c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753705c709de4100 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c028020 con 0x55ca584fe190 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c4c012660 con 0x55ca584fe190 2024-09-10T23:07:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c012aa0 con 0x55ca584fe190 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c004490 con 0x55ca584fe190 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c5b3fd740 10 client.18396 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca58237620 con 0x55ca584fe190 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.446+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c012aa0 con 0x55ca584fe190 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4c4c0038c0 con 0x55ca584fe190 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 10 client.18396 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 10 client.18396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 20 client.18396 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca585104d0 0x55ca585128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 4 client.18396 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca5846f640 con 0x55ca585104d0 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c55aed640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca585104d0 0x55ca585128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c55aed640 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca585104d0 0x55ca585128c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec1e0ed7a25e489 server_cookie=f4e1f0b86781cc28 in_seq=0 out_seq=0 2024-09-10T23:07:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.447+0000 7f4c39ffb640 10 client.18396 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.449+0000 7f4c39ffb640 1 -- 172.21.15.120:0/3830678899 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca5846f640 con 0x55ca585104d0 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.449+0000 7f4c39ffb640 10 client.18396 handle_command_reply: tid=0 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.449+0000 7f4c5b3fd740 1 client.18396 shutdown 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 msgr2=0x55ca58505560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d20 tx=0x7f4c4c009a70 comp rx=0 tx=0).stop 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 shutdown_connections 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca585104d0 0x55ca585128c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca58506130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca58505560 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c39ffb640 0 client.18396 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/3830678899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca58504ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 >> 172.21.15.120:0/3830678899 conn(0x55ca584f69b0 msgr2=0x55ca584f5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 shutdown_connections 2024-09-10T23:07:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/3830678899 wait complete. 2024-09-10T23:07:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.450+0000 7f4c5b3fd740 20 client.18396 trim_cache size 0 max 16384 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 Processor -- start 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 -- start start 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 0x55ca585014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca58500bd0 0x55ca58504ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501a7e60 con 0x55ca584fb860 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501aa6e0 con 0x55ca58500bd0 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501a84d0 con 0x55ca584fe190 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c55aed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c55aed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39656/0 (socket says 172.21.15.120:39656) 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c55aed640 1 -- 172.21.15.120:0/1416840071 learned_addr learned my addr 172.21.15.120:0/1416840071 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 0x55ca585014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.465+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca58500bd0 0x55ca58504ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 -- 172.21.15.120:0/1416840071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca58500bd0 msgr2=0x55ca58504ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca58500bd0 0x55ca58504ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 -- 172.21.15.120:0/1416840071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 msgr2=0x55ca58500fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 -- 172.21.15.120:0/1416840071 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca58503a90 con 0x55ca584fe190 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 0x55ca585014f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ae0 tx=0x7f4c4c009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299e08e6663d341e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c3bfff640 1 -- 172.21.15.120:0/1416840071 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c028040 con 0x55ca584fe190 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c3bfff640 1 -- 172.21.15.120:0/1416840071 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c02d440 con 0x55ca584fe190 2024-09-10T23:07:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.466+0000 7f4c3bfff640 1 -- 172.21.15.120:0/1416840071 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c005070 con 0x55ca584fe190 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1416840071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 msgr2=0x55ca585014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 0x55ca585014f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ae0 tx=0x7f4c4c009300 comp rx=0 tx=0).stop 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1416840071 shutdown_connections 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca58500bd0 0x55ca58504ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca584fe190 0x55ca585014f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1416840071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fb860 0x55ca58500fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1416840071 >> 172.21.15.120:0/1416840071 conn(0x55ca584f69b0 msgr2=0x55ca584facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1416840071 shutdown_connections 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1416840071 wait complete. 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 -- start start 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.467+0000 7f4c5b3fd740 1 Processor -- start 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca5850c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca5850b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501a8ba0 con 0x55ca584fe190 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501aa010 con 0x55ca584fb860 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501a9270 con 0x55ca58500bd0 2024-09-10T23:07:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c5b3fd740 10 client.0 _ll_register_callbacks cb 0x7f4c40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39658/0 (socket says 172.21.15.120:39658) 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 -- 172.21.15.120:0/1615253520 learned_addr learned my addr 172.21.15.120:0/1615253520 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca5850b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca5850c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 -- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 msgr2=0x55ca5850b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca5850b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 -- 172.21.15.120:0/1615253520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 msgr2=0x55ca5850c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca5850c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 -- 172.21.15.120:0/1615253520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5850ba20 con 0x55ca584fe190 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca5850c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c562ee640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a795e454252db55f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.468+0000 7f4c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c028040 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c02d440 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c4c02d5f0 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c012a80 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 10 client.18402 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca5850a840 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c02d440 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4c4c038930 con 0x55ca584fe190 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 10 client.18402 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 20 client.18402 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca5857d820 0x55ca5857fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 4 client.18402 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:07:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca5846b0f0 con 0x55ca5857d820 2024-09-10T23:07:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.469+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca5857d820 0x55ca5857fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.470+0000 7f4c39ffb640 10 client.18402 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:36.470+0000 7f4c55aed640 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca5857d820 0x55ca5857fc40 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e03b72533e329e server_cookie=6567bdb16610186c in_seq=0 out_seq=0 2024-09-10T23:07:36.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:36 smithi176 ceph-mon[34982]: pgmap v801: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-09-10T23:07:36.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:36 smithi120 ceph-mon[31247]: pgmap v801: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-09-10T23:07:36.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:36 smithi153 ceph-mon[34951]: pgmap v801: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 566 op/s 2024-09-10T23:07:36.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 63811 22.76 MB/sec execute 31 sec latency 293.648 ms 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.193+0000 7f4c39ffb640 1 -- 172.21.15.120:0/1615253520 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca5846b0f0 con 0x55ca5857d820 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.193+0000 7f4c39ffb640 10 client.18402 handle_command_reply: tid=0 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.193+0000 7f4c5b3fd740 1 client.18402 shutdown 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 msgr2=0x55ca5850cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c000c00 tx=0x7f4c4c009f60 comp rx=0 tx=0).stop 2024-09-10T23:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 shutdown_connections 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ca5857d820 0x55ca5857fc40 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c39ffb640 0 client.18402 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca58500bd0 0x55ca5850b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ca584fe190 0x55ca5850cb50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 --2- 172.21.15.120:0/1615253520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca584fb860 0x55ca5850c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 >> 172.21.15.120:0/1615253520 conn(0x55ca584f69b0 msgr2=0x55ca584fabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 shutdown_connections 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 1 -- 172.21.15.120:0/1615253520 wait complete. 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.194+0000 7f4c5b3fd740 20 client.18402 trim_cache size 0 max 16384 2024-09-10T23:07:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 msgr2=0x7f4c3c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 0x7f4c3c0b77e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c480a9000 tx=0x7f4c4800cee0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 msgr2=0x7f4c50199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800eb10 tx=0x7f4c4800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 shutdown_connections 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c3c0b5320 0x7f4c3c0b77e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c50108670 0x7f4c50199fc0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c50105ac0 0x7f4c50199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 --2- 172.21.15.120:0/1968640377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c5006b470 0x7f4c50198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 >> 172.21.15.120:0/1968640377 conn(0x7f4c500fd140 msgr2=0x7f4c50103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 shutdown_connections 2024-09-10T23:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:37.195+0000 7f4c58552640 1 -- 172.21.15.120:0/1968640377 wait complete. 2024-09-10T23:07:37.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 64057 22.56 MB/sec execute 32 sec latency 362.722 ms 2024-09-10T23:07:38.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:38 smithi176 ceph-mon[34982]: pgmap v802: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-09-10T23:07:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:38 smithi120 ceph-mon[31247]: pgmap v802: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-09-10T23:07:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:38 smithi153 ceph-mon[34951]: pgmap v802: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-09-10T23:07:38.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 64336 22.18 MB/sec execute 33 sec latency 93.546 ms 2024-09-10T23:07:39.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 64700 21.98 MB/sec execute 34 sec latency 69.814 ms 2024-09-10T23:07:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:40 smithi176 ceph-mon[34982]: pgmap v803: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-09-10T23:07:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:40 smithi120 ceph-mon[31247]: pgmap v803: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-09-10T23:07:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:40 smithi153 ceph-mon[34951]: pgmap v803: 97 pgs: 97 active+clean; 248 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 402 op/s 2024-09-10T23:07:40.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 65221 21.98 MB/sec execute 35 sec latency 77.734 ms 2024-09-10T23:07:41.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 65649 21.83 MB/sec execute 36 sec latency 116.420 ms 2024-09-10T23:07:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:42 smithi176 ceph-mon[34982]: pgmap v804: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 516 op/s 2024-09-10T23:07:42.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:42 smithi120 ceph-mon[31247]: pgmap v804: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 516 op/s 2024-09-10T23:07:42.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:42 smithi153 ceph-mon[34951]: pgmap v804: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.5 MiB/s wr, 516 op/s 2024-09-10T23:07:42.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 66226 22.19 MB/sec execute 37 sec latency 140.356 ms 2024-09-10T23:07:42.923 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:07:43.202 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 -- 172.21.15.120:0/2802750816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de4101c90 msgr2=0x7f3de4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 --2- 172.21.15.120:0/2802750816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de4101c90 0x7f3de4102070 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002a410 comp rx=0 tx=0).stop 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 -- 172.21.15.120:0/2802750816 shutdown_connections 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 --2- 172.21.15.120:0/2802750816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4108020 0x7f3de410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 --2- 172.21.15.120:0/2802750816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de41076a0 0x7f3de4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 --2- 172.21.15.120:0/2802750816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de4101c90 0x7f3de4102070 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 -- 172.21.15.120:0/2802750816 >> 172.21.15.120:0/2802750816 conn(0x7f3de40fd700 msgr2=0x7f3de40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 -- 172.21.15.120:0/2802750816 shutdown_connections 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.743+0000 7f3debac7640 1 -- 172.21.15.120:0/2802750816 wait complete. 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 Processor -- start 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 -- start start 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4101c90 0x7f3de4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 0x7f3de419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112e00 con 0x7f3de41076a0 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112c80 con 0x7f3de4108020 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3debac7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4112f80 con 0x7f3de4101c90 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3de9062640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3de9062640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44112/0 (socket says 172.21.15.120:44112) 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.744+0000 7f3de9062640 1 -- 172.21.15.120:0/3707722271 learned_addr learned my addr 172.21.15.120:0/3707722271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9863640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4101c90 0x7f3de4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3dea064640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 0x7f3de419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 -- 172.21.15.120:0/3707722271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4101c90 msgr2=0x7f3de4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4101c90 0x7f3de4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 -- 172.21.15.120:0/3707722271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 msgr2=0x7f3de419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 0x7f3de419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 -- 172.21.15.120:0/3707722271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de419f320 con 0x7f3de41076a0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3dea064640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 0x7f3de419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3de9062640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3ddaffd640 1 -- 172.21.15.120:0/3707722271 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3de41076a0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3ddaffd640 1 -- 172.21.15.120:0/3707722271 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dd4009c30 con 0x7f3de41076a0 2024-09-10T23:07:43.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de419f610 con 0x7f3de41076a0 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3ddaffd640 1 -- 172.21.15.120:0/3707722271 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400f620 con 0x7f3de41076a0 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.745+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a5ea0 con 0x7f3de41076a0 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 Processor -- start 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 -- start start 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3ddaffd640 1 -- 172.21.15.120:0/3707722271 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3dd4010460 con 0x7f3de41076a0 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd0c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55beccc82e50 con 0x55beccd1d140 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55beccc72280 con 0x55beccd0cda0 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dede9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55beccc76090 con 0x55beccd0c480 2024-09-10T23:07:43.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3de9863640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3de9062640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3dea064640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd0c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3de9863640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd1cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34324/0 (socket says 172.21.15.120:34324) 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.748+0000 7f3de9863640 1 -- 172.21.15.120:0/2134001687 learned_addr learned my addr 172.21.15.120:0/2134001687 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 -- 172.21.15.120:0/2134001687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 msgr2=0x55beccd1cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd1cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 -- 172.21.15.120:0/2134001687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 msgr2=0x55beccd1f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 -- 172.21.15.120:0/2134001687 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becc9d0650 con 0x55beccd0c480 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dea064640 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd0c860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009fc0 tx=0x7f3ddc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6038b51ad65949 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3ddaffd640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 0x7f3dd00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dcffff640 1 -- 172.21.15.120:0/2134001687 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004d20 con 0x55beccd0c480 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dcffff640 1 -- 172.21.15.120:0/2134001687 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc0119b0 con 0x55beccd0c480 2024-09-10T23:07:43.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3dcffff640 1 -- 172.21.15.120:0/2134001687 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0054f0 con 0x55beccd0c480 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3ddaffd640 1 -- 172.21.15.120:0/3707722271 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3dd40e57c0 con 0x7f3de41076a0 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.749+0000 7f3de9863640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 0x7f3dd00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- 172.21.15.120:0/2134001687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 msgr2=0x55beccd0c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3de9863640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 0x7f3dd00b7820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005c00 tx=0x7f3de0005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd0c860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009fc0 tx=0x7f3ddc010860 comp rx=0 tx=0).stop 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- 172.21.15.120:0/2134001687 shutdown_connections 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd1cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2134001687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd0c860 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- 172.21.15.120:0/2134001687 >> 172.21.15.120:0/2134001687 conn(0x55beccd0a2b0 msgr2=0x55beccd0b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- 172.21.15.120:0/2134001687 shutdown_connections 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- 172.21.15.120:0/2134001687 wait complete. 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.750+0000 7f3dede9c740 1 -- start start 2024-09-10T23:07:43.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 Processor -- start 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd19e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55beccad3700 con 0x55beccd1d140 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55beccc0d110 con 0x55beccd0cda0 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55beccd1fdf0 con 0x55beccd0c480 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dede9c740 10 client.0 _ll_register_callbacks cb 0x55beccc26990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44118/0 (socket says 172.21.15.120:44118) 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3de9863640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 -- 172.21.15.120:0/2058837469 learned_addr learned my addr 172.21.15.120:0/2058837469 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3de9062640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd19e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 -- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 msgr2=0x55beccd1ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd1ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 -- 172.21.15.120:0/2058837469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 msgr2=0x55beccd19e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd19e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.751+0000 7f3dea064640 1 -- 172.21.15.120:0/2058837469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beccc3ac40 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dea064640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005170 tx=0x7f3ddc011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b0ceda96f6f50d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc02a060 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3ddc004c50 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc0119b0 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc004e50 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 10 client.18414 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55beccd2b770 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc0119b0 con 0x55beccd1d140 2024-09-10T23:07:43.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3ddc01b6c0 con 0x55beccd1d140 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 10 client.18414 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 20 client.18414 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd2cb70 0x55beccd2ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 4 client.18414 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.752+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beccc84da0 con 0x55beccd2cb70 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.753+0000 7f3de9863640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd2cb70 0x55beccd2ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.753+0000 7f3dcd7fa640 10 client.18414 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.753+0000 7f3de9863640 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd2cb70 0x55beccd2ef60 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8ef8a7c09246e5 server_cookie=605438ac715a10c5 in_seq=0 out_seq=0 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2058837469 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55beccc84da0 con 0x55beccd2cb70 2024-09-10T23:07:43.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dcd7fa640 10 client.18414 handle_command_reply: tid=0 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 client.18414 shutdown 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 msgr2=0x55beccd1a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005170 tx=0x7f3ddc011730 comp rx=0 tx=0).stop 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 shutdown_connections 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd2cb70 0x55beccd2ef60 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd1a4a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dcd7fa640 0 client.18414 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd19e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2058837469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 >> 172.21.15.120:0/2058837469 conn(0x55beccd0a2b0 msgr2=0x55beccd093d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 shutdown_connections 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 1 -- 172.21.15.120:0/2058837469 wait complete. 2024-09-10T23:07:43.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.757+0000 7f3dede9c740 20 client.18414 trim_cache size 0 max 16384 2024-09-10T23:07:43.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.775+0000 7f3dede9c740 1 Processor -- start 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.775+0000 7f3dede9c740 1 -- start start 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd0ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41aa080 con 0x55beccd1d140 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac900 con 0x55beccd0cda0 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dede9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41aa6f0 con 0x55beccd0c480 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dea064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dea064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44122/0 (socket says 172.21.15.120:44122) 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dea064640 1 -- 172.21.15.120:0/3435417845 learned_addr learned my addr 172.21.15.120:0/3435417845 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9062640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd0ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 -- 172.21.15.120:0/3435417845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 msgr2=0x55beccd0ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd0ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 -- 172.21.15.120:0/3435417845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 msgr2=0x55beccd09bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 -- 172.21.15.120:0/3435417845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beccd18390 con 0x55beccd0c480 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3dea064640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.776+0000 7f3de9863640 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd290c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3de0031440 tx=0x7f3de003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ee5c2f7213494d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dcf7fe640 1 -- 172.21.15.120:0/3435417845 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0035040 con 0x55beccd0c480 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dcf7fe640 1 -- 172.21.15.120:0/3435417845 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0056070 con 0x55beccd0c480 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dcf7fe640 1 -- 172.21.15.120:0/3435417845 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0004b40 con 0x55beccd0c480 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dede9c740 1 -- 172.21.15.120:0/3435417845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 msgr2=0x55beccd290c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dede9c740 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd290c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3de0031440 tx=0x7f3de003f000 comp rx=0 tx=0).stop 2024-09-10T23:07:43.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dede9c740 1 -- 172.21.15.120:0/3435417845 shutdown_connections 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.777+0000 7f3dede9c740 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd09bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd0ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 --2- 172.21.15.120:0/3435417845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd290c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- 172.21.15.120:0/3435417845 >> 172.21.15.120:0/3435417845 conn(0x55beccd0a2b0 msgr2=0x55beccd0b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- 172.21.15.120:0/3435417845 shutdown_connections 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- 172.21.15.120:0/3435417845 wait complete. 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- start start 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 Processor -- start 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd92720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41aadc0 con 0x55beccd1d140 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac230 con 0x55beccd0cda0 2024-09-10T23:07:43.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.778+0000 7f3dede9c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ab490 con 0x55beccd0c480 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3dede9c740 10 client.0 _ll_register_callbacks cb 0x7f3dd40eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3dea064640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd92720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38624/0 (socket says 172.21.15.120:38624) 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 -- 172.21.15.120:0/2795396926 learned_addr learned my addr 172.21.15.120:0/2795396926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9062640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 -- 172.21.15.120:0/2795396926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 msgr2=0x55beccd30ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd30ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 -- 172.21.15.120:0/2795396926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 msgr2=0x55beccd92720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd92720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 -- 172.21.15.120:0/2795396926 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beccd92e30 con 0x55beccd0c480 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3dea064640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd92720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3de9863640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3de0031820 tx=0x7f3de0004100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1fe0d2af43099ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:43.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.779+0000 7f3dcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0063060 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3de0005230 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de0056070 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3de0005430 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 10 client.27886 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55beccd9a1e0 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0056070 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3de0003890 con 0x55beccd0c480 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 10 client.27886 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 10 client.27886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 20 client.27886 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd9a900 0x55beccd9cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 4 client.27886 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:07:43.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.780+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55beccc80850 con 0x55beccd9a900 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.781+0000 7f3de9062640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd9a900 0x55beccd9cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.781+0000 7f3dcd7fa640 10 client.27886 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.781+0000 7f3de9062640 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd9a900 0x55beccd9cd20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e7ad9750cfc902 server_cookie=bb002ee5a6100d7e in_seq=0 out_seq=0 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dcd7fa640 1 -- 172.21.15.120:0/2795396926 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x55beccc80850 con 0x55beccd9a900 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dcd7fa640 10 client.27886 handle_command_reply: tid=0 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 client.27886 shutdown 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 msgr2=0x55beccd30940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3de0031820 tx=0x7f3de0004100 comp rx=0 tx=0).stop 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 shutdown_connections 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55beccd9a900 0x55beccd9cd20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55beccd1d140 0x55beccd92720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55beccd0cda0 0x55beccd30ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 --2- 172.21.15.120:0/2795396926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55beccd0c480 0x55beccd30940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 >> 172.21.15.120:0/2795396926 conn(0x55beccd0a2b0 msgr2=0x55beccd1b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dcd7fa640 0 client.27886 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 shutdown_connections 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.783+0000 7f3dede9c740 1 -- 172.21.15.120:0/2795396926 wait complete. 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3dede9c740 20 client.27886 trim_cache size 0 max 16384 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:07:43.797 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.803 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:07:43.805 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.807 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:07:43.808 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.810 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.811 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:07:43.812 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:43.813 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 msgr2=0x7f3dd00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 0x7f3dd00b7820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005c00 tx=0x7f3de0005b90 comp rx=0 tx=0).stop 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 msgr2=0x7f3de41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d950 tx=0x7f3dd400de10 comp rx=0 tx=0).stop 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 shutdown_connections 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3dd00b5360 0x7f3dd00b7820 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4108020 0x7f3de419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3de41076a0 0x7f3de41997b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 --2- 172.21.15.120:0/3707722271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3de4101c90 0x7f3de4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 >> 172.21.15.120:0/3707722271 conn(0x7f3de40fd700 msgr2=0x7f3de40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 shutdown_connections 2024-09-10T23:07:43.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:43.784+0000 7f3debac7640 1 -- 172.21.15.120:0/3707722271 wait complete. 2024-09-10T23:07:43.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 66854 22.84 MB/sec execute 38 sec latency 62.092 ms 2024-09-10T23:07:44.576 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:44 smithi176 ceph-mon[34982]: pgmap v805: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:07:44.632 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:44 smithi120 ceph-mon[31247]: pgmap v805: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:07:44.665 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:07:44.666 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:07:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:44 smithi153 ceph-mon[34951]: pgmap v805: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:07:44.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 67332 22.96 MB/sec execute 39 sec latency 64.724 ms 2024-09-10T23:07:44.924 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:45.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.412+0000 7f85e5929640 1 -- 172.21.15.120:0/801787724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 msgr2=0x7f85e01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:45.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.412+0000 7f85e5929640 1 --2- 172.21.15.120:0/801787724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01031b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009900 tx=0x7f85d002a410 comp rx=0 tx=0).stop 2024-09-10T23:07:45.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 -- 172.21.15.120:0/801787724 shutdown_connections 2024-09-10T23:07:45.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 --2- 172.21.15.120:0/801787724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 0x7f85e010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 --2- 172.21.15.120:0/801787724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 0x7f85e0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 --2- 172.21.15.120:0/801787724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01031b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 -- 172.21.15.120:0/801787724 >> 172.21.15.120:0/801787724 conn(0x7f85e00fe860 msgr2=0x7f85e0100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 -- 172.21.15.120:0/801787724 shutdown_connections 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 -- 172.21.15.120:0/801787724 wait complete. 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 Processor -- start 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.413+0000 7f85e5929640 1 -- start start 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 0x7f85e0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 0x7f85e019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0112e20 con 0x7f85e0102dd0 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0112ca0 con 0x7f85e0109160 2024-09-10T23:07:45.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e5929640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0112fa0 con 0x7f85e01087e0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44164/0 (socket says 172.21.15.120:44164) 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 -- 172.21.15.120:0/373025581 learned_addr learned my addr 172.21.15.120:0/373025581 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85d7fff640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 0x7f85e0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85df7fe640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 0x7f85e019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 -- 172.21.15.120:0/373025581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 msgr2=0x7f85e0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 0x7f85e0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 -- 172.21.15.120:0/373025581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 msgr2=0x7f85e019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 0x7f85e019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 -- 172.21.15.120:0/373025581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e019f3a0 con 0x7f85e0102dd0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85deffd640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005940 tx=0x7f85d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:45.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.414+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d003f030 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.415+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0003cb0 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.415+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e019f630 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.415+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0035070 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.415+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e01a5d50 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.416+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e01031b0 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.417+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f85d003bbf0 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.418+0000 7f85e4927640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 0x7f85cc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.418+0000 7f85d7fff640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 0x7f85cc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.418+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f85d0106670 con 0x7f85e0102dd0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.418+0000 7f85d7fff640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 0x7f85cc0b77a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85e0101e00 tx=0x7f85c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:45.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.422+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f85d00ccf10 con 0x7f85e0102dd0 2024-09-10T23:07:45.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.653+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85e010a9b0 con 0x7f85e0102dd0 2024-09-10T23:07:45.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.653+0000 7f85e4927640 1 -- 172.21.15.120:0/373025581 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f85d00d1940 con 0x7f85e0102dd0 2024-09-10T23:07:45.656 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:07:45.656 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:07:45.657 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:07:45.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 msgr2=0x7f85cc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:45.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 0x7f85cc0b77a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85e0101e00 tx=0x7f85c800b390 comp rx=0 tx=0).stop 2024-09-10T23:07:45.658 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 msgr2=0x7f85e01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005940 tx=0x7f85d00042f0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 shutdown_connections 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f85cc0b52e0 0x7f85cc0b77a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85e0109160 0x7f85e019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85e01087e0 0x7f85e0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 --2- 172.21.15.120:0/373025581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85e0102dd0 0x7f85e01992f0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 >> 172.21.15.120:0/373025581 conn(0x7f85e00fe860 msgr2=0x7f85e0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 shutdown_connections 2024-09-10T23:07:45.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:45.656+0000 7f85e5929640 1 -- 172.21.15.120:0/373025581 wait complete. 2024-09-10T23:07:45.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 67650 22.88 MB/sec execute 40 sec latency 89.283 ms 2024-09-10T23:07:46.369 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:07:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:46 smithi176 ceph-mon[34982]: pgmap v806: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-09-10T23:07:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:46 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/373025581' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:46.657 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:46 smithi120 ceph-mon[31247]: pgmap v806: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-09-10T23:07:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/373025581' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:46 smithi153 ceph-mon[34951]: pgmap v806: 97 pgs: 97 active+clean; 236 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-09-10T23:07:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:46 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/373025581' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:07:46.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 67858 22.51 MB/sec execute 41 sec latency 175.304 ms 2024-09-10T23:07:47.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 -- 172.21.15.120:0/2748664997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c105f40 msgr2=0x7fa46c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2748664997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c105f40 0x7fa46c10f960 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa464009ca0 tx=0x7fa46402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 -- 172.21.15.120:0/2748664997 shutdown_connections 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2748664997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c105f40 0x7fa46c10f960 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2748664997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c1055a0 0x7fa46c105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2748664997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c101250 0x7fa46c101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 -- 172.21.15.120:0/2748664997 >> 172.21.15.120:0/2748664997 conn(0x7fa46c078140 msgr2=0x7fa46c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 -- 172.21.15.120:0/2748664997 shutdown_connections 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.184+0000 7fa472bcf640 1 -- 172.21.15.120:0/2748664997 wait complete. 2024-09-10T23:07:47.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 Processor -- start 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 -- start start 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c101250 0x7fa46c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 0x7fa46c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c112db0 con 0x7fa46c1055a0 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c112c30 con 0x7fa46c105f40 2024-09-10T23:07:47.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa472bcf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c112f30 con 0x7fa46c101250 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44186/0 (socket says 172.21.15.120:44186) 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa47096b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c101250 0x7fa46c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 -- 172.21.15.120:0/2856830158 learned_addr learned my addr 172.21.15.120:0/2856830158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa47116c640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 0x7fa46c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 -- 172.21.15.120:0/2856830158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c101250 msgr2=0x7fa46c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c101250 0x7fa46c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 -- 172.21.15.120:0/2856830158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 msgr2=0x7fa46c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.185+0000 7fa46bfff640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 0x7fa46c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa46bfff640 1 -- 172.21.15.120:0/2856830158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c19f310 con 0x7fa46c1055a0 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa47116c640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 0x7fa46c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa46bfff640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e9f0 tx=0x7fa45400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa469ffb640 1 -- 172.21.15.120:0/2856830158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400cd80 con 0x7fa46c1055a0 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c19f600 con 0x7fa46c1055a0 2024-09-10T23:07:47.391 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa469ffb640 1 -- 172.21.15.120:0/2856830158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4540054d0 con 0x7fa46c1055a0 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa469ffb640 1 -- 172.21.15.120:0/2856830158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454010690 con 0x7fa46c1055a0 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.186+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c1a5cc0 con 0x7fa46c1055a0 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa469ffb640 1 -- 172.21.15.120:0/2856830158 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa454004e90 con 0x7fa46c1055a0 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 Processor -- start 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 -- start start 2024-09-10T23:07:47.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca347b9a0 0x557ca347dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ca33e5ea0 con 0x557ca346fa10 2024-09-10T23:07:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ca33d52d0 con 0x557ca347b9a0 2024-09-10T23:07:47.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa474fa4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557ca33d90e0 con 0x557ca346f0f0 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa47116c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa47096b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa47116c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38658/0 (socket says 172.21.15.120:38658) 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa47096b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44190/0 (socket says 172.21.15.120:44190) 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa47116c640 1 -- 172.21.15.120:0/4220778276 learned_addr learned my addr 172.21.15.120:0/4220778276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.189+0000 7fa46bfff640 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca347b9a0 0x557ca347dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 -- 172.21.15.120:0/4220778276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca347b9a0 msgr2=0x557ca347dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca347b9a0 0x557ca347dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 -- 172.21.15.120:0/4220778276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 msgr2=0x557ca347b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa469ffb640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 0x7fa4580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 -- 172.21.15.120:0/4220778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca3133650 con 0x557ca346f0f0 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47096b640 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:47.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 0x7fa4580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa469ffb640 1 -- 172.21.15.120:0/2856830158 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa4540e5fc0 con 0x7fa46c1055a0 2024-09-10T23:07:47.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa464009400 tx=0x7fa464009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03ee445e1013455 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa462ffd640 1 -- 172.21.15.120:0/4220778276 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464003b70 con 0x557ca346f0f0 2024-09-10T23:07:47.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa47116c640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 0x7fa4580b77a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa46404c000 tx=0x7fa46404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa462ffd640 1 -- 172.21.15.120:0/4220778276 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464036040 con 0x557ca346f0f0 2024-09-10T23:07:47.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.190+0000 7fa462ffd640 1 -- 172.21.15.120:0/4220778276 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46402f8c0 con 0x557ca346f0f0 2024-09-10T23:07:47.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 -- 172.21.15.120:0/4220778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 msgr2=0x557ca346f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa464009400 tx=0x7fa464009800 comp rx=0 tx=0).stop 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 -- 172.21.15.120:0/4220778276 shutdown_connections 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca347b9a0 0x557ca347dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346fa10 0x557ca347b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 --2- 172.21.15.120:0/4220778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346f0f0 0x557ca346f4d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 -- 172.21.15.120:0/4220778276 >> 172.21.15.120:0/4220778276 conn(0x557ca346cf20 msgr2=0x557ca346e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 -- 172.21.15.120:0/4220778276 shutdown_connections 2024-09-10T23:07:47.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.191+0000 7fa474fa4740 1 -- 172.21.15.120:0/4220778276 wait complete. 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 -- start start 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 Processor -- start 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca347b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca3478d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.399 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557ca3236700 con 0x557ca347b9a0 2024-09-10T23:07:47.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557ca3370160 con 0x557ca346f0f0 2024-09-10T23:07:47.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557ca347e620 con 0x557ca346fa10 2024-09-10T23:07:47.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa474fa4740 10 client.0 _ll_register_callbacks cb 0x557ca33899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:47.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.400 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38662/0 (socket says 172.21.15.120:38662) 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa46bfff640 1 -- 172.21.15.120:0/3956330057 learned_addr learned my addr 172.21.15.120:0/3956330057 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa47096b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca347b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.192+0000 7fa47116c640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca3478d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 -- 172.21.15.120:0/3956330057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 msgr2=0x557ca347b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca347b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 -- 172.21.15.120:0/3956330057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 msgr2=0x557ca3478d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.401 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca3478d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 -- 172.21.15.120:0/3956330057 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca339dc90 con 0x557ca346fa10 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa46bfff640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4540a9ac0 tx=0x7fa454009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e965c07988eb912a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540a86f0 con 0x557ca346fa10 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa4540e80d0 con 0x557ca346fa10 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa464036040 con 0x557ca346fa10 2024-09-10T23:07:47.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa4540abfc0 con 0x557ca346fa10 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa474fa4740 10 client.27904 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.193+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ca348a050 con 0x557ca346fa10 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464036040 con 0x557ca346fa10 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa4540b5950 con 0x557ca346fa10 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 10 client.27904 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 10 client.27904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:47.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 20 client.27904 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca348b510 0x557ca348d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 4 client.27904 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ca33e7df0 con 0x557ca348b510 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa47096b640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca348b510 0x557ca348d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa44bfff640 10 client.27904 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.194+0000 7fa47096b640 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca348b510 0x557ca348d930 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8442142e8b6ff426 server_cookie=393f765185e4a614 in_seq=0 out_seq=0 2024-09-10T23:07:47.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.195+0000 7fa44bfff640 1 -- 172.21.15.120:0/3956330057 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ca33e7df0 con 0x557ca348b510 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.195+0000 7fa44bfff640 10 client.27904 handle_command_reply: tid=0 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.195+0000 7fa474fa4740 1 client.27904 shutdown 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 msgr2=0x557ca34786d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4540a9ac0 tx=0x7fa454009310 comp rx=0 tx=0).stop 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 shutdown_connections 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca348b510 0x557ca348d930 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca3478d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa44bfff640 0 client.27904 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca34786d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 --2- 172.21.15.120:0/3956330057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca347b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 >> 172.21.15.120:0/3956330057 conn(0x557ca346cf20 msgr2=0x557ca3472ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 shutdown_connections 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 1 -- 172.21.15.120:0/3956330057 wait complete. 2024-09-10T23:07:47.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.196+0000 7fa474fa4740 20 client.27904 trim_cache size 0 max 16384 2024-09-10T23:07:47.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.210+0000 7fa474fa4740 1 Processor -- start 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.210+0000 7fa474fa4740 1 -- start start 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 0x557ca3487900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ac2c0 con 0x557ca346f0f0 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1aeb40 con 0x557ca346fa10 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa474fa4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ac930 con 0x557ca347b9a0 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47096b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 0x557ca3487900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34428/0 (socket says 172.21.15.120:34428) 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38670/0 (socket says 172.21.15.120:38670) 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa46bfff640 1 -- 172.21.15.120:0/2400286779 learned_addr learned my addr 172.21.15.120:0/2400286779 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 -- 172.21.15.120:0/2400286779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 msgr2=0x557ca347cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 -- 172.21.15.120:0/2400286779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 msgr2=0x557ca3487900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 0x557ca3487900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 -- 172.21.15.120:0/2400286779 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca348dca0 con 0x557ca347b9a0 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47096b640 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 0x557ca3487900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa46bfff640 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.211+0000 7fa47116c640 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa464031b30 tx=0x7fa464033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f066a4c8040a38f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa4627fc640 1 -- 172.21.15.120:0/2400286779 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464060020 con 0x557ca347b9a0 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa4627fc640 1 -- 172.21.15.120:0/2400286779 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464036050 con 0x557ca347b9a0 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa4627fc640 1 -- 172.21.15.120:0/2400286779 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640308b0 con 0x557ca347b9a0 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 -- 172.21.15.120:0/2400286779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 msgr2=0x557ca347d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa464031b30 tx=0x7fa464033040 comp rx=0 tx=0).stop 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 -- 172.21.15.120:0/2400286779 shutdown_connections 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca347b9a0 0x557ca347d2d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346fa10 0x557ca347cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 --2- 172.21.15.120:0/2400286779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca346f0f0 0x557ca3487900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.212+0000 7fa474fa4740 1 -- 172.21.15.120:0/2400286779 >> 172.21.15.120:0/2400286779 conn(0x557ca346cf20 msgr2=0x557ca3473950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.213+0000 7fa474fa4740 1 -- 172.21.15.120:0/2400286779 shutdown_connections 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.213+0000 7fa474fa4740 1 -- 172.21.15.120:0/2400286779 wait complete. 2024-09-10T23:07:47.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.213+0000 7fa474fa4740 1 -- start start 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.213+0000 7fa474fa4740 1 Processor -- start 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca3484090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47096b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47096b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34444/0 (socket says 172.21.15.120:34444) 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca347e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47096b640 1 -- 172.21.15.120:0/1540897394 learned_addr learned my addr 172.21.15.120:0/1540897394 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ad000 con 0x557ca347b9a0 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ae470 con 0x557ca346f0f0 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1ad6d0 con 0x557ca346fa10 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca3484090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa474fa4740 10 client.0 _ll_register_callbacks cb 0x7fa4540a6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 -- 172.21.15.120:0/1540897394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 msgr2=0x557ca3483b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47116c640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca347e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 -- 172.21.15.120:0/1540897394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 msgr2=0x557ca347e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:47.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca347e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca347f060 con 0x557ca346fa10 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47096b640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa47116c640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca347e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa46bfff640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca3484090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa4540ac7e0 tx=0x7fa4540a8cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11432c9d5d3fd163 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.214+0000 7fa44bfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540ac1a0 con 0x557ca346fa10 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa4540adb80 con 0x557ca346fa10 2024-09-10T23:07:47.421 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa464036050 con 0x557ca346fa10 2024-09-10T23:07:47.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa4540ac620 con 0x557ca346fa10 2024-09-10T23:07:47.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa474fa4740 10 client.27910 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:47.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ca34f8dc0 con 0x557ca346fa10 2024-09-10T23:07:47.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4540ac880 con 0x557ca346fa10 2024-09-10T23:07:47.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.215+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa4540a81a0 con 0x557ca346fa10 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 10 client.27910 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 10 client.27910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 20 client.27910 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca34f9540 0x557ca34fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 4 client.27910 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa464036050 con 0x557ca34f9540 2024-09-10T23:07:47.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa47096b640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca34f9540 0x557ca34fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa47096b640 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca34f9540 0x557ca34fb960 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104e22efb276fa1b server_cookie=9a85f8b138ff21db in_seq=0 out_seq=0 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.216+0000 7fa44bfff640 10 client.27910 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa44bfff640 1 -- 172.21.15.120:0/1540897394 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa464036050 con 0x557ca34f9540 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa44bfff640 10 client.27910 handle_command_reply: tid=0 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 client.27910 shutdown 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 msgr2=0x557ca3484090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca3484090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa4540ac7e0 tx=0x7fa4540a8cf0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 shutdown_connections 2024-09-10T23:07:47.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557ca34f9540 0x557ca34fb960 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557ca347b9a0 0x557ca347e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557ca346fa10 0x557ca3484090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa44bfff640 0 client.27910 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 --2- 172.21.15.120:0/1540897394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557ca346f0f0 0x557ca3483b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 >> 172.21.15.120:0/1540897394 conn(0x557ca346cf20 msgr2=0x557ca3479d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 shutdown_connections 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 1 -- 172.21.15.120:0/1540897394 wait complete. 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.218+0000 7fa474fa4740 20 client.27910 trim_cache size 0 max 16384 2024-09-10T23:07:47.425 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:07:47.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 msgr2=0x7fa4580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 0x7fa4580b77a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa46404c000 tx=0x7fa46404d000 comp rx=0 tx=0).stop 2024-09-10T23:07:47.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 msgr2=0x7fa46c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e9f0 tx=0x7fa45400edf0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 shutdown_connections 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa4580b52e0 0x7fa4580b77a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa46c105f40 0x7fa46c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa46c1055a0 0x7fa46c1997a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 --2- 172.21.15.120:0/2856830158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa46c101250 0x7fa46c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 >> 172.21.15.120:0/2856830158 conn(0x7fa46c078140 msgr2=0x7fa46c108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 shutdown_connections 2024-09-10T23:07:47.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:47.219+0000 7fa472bcf640 1 -- 172.21.15.120:0/2856830158 wait complete. 2024-09-10T23:07:47.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:47 smithi120 ceph-mon[31247]: pgmap v807: 97 pgs: 97 active+clean; 241 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:07:47.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:47 smithi153 ceph-mon[34951]: pgmap v807: 97 pgs: 97 active+clean; 241 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:07:47.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:47 smithi176 ceph-mon[34982]: pgmap v807: 97 pgs: 97 active+clean; 241 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:07:47.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 68215 22.34 MB/sec execute 42 sec latency 62.511 ms 2024-09-10T23:07:48.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 68673 22.26 MB/sec execute 43 sec latency 57.895 ms 2024-09-10T23:07:49.061 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:07:49.062 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:07:49.358 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:49.800 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:49 smithi120 ceph-mon[31247]: pgmap v808: 97 pgs: 97 active+clean; 240 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-09-10T23:07:49.800 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:49 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:49.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:49 smithi176 ceph-mon[34982]: pgmap v808: 97 pgs: 97 active+clean; 240 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-09-10T23:07:49.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:49 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:49.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 69208 22.21 MB/sec execute 44 sec latency 92.346 ms 2024-09-10T23:07:49.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/2767443737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc1087d0 msgr2=0x7fb0bc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/2767443737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc108c10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009900 tx=0x7fb0ac02a410 comp rx=0 tx=0).stop 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/2767443737 shutdown_connections 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/2767443737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/2767443737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc108c10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/2767443737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/2767443737 >> 172.21.15.120:0/2767443737 conn(0x7fb0bc0fe870 msgr2=0x7fb0bc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/2767443737 shutdown_connections 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.897+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/2767443737 wait complete. 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 Processor -- start 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 -- start start 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc112da0 con 0x7fb0bc102dc0 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc112c20 con 0x7fb0bc1087d0 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c3ffd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc112f20 con 0x7fb0bc109150 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44232/0 (socket says 172.21.15.120:44232) 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 -- 172.21.15.120:0/3285844352 learned_addr learned my addr 172.21.15.120:0/3285844352 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34456/0 (socket says 172.21.15.120:34456) 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c259a640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 -- 172.21.15.120:0/3285844352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 msgr2=0x7fb0bc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 -- 172.21.15.120:0/3285844352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 msgr2=0x7fb0bc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c1d99640 1 -- 172.21.15.120:0/3285844352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc19f340 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.898+0000 7fb0c259a640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0b2ffd640 1 -- 172.21.15.120:0/3285844352 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800cd80 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc19f630 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0b2ffd640 1 -- 172.21.15.120:0/3285844352 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b80108e0 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0b2ffd640 1 -- 172.21.15.120:0/3285844352 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8018980 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.899+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a5ec0 con 0x7fb0bc102dc0 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.901+0000 7fb0c63d2740 1 Processor -- start 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 -- start start 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23dfc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bf23d62e60 con 0x55bf23dec800 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf23d52290 con 0x55bf23dfcba0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bf23d560a0 con 0x55bf23debee0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0b2ffd640 1 -- 172.21.15.120:0/3285844352 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb0b8010430 con 0x7fb0bc102dc0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38694/0 (socket says 172.21.15.120:38694) 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 -- 172.21.15.120:0/2409672425 learned_addr learned my addr 172.21.15.120:0/2409672425 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c1598640 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23dfc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 -- 172.21.15.120:0/2409672425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 msgr2=0x55bf23dfef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 -- 172.21.15.120:0/2409672425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 msgr2=0x55bf23dfc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23dfc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.902+0000 7fb0c259a640 1 -- 172.21.15.120:0/2409672425 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf23ab0930 con 0x55bf23debee0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0c259a640 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40117e0 tx=0x7fb0b4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3c157f145ff956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0a7fff640 1 -- 172.21.15.120:0/2409672425 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011310 con 0x55bf23debee0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0a7fff640 1 -- 172.21.15.120:0/2409672425 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b4003e20 con 0x55bf23debee0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0a7fff640 1 -- 172.21.15.120:0/2409672425 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4004460 con 0x55bf23debee0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0b2ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 0x7fb0a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.903+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 0x7fb0a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0b2ffd640 1 -- 172.21.15.120:0/3285844352 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0b80e5ef0 con 0x7fb0bc102dc0 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- 172.21.15.120:0/2409672425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 msgr2=0x55bf23dec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40117e0 tx=0x7fb0b4011ca0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- 172.21.15.120:0/2409672425 shutdown_connections 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23dfc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/2409672425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23dec2c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- 172.21.15.120:0/2409672425 >> 172.21.15.120:0/2409672425 conn(0x55bf23de9d10 msgr2=0x55bf23deaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c1d99640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 0x7fb0a80b77a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80e3d90 tx=0x7fb0b8018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- 172.21.15.120:0/2409672425 shutdown_connections 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- 172.21.15.120:0/2409672425 wait complete. 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 -- start start 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.904+0000 7fb0c63d2740 1 Processor -- start 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 0x55bf23df9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bf23bb3890 con 0x55bf23debee0 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bf23cecd40 con 0x55bf23dfcba0 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bf23dff850 con 0x55bf23dec800 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c63d2740 10 client.0 _ll_register_callbacks cb 0x55bf23d50c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 0x55bf23df9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44250/0 (socket says 172.21.15.120:44250) 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c259a640 1 -- 172.21.15.120:0/724197676 learned_addr learned my addr 172.21.15.120:0/724197676 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c259a640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 -- 172.21.15.120:0/724197676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 msgr2=0x55bf23dfa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 -- 172.21.15.120:0/724197676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 msgr2=0x55bf23dfc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c1598640 1 -- 172.21.15.120:0/724197676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf23e08bb0 con 0x55bf23dec800 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.905+0000 7fb0c259a640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c1598640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 0x55bf23df9af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009230 tx=0x7fb0ac032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662e13a09d5e7ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac035040 con 0x55bf23dec800 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b4003e20 con 0x55bf23dec800 2024-09-10T23:07:49.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0ac0317a0 con 0x55bf23dec800 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac04d070 con 0x55bf23dec800 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 10 client.27922 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf23e0b560 con 0x55bf23dec800 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b4003e20 con 0x55bf23dec800 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb0ac004750 con 0x55bf23dec800 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 10 client.27922 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 10 client.27922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.906+0000 7fb0c63d2740 20 client.27922 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e0cda0 0x55bf23e0f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0c63d2740 4 client.27922 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf23d64db0 con 0x55bf23e0cda0 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0c259a640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e0cda0 0x55bf23e0f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0c259a640 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e0cda0 0x55bf23e0f220 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f33d27a89d452b server_cookie=9ccea42411bc8c71 in_seq=0 out_seq=0 2024-09-10T23:07:49.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.907+0000 7fb0a57fa640 10 client.27922 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.908+0000 7fb0a57fa640 1 -- 172.21.15.120:0/724197676 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf23d64db0 con 0x55bf23e0cda0 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.908+0000 7fb0a57fa640 10 client.27922 handle_command_reply: tid=0 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.908+0000 7fb0c63d2740 1 client.27922 shutdown 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.908+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 msgr2=0x55bf23df9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.908+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 0x55bf23df9af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009230 tx=0x7fb0ac032f20 comp rx=0 tx=0).stop 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 shutdown_connections 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e0cda0 0x55bf23e0f220 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dfcba0 0x55bf23dfa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dec800 0x55bf23df9af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0a57fa640 0 client.27922 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/724197676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23debee0 0x55bf23dfc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 >> 172.21.15.120:0/724197676 conn(0x55bf23de9d10 msgr2=0x55bf23de8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 shutdown_connections 2024-09-10T23:07:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 1 -- 172.21.15.120:0/724197676 wait complete. 2024-09-10T23:07:49.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.909+0000 7fb0c63d2740 20 client.27922 trim_cache size 0 max 16384 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.928+0000 7fb0c63d2740 1 Processor -- start 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.928+0000 7fb0c63d2740 1 -- start start 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 0x55bf23df7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23deae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 0x55bf23deb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1aa130 con 0x55bf23dec800 2024-09-10T23:07:49.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ac990 con 0x55bf23debee0 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1aa780 con 0x55bf23dfcba0 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23deae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 0x55bf23df7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 0x55bf23df7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34476/0 (socket says 172.21.15.120:34476) 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c259a640 1 -- 172.21.15.120:0/3235026029 learned_addr learned my addr 172.21.15.120:0/3235026029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c259a640 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 0x55bf23deb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 -- 172.21.15.120:0/3235026029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 msgr2=0x55bf23deb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 0x55bf23deb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 -- 172.21.15.120:0/3235026029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 msgr2=0x55bf23df7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 0x55bf23df7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 -- 172.21.15.120:0/3235026029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf23de5640 con 0x55bf23dec800 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c259a640 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 0x55bf23deb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.929+0000 7fb0c1598640 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23deae40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009340 tx=0x7fb0ac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e9b26fb9de5ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0a77fe640 1 -- 172.21.15.120:0/3235026029 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac035030 con 0x55bf23dec800 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0a77fe640 1 -- 172.21.15.120:0/3235026029 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac0051f0 con 0x55bf23dec800 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0a77fe640 1 -- 172.21.15.120:0/3235026029 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0035f0 con 0x55bf23dec800 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 -- 172.21.15.120:0/3235026029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 msgr2=0x55bf23deae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23deae40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009340 tx=0x7fb0ac02a410 comp rx=0 tx=0).stop 2024-09-10T23:07:49.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 -- 172.21.15.120:0/3235026029 shutdown_connections 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23dfcba0 0x55bf23deb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dec800 0x55bf23deae40 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/3235026029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23debee0 0x55bf23df7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 -- 172.21.15.120:0/3235026029 >> 172.21.15.120:0/3235026029 conn(0x55bf23de9d10 msgr2=0x55bf23dfdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 -- 172.21.15.120:0/3235026029 shutdown_connections 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.930+0000 7fb0c63d2740 1 -- 172.21.15.120:0/3235026029 wait complete. 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 -- start start 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 Processor -- start 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23e04b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dfcba0 0x55bf23e138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1aae50 con 0x55bf23dfcba0 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ac2c0 con 0x55bf23dec800 2024-09-10T23:07:49.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ab520 con 0x55bf23debee0 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c63d2740 10 client.0 _ll_register_callbacks cb 0x7fb0ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c259a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dfcba0 0x55bf23e138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c1598640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34484/0 (socket says 172.21.15.120:34484) 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c1598640 1 -- 172.21.15.120:0/4095297125 learned_addr learned my addr 172.21.15.120:0/4095297125 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.931+0000 7fb0c259a640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23e04b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 -- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 msgr2=0x55bf23e04b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23e04b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 -- 172.21.15.120:0/4095297125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dfcba0 msgr2=0x55bf23e138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dfcba0 0x55bf23e138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 -- 172.21.15.120:0/4095297125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf23e13e00 con 0x55bf23dec800 2024-09-10T23:07:49.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c1598640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a930 tx=0x7fb0ac032f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77eda80847b0af38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac05e020 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ac0051f0 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb0ac004030 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac035070 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c63d2740 10 client.27728 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf23e15140 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.932+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac0051f0 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb0ac003aa0 con 0x55bf23dec800 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 10 client.27728 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 10 client.27728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 20 client.27728 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e7aae0 0x55bf23e7ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:49.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 4 client.27728 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:07:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf23d60860 con 0x55bf23e7aae0 2024-09-10T23:07:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c259a640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e7aae0 0x55bf23e7ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0a57fa640 10 client.27728 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:49.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.933+0000 7fb0c259a640 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e7aae0 0x55bf23e7ced0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abc773e6b80331d server_cookie=777aa36bcb154405 in_seq=0 out_seq=0 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0a57fa640 1 -- 172.21.15.120:0/4095297125 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf23d60860 con 0x55bf23e7aae0 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0a57fa640 10 client.27728 handle_command_reply: tid=0 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 client.27728 shutdown 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 msgr2=0x55bf23e05050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a930 tx=0x7fb0ac032f00 comp rx=0 tx=0).stop 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 shutdown_connections 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bf23e7aae0 0x55bf23e7ced0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bf23dfcba0 0x55bf23e138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0a57fa640 0 client.27728 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf23dec800 0x55bf23e05050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 --2- 172.21.15.120:0/4095297125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bf23debee0 0x55bf23e04b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 >> 172.21.15.120:0/4095297125 conn(0x55bf23de9d10 msgr2=0x55bf23df8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 shutdown_connections 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 1 -- 172.21.15.120:0/4095297125 wait complete. 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.942+0000 7fb0c63d2740 20 client.27728 trim_cache size 0 max 16384 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "2e936509-53bd-4750-adb6-e64711626a8e", 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:07:49.946 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 msgr2=0x7fb0a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 0x7fb0a80b77a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80e3d90 tx=0x7fb0b8018420 comp rx=0 tx=0).stop 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 msgr2=0x7fb0bc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800e9f0 tx=0x7fb0b800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 shutdown_connections 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb0a80b52e0 0x7fb0a80b77a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0bc109150 0x7fb0bc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0bc1087d0 0x7fb0bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 --2- 172.21.15.120:0/3285844352 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0bc102dc0 0x7fb0bc199290 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 >> 172.21.15.120:0/3285844352 conn(0x7fb0bc0fe870 msgr2=0x7fb0bc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 shutdown_connections 2024-09-10T23:07:49.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:49.943+0000 7fb0c3ffd640 1 -- 172.21.15.120:0/3285844352 wait complete. 2024-09-10T23:07:49.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:49 smithi153 ceph-mon[34951]: pgmap v808: 97 pgs: 97 active+clean; 240 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 552 op/s 2024-09-10T23:07:49.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:49 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 -- 172.21.15.120:0/1923477485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f0101890 msgr2=0x7f30f0101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 --2- 172.21.15.120:0/1923477485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f0101890 0x7f30f0101c90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009930 tx=0x7f30ec02a420 comp rx=0 tx=0).stop 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 -- 172.21.15.120:0/1923477485 shutdown_connections 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 --2- 172.21.15.120:0/1923477485 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f01033a0 0x7f30f0109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 --2- 172.21.15.120:0/1923477485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 0x7f30f0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 --2- 172.21.15.120:0/1923477485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f0101890 0x7f30f0101c90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 -- 172.21.15.120:0/1923477485 >> 172.21.15.120:0/1923477485 conn(0x7f30f00fd130 msgr2=0x7f30f00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 -- 172.21.15.120:0/1923477485 shutdown_connections 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.291+0000 7f30f757c640 1 -- 172.21.15.120:0/1923477485 wait complete. 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 Processor -- start 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 -- start start 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f0101890 0x7f30f006bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 0x7f30f019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010d4e0 con 0x7f30f01033a0 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010d360 con 0x7f30f0101890 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f757c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f30f010d660 con 0x7f30f01029e0 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44296/0 (socket says 172.21.15.120:44296) 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 -- 172.21.15.120:0/546932436 learned_addr learned my addr 172.21.15.120:0/546932436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5318640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f0101890 0x7f30f006bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f4b17640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 0x7f30f019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 -- 172.21.15.120:0/546932436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 msgr2=0x7f30f019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 0x7f30f019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 -- 172.21.15.120:0/546932436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f0101890 msgr2=0x7f30f006bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.292+0000 7f30f5b19640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f0101890 0x7f30f006bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30f5b19640 1 -- 172.21.15.120:0/546932436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f01a0930 con 0x7f30f01033a0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30f5b19640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cce0 tx=0x7f30e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30e67fc640 1 -- 172.21.15.120:0/546932436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800f450 con 0x7f30f01033a0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30e67fc640 1 -- 172.21.15.120:0/546932436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e800faa0 con 0x7f30f01033a0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f01a0c20 con 0x7f30f01033a0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30e67fc640 1 -- 172.21.15.120:0/546932436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8018b70 con 0x7f30f01033a0 2024-09-10T23:07:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.293+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01a1120 con 0x7f30f01033a0 2024-09-10T23:07:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.296+0000 7f30e67fc640 1 -- 172.21.15.120:0/546932436 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f30e800f5f0 con 0x7f30f01033a0 2024-09-10T23:07:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.296+0000 7f30fa425740 1 Processor -- start 2024-09-10T23:07:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 -- start start 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac54860 0x56459ac56c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac59580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56459abc66f0 con 0x56459ac59bd0 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56459abb5b20 con 0x56459ac57190 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30fa425740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56459abb9930 con 0x56459ac54860 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44302/0 (socket says 172.21.15.120:44302) 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30f4b17640 1 -- 172.21.15.120:0/4046996107 learned_addr learned my addr 172.21.15.120:0/4046996107 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30f5318640 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac59580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac54860 0x56459ac56c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.297+0000 7f30e67fc640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 0x7f30dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f5b19640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 0x7f30dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 -- 172.21.15.120:0/4046996107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac54860 msgr2=0x56459ac56c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac54860 0x56459ac56c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 -- 172.21.15.120:0/4046996107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 msgr2=0x56459ac59580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac59580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 -- 172.21.15.120:0/4046996107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459ab7b530 con 0x56459ac59bd0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30e67fc640 1 -- 172.21.15.120:0/546932436 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f30e80e6220 con 0x7f30f01033a0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bf20 tx=0x7f30e000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6716256a21e7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30db7fe640 1 -- 172.21.15.120:0/4046996107 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0010870 con 0x56459ac59bd0 2024-09-10T23:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30db7fe640 1 -- 172.21.15.120:0/4046996107 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e001d3e0 con 0x56459ac59bd0 2024-09-10T23:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30f5b19640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 0x7f30dc0b77a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f30e801b000 tx=0x7f30e80a12c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.298+0000 7f30db7fe640 1 -- 172.21.15.120:0/4046996107 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e001c5f0 con 0x56459ac59bd0 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- 172.21.15.120:0/4046996107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 msgr2=0x56459ac5c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bf20 tx=0x7f30e000b7e0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- 172.21.15.120:0/4046996107 shutdown_connections 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac59bd0 0x56459ac5c090 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac59580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 --2- 172.21.15.120:0/4046996107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac54860 0x56459ac56c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- 172.21.15.120:0/4046996107 >> 172.21.15.120:0/4046996107 conn(0x56459ac4f9b0 msgr2=0x56459ac50740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- 172.21.15.120:0/4046996107 shutdown_connections 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- 172.21.15.120:0/4046996107 wait complete. 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 -- start start 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.299+0000 7f30fa425740 1 Processor -- start 2024-09-10T23:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459ac5f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56459aaedab0 con 0x56459ac54860 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56459ab5b730 con 0x56459ac57190 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56459ac5cbb0 con 0x56459ac59bd0 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30fa425740 10 client.0 _ll_register_callbacks cb 0x56459abaf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44304/0 (socket says 172.21.15.120:44304) 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f4b17640 1 -- 172.21.15.120:0/86160539 learned_addr learned my addr 172.21.15.120:0/86160539 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f4b17640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459ac5f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 -- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 msgr2=0x56459ac5f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459ac5f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 -- 172.21.15.120:0/86160539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 msgr2=0x56459ac5df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 -- 172.21.15.120:0/86160539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459a917790 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30f5318640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac5e5f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005ab0 tx=0x7f30ec0317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5bd4df2479f5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.300+0000 7f30d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec035040 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f30ec004450 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e001d3e0 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30ec0035f0 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 10 client.27737 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56459a990620 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e001d3e0 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f30ec03c8d0 con 0x56459ac57190 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 10 client.27737 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 10 client.27737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 20 client.27737 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459ac69830 0x56459ac6bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 4 client.27737 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56459abc8640 con 0x56459ac69830 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.301+0000 7f30f4b17640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459ac69830 0x56459ac6bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.302+0000 7f30f4b17640 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459ac69830 0x56459ac6bc20 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e665d89ab781c9 server_cookie=4b07b3e8a14b4e44 in_seq=0 out_seq=0 2024-09-10T23:07:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.302+0000 7f30d8ff9640 10 client.27737 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.304+0000 7f30d8ff9640 1 -- 172.21.15.120:0/86160539 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56459abc8640 con 0x56459ac69830 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.304+0000 7f30d8ff9640 10 client.27737 handle_command_reply: tid=0 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 client.27737 shutdown 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 msgr2=0x56459ac5e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac5e5f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005ab0 tx=0x7f30ec0317d0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 shutdown_connections 2024-09-10T23:07:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459ac69830 0x56459ac6bc20 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459ac5f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30d8ff9640 0 client.27737 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac5e5f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 --2- 172.21.15.120:0/86160539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 >> 172.21.15.120:0/86160539 conn(0x56459ac4f9b0 msgr2=0x56459ac4ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 shutdown_connections 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 1 -- 172.21.15.120:0/86160539 wait complete. 2024-09-10T23:07:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.305+0000 7f30fa425740 20 client.27737 trim_cache size 0 max 16384 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 Processor -- start 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 -- start start 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac57190 0x56459ac5d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac59bd0 0x56459ac5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01a7da0 con 0x56459ac54860 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01aa620 con 0x56459ac59bd0 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30fa425740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01a8410 con 0x56459ac57190 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44312/0 (socket says 172.21.15.120:44312) 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 -- 172.21.15.120:0/4203689735 learned_addr learned my addr 172.21.15.120:0/4203689735 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f5318640 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac57190 0x56459ac5d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac59bd0 0x56459ac5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 -- 172.21.15.120:0/4203689735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac57190 msgr2=0x56459ac5d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac57190 0x56459ac5d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 -- 172.21.15.120:0/4203689735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac59bd0 msgr2=0x56459ac5a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac59bd0 0x56459ac5a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.321+0000 7f30f4b17640 1 -- 172.21.15.120:0/4203689735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459ac6bf90 con 0x56459ac54860 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.322+0000 7f30f4b17640 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bf20 tx=0x7f30e0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36bc853a4adfb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.322+0000 7f30daffd640 1 -- 172.21.15.120:0/4203689735 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0023040 con 0x56459ac54860 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.322+0000 7f30daffd640 1 -- 172.21.15.120:0/4203689735 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e00050b0 con 0x56459ac54860 2024-09-10T23:07:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.322+0000 7f30daffd640 1 -- 172.21.15.120:0/4203689735 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0010a30 con 0x56459ac54860 2024-09-10T23:07:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- 172.21.15.120:0/4203689735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 msgr2=0x56459ac5cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bf20 tx=0x7f30e0005c90 comp rx=0 tx=0).stop 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- 172.21.15.120:0/4203689735 shutdown_connections 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac59bd0 0x56459ac5a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac57190 0x56459ac5d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 --2- 172.21.15.120:0/4203689735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac5cfb0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- 172.21.15.120:0/4203689735 >> 172.21.15.120:0/4203689735 conn(0x56459ac4f9b0 msgr2=0x56459ac53d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- 172.21.15.120:0/4203689735 shutdown_connections 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- 172.21.15.120:0/4203689735 wait complete. 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 -- start start 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.323+0000 7f30fa425740 1 Processor -- start 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459acd46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01a8ae0 con 0x56459ac54860 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01a9f50 con 0x56459ac57190 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01a91b0 con 0x56459ac59bd0 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30fa425740 10 client.0 _ll_register_callbacks cb 0x7f30ec031ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44328/0 (socket says 172.21.15.120:44328) 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 -- 172.21.15.120:0/415332162 learned_addr learned my addr 172.21.15.120:0/415332162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f5318640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459acd46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 -- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 msgr2=0x56459acd46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459acd46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 -- 172.21.15.120:0/415332162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 msgr2=0x56459ac657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 -- 172.21.15.120:0/415332162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459acd4c20 con 0x56459ac54860 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30f4b17640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f30e000b620 tx=0x7f30e000b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0042b56639fb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.324+0000 7f30d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0023040 con 0x56459ac54860 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f30e0005260 con 0x56459ac54860 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e00050b0 con 0x56459ac54860 2024-09-10T23:07:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30e0005460 con 0x56459ac54860 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 10 client.18477 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56459acd5e60 con 0x56459ac54860 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e00050b0 con 0x56459ac54860 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f30e0029040 con 0x56459ac54860 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 10 client.18477 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 20 client.18477 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459acd6470 0x56459acd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 4 client.18477 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.325+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56459abc40f0 con 0x56459acd6470 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.326+0000 7f30f5318640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459acd6470 0x56459acd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.326+0000 7f30d8ff9640 10 client.18477 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:50.326+0000 7f30f5318640 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459acd6470 0x56459acd8890 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b501e4ca4abeffe server_cookie=bd61baa840c6fda8 in_seq=0 out_seq=0 2024-09-10T23:07:50.710 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:07:50.769 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:50.770 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:50.770 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:07:50.770 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:07:50.770 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:50.770 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:50 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:50.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:50 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:07:50.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 69473 22.23 MB/sec execute 45 sec latency 242.657 ms 2024-09-10T23:07:50.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:07:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:07:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:07:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:07:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:50.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:50 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:07:51.067 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30d8ff9640 1 -- 172.21.15.120:0/415332162 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56459abc40f0 con 0x56459acd6470 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30d8ff9640 10 client.18477 handle_command_reply: tid=0 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 client.18477 shutdown 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 msgr2=0x56459ac65240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f30e000b620 tx=0x7f30e000b570 comp rx=0 tx=0).stop 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 shutdown_connections 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56459acd6470 0x56459acd8890 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56459ac59bd0 0x56459acd46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30d8ff9640 0 client.18477 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56459ac57190 0x56459ac657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 --2- 172.21.15.120:0/415332162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56459ac54860 0x56459ac65240 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 >> 172.21.15.120:0/415332162 conn(0x56459ac4f9b0 msgr2=0x56459ac5a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 shutdown_connections 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 1 -- 172.21.15.120:0/415332162 wait complete. 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.131+0000 7f30fa425740 20 client.18477 trim_cache size 0 max 16384 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:07:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:07:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 msgr2=0x7f30dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 0x7f30dc0b77a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f30e801b000 tx=0x7f30e80a12c0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 msgr2=0x7f30f01a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cce0 tx=0x7f30e800a620 comp rx=0 tx=0).stop 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 shutdown_connections 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f30dc0b52e0 0x7f30dc0b77a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30f01033a0 0x7f30f01a0070 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30f01029e0 0x7f30f019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 --2- 172.21.15.120:0/546932436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f0101890 0x7f30f006bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 >> 172.21.15.120:0/546932436 conn(0x7f30f00fd130 msgr2=0x7f30f0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 shutdown_connections 2024-09-10T23:07:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:07:51.132+0000 7f30f757c640 1 -- 172.21.15.120:0/546932436 wait complete. 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.619+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/3521570770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 msgr2=0x7f6b9810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.619+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/3521570770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9810e820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0108e0 tx=0x7f6b8c02e2c0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.619+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/3521570770 shutdown_connections 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.619+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/3521570770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9810e820 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.620+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/3521570770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b981076a0 0x7f6b98107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.620+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/3521570770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b98101c90 0x7f6b98102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.620+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/3521570770 >> 172.21.15.120:0/3521570770 conn(0x7f6b980fd700 msgr2=0x7f6b980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.620+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/3521570770 shutdown_connections 2024-09-10T23:07:51.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.621+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/3521570770 wait complete. 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.622+0000 7f6b9ebbb640 1 Processor -- start 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.622+0000 7f6b9ebbb640 1 -- start start 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b98101c90 0x7f6b98074730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b981076a0 0x7f6b98074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98121bf0 con 0x7f6b98101c90 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98121a70 con 0x7f6b981076a0 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.623+0000 7f6b9ebbb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98121d70 con 0x7f6b98108020 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.627+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.627+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38800/0 (socket says 172.21.15.120:38800) 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.627+0000 7f6b9d158640 1 -- 172.21.15.120:0/899389984 learned_addr learned my addr 172.21.15.120:0/899389984 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:51.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.628+0000 7f6b97fff640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b981076a0 0x7f6b98074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.628+0000 7f6b9c957640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b98101c90 0x7f6b98074730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.629+0000 7f6b9d158640 1 -- 172.21.15.120:0/899389984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b981076a0 msgr2=0x7f6b98074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.629+0000 7f6b9d158640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b981076a0 0x7f6b98074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.629+0000 7f6b9d158640 1 -- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b98101c90 msgr2=0x7f6b98074730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.629+0000 7f6b9d158640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b98101c90 0x7f6b98074730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.629+0000 7f6b9d158640 1 -- 172.21.15.120:0/899389984 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9806e340 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.630+0000 7f6b9d158640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c02e8b0 tx=0x7f6b8c009690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.630+0000 7f6b95ffb640 1 -- 172.21.15.120:0/899389984 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c03b070 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.630+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9806e5d0 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.630+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b98073570 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.631+0000 7f6b95ffb640 1 -- 172.21.15.120:0/899389984 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c0054d0 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.631+0000 7f6b95ffb640 1 -- 172.21.15.120:0/899389984 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0035f0 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.634+0000 7f6b95ffb640 1 -- 172.21.15.120:0/899389984 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6b8c003790 con 0x7f6b98108020 2024-09-10T23:07:51.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.635+0000 7f6b95ffb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 0x7f6b800b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.637+0000 7f6b95ffb640 1 -- 172.21.15.120:0/899389984 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6b8c106360 con 0x7f6b98108020 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.637+0000 7f6b9c957640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 0x7f6b800b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6ba0f90740 1 Processor -- start 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6ba0f90740 1 -- start start 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6b9c957640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 0x7f6b800b7960 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6b880058c0 tx=0x7f6b8800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa84860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa84da0 0x5626bfa907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.638+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626bf9fae50 con 0x5626bfa90d30 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626bf9ea280 con 0x5626bfa84da0 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5626bf9ee090 con 0x5626bfa84480 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44350/0 (socket says 172.21.15.120:44350) 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 -- 172.21.15.120:0/2780209680 learned_addr learned my addr 172.21.15.120:0/2780209680 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9c957640 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa84860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 -- 172.21.15.120:0/2780209680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 msgr2=0x5626bfa84860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa84860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 -- 172.21.15.120:0/2780209680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa84da0 msgr2=0x5626bfa907f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa84da0 0x5626bfa907f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.639+0000 7f6b9d158640 1 -- 172.21.15.120:0/2780209680 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bf748650 con 0x5626bfa90d30 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.640+0000 7f6b9d158640 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0c90c0 tx=0x7f6b8c02e2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944eede1428d8b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.640+0000 7f6b7effd640 1 -- 172.21.15.120:0/2780209680 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0cc260 con 0x5626bfa90d30 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.640+0000 7f6b7effd640 1 -- 172.21.15.120:0/2780209680 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c0d1070 con 0x5626bfa90d30 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2780209680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 msgr2=0x5626bfa93120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0c90c0 tx=0x7f6b8c02e2c0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2780209680 shutdown_connections 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa90d30 0x5626bfa93120 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa84da0 0x5626bfa907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2780209680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa84860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.641+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2780209680 >> 172.21.15.120:0/2780209680 conn(0x5626bfa822b0 msgr2=0x5626bfa83450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.649+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2780209680 shutdown_connections 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.649+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2780209680 wait complete. 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.649+0000 7f6ba0f90740 1 -- start start 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.649+0000 7f6ba0f90740 1 Processor -- start 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa8e1b0 0x5626bfa8e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:38810/0 (socket says 172.21.15.120:38810) 2024-09-10T23:07:51.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6b9c957640 1 -- 172.21.15.120:0/3681842856 learned_addr learned my addr 172.21.15.120:0/3681842856 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5626bf84b700 con 0x5626bfa8e1b0 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6b97fff640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5626bf985110 con 0x5626bfa90d30 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5626bfa93770 con 0x5626bfa84480 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.650+0000 7f6ba0f90740 10 client.0 _ll_register_callbacks cb 0x5626bf99e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9d158640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa8e1b0 0x5626bfa8e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9c957640 1 -- 172.21.15.120:0/3681842856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 msgr2=0x5626bfa8db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9c957640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9c957640 1 -- 172.21.15.120:0/3681842856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa8e1b0 msgr2=0x5626bfa8e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9c957640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa8e1b0 0x5626bfa8e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.651+0000 7f6b9c957640 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bf9b2c40 con 0x5626bfa84480 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.652+0000 7f6b9c957640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b880109e0 tx=0x7f6b880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e21000598935a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.654+0000 7f6b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.654+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8801dd50 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.654+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6b88005020 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.655+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c0d1070 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.655+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b88005220 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.656+0000 7f6ba0f90740 10 client.27958 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.656+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626bfa9f660 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.656+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b88004c50 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6b880047a0 con 0x5626bfa84480 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 10 client.27958 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 10 client.27958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 20 client.27958 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:51.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfaa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 4 client.27958 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.657+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6b8c0d1070 con 0x5626bfaa0b10 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.659+0000 7f6b97fff640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfaa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.675+0000 7f6b77fff640 10 client.27958 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.675+0000 7f6b97fff640 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfaa2f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b658198dcaa2593 server_cookie=a1abafab1687fb74 in_seq=0 out_seq=0 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.681+0000 7f6b77fff640 1 -- 172.21.15.120:0/3681842856 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6b8c0d1070 con 0x5626bfaa0b10 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.681+0000 7f6b77fff640 10 client.27958 handle_command_reply: tid=0 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.681+0000 7f6ba0f90740 1 client.27958 shutdown 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 msgr2=0x5626bfa90710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b880109e0 tx=0x7f6b880046c0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 shutdown_connections 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfaa2f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6b77fff640 0 client.27958 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa8e1b0 0x5626bfa8e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/3681842856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa84480 0x5626bfa90710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.682+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 >> 172.21.15.120:0/3681842856 conn(0x5626bfa822b0 msgr2=0x5626bfa87e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.683+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 shutdown_connections 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.683+0000 7f6ba0f90740 1 -- 172.21.15.120:0/3681842856 wait complete. 2024-09-10T23:07:51.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.683+0000 7f6ba0f90740 20 client.27958 trim_cache size 0 max 16384 2024-09-10T23:07:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 Processor -- start 2024-09-10T23:07:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 -- start start 2024-09-10T23:07:51.719 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa92120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa92660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400c490 con 0x5626bfa84480 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400ed10 con 0x5626bfa90d30 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.711+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400cb00 con 0x5626bfa8e1b0 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa92660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa92120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44376/0 (socket says 172.21.15.120:44376) 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9c957640 1 -- 172.21.15.120:0/2030938413 learned_addr learned my addr 172.21.15.120:0/2030938413 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9d158640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa92660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:34550/0 (socket says 172.21.15.120:34550) 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 -- 172.21.15.120:0/2030938413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 msgr2=0x5626bfa92660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.720 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa92660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 -- 172.21.15.120:0/2030938413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 msgr2=0x5626bfa8d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b97fff640 1 -- 172.21.15.120:0/2030938413 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bfa82d30 con 0x5626bfa8e1b0 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.712+0000 7f6b9c957640 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.713+0000 7f6b97fff640 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa92120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5626bfa83c00 tx=0x7f6b90009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95058668e78bbfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.713+0000 7f6b7e7fc640 1 -- 172.21.15.120:0/2030938413 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90003d50 con 0x5626bfa8e1b0 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.713+0000 7f6b7e7fc640 1 -- 172.21.15.120:0/2030938413 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b90022920 con 0x5626bfa8e1b0 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.713+0000 7f6b7e7fc640 1 -- 172.21.15.120:0/2030938413 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000f8c0 con 0x5626bfa8e1b0 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2030938413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 msgr2=0x5626bfa92120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa92120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5626bfa83c00 tx=0x7f6b90009680 comp rx=0 tx=0).stop 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2030938413 shutdown_connections 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa92660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.721 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa92120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/2030938413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa8d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2030938413 >> 172.21.15.120:0/2030938413 conn(0x5626bfa822b0 msgr2=0x5626bfa8f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2030938413 shutdown_connections 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- 172.21.15.120:0/2030938413 wait complete. 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 -- start start 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.714+0000 7f6ba0f90740 1 Processor -- start 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa99000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400d1d0 con 0x5626bfa84480 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400e640 con 0x5626bfa90d30 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400d8a0 con 0x5626bfa8e1b0 2024-09-10T23:07:51.722 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6ba0f90740 10 client.0 _ll_register_callbacks cb 0x7f6b8800a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa99000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44378/0 (socket says 172.21.15.120:44378) 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 -- 172.21.15.120:0/1339809045 learned_addr learned my addr 172.21.15.120:0/1339809045 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9d158640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 -- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 msgr2=0x5626bfa99000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa99000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 -- 172.21.15.120:0/1339809045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 msgr2=0x5626bfa8a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:07:51.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 -- 172.21.15.120:0/1339809045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bfa8a970 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b9c957640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6b880109e0 tx=0x7f6b880056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2dc3703c3dceff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.715+0000 7f6b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8803e060 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6b88004c50 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b90022920 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b88004940 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6ba0f90740 10 client.18492 fetch_fsmap learned FSMap version 26 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626bfb0f300 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b88004e50 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.716+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6b88039450 con 0x5626bfa84480 2024-09-10T23:07:51.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 10 client.18492 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 20 client.18492 populate_metadata read hostname 'smithi120' 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfb119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 4 client.18492 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6b90022920 con 0x5626bfaa0b10 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6b97fff640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfb119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6b77fff640 10 client.18492 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:51.725 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.717+0000 7f6b97fff640 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfb119e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3159aaec2662e223 server_cookie=f4555efcd04cb8e5 in_seq=0 out_seq=0 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.718+0000 7f6b77fff640 1 -- 172.21.15.120:0/1339809045 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7954 (crc 0 0 0) 0x7f6b90022920 con 0x5626bfaa0b10 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.718+0000 7f6b77fff640 10 client.18492 handle_command_reply: tid=0 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.718+0000 7f6ba0f90740 1 client.18492 shutdown 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.718+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 msgr2=0x5626bfa98aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.718+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6b880109e0 tx=0x7f6b880056f0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 shutdown_connections 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5626bfaa0b10 0x5626bfb119e0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626bfa90d30 0x5626bfa8a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5626bfa8e1b0 0x5626bfa99000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 --2- 172.21.15.120:0/1339809045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5626bfa84480 0x5626bfa98aa0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 >> 172.21.15.120:0/1339809045 conn(0x5626bfa822b0 msgr2=0x5626bfa9ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6b77fff640 0 client.18492 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:07:51.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 shutdown_connections 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 1 -- 172.21.15.120:0/1339809045 wait complete. 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.719+0000 7f6ba0f90740 20 client.18492 trim_cache size 0 max 16384 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "2e936509-53bd-4750-adb6-e64711626a8e", 2024-09-10T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:07:51.729 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.730 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.731 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:07:51.732 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.733 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.734 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:07:51.735 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.736 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.738 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.741 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.742 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:07:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:07:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.744 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:07:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:07:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.746 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.747 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.748 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.749 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.751 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:07:51.752 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:07:51.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 msgr2=0x7f6b800b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 0x7f6b800b7960 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6b880058c0 tx=0x7f6b8800a4c0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 msgr2=0x7f6b9806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c02e8b0 tx=0x7f6b8c009690 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 shutdown_connections 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6b800b54a0 0x7f6b800b7960 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b98108020 0x7f6b9806da80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b981076a0 0x7f6b98074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 --2- 172.21.15.120:0/899389984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b98101c90 0x7f6b98074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 >> 172.21.15.120:0/899389984 conn(0x7f6b980fd700 msgr2=0x7f6b980fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 shutdown_connections 2024-09-10T23:07:51.754 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:07:51.720+0000 7f6b9ebbb640 1 -- 172.21.15.120:0/899389984 wait complete. 2024-09-10T23:07:51.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 69535 21.81 MB/sec execute 46 sec latency 381.354 ms 2024-09-10T23:07:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:51 smithi120 ceph-mon[31247]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 597 op/s 2024-09-10T23:07:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:51 smithi153 ceph-mon[34951]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 597 op/s 2024-09-10T23:07:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:51 smithi176 ceph-mon[34982]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.6 MiB/s wr, 597 op/s 2024-09-10T23:07:52.554 INFO:tasks.cephfs.filesystem:scrub status for tag:2e936509-53bd-4750-adb6-e64711626a8e - {'path': '/', 'tag': '2e936509-53bd-4750-adb6-e64711626a8e', 'options': 'recursive,force'} 2024-09-10T23:07:52.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 70051 21.85 MB/sec execute 47 sec latency 516.161 ms 2024-09-10T23:07:53.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:52 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:53.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:52 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:53.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:52 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:07:53.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 70550 22.22 MB/sec execute 48 sec latency 41.785 ms 2024-09-10T23:07:54.102 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:53 smithi120 ceph-mon[31247]: pgmap v810: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T23:07:54.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:53 smithi153 ceph-mon[34951]: pgmap v810: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T23:07:54.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:53 smithi176 ceph-mon[34982]: pgmap v810: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T23:07:54.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 70957 22.30 MB/sec execute 49 sec latency 68.926 ms 2024-09-10T23:07:55.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 71270 22.22 MB/sec execute 50 sec latency 70.912 ms 2024-09-10T23:07:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:55 smithi120 ceph-mon[31247]: pgmap v811: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 410 op/s 2024-09-10T23:07:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:07:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:55 smithi153 ceph-mon[34951]: pgmap v811: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 410 op/s 2024-09-10T23:07:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:07:56.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:55 smithi176 ceph-mon[34982]: pgmap v811: 97 pgs: 97 active+clean; 242 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 410 op/s 2024-09-10T23:07:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:07:56.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 71537 21.99 MB/sec execute 51 sec latency 190.318 ms 2024-09-10T23:07:57.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 71801 21.76 MB/sec execute 52 sec latency 278.035 ms 2024-09-10T23:07:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:57 smithi120 ceph-mon[31247]: pgmap v812: 97 pgs: 97 active+clean; 236 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-10T23:07:58.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:57 smithi153 ceph-mon[34951]: pgmap v812: 97 pgs: 97 active+clean; 236 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-10T23:07:58.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:57 smithi176 ceph-mon[34982]: pgmap v812: 97 pgs: 97 active+clean; 236 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 699 op/s 2024-09-10T23:07:58.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 72298 21.73 MB/sec execute 53 sec latency 62.981 ms 2024-09-10T23:07:59.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 72857 21.81 MB/sec execute 54 sec latency 99.662 ms 2024-09-10T23:08:00.125 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:07:59 smithi176 ceph-mon[34982]: pgmap v813: 97 pgs: 97 active+clean; 234 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 601 op/s 2024-09-10T23:08:00.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:07:59 smithi120 ceph-mon[31247]: pgmap v813: 97 pgs: 97 active+clean; 234 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 601 op/s 2024-09-10T23:08:00.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:07:59 smithi153 ceph-mon[34951]: pgmap v813: 97 pgs: 97 active+clean; 234 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 601 op/s 2024-09-10T23:08:00.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 73580 22.15 MB/sec execute 55 sec latency 39.760 ms 2024-09-10T23:08:01.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74199 22.57 MB/sec execute 56 sec latency 63.128 ms 2024-09-10T23:08:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:01 smithi120 ceph-mon[31247]: pgmap v814: 97 pgs: 97 active+clean; 238 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-09-10T23:08:02.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:01 smithi153 ceph-mon[34951]: pgmap v814: 97 pgs: 97 active+clean; 238 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-09-10T23:08:02.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:01 smithi176 ceph-mon[34982]: pgmap v814: 97 pgs: 97 active+clean; 238 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 683 op/s 2024-09-10T23:08:02.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74526 22.52 MB/sec execute 57 sec latency 191.571 ms 2024-09-10T23:08:03.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74829 22.41 MB/sec execute 58 sec latency 63.936 ms 2024-09-10T23:08:04.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:03 smithi120 ceph-mon[31247]: pgmap v815: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 698 op/s 2024-09-10T23:08:04.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:03 smithi153 ceph-mon[34951]: pgmap v815: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 698 op/s 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 -- 172.21.15.120:0/1159421252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201033a0 msgr2=0x7f0b20109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 --2- 172.21.15.120:0/1159421252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201033a0 0x7f0b20109d20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009cd0 tx=0x7f0b1802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 -- 172.21.15.120:0/1159421252 shutdown_connections 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 --2- 172.21.15.120:0/1159421252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201033a0 0x7f0b20109d20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 --2- 172.21.15.120:0/1159421252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201029e0 0x7f0b20102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 --2- 172.21.15.120:0/1159421252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 -- 172.21.15.120:0/1159421252 >> 172.21.15.120:0/1159421252 conn(0x7f0b200fd150 msgr2=0x7f0b200ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 -- 172.21.15.120:0/1159421252 shutdown_connections 2024-09-10T23:08:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.225+0000 7f0b24e60640 1 -- 172.21.15.120:0/1159421252 wait complete. 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 Processor -- start 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 -- start start 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 0x7f0b20199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e630 con 0x7f0b201029e0 2024-09-10T23:08:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e4b0 con 0x7f0b201033a0 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b24e60640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010e7b0 con 0x7f0b20101890 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59214/0 (socket says 172.21.15.120:59214) 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:35858/0 (socket says 172.21.15.120:35858) 2024-09-10T23:08:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 -- 172.21.15.120:0/833205538 learned_addr learned my addr 172.21.15.120:0/833205538 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 0x7f0b20199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 -- 172.21.15.120:0/833205538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 msgr2=0x7f0b20198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 -- 172.21.15.120:0/833205538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 msgr2=0x7f0b20199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 0x7f0b20199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 -- 172.21.15.120:0/833205538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019a800 con 0x7f0b201029e0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 0x7f0b20199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.226+0000 7f0b17fff640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c920 tx=0x7f0b0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.227+0000 7f0b177fe640 1 -- 172.21.15.120:0/833205538 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08007c70 con 0x7f0b201029e0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.227+0000 7f0b177fe640 1 -- 172.21.15.120:0/833205538 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b08007e10 con 0x7f0b201029e0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.227+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019aaf0 con 0x7f0b201029e0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.227+0000 7f0b177fe640 1 -- 172.21.15.120:0/833205538 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08018820 con 0x7f0b201029e0 2024-09-10T23:08:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.227+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a11c0 con 0x7f0b201029e0 2024-09-10T23:08:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 Processor -- start 2024-09-10T23:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 -- start start 2024-09-10T23:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b62090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b742acc6f0 con 0x55b742b5d190 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b742abbb20 con 0x55b742b5fbd0 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b27c29740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b742abf930 con 0x55b742b5a860 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b177fe640 1 -- 172.21.15.120:0/833205538 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0b0800f450 con 0x7f0b201029e0 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b1ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59218/0 (socket says 172.21.15.120:59218) 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b1e59c640 1 -- 172.21.15.120:0/3683879782 learned_addr learned my addr 172.21.15.120:0/3683879782 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.230+0000 7f0b17fff640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b62090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 -- 172.21.15.120:0/3683879782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 msgr2=0x55b742b62090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b62090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 -- 172.21.15.120:0/3683879782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 msgr2=0x55b742b5f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 -- 172.21.15.120:0/3683879782 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b742a81530 con 0x55b742b5a860 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b17fff640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b62090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b5cc50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ca0 tx=0x7f0b18009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361e00ab7f04aee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b177fe640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 0x7f0b0c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b14ff9640 1 -- 172.21.15.120:0/3683879782 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18031b50 con 0x55b742b5a860 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b14ff9640 1 -- 172.21.15.120:0/3683879782 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18036040 con 0x55b742b5a860 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b14ff9640 1 -- 172.21.15.120:0/3683879782 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004630 con 0x55b742b5a860 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 0x7f0b0c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.231+0000 7f0b177fe640 1 -- 172.21.15.120:0/833205538 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b080e5c50 con 0x7f0b201029e0 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 0x7f0b0c0b7820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005bc0 tx=0x7f0b10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 -- 172.21.15.120:0/3683879782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 msgr2=0x55b742b5cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b5cc50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ca0 tx=0x7f0b18009a20 comp rx=0 tx=0).stop 2024-09-10T23:08:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 -- 172.21.15.120:0/3683879782 shutdown_connections 2024-09-10T23:08:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b62090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3683879782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b5cc50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 -- 172.21.15.120:0/3683879782 >> 172.21.15.120:0/3683879782 conn(0x55b742b559b0 msgr2=0x55b742b56740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 -- 172.21.15.120:0/3683879782 shutdown_connections 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.232+0000 7f0b27c29740 1 -- 172.21.15.120:0/3683879782 wait complete. 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 -- start start 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 Processor -- start 2024-09-10T23:08:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b7429f3ab0 con 0x55b742b5a860 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b742a61730 con 0x55b742b5d190 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b742b62b50 con 0x55b742b5fbd0 2024-09-10T23:08:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b27c29740 10 client.0 _ll_register_callbacks cb 0x55b742ab56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59226/0 (socket says 172.21.15.120:59226) 2024-09-10T23:08:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 -- 172.21.15.120:0/1108517348 learned_addr learned my addr 172.21.15.120:0/1108517348 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b17fff640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 -- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 msgr2=0x55b742b650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 -- 172.21.15.120:0/1108517348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 msgr2=0x55b742b644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.233+0000 7f0b1e59c640 1 -- 172.21.15.120:0/1108517348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74281d790 con 0x55b742b5a860 2024-09-10T23:08:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000f9a0 tx=0x7f0b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878f463400593da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1002d060 con 0x55b742b5a860 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b10018030 con 0x55b742b5a860 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18036040 con 0x55b742b5a860 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b1001d620 con 0x55b742b5a860 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 10 client.18504 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b742896620 con 0x55b742b5a860 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18036040 con 0x55b742b5a860 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0b10033910 con 0x55b742b5a860 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 10 client.18504 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 10 client.18504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 20 client.18504 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742b6f7d0 0x55b742b71c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 4 client.18504 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.234+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b742ace640 con 0x55b742b6f7d0 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.235+0000 7f0b17fff640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742b6f7d0 0x55b742b71c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.235+0000 7f0b05ffb640 10 client.18504 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.235+0000 7f0b17fff640 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742b6f7d0 0x55b742b71c10 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24209b1fa0ce9df6 server_cookie=4b9b5a5dda278d53 in_seq=0 out_seq=0 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b05ffb640 1 -- 172.21.15.120:0/1108517348 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b742ace640 con 0x55b742b6f7d0 2024-09-10T23:08:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b05ffb640 10 client.18504 handle_command_reply: tid=0 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 client.18504 shutdown 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 msgr2=0x55b742b63e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000f9a0 tx=0x7f0b100046c0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 shutdown_connections 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742b6f7d0 0x55b742b71c10 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b05ffb640 0 client.18504 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 --2- 172.21.15.120:0/1108517348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b63e40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 >> 172.21.15.120:0/1108517348 conn(0x55b742b559b0 msgr2=0x55b742b54bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 shutdown_connections 2024-09-10T23:08:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.237+0000 7f0b27c29740 1 -- 172.21.15.120:0/1108517348 wait complete. 2024-09-10T23:08:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.238+0000 7f0b27c29740 20 client.18504 trim_cache size 0 max 16384 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 Processor -- start 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 -- start start 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a7ce0 con 0x55b742b5d190 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201aa560 con 0x55b742b5fbd0 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.252+0000 7f0b27c29740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a8350 con 0x55b742b5a860 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59238/0 (socket says 172.21.15.120:59238) 2024-09-10T23:08:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 -- 172.21.15.120:0/201596913 learned_addr learned my addr 172.21.15.120:0/201596913 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 -- 172.21.15.120:0/201596913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 msgr2=0x55b742b59d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b59d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 -- 172.21.15.120:0/201596913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 msgr2=0x55b742b5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 -- 172.21.15.120:0/201596913 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b742b65420 con 0x55b742b5d190 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b17fff640 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0b080ac3d0 tx=0x7f0b080cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763eef96e2cd482f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b07fff640 1 -- 172.21.15.120:0/201596913 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080e9040 con 0x55b742b5d190 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b07fff640 1 -- 172.21.15.120:0/201596913 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b080abf50 con 0x55b742b5d190 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.253+0000 7f0b07fff640 1 -- 172.21.15.120:0/201596913 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800f770 con 0x55b742b5d190 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 -- 172.21.15.120:0/201596913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 msgr2=0x55b742b5a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0b080ac3d0 tx=0x7f0b080cf040 comp rx=0 tx=0).stop 2024-09-10T23:08:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 -- 172.21.15.120:0/201596913 shutdown_connections 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5fbd0 0x55b742b5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5d190 0x55b742b5a290 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 --2- 172.21.15.120:0/201596913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5a860 0x55b742b59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 -- 172.21.15.120:0/201596913 >> 172.21.15.120:0/201596913 conn(0x55b742b559b0 msgr2=0x55b742b62ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 -- 172.21.15.120:0/201596913 shutdown_connections 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.254+0000 7f0b27c29740 1 -- 172.21.15.120:0/201596913 wait complete. 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 -- start start 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 Processor -- start 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b6bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a8a20 con 0x55b742b5a860 2024-09-10T23:08:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a9e90 con 0x55b742b5d190 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a90f0 con 0x55b742b5fbd0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b27c29740 10 client.0 _ll_register_callbacks cb 0x7f0b10009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59240/0 (socket says 172.21.15.120:59240) 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 -- 172.21.15.120:0/3388771932 learned_addr learned my addr 172.21.15.120:0/3388771932 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1ed9d640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b17fff640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b6bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 -- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 msgr2=0x55b742b6a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b6a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 -- 172.21.15.120:0/3388771932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 msgr2=0x55b742b6bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b6bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 -- 172.21.15.120:0/3388771932 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b742b6ac10 con 0x55b742b5a860 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b1e59c640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005eb0 tx=0x7f0b10033f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b046c0fb293da67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.255+0000 7f0b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10038070 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0b1001d9d0 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b080abf50 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b10044050 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 10 client.18510 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b742b69b20 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b080abf50 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0b1001dcd0 con 0x55b742b5a860 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 10 client.18510 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 10 client.18510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 20 client.18510 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742bdce30 0x55b742bdf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 4 client.18510 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:08:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b742aca0f0 con 0x55b742bdce30 2024-09-10T23:08:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.256+0000 7f0b17fff640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742bdce30 0x55b742bdf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.257+0000 7f0b17fff640 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742bdce30 0x55b742bdf250 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab1005d8a5dc6cf server_cookie=3ea411ac107924eb in_seq=0 out_seq=0 2024-09-10T23:08:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.257+0000 7f0b05ffb640 10 client.18510 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:04.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:03 smithi176 ceph-mon[34982]: pgmap v815: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 698 op/s 2024-09-10T23:08:04.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 74986 22.12 MB/sec execute 59 sec latency 284.133 ms 2024-09-10T23:08:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b05ffb640 1 -- 172.21.15.120:0/3388771932 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b742aca0f0 con 0x55b742bdce30 2024-09-10T23:08:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b05ffb640 10 client.18510 handle_command_reply: tid=0 2024-09-10T23:08:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 client.18510 shutdown 2024-09-10T23:08:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 msgr2=0x55b742b6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005eb0 tx=0x7f0b10033f70 comp rx=0 tx=0).stop 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 shutdown_connections 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b742bdce30 0x55b742bdf250 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b742b5fbd0 0x55b742b6a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b742b5d190 0x55b742b6bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 --2- 172.21.15.120:0/3388771932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b742b5a860 0x55b742b6b570 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b05ffb640 0 client.18510 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 >> 172.21.15.120:0/3388771932 conn(0x55b742b559b0 msgr2=0x55b742b543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 shutdown_connections 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 1 -- 172.21.15.120:0/3388771932 wait complete. 2024-09-10T23:08:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.981+0000 7f0b27c29740 20 client.18510 trim_cache size 0 max 16384 2024-09-10T23:08:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:08:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:08:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:08:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 msgr2=0x7f0b0c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 0x7f0b0c0b7820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005bc0 tx=0x7f0b10005f90 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 msgr2=0x7f0b20199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c920 tx=0x7f0b0800cde0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 shutdown_connections 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0b0c0b5360 0x7f0b0c0b7820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b201033a0 0x7f0b20199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b201029e0 0x7f0b20199310 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 --2- 172.21.15.120:0/833205538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b20101890 0x7f0b20198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 >> 172.21.15.120:0/833205538 conn(0x7f0b200fd150 msgr2=0x7f0b20105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 shutdown_connections 2024-09-10T23:08:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:04.982+0000 7f0b24e60640 1 -- 172.21.15.120:0/833205538 wait complete. 2024-09-10T23:08:05.871 INFO:tasks.workunit.client.0.smithi120.stdout: 10 75286 22.00 MB/sec execute 60 sec latency 225.228 ms 2024-09-10T23:08:06.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:05 smithi120 ceph-mon[31247]: pgmap v816: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 514 op/s 2024-09-10T23:08:06.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:05 smithi153 ceph-mon[34951]: pgmap v816: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 514 op/s 2024-09-10T23:08:06.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:05 smithi176 ceph-mon[34982]: pgmap v816: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 514 op/s 2024-09-10T23:08:06.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 75714 21.97 MB/sec execute 61 sec latency 67.909 ms 2024-09-10T23:08:07.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 76108 21.77 MB/sec execute 62 sec latency 385.346 ms 2024-09-10T23:08:08.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:07 smithi120 ceph-mon[31247]: pgmap v817: 97 pgs: 97 active+clean; 228 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.6 MiB/s wr, 722 op/s 2024-09-10T23:08:08.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:07 smithi153 ceph-mon[34951]: pgmap v817: 97 pgs: 97 active+clean; 228 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.6 MiB/s wr, 722 op/s 2024-09-10T23:08:08.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:07 smithi176 ceph-mon[34982]: pgmap v817: 97 pgs: 97 active+clean; 228 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.6 MiB/s wr, 722 op/s 2024-09-10T23:08:08.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 76589 21.80 MB/sec execute 63 sec latency 82.724 ms 2024-09-10T23:08:09.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 77285 22.22 MB/sec execute 64 sec latency 98.370 ms 2024-09-10T23:08:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:09 smithi120 ceph-mon[31247]: pgmap v818: 97 pgs: 97 active+clean; 227 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 495 op/s 2024-09-10T23:08:10.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:09 smithi153 ceph-mon[34951]: pgmap v818: 97 pgs: 97 active+clean; 227 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 495 op/s 2024-09-10T23:08:10.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:09 smithi176 ceph-mon[34982]: pgmap v818: 97 pgs: 97 active+clean; 227 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 495 op/s 2024-09-10T23:08:10.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 77837 22.45 MB/sec execute 65 sec latency 44.095 ms 2024-09-10T23:08:11.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 78174 22.49 MB/sec execute 66 sec latency 104.389 ms 2024-09-10T23:08:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:11 smithi120 ceph-mon[31247]: pgmap v819: 97 pgs: 97 active+clean; 230 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2024-09-10T23:08:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:11 smithi153 ceph-mon[34951]: pgmap v819: 97 pgs: 97 active+clean; 230 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2024-09-10T23:08:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:11 smithi176 ceph-mon[34982]: pgmap v819: 97 pgs: 97 active+clean; 230 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 580 op/s 2024-09-10T23:08:12.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 78406 22.31 MB/sec execute 67 sec latency 197.186 ms 2024-09-10T23:08:13.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 78669 22.11 MB/sec execute 68 sec latency 76.440 ms 2024-09-10T23:08:14.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:13 smithi120 ceph-mon[31247]: pgmap v820: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 574 op/s 2024-09-10T23:08:14.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:13 smithi153 ceph-mon[34951]: pgmap v820: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 574 op/s 2024-09-10T23:08:14.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:13 smithi176 ceph-mon[34982]: pgmap v820: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 574 op/s 2024-09-10T23:08:14.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 79037 22.06 MB/sec execute 69 sec latency 58.981 ms 2024-09-10T23:08:15.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 79660 22.04 MB/sec execute 70 sec latency 47.285 ms 2024-09-10T23:08:16.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:15 smithi120 ceph-mon[31247]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-09-10T23:08:16.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:15 smithi153 ceph-mon[34951]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-09-10T23:08:16.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:15 smithi176 ceph-mon[34982]: pgmap v821: 97 pgs: 97 active+clean; 235 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-09-10T23:08:16.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 80233 22.15 MB/sec execute 71 sec latency 70.134 ms 2024-09-10T23:08:17.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 80811 22.30 MB/sec execute 72 sec latency 215.215 ms 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 -- 172.21.15.120:0/3719575659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1033a0 msgr2=0x7f987c109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 --2- 172.21.15.120:0/3719575659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1033a0 0x7f987c109d20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cd0 tx=0x7f987402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 -- 172.21.15.120:0/3719575659 shutdown_connections 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 --2- 172.21.15.120:0/3719575659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1033a0 0x7f987c109d20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 --2- 172.21.15.120:0/3719575659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c1029e0 0x7f987c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 --2- 172.21.15.120:0/3719575659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c101890 0x7f987c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 -- 172.21.15.120:0/3719575659 >> 172.21.15.120:0/3719575659 conn(0x7f987c0fd170 msgr2=0x7f987c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 -- 172.21.15.120:0/3719575659 shutdown_connections 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.073+0000 7f988353d640 1 -- 172.21.15.120:0/3719575659 wait complete. 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 Processor -- start 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 -- start start 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c1033a0 0x7f987c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e690 con 0x7f987c1029e0 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e510 con 0x7f987c101890 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f988353d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e810 con 0x7f987c1033a0 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f98812d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52986/0 (socket says 172.21.15.120:52986) 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f98812d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51052/0 (socket says 172.21.15.120:51052) 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 -- 172.21.15.120:0/4109034331 learned_addr learned my addr 172.21.15.120:0/4109034331 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9881ada640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c1033a0 0x7f987c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 -- 172.21.15.120:0/4109034331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c1033a0 msgr2=0x7f987c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c1033a0 0x7f987c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 -- 172.21.15.120:0/4109034331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 msgr2=0x7f987c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f9880ad8640 1 -- 172.21.15.120:0/4109034331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c19a8b0 con 0x7f987c1029e0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.074+0000 7f98812d9640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f9880ad8640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f986c00e9f0 tx=0x7f986c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f98727fc640 1 -- 172.21.15.120:0/4109034331 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c00cd80 con 0x7f987c1029e0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c19aba0 con 0x7f987c1029e0 2024-09-10T23:08:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f98727fc640 1 -- 172.21.15.120:0/4109034331 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f986c0108e0 con 0x7f987c1029e0 2024-09-10T23:08:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f98727fc640 1 -- 172.21.15.120:0/4109034331 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c018980 con 0x7f987c1029e0 2024-09-10T23:08:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.075+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1a1270 con 0x7f987c1029e0 2024-09-10T23:08:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.077+0000 7f98727fc640 1 -- 172.21.15.120:0/4109034331 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f986c010430 con 0x7f987c1029e0 2024-09-10T23:08:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.077+0000 7f98863ea740 1 Processor -- start 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.077+0000 7f98863ea740 1 -- start start 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f132c70 0x55634f135060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 0x55634f13a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55634f0a06f0 con 0x55634f137fe0 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55634f08fb20 con 0x55634f1355a0 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98863ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55634f093930 con 0x55634f132c70 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51066/0 (socket says 172.21.15.120:51066) 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 -- 172.21.15.120:0/2490661641 learned_addr learned my addr 172.21.15.120:0/2490661641 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f9880ad8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 0x55634f13a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f9881ada640 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f132c70 0x55634f135060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 -- 172.21.15.120:0/2490661641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f132c70 msgr2=0x55634f135060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f132c70 0x55634f135060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 -- 172.21.15.120:0/2490661641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 msgr2=0x55634f13a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 0x55634f13a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 -- 172.21.15.120:0/2490661641 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634f055530 con 0x55634f1355a0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f9880ad8640 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 0x55634f13a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98727fc640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 0x7f98680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98812d9640 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98780115c0 tx=0x7f9878011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c954359249ca358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f9881ada640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 0x7f98680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.078+0000 7f98727fc640 1 -- 172.21.15.120:0/4109034331 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f986c0e5fa0 con 0x7f987c1029e0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98677fe640 1 -- 172.21.15.120:0/2490661641 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987800f640 con 0x55634f1355a0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98677fe640 1 -- 172.21.15.120:0/2490661641 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f987800fc90 con 0x55634f1355a0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98677fe640 1 -- 172.21.15.120:0/2490661641 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878019c60 con 0x55634f1355a0 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 -- 172.21.15.120:0/2490661641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 msgr2=0x55634f137990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98780115c0 tx=0x7f9878011a80 comp rx=0 tx=0).stop 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f9881ada640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 0x7f98680b77a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9874006010 tx=0x7f9874005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 -- 172.21.15.120:0/2490661641 shutdown_connections 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f137fe0 0x55634f13a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f1355a0 0x55634f137990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 --2- 172.21.15.120:0/2490661641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f132c70 0x55634f135060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 -- 172.21.15.120:0/2490661641 >> 172.21.15.120:0/2490661641 conn(0x55634f1299b0 msgr2=0x55634f12a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 -- 172.21.15.120:0/2490661641 shutdown_connections 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.079+0000 7f98863ea740 1 -- 172.21.15.120:0/2490661641 wait complete. 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 -- start start 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 Processor -- start 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f13c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f13d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55634efc7ab0 con 0x55634f132c70 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55634f035730 con 0x55634f137fe0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55634f13afc0 con 0x55634f1355a0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98863ea740 10 client.0 _ll_register_callbacks cb 0x55634f0896b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98812d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98812d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52996/0 (socket says 172.21.15.120:52996) 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f98812d9640 1 -- 172.21.15.120:0/3038596251 learned_addr learned my addr 172.21.15.120:0/3038596251 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f9881ada640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.080+0000 7f9880ad8640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f13c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 -- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 msgr2=0x55634f13c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f13c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 -- 172.21.15.120:0/3038596251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 msgr2=0x55634f13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f13d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 -- 172.21.15.120:0/3038596251 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634edf1790 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98812d9640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f987800a010 tx=0x7f98780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625570e2a5783615 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878026020 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f987800fe40 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f987800fc90 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98780044e0 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 10 client.18522 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55634ee6a620 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f987800fc90 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9878003a10 con 0x55634f132c70 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 10 client.18522 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 10 client.18522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 20 client.18522 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1479d0 0x55634f149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 4 client.18522 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.081+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55634f0a2640 con 0x55634f1479d0 2024-09-10T23:08:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.082+0000 7f9880ad8640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1479d0 0x55634f149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.082+0000 7f9864ff9640 10 client.18522 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.082+0000 7f9880ad8640 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1479d0 0x55634f149dc0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4252461145df39b server_cookie=374efe7a690efaf1 in_seq=0 out_seq=0 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f9864ff9640 1 -- 172.21.15.120:0/3038596251 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55634f0a2640 con 0x55634f1479d0 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f9864ff9640 10 client.18522 handle_command_reply: tid=0 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 client.18522 shutdown 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 msgr2=0x55634f13c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f987800a010 tx=0x7f98780042f0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 shutdown_connections 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1479d0 0x55634f149dc0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f13d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f9864ff9640 0 client.18522 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 --2- 172.21.15.120:0/3038596251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f132c70 0x55634f13c2e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.086+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 >> 172.21.15.120:0/3038596251 conn(0x55634f1299b0 msgr2=0x55634f128bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.087+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 shutdown_connections 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.087+0000 7f98863ea740 1 -- 172.21.15.120:0/3038596251 wait complete. 2024-09-10T23:08:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.087+0000 7f98863ea740 20 client.18522 trim_cache size 0 max 16384 2024-09-10T23:08:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 Processor -- start 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 -- start start 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f137fe0 0x55634f14e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f12d220 0x55634f14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a5cf0 con 0x55634f1355a0 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a8570 con 0x55634f12d220 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98863ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a6360 con 0x55634f137fe0 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53000/0 (socket says 172.21.15.120:53000) 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 -- 172.21.15.120:0/1982976145 learned_addr learned my addr 172.21.15.120:0/1982976145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f9881ada640 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f12d220 0x55634f14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f9880ad8640 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f137fe0 0x55634f14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 -- 172.21.15.120:0/1982976145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f137fe0 msgr2=0x55634f14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f137fe0 0x55634f14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 -- 172.21.15.120:0/1982976145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f12d220 msgr2=0x55634f14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f12d220 0x55634f14eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 -- 172.21.15.120:0/1982976145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634f12a450 con 0x55634f1355a0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.102+0000 7f98812d9640 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9878003fd0 tx=0x7f9878012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc929a7f87a747f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f9866ffd640 1 -- 172.21.15.120:0/1982976145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878026040 con 0x55634f1355a0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f9866ffd640 1 -- 172.21.15.120:0/1982976145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9878022480 con 0x55634f1355a0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f9866ffd640 1 -- 172.21.15.120:0/1982976145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878022b90 con 0x55634f1355a0 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 -- 172.21.15.120:0/1982976145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 msgr2=0x55634f13bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9878003fd0 tx=0x7f9878012f90 comp rx=0 tx=0).stop 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 -- 172.21.15.120:0/1982976145 shutdown_connections 2024-09-10T23:08:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f12d220 0x55634f14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f137fe0 0x55634f14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 --2- 172.21.15.120:0/1982976145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f1355a0 0x55634f13bfb0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 -- 172.21.15.120:0/1982976145 >> 172.21.15.120:0/1982976145 conn(0x55634f1299b0 msgr2=0x55634f13b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 -- 172.21.15.120:0/1982976145 shutdown_connections 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.103+0000 7f98863ea740 1 -- 172.21.15.120:0/1982976145 wait complete. 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 -- start start 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 Processor -- start 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a6a30 con 0x55634f141e90 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a7ea0 con 0x55634f137fe0 2024-09-10T23:08:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1a7100 con 0x55634f1355a0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98863ea740 10 client.0 _ll_register_callbacks cb 0x7f9874031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f9881ada640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f9881ada640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53016/0 (socket says 172.21.15.120:53016) 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f9881ada640 1 -- 172.21.15.120:0/889397525 learned_addr learned my addr 172.21.15.120:0/889397525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f98812d9640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.104+0000 7f9880ad8640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 -- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 msgr2=0x55634f143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 -- 172.21.15.120:0/889397525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 msgr2=0x55634f1439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f1439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 -- 172.21.15.120:0/889397525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634f142850 con 0x55634f141e90 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9881ada640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cd0 tx=0x7f98740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8da15a44820ddfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874057030 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9874041030 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55634f09e0f0 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9874004250 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 10 client.18528 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55634f1b5560 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55634f09e0f0 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9874030070 con 0x55634f141e90 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 10 client.18528 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 10 client.18528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 20 client.18528 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1b5f90 0x55634f1b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 4 client.18528 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:08:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.105+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55634f09b520 con 0x55634f1b5f90 2024-09-10T23:08:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.106+0000 7f98812d9640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1b5f90 0x55634f1b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.106+0000 7f9864ff9640 10 client.18528 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.106+0000 7f98812d9640 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1b5f90 0x55634f1b83b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df03ddad719151e6 server_cookie=d903cd252d16ec6f in_seq=0 out_seq=0 2024-09-10T23:08:18.198 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:17 smithi120 ceph-mon[31247]: pgmap v822: 97 pgs: 97 active+clean; 241 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.1 MiB/s wr, 623 op/s 2024-09-10T23:08:18.263 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:17 smithi153 ceph-mon[34951]: pgmap v822: 97 pgs: 97 active+clean; 241 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.1 MiB/s wr, 623 op/s 2024-09-10T23:08:18.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:17 smithi176 ceph-mon[34982]: pgmap v822: 97 pgs: 97 active+clean; 241 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.1 MiB/s wr, 623 op/s 2024-09-10T23:08:18.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 81078 22.34 MB/sec execute 73 sec latency 345.779 ms 2024-09-10T23:08:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f9864ff9640 1 -- 172.21.15.120:0/889397525 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55634f09b520 con 0x55634f1b5f90 2024-09-10T23:08:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f9864ff9640 10 client.18528 handle_command_reply: tid=0 2024-09-10T23:08:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 client.18528 shutdown 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 msgr2=0x55634f142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9874009cd0 tx=0x7f98740095c0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 shutdown_connections 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55634f1b5f90 0x55634f1b83b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f9864ff9640 0 client.18528 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55634f141e90 0x55634f142310 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55634f137fe0 0x55634f1439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 --2- 172.21.15.120:0/889397525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55634f1355a0 0x55634f143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 >> 172.21.15.120:0/889397525 conn(0x55634f1299b0 msgr2=0x55634f128bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 shutdown_connections 2024-09-10T23:08:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.886+0000 7f98863ea740 1 -- 172.21.15.120:0/889397525 wait complete. 2024-09-10T23:08:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f98863ea740 20 client.18528 trim_cache size 0 max 16384 2024-09-10T23:08:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:08:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:08:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:08:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:08:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 msgr2=0x7f98680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 0x7f98680b77a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9874006010 tx=0x7f9874005d80 comp rx=0 tx=0).stop 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 msgr2=0x7f987c1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f986c00e9f0 tx=0x7f986c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 shutdown_connections 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98680b52e0 0x7f98680b77a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f987c1033a0 0x7f987c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c1029e0 0x7f987c1993c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 --2- 172.21.15.120:0/4109034331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c101890 0x7f987c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 >> 172.21.15.120:0/4109034331 conn(0x7f987c0fd170 msgr2=0x7f987c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 shutdown_connections 2024-09-10T23:08:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:18.887+0000 7f988353d640 1 -- 172.21.15.120:0/4109034331 wait complete. 2024-09-10T23:08:19.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 81501 22.42 MB/sec execute 74 sec latency 120.232 ms 2024-09-10T23:08:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:19 smithi120 ceph-mon[31247]: pgmap v823: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-09-10T23:08:20.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:19 smithi153 ceph-mon[34951]: pgmap v823: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-09-10T23:08:20.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:19 smithi176 ceph-mon[34982]: pgmap v823: 97 pgs: 97 active+clean; 242 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 399 op/s 2024-09-10T23:08:20.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 81863 22.45 MB/sec execute 75 sec latency 63.815 ms 2024-09-10T23:08:21.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 82125 22.28 MB/sec execute 76 sec latency 92.006 ms 2024-09-10T23:08:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:21 smithi120 ceph-mon[31247]: pgmap v824: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-09-10T23:08:22.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:21 smithi153 ceph-mon[34951]: pgmap v824: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-09-10T23:08:22.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:21 smithi176 ceph-mon[34982]: pgmap v824: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-09-10T23:08:22.555 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:08:22.816 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:22.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 82374 22.11 MB/sec execute 77 sec latency 253.815 ms 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 -- 172.21.15.120:0/2961712999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f0109130 msgr2=0x7f53f010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 --2- 172.21.15.120:0/2961712999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f0109130 0x7f53f010f930 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009ca0 tx=0x7f53e802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 -- 172.21.15.120:0/2961712999 shutdown_connections 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 --2- 172.21.15.120:0/2961712999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f0109130 0x7f53f010f930 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 --2- 172.21.15.120:0/2961712999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f01087b0 0x7f53f0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 --2- 172.21.15.120:0/2961712999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.312 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 -- 172.21.15.120:0/2961712999 >> 172.21.15.120:0/2961712999 conn(0x7f53f00fe850 msgr2=0x7f53f0100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 -- 172.21.15.120:0/2961712999 shutdown_connections 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.309+0000 7f53f573c640 1 -- 172.21.15.120:0/2961712999 wait complete. 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 Processor -- start 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 -- start start 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 0x7f53f019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0112db0 con 0x7f53f01087b0 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0112c30 con 0x7f53f0102da0 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53f573c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0112f30 con 0x7f53f0109130 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53592/0 (socket says 172.21.15.120:53592) 2024-09-10T23:08:23.313 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53ee7fc640 1 -- 172.21.15.120:0/746041153 learned_addr learned my addr 172.21.15.120:0/746041153 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53eeffd640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.310+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 0x7f53f019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 -- 172.21.15.120:0/746041153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 msgr2=0x7f53f019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 0x7f53f019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 -- 172.21.15.120:0/746041153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 msgr2=0x7f53f01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 -- 172.21.15.120:0/746041153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f019f350 con 0x7f53f01087b0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53eeffd640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 0x7f53f019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53d7fff640 1 -- 172.21.15.120:0/746041153 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017030 con 0x7f53f01087b0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f019f640 con 0x7f53f01087b0 2024-09-10T23:08:23.314 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53d7fff640 1 -- 172.21.15.120:0/746041153 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e4004020 con 0x7f53f01087b0 2024-09-10T23:08:23.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53d7fff640 1 -- 172.21.15.120:0/746041153 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40047a0 con 0x7f53f01087b0 2024-09-10T23:08:23.315 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.311+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01a5ed0 con 0x7f53f01087b0 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.314+0000 7f53f7b11740 1 Processor -- start 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.314+0000 7f53d7fff640 1 -- 172.21.15.120:0/746041153 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f53e4009a30 con 0x7f53f01087b0 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.314+0000 7f53f7b11740 1 -- start start 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 0x557563c15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557563b7ce50 con 0x557563c06da0 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557563b6c280 con 0x557563c12d30 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53f7b11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557563b70090 con 0x557563c06480 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50110/0 (socket says 172.21.15.120:50110) 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 -- 172.21.15.120:0/1734791473 learned_addr learned my addr 172.21.15.120:0/1734791473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 0x557563c15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53d7fff640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 0x7f53d80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 -- 172.21.15.120:0/1734791473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 msgr2=0x557563c15120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 0x557563c15120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.322 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 -- 172.21.15.120:0/1734791473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 msgr2=0x557563c127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ef7fe640 1 -- 172.21.15.120:0/1734791473 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575638ca650 con 0x557563c06480 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 0x557563c15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.315+0000 7f53eeffd640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53d7fff640 1 -- 172.21.15.120:0/746041153 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53e40e5450 con 0x7f53f01087b0 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53e8035000 tx=0x7f53e802f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d509e35ba49200 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 0x7f53d80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53d57fa640 1 -- 172.21.15.120:0/1734791473 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e804b040 con 0x557563c06480 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53d57fa640 1 -- 172.21.15.120:0/1734791473 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8036040 con 0x557563c06480 2024-09-10T23:08:23.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.316+0000 7f53d57fa640 1 -- 172.21.15.120:0/1734791473 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e802f8c0 con 0x557563c06480 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- 172.21.15.120:0/1734791473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 msgr2=0x557563c06860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 0x7f53d80b7820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009a60 tx=0x7f53e802fb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53e8035000 tx=0x7f53e802f430 comp rx=0 tx=0).stop 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- 172.21.15.120:0/1734791473 shutdown_connections 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c12d30 0x557563c15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1734791473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c06480 0x557563c06860 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- 172.21.15.120:0/1734791473 >> 172.21.15.120:0/1734791473 conn(0x557563c042b0 msgr2=0x557563c05450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- 172.21.15.120:0/1734791473 shutdown_connections 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- 172.21.15.120:0/1734791473 wait complete. 2024-09-10T23:08:23.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 -- start start 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.317+0000 7f53f7b11740 1 Processor -- start 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 0x557563c12610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 0x557563c0fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575639cd700 con 0x557563c06480 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557563b07110 con 0x557563c06da0 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557563c157d0 con 0x557563c12d30 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53f7b11740 10 client.0 _ll_register_callbacks cb 0x557563b20990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 0x557563c12610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.325 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 0x557563c0fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50124/0 (socket says 172.21.15.120:50124) 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ef7fe640 1 -- 172.21.15.120:0/732953496 learned_addr learned my addr 172.21.15.120:0/732953496 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 0x557563c12610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53610/0 (socket says 172.21.15.120:53610) 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 -- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 msgr2=0x557563c10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 -- 172.21.15.120:0/732953496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 msgr2=0x557563c12610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 0x557563c12610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 -- 172.21.15.120:0/732953496 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557563b34c40 con 0x557563c06da0 2024-09-10T23:08:23.326 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 0x557563c0fa60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53e40e2fd0 tx=0x7f53e40a7ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58fb5545554a9876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.318+0000 7f53d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40ab3b0 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53e40041d0 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8036040 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53e40a73b0 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 10 client.27800 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557563c21120 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e8036040 con 0x557563c06da0 2024-09-10T23:08:23.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f53e40a7eb0 con 0x557563c06da0 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 10 client.27800 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 10 client.27800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 20 client.27800 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c22650 0x557563c24a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 4 client.27800 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557563b7eda0 con 0x557563c22650 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.319+0000 7f53eeffd640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c22650 0x557563c24a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.320+0000 7f53d2ff5640 10 client.27800 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:23.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.320+0000 7f53eeffd640 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c22650 0x557563c24a90 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e339474afe0db7e4 server_cookie=ff6d395c39f0c261 in_seq=0 out_seq=0 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53d2ff5640 1 -- 172.21.15.120:0/732953496 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557563b7eda0 con 0x557563c22650 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53d2ff5640 10 client.27800 handle_command_reply: tid=0 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 client.27800 shutdown 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 msgr2=0x557563c0fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 0x557563c0fa60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53e40e2fd0 tx=0x7f53e40a7ae0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 shutdown_connections 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c22650 0x557563c24a90 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06da0 0x557563c0fa60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53d2ff5640 0 client.27800 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 --2- 172.21.15.120:0/732953496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06480 0x557563c12610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 >> 172.21.15.120:0/732953496 conn(0x557563c042b0 msgr2=0x557563c09e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.322+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 shutdown_connections 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.323+0000 7f53f7b11740 1 -- 172.21.15.120:0/732953496 wait complete. 2024-09-10T23:08:23.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.323+0000 7f53f7b11740 20 client.27800 trim_cache size 0 max 16384 2024-09-10T23:08:23.343 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 Processor -- start 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 -- start start 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01ac2d0 con 0x557563c06da0 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01aeb50 con 0x557563c06480 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53f7b11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01ac940 con 0x557563c12d30 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53624/0 (socket says 172.21.15.120:53624) 2024-09-10T23:08:23.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 -- 172.21.15.120:0/402432977 learned_addr learned my addr 172.21.15.120:0/402432977 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 -- 172.21.15.120:0/402432977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 msgr2=0x557563c14120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c14120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.342+0000 7f53ee7fc640 1 -- 172.21.15.120:0/402432977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 msgr2=0x557563c0ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c0ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53ee7fc640 1 -- 172.21.15.120:0/402432977 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557563c04d30 con 0x557563c06da0 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53eeffd640 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c0ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f53e40e32f0 tx=0x7f53e4097150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b022415d715c7f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53d4ff9640 1 -- 172.21.15.120:0/402432977 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40ab3b0 con 0x557563c06da0 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53d4ff9640 1 -- 172.21.15.120:0/402432977 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e40056a0 con 0x557563c06da0 2024-09-10T23:08:23.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.343+0000 7f53d4ff9640 1 -- 172.21.15.120:0/402432977 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e40b5220 con 0x557563c06da0 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- 172.21.15.120:0/402432977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 msgr2=0x557563c032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f53e40e32f0 tx=0x7f53e4097150 comp rx=0 tx=0).stop 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- 172.21.15.120:0/402432977 shutdown_connections 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c14120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c032c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 --2- 172.21.15.120:0/402432977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c0ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- 172.21.15.120:0/402432977 >> 172.21.15.120:0/402432977 conn(0x557563c042b0 msgr2=0x557563c02bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- 172.21.15.120:0/402432977 shutdown_connections 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- 172.21.15.120:0/402432977 wait complete. 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 -- start start 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.344+0000 7f53f7b11740 1 Processor -- start 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c15a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c15ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01ad010 con 0x557563c06da0 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01ae480 con 0x557563c06480 2024-09-10T23:08:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01ad6e0 con 0x557563c12d30 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53f7b11740 10 client.0 _ll_register_callbacks cb 0x7f53e40044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51246/0 (socket says 172.21.15.120:51246) 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 -- 172.21.15.120:0/1485925928 learned_addr learned my addr 172.21.15.120:0/1485925928 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c15a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53ef7fe640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c15ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 -- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 msgr2=0x557563c15ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c15ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 -- 172.21.15.120:0/1485925928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 msgr2=0x557563c15a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c15a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 -- 172.21.15.120:0/1485925928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557563c16820 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.345+0000 7f53eeffd640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005fa0 tx=0x7f53e0012dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d920198e33bca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0028030 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e40056a0 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53e0010cf0 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53e0005020 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 10 client.27806 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557563c98e10 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e40056a0 con 0x557563c06480 2024-09-10T23:08:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f53e002d450 con 0x557563c06480 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 10 client.27806 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 10 client.27806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.346+0000 7f53f7b11740 20 client.27806 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c995e0 0x557563c9ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53f7b11740 4 client.27806 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557563b7a850 con 0x557563c995e0 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c995e0 0x557563c9ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53d2ff5640 10 client.27806 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:23.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.347+0000 7f53ee7fc640 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c995e0 0x557563c9ba00 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dceabae57980e9ef server_cookie=ec31ee295351d8e9 in_seq=0 out_seq=0 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.351+0000 7f53d2ff5640 1 -- 172.21.15.120:0/1485925928 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7758 (crc 0 0 0) 0x557563b7a850 con 0x557563c995e0 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.351+0000 7f53d2ff5640 10 client.27806 handle_command_reply: tid=0 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.351+0000 7f53f7b11740 1 client.27806 shutdown 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 msgr2=0x557563c1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005fa0 tx=0x7f53e0012dc0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 shutdown_connections 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557563c995e0 0x557563c9ba00 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557563c12d30 0x557563c15ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557563c06da0 0x557563c15a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 --2- 172.21.15.120:0/1485925928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557563c06480 0x557563c1b2e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53d2ff5640 0 client.27806 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:23.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 >> 172.21.15.120:0/1485925928 conn(0x557563c042b0 msgr2=0x557563c033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 shutdown_connections 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 1 -- 172.21.15.120:0/1485925928 wait complete. 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f7b11740 20 client.27806 trim_cache size 0 max 16384 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:08:23.355 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:08:23.356 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.357 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.359 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.364 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:08:23.365 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:08:23.368 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:08:23.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 msgr2=0x7f53d80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 0x7f53d80b7820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009a60 tx=0x7f53e802fb40 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 msgr2=0x7f53f01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.352+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400de10 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 shutdown_connections 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f53d80b5360 0x7f53d80b7820 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53f0109130 0x7f53f019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53f01087b0 0x7f53f01997e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 --2- 172.21.15.120:0/746041153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53f0102da0 0x7f53f01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 >> 172.21.15.120:0/746041153 conn(0x7f53f00fe850 msgr2=0x7f53f00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 shutdown_connections 2024-09-10T23:08:23.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:23.353+0000 7f53f573c640 1 -- 172.21.15.120:0/746041153 wait complete. 2024-09-10T23:08:23.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 82756 22.07 MB/sec execute 78 sec latency 74.668 ms 2024-09-10T23:08:24.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:23 smithi120 ceph-mon[31247]: pgmap v825: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-10T23:08:24.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:23 smithi153 ceph-mon[34951]: pgmap v825: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-10T23:08:24.304 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:08:24.304 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:08:24.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:23 smithi176 ceph-mon[34982]: pgmap v825: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-10T23:08:24.586 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:24.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 83376 22.02 MB/sec execute 79 sec latency 61.489 ms 2024-09-10T23:08:25.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 -- 172.21.15.120:0/137709946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd410d130 msgr2=0x7f5cd410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:25.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 --2- 172.21.15.120:0/137709946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd410d130 0x7f5cd410f560 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009ca0 tx=0x7f5ccc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 -- 172.21.15.120:0/137709946 shutdown_connections 2024-09-10T23:08:25.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 --2- 172.21.15.120:0/137709946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd410d130 0x7f5cd410f560 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 --2- 172.21.15.120:0/137709946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4104580 0x7f5cd410cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 --2- 172.21.15.120:0/137709946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd4103b80 0x7f5cd4103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 -- 172.21.15.120:0/137709946 >> 172.21.15.120:0/137709946 conn(0x7f5cd40fd730 msgr2=0x7f5cd40ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 -- 172.21.15.120:0/137709946 shutdown_connections 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.164+0000 7f5cd891b640 1 -- 172.21.15.120:0/137709946 wait complete. 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 Processor -- start 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 -- start start 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 0x7f5cd4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd410d130 0x7f5cd4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4112d70 con 0x7f5cd4104580 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4112bf0 con 0x7f5cd4103b80 2024-09-10T23:08:25.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd891b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4112ef0 con 0x7f5cd410d130 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 0x7f5cd4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53652/0 (socket says 172.21.15.120:53652) 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.165+0000 7f5cd1d9b640 1 -- 172.21.15.120:0/2923123194 learned_addr learned my addr 172.21.15.120:0/2923123194 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd2d9d640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd410d130 0x7f5cd4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 -- 172.21.15.120:0/2923123194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd410d130 msgr2=0x7f5cd4102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd410d130 0x7f5cd4102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 -- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 msgr2=0x7f5cd4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 -- 172.21.15.120:0/2923123194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41a7620 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd1d9b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd259c640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 0x7f5cd4101a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400e9f0 tx=0x7f5cc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400ccc0 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41a7910 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc40108e0 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4018880 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.166+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a7e50 con 0x7f5cd4103b80 2024-09-10T23:08:25.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.167+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd4103f60 con 0x7f5cd4103b80 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.169+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5cc4010430 con 0x7f5cd4103b80 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.169+0000 7f5cc37fe640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 0x7f5cbc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.169+0000 7f5cd1d9b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 0x7f5cbc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.169+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5cc40e5f30 con 0x7f5cd4103b80 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.170+0000 7f5cd1d9b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 0x7f5cbc0b77a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5cd41034f0 tx=0x7f5cc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:25.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.172+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5cc4117220 con 0x7f5cd4103b80 2024-09-10T23:08:25.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.427+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cd4102d00 con 0x7f5cd4103b80 2024-09-10T23:08:25.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.428+0000 7f5cc37fe640 1 -- 172.21.15.120:0/2923123194 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f5cc40ac770 con 0x7f5cd4103b80 2024-09-10T23:08:25.430 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:08:25.430 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:08:25.430 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 msgr2=0x7f5cbc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 0x7f5cbc0b77a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5cd41034f0 tx=0x7f5cc8005f90 comp rx=0 tx=0).stop 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 msgr2=0x7f5cd4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 0x7f5cd4101a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400e9f0 tx=0x7f5cc400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 shutdown_connections 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5cbc0b52e0 0x7f5cbc0b77a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cd410d130 0x7f5cd4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cd4104580 0x7f5cd4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 --2- 172.21.15.120:0/2923123194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4103b80 0x7f5cd4101a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:25.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 >> 172.21.15.120:0/2923123194 conn(0x7f5cd40fd730 msgr2=0x7f5cd410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:25.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 shutdown_connections 2024-09-10T23:08:25.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:25.431+0000 7f5cd891b640 1 -- 172.21.15.120:0/2923123194 wait complete. 2024-09-10T23:08:25.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 83988 22.22 MB/sec execute 80 sec latency 43.659 ms 2024-09-10T23:08:26.068 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:25 smithi120 ceph-mon[31247]: pgmap v826: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-09-10T23:08:26.069 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2923123194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:26.098 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:08:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:25 smithi153 ceph-mon[34951]: pgmap v826: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-09-10T23:08:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2923123194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:26.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:25 smithi176 ceph-mon[34982]: pgmap v826: 97 pgs: 97 active+clean; 234 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-09-10T23:08:26.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2923123194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:08:26.361 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:26.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 84643 22.48 MB/sec execute 81 sec latency 57.748 ms 2024-09-10T23:08:26.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 -- 172.21.15.120:0/2593292340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560108020 msgr2=0x7f556010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 --2- 172.21.15.120:0/2593292340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560108020 0x7f556010e820 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5558009ca0 tx=0x7f555802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 -- 172.21.15.120:0/2593292340 shutdown_connections 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 --2- 172.21.15.120:0/2593292340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5560108020 0x7f556010e820 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 --2- 172.21.15.120:0/2593292340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55601076a0 0x7f5560107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 --2- 172.21.15.120:0/2593292340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560101c90 0x7f5560102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 -- 172.21.15.120:0/2593292340 >> 172.21.15.120:0/2593292340 conn(0x7f55600fd700 msgr2=0x7f55600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 -- 172.21.15.120:0/2593292340 shutdown_connections 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.922+0000 7f556655d640 1 -- 172.21.15.120:0/2593292340 wait complete. 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 Processor -- start 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 -- start start 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 0x7f556019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 0x7f55601a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111ca0 con 0x7f55601076a0 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111b20 con 0x7f5560108020 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f556655d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111e20 con 0x7f5560101c90 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.923+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53668/0 (socket says 172.21.15.120:53668) 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 -- 172.21.15.120:0/3961668578 learned_addr learned my addr 172.21.15.120:0/3961668578 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555ffff640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 0x7f556019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f5564afa640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 0x7f55601a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 -- 172.21.15.120:0/3961668578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 msgr2=0x7f556019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 0x7f556019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 -- 172.21.15.120:0/3961668578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 msgr2=0x7f55601a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 0x7f55601a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 -- 172.21.15.120:0/3961668578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55601a55f0 con 0x7f55601076a0 2024-09-10T23:08:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f5564afa640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 0x7f55601a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555ffff640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 0x7f556019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555d7fa640 1 -- 172.21.15.120:0/3961668578 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400cd80 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555d7fa640 1 -- 172.21.15.120:0/3961668578 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55540054d0 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55601a57c0 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f555d7fa640 1 -- 172.21.15.120:0/3961668578 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554010690 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.924+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55601a5c50 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f555d7fa640 1 -- 172.21.15.120:0/3961668578 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5554004e90 con 0x7f55601076a0 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 Processor -- start 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 -- start start 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55568332bea0 con 0x5556833c1d80 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55568331b2d0 con 0x5556833b54d0 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5568932740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55568331f0e0 con 0x5556833b5df0 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f555ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f5564afa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f555ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833c1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50174/0 (socket says 172.21.15.120:50174) 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.927+0000 7f555ffff640 1 -- 172.21.15.120:0/402677139 learned_addr learned my addr 172.21.15.120:0/402677139 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 -- 172.21.15.120:0/402677139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 msgr2=0x5556833c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 -- 172.21.15.120:0/402677139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 msgr2=0x5556833c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 -- 172.21.15.120:0/402677139 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555683079650 con 0x5556833b54d0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f555d7fa640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 0x7f55480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f5564afa640 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b58b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5558005e30 tx=0x7f555802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c0caf48795e1910 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f555ffff640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 0x7f55480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f555d7fa640 1 -- 172.21.15.120:0/3961668578 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f55540e5f60 con 0x7f55601076a0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f55467fc640 1 -- 172.21.15.120:0/402677139 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558031450 con 0x5556833b54d0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f55467fc640 1 -- 172.21.15.120:0/402677139 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5558036040 con 0x5556833b54d0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.928+0000 7f55467fc640 1 -- 172.21.15.120:0/402677139 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004330 con 0x5556833b54d0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- 172.21.15.120:0/402677139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 msgr2=0x5556833b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f555ffff640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 0x7f55480b77a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5556833b41e0 tx=0x7f5550005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b58b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5558005e30 tx=0x7f555802f310 comp rx=0 tx=0).stop 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- 172.21.15.120:0/402677139 shutdown_connections 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 --2- 172.21.15.120:0/402677139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b58b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- 172.21.15.120:0/402677139 >> 172.21.15.120:0/402677139 conn(0x5556833b3300 msgr2=0x5556833b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- 172.21.15.120:0/402677139 shutdown_connections 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- 172.21.15.120:0/402677139 wait complete. 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 -- start start 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.929+0000 7f5568932740 1 Processor -- start 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55568317c700 con 0x5556833b5df0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5556832b6160 con 0x5556833b54d0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5556833c4820 con 0x5556833c1d80 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 10 client.0 _ll_register_callbacks cb 0x5556832cf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53686/0 (socket says 172.21.15.120:53686) 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 -- 172.21.15.120:0/3406754518 learned_addr learned my addr 172.21.15.120:0/3406754518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5564afa640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 -- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 msgr2=0x5556833bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555ffff640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 -- 172.21.15.120:0/3406754518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 msgr2=0x5556833c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 -- 172.21.15.120:0/3406754518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556832e3c90 con 0x5556833b5df0 2024-09-10T23:08:26.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555ffff640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f555f7fe640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f555400c6a0 tx=0x7f55540e9630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b86385a83a3edba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f553f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540ac0a0 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f55540ac9b0 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.930+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5558036040 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f55540abec0 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 10 client.18564 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556833d0130 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5558036040 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f55540b5850 con 0x5556833b5df0 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 10 client.18564 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:26.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 20 client.18564 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556833d1540 0x5556833d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 4 client.18564 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55568332ddf0 con 0x5556833d1540 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.931+0000 7f555ffff640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556833d1540 0x5556833d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.932+0000 7f553f7fe640 10 client.18564 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.932+0000 7f555ffff640 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556833d1540 0x5556833d3950 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450b04a69891d90c server_cookie=586bdacec0268e22 in_seq=0 out_seq=0 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.934+0000 7f553f7fe640 1 -- 172.21.15.120:0/3406754518 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55568332ddf0 con 0x5556833d1540 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.934+0000 7f553f7fe640 10 client.18564 handle_command_reply: tid=0 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.934+0000 7f5568932740 1 client.18564 shutdown 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.934+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 msgr2=0x5556833beaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.934+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f555400c6a0 tx=0x7f55540e9630 comp rx=0 tx=0).stop 2024-09-10T23:08:26.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 shutdown_connections 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5556833d1540 0x5556833d3950 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833beaa0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 --2- 172.21.15.120:0/3406754518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 >> 172.21.15.120:0/3406754518 conn(0x5556833b3300 msgr2=0x5556833b8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f553f7fe640 0 client.18564 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 shutdown_connections 2024-09-10T23:08:26.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 1 -- 172.21.15.120:0/3406754518 wait complete. 2024-09-10T23:08:26.951 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.935+0000 7f5568932740 20 client.18564 trim_cache size 0 max 16384 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 Processor -- start 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 -- start start 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ac220 con 0x5556833c1d80 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55601aeaa0 con 0x5556833b54d0 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5568932740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ac890 con 0x5556833b5df0 2024-09-10T23:08:26.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.949+0000 7f5564afa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53702/0 (socket says 172.21.15.120:53702) 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 -- 172.21.15.120:0/2508116051 learned_addr learned my addr 172.21.15.120:0/2508116051 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f555f7fe640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f555ffff640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 -- 172.21.15.120:0/2508116051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 msgr2=0x5556833b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 -- 172.21.15.120:0/2508116051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 msgr2=0x5556833b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 -- 172.21.15.120:0/2508116051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556833bcfa0 con 0x5556833c1d80 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f555ffff640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5564afa640 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f80 tx=0x7f55580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e10fe8263602b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5545ffb640 1 -- 172.21.15.120:0/2508116051 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558009800 con 0x5556833c1d80 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5545ffb640 1 -- 172.21.15.120:0/2508116051 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5558053070 con 0x5556833c1d80 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.950+0000 7f5545ffb640 1 -- 172.21.15.120:0/2508116051 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004290 con 0x5556833c1d80 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- 172.21.15.120:0/2508116051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 msgr2=0x5556833b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f80 tx=0x7f55580095c0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- 172.21.15.120:0/2508116051 shutdown_connections 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833c1d80 0x5556833b4900 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833b5df0 0x5556833b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 --2- 172.21.15.120:0/2508116051 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- 172.21.15.120:0/2508116051 >> 172.21.15.120:0/2508116051 conn(0x5556833b3300 msgr2=0x5556833b9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- 172.21.15.120:0/2508116051 shutdown_connections 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- 172.21.15.120:0/2508116051 wait complete. 2024-09-10T23:08:26.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.951+0000 7f5568932740 1 -- start start 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 Processor -- start 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55601acf60 con 0x5556833b5df0 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ae3d0 con 0x5556833b54d0 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ad630 con 0x5556833c1d80 2024-09-10T23:08:26.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5568932740 10 client.0 _ll_register_callbacks cb 0x7f55540e9730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53716/0 (socket says 172.21.15.120:53716) 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555f7fe640 1 -- 172.21.15.120:0/1870498350 learned_addr learned my addr 172.21.15.120:0/1870498350 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f5564afa640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 -- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 msgr2=0x5556833c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 -- 172.21.15.120:0/1870498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 msgr2=0x5556833ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.952+0000 7f555ffff640 1 -- 172.21.15.120:0/1870498350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556833c5320 con 0x5556833b54d0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f555ffff640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c9dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55568317c850 tx=0x7f55500124c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b1fde3b8627eac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:26.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550005020 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5550018030 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5558053070 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f55500051c0 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 10 client.27827 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55568343f020 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5558053070 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f55540051b0 con 0x5556833b54d0 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 10 client.27827 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 10 client.27827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 20 client.27827 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55568343f740 0x555683441b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 4 client.27827 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.953+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556833298a0 con 0x55568343f740 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.954+0000 7f555f7fe640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55568343f740 0x555683441b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.954+0000 7f555f7fe640 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55568343f740 0x555683441b30 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e94e642a8e05065 server_cookie=154d296ea481dcf1 in_seq=0 out_seq=0 2024-09-10T23:08:26.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.954+0000 7f553f7fe640 10 client.27827 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f553f7fe640 1 -- 172.21.15.120:0/1870498350 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556833298a0 con 0x55568343f740 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f553f7fe640 10 client.27827 handle_command_reply: tid=0 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 client.27827 shutdown 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 msgr2=0x5556833c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c9dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55568317c850 tx=0x7f55500124c0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 shutdown_connections 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55568343f740 0x555683441b30 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556833c1d80 0x5556833c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f553f7fe640 0 client.27827 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5556833b5df0 0x5556833ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 --2- 172.21.15.120:0/1870498350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5556833b54d0 0x5556833c9dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 >> 172.21.15.120:0/1870498350 conn(0x5556833b3300 msgr2=0x5556833b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 shutdown_connections 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 1 -- 172.21.15.120:0/1870498350 wait complete. 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.959+0000 7f5568932740 20 client.27827 trim_cache size 0 max 16384 2024-09-10T23:08:26.963 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:08:26.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 msgr2=0x7f55480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 0x7f55480b77a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5556833b41e0 tx=0x7f5550005f90 comp rx=0 tx=0).stop 2024-09-10T23:08:26.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 msgr2=0x7f556019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:26.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400edf0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 shutdown_connections 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f55480b52e0 0x7f55480b77a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5560108020 0x7f55601a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55601076a0 0x7f556019a570 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 --2- 172.21.15.120:0/3961668578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5560101c90 0x7f556019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 >> 172.21.15.120:0/3961668578 conn(0x7f55600fd700 msgr2=0x7f55600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.960+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 shutdown_connections 2024-09-10T23:08:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:26.961+0000 7f556655d640 1 -- 172.21.15.120:0/3961668578 wait complete. 2024-09-10T23:08:27.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 85024 22.49 MB/sec execute 82 sec latency 239.106 ms 2024-09-10T23:08:28.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:27 smithi120 ceph-mon[31247]: pgmap v827: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:08:28.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:27 smithi153 ceph-mon[34951]: pgmap v827: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:08:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:27 smithi176 ceph-mon[34982]: pgmap v827: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:08:28.696 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:08:28.696 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:08:28.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 85391 22.55 MB/sec execute 83 sec latency 83.675 ms 2024-09-10T23:08:28.982 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:29.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 -- 172.21.15.120:0/2167650211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 msgr2=0x7f7374102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 --2- 172.21.15.120:0/2167650211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f7374102070 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7368009900 tx=0x7f736802a410 comp rx=0 tx=0).stop 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 -- 172.21.15.120:0/2167650211 shutdown_connections 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 --2- 172.21.15.120:0/2167650211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 0x7f737410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 --2- 172.21.15.120:0/2167650211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 --2- 172.21.15.120:0/2167650211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f7374102070 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 -- 172.21.15.120:0/2167650211 >> 172.21.15.120:0/2167650211 conn(0x7f73740fd700 msgr2=0x7f73740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 -- 172.21.15.120:0/2167650211 shutdown_connections 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.498+0000 7f737aad5640 1 -- 172.21.15.120:0/2167650211 wait complete. 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.499+0000 7f737aad5640 1 Processor -- start 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.499+0000 7f737aad5640 1 -- start start 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 0x7f737419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7374110be0 con 0x7f7374101c90 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7374110a60 con 0x7f73741076a0 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f737aad5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7374110d60 con 0x7f7374108020 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7378871640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7378871640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53732/0 (socket says 172.21.15.120:53732) 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51316/0 (socket says 172.21.15.120:51316) 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7378871640 1 -- 172.21.15.120:0/1620720795 learned_addr learned my addr 172.21.15.120:0/1620720795 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7379072640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 0x7f737419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 -- 172.21.15.120:0/1620720795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 msgr2=0x7f737419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 0x7f737419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 -- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 msgr2=0x7f73741970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 -- 172.21.15.120:0/1620720795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737419d180 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7378871640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.500+0000 7f7373fff640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9f0 tx=0x7f736000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.501+0000 7f7371ffb640 1 -- 172.21.15.120:0/1620720795 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000ccc0 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.501+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737419d470 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.501+0000 7f7371ffb640 1 -- 172.21.15.120:0/1620720795 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73600108e0 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.501+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73741a3d00 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.501+0000 7f7371ffb640 1 -- 172.21.15.120:0/1620720795 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360018880 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.504+0000 7f7371ffb640 1 -- 172.21.15.120:0/1620720795 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7360010430 con 0x7f73741076a0 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 Processor -- start 2024-09-10T23:08:29.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 -- start start 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7371ffb640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 0x7f73640b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4ddad9e60 con 0x55d4ddb62ee0 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4ddac9290 con 0x55d4ddb6f790 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f737ceaa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d4ddacd0a0 con 0x55d4ddb63800 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7379072640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 0x7f73640b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53744/0 (socket says 172.21.15.120:53744) 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 -- 172.21.15.120:0/1021306888 learned_addr learned my addr 172.21.15.120:0/1021306888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7379072640 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7373fff640 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7371ffb640 1 -- 172.21.15.120:0/1620720795 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f73600e5ec0 con 0x7f73741076a0 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 -- 172.21.15.120:0/1021306888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 msgr2=0x55d4ddb6f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 -- 172.21.15.120:0/1021306888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 msgr2=0x55d4ddb71b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb71b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.505+0000 7f7378871640 1 -- 172.21.15.120:0/1021306888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4dd827930 con 0x55d4ddb62ee0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f7379072640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 0x7f73640b99b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c009e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f7378871640 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7368009a20 tx=0x7f7368032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32f3a797695d89c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f735effd640 1 -- 172.21.15.120:0/1021306888 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368035040 con 0x55d4ddb62ee0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f735effd640 1 -- 172.21.15.120:0/1021306888 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73680317a0 con 0x55d4ddb62ee0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f735effd640 1 -- 172.21.15.120:0/1021306888 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680049c0 con 0x55d4ddb62ee0 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f737ceaa740 1 -- 172.21.15.120:0/1021306888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 msgr2=0x55d4ddb632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.506+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7368009a20 tx=0x7f7368032ed0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- 172.21.15.120:0/1021306888 shutdown_connections 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb71b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1021306888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb632c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- 172.21.15.120:0/1021306888 >> 172.21.15.120:0/1021306888 conn(0x55d4ddb60d10 msgr2=0x55d4ddb61eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- 172.21.15.120:0/1021306888 shutdown_connections 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- 172.21.15.120:0/1021306888 wait complete. 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- start start 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 Processor -- start 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb6c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4dd92a890 con 0x55d4ddb6f790 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4dda63d40 con 0x55d4ddb62ee0 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.507+0000 7f737ceaa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d4ddb723e0 con 0x55d4ddb63800 2024-09-10T23:08:29.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f737ceaa740 10 client.0 _ll_register_callbacks cb 0x55d4ddac7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53752/0 (socket says 172.21.15.120:53752) 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 -- 172.21.15.120:0/1160519618 learned_addr learned my addr 172.21.15.120:0/1160519618 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7373fff640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7378871640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb6c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 -- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 msgr2=0x55d4ddb6cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 -- 172.21.15.120:0/1160519618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb62ee0 msgr2=0x55d4ddb6c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb6c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 -- 172.21.15.120:0/1160519618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ddb7b790 con 0x55d4ddb6f790 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7373fff640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7379072640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f736c013790 tx=0x7f736c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de318ff9974d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7353fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c004b20 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f736c01c040 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73680317a0 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f736c031050 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f737ceaa740 10 client.18582 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.508+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4ddb7e140 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73680317a0 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f736c028070 con 0x55d4ddb6f790 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 10 client.18582 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 20 client.18582 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddb81ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 4 client.18582 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ddadbdb0 con 0x55d4ddb7f6a0 2024-09-10T23:08:29.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f7378871640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddb81ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f7353fff640 10 client.18582 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.509+0000 7f7378871640 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddb81ad0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfdedcc18165e815 server_cookie=b4b02339dcf3a34c in_seq=0 out_seq=0 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.512+0000 7f7353fff640 1 -- 172.21.15.120:0/1160519618 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4ddadbdb0 con 0x55d4ddb7f6a0 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.512+0000 7f7353fff640 10 client.18582 handle_command_reply: tid=0 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.512+0000 7f737ceaa740 1 client.18582 shutdown 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 msgr2=0x55d4ddb6d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f736c013790 tx=0x7f736c01d000 comp rx=0 tx=0).stop 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 shutdown_connections 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddb81ad0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f7353fff640 0 client.18582 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb6d1a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1160519618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb6c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 >> 172.21.15.120:0/1160519618 conn(0x55d4ddb60d10 msgr2=0x55d4ddb5ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 shutdown_connections 2024-09-10T23:08:29.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 1 -- 172.21.15.120:0/1160519618 wait complete. 2024-09-10T23:08:29.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.513+0000 7f737ceaa740 20 client.18582 trim_cache size 0 max 16384 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 Processor -- start 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 -- start start 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb69950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 0x55d4ddb69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.534+0000 7f737ceaa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73741aa110 con 0x55d4ddb6f790 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f737ceaa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73741ac990 con 0x55d4ddb63800 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f737ceaa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73741aa780 con 0x55d4ddb62ee0 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53766/0 (socket says 172.21.15.120:53766) 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 -- 172.21.15.120:0/1300797245 learned_addr learned my addr 172.21.15.120:0/1300797245 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7378871640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb69950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7373fff640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 0x55d4ddb69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 -- 172.21.15.120:0/1300797245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 msgr2=0x55d4ddb69950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb69950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 -- 172.21.15.120:0/1300797245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 msgr2=0x55d4ddb69e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 0x55d4ddb69e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 -- 172.21.15.120:0/1300797245 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ddb63630 con 0x55d4ddb6f790 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7378871640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb69950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7373fff640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 0x55d4ddb69e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.535+0000 7f7379072640 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f736c00fcc0 tx=0x7f736c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3778877cb62cdd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.536+0000 7f735e7fc640 1 -- 172.21.15.120:0/1300797245 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c02e030 con 0x55d4ddb6f790 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.536+0000 7f735e7fc640 1 -- 172.21.15.120:0/1300797245 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c009040 con 0x55d4ddb6f790 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.536+0000 7f735e7fc640 1 -- 172.21.15.120:0/1300797245 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c004040 con 0x55d4ddb6f790 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- 172.21.15.120:0/1300797245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 msgr2=0x55d4ddb873a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f736c00fcc0 tx=0x7f736c005bd0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- 172.21.15.120:0/1300797245 shutdown_connections 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb6f790 0x55d4ddb873a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb63800 0x55d4ddb69e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1300797245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb69950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- 172.21.15.120:0/1300797245 >> 172.21.15.120:0/1300797245 conn(0x55d4ddb60d10 msgr2=0x55d4ddb6e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- 172.21.15.120:0/1300797245 shutdown_connections 2024-09-10T23:08:29.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- 172.21.15.120:0/1300797245 wait complete. 2024-09-10T23:08:29.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 -- start start 2024-09-10T23:08:29.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.537+0000 7f737ceaa740 1 Processor -- start 2024-09-10T23:08:29.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb77b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73741aae50 con 0x55d4ddb62ee0 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73741ac2c0 con 0x55d4ddb6f790 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f73741ab520 con 0x55d4ddb63800 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f737ceaa740 10 client.0 _ll_register_callbacks cb 0x7f736c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7378871640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb77b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50272/0 (socket says 172.21.15.120:50272) 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 -- 172.21.15.120:0/1327550077 learned_addr learned my addr 172.21.15.120:0/1327550077 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7379072640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 -- 172.21.15.120:0/1327550077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 msgr2=0x55d4ddb765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 -- 172.21.15.120:0/1327550077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 msgr2=0x55d4ddb77b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb77b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.538+0000 7f7373fff640 1 -- 172.21.15.120:0/1327550077 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ddb76b10 con 0x55d4ddb63800 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7373fff640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73600a8ce0 tx=0x7f73600951e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2db8c116c5ed22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7353fff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600a6e60 con 0x55d4ddb63800 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f73600ab9c0 con 0x55d4ddb63800 2024-09-10T23:08:29.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736c009040 con 0x55d4ddb63800 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73600b5c40 con 0x55d4ddb63800 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f737ceaa740 10 client.28060 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4ddbed3a0 con 0x55d4ddb63800 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.539+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73600abbc0 con 0x55d4ddb63800 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f73600ab610 con 0x55d4ddb63800 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 10 client.28060 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 10 client.28060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 20 client.28060 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddbefb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 4 client.28060 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f736c009040 con 0x55d4ddb7f6a0 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f7378871640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddbefb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f7378871640 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddbefb60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10344ff6f40ad4ac server_cookie=fad4e9c775307d98 in_seq=0 out_seq=0 2024-09-10T23:08:29.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.540+0000 7f7353fff640 10 client.28060 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f7353fff640 1 -- 172.21.15.120:0/1327550077 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f736c009040 con 0x55d4ddb7f6a0 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f7353fff640 10 client.28060 handle_command_reply: tid=0 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 client.28060 shutdown 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 msgr2=0x55d4ddb76060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73600a8ce0 tx=0x7f73600951e0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 shutdown_connections 2024-09-10T23:08:29.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4ddb7f6a0 0x55d4ddbefb60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4ddb6f790 0x55d4ddb765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f7353fff640 0 client.28060 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4ddb63800 0x55d4ddb76060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 --2- 172.21.15.120:0/1327550077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4ddb62ee0 0x55d4ddb77b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 >> 172.21.15.120:0/1327550077 conn(0x55d4ddb60d10 msgr2=0x55d4ddb6dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 shutdown_connections 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 1 -- 172.21.15.120:0/1327550077 wait complete. 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.544+0000 7f737ceaa740 20 client.28060 trim_cache size 0 max 16384 2024-09-10T23:08:29.549 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:08:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:08:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "da14d37b-643f-43a0-a252-93f0ac5bb9b1", 2024-09-10T23:08:29.550 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:08:29.550 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:08:29.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 msgr2=0x7f73640b99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 0x7f73640b99b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c009e00 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 msgr2=0x7f7374197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9f0 tx=0x7f736000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 shutdown_connections 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f73640b74f0 0x7f73640b99b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7374108020 0x7f737419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73741076a0 0x7f7374197610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 --2- 172.21.15.120:0/1620720795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7374101c90 0x7f73741970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 >> 172.21.15.120:0/1620720795 conn(0x7f73740fd700 msgr2=0x7f73740ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 shutdown_connections 2024-09-10T23:08:29.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:29.545+0000 7f737aad5640 1 -- 172.21.15.120:0/1620720795 wait complete. 2024-09-10T23:08:29.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 85656 22.42 MB/sec execute 84 sec latency 68.771 ms 2024-09-10T23:08:30.141 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:29 smithi120 ceph-mon[31247]: pgmap v828: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:08:30.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:29 smithi153 ceph-mon[34951]: pgmap v828: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:08:30.325 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:29 smithi176 ceph-mon[34982]: pgmap v828: 97 pgs: 97 active+clean; 239 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-09-10T23:08:30.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:08:30.809 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:08:30.872 INFO:tasks.workunit.client.0.smithi120.stdout: 10 85744 22.19 MB/sec execute 85 sec latency 226.373 ms 2024-09-10T23:08:31.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:30 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:08:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:30 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:31.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:30 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:08:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:30 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:08:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:30 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:30 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:08:31.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:30 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:08:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:30 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:31.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:30 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 -- 172.21.15.120:0/861499189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd8104530 msgr2=0x7f7dd810cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/861499189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd8104530 0x7f7dd810cb10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009900 tx=0x7f7dc802a410 comp rx=0 tx=0).stop 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 -- 172.21.15.120:0/861499189 shutdown_connections 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/861499189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd810d0e0 0x7f7dd810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/861499189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd8104530 0x7f7dd810cb10 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/861499189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8103b30 0x7f7dd8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 -- 172.21.15.120:0/861499189 >> 172.21.15.120:0/861499189 conn(0x7f7dd80fd700 msgr2=0x7f7dd80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 -- 172.21.15.120:0/861499189 shutdown_connections 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.341+0000 7f7ddf4da640 1 -- 172.21.15.120:0/861499189 wait complete. 2024-09-10T23:08:31.347 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.342+0000 7f7ddf4da640 1 Processor -- start 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.342+0000 7f7ddf4da640 1 -- start start 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8104530 0x7f7dd8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd810d0e0 0x7f7dd8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8112d60 con 0x7f7dd810d0e0 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8112be0 con 0x7f7dd8104530 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddf4da640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8112ee0 con 0x7f7dd8103b30 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50290/0 (socket says 172.21.15.120:50290) 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 -- 172.21.15.120:0/3497567686 learned_addr learned my addr 172.21.15.120:0/3497567686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:31.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddda77640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd810d0e0 0x7f7dd8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddca75640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8104530 0x7f7dd8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 -- 172.21.15.120:0/3497567686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8104530 msgr2=0x7f7dd8101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8104530 0x7f7dd8101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 -- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd810d0e0 msgr2=0x7f7dd8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd810d0e0 0x7f7dd8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 -- 172.21.15.120:0/3497567686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd81a7690 con 0x7f7dd8103b30 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.343+0000 7f7ddd276640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d950 tx=0x7f7dd400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.344+0000 7f7dce7fc640 1 -- 172.21.15.120:0/3497567686 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4017070 con 0x7f7dd8103b30 2024-09-10T23:08:31.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.344+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd81a7980 con 0x7f7dd8103b30 2024-09-10T23:08:31.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.344+0000 7f7dce7fc640 1 -- 172.21.15.120:0/3497567686 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd4009c30 con 0x7f7dd8103b30 2024-09-10T23:08:31.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.344+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd81a7ee0 con 0x7f7dd8103b30 2024-09-10T23:08:31.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.344+0000 7f7dce7fc640 1 -- 172.21.15.120:0/3497567686 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400f620 con 0x7f7dd8103b30 2024-09-10T23:08:31.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 Processor -- start 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 -- start start 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb7500410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7dce7fc640 1 -- 172.21.15.120:0/3497567686 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7dd4010460 con 0x7f7dd8103b30 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb7502d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562cb746ae50 con 0x562cb74f40a0 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562cb745a280 con 0x562cb7500950 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.347+0000 7f7de18af740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562cb745e090 con 0x562cb74f49c0 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53816/0 (socket says 172.21.15.120:53816) 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 -- 172.21.15.120:0/4022816417 learned_addr learned my addr 172.21.15.120:0/4022816417 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddd276640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb7500410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddca75640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb7502d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 -- 172.21.15.120:0/4022816417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 msgr2=0x562cb7500410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb7500410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 -- 172.21.15.120:0/4022816417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 msgr2=0x562cb7502d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb7502d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddda77640 1 -- 172.21.15.120:0/4022816417 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb71b8650 con 0x562cb74f40a0 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddca75640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb7502d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.348+0000 7f7ddd276640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb7500410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7ddda77640 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009f90 tx=0x7f7dd00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c0cc90466ad65a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7dc37fe640 1 -- 172.21.15.120:0/4022816417 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0004d20 con 0x562cb74f40a0 2024-09-10T23:08:31.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7dc37fe640 1 -- 172.21.15.120:0/4022816417 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd0004050 con 0x562cb74f40a0 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7dce7fc640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 0x7f7dc40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7dc37fe640 1 -- 172.21.15.120:0/4022816417 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0005280 con 0x562cb74f40a0 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.349+0000 7f7ddd276640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 0x7f7dc40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7dce7fc640 1 -- 172.21.15.120:0/3497567686 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7dd40e5710 con 0x7f7dd8103b30 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 -- 172.21.15.120:0/4022816417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 msgr2=0x562cb74f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009f90 tx=0x7f7dd00118d0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 -- 172.21.15.120:0/4022816417 shutdown_connections 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb7502d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb7500410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.360 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 --2- 172.21.15.120:0/4022816417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f4480 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.350+0000 7f7de18af740 1 -- 172.21.15.120:0/4022816417 >> 172.21.15.120:0/4022816417 conn(0x562cb74f1ed0 msgr2=0x562cb74f3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.351+0000 7f7de18af740 1 -- 172.21.15.120:0/4022816417 shutdown_connections 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.351+0000 7f7ddd276640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 0x7f7dc40b7820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40abc50 tx=0x7f7dd40eb530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.351+0000 7f7de18af740 1 -- 172.21.15.120:0/4022816417 wait complete. 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.351+0000 7f7de18af740 1 -- start start 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.351+0000 7f7de18af740 1 Processor -- start 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb74fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53826/0 (socket says 172.21.15.120:53826) 2024-09-10T23:08:31.361 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 -- 172.21.15.120:0/1212628848 learned_addr learned my addr 172.21.15.120:0/1212628848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb74fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562cb72bb700 con 0x562cb74f40a0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562cb73f5110 con 0x562cb7500950 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562cb7503670 con 0x562cb74f49c0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddca75640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb74fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb74fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 -- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 msgr2=0x562cb74fd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb74fd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 -- 172.21.15.120:0/1212628848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 msgr2=0x562cb74fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb74fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb7422c40 con 0x562cb74f40a0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7ddda77640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009b00 tx=0x7f7dd0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767f974e0b506857 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7dc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 10 client.0 _ll_register_callbacks cb 0x562cb740e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd0004050 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.352+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/1212628848 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002b020 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/1212628848 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7dd000f4e0 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/1212628848 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7dd0003e40 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 10 client.18600 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=26}) -- 0x562cb750d3d0 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/1212628848 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7dd000f6e0 con 0x562cb74f40a0 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 10 client.18600 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 20 client.18600 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb750e780 0x562cb7510c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 4 client.18600 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:31.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7dd0004050 con 0x562cb750e780 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.353+0000 7f7ddca75640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb750e780 0x562cb7510c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.354+0000 7f7dc0ff9640 10 client.18600 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.354+0000 7f7ddca75640 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb750e780 0x562cb7510c40 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65711469b023934b server_cookie=2aa2893c7f365005 in_seq=0 out_seq=0 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/1212628848 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7dd0004050 con 0x562cb750e780 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7dc0ff9640 10 client.18600 handle_command_reply: tid=0 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 client.18600 shutdown 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 msgr2=0x562cb7500210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009b00 tx=0x7f7dd0009be0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 shutdown_connections 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb750e780 0x562cb7510c40 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb7500950 0x562cb74fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb74f49c0 0x562cb74fd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 --2- 172.21.15.120:0/1212628848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7500210 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 >> 172.21.15.120:0/1212628848 conn(0x562cb74f1ed0 msgr2=0x562cb74f7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7dc0ff9640 0 client.18600 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:31.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 shutdown_connections 2024-09-10T23:08:31.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 1 -- 172.21.15.120:0/1212628848 wait complete. 2024-09-10T23:08:31.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.355+0000 7f7de18af740 20 client.18600 trim_cache size 0 max 16384 2024-09-10T23:08:31.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 Processor -- start 2024-09-10T23:08:31.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 -- start start 2024-09-10T23:08:31.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7515260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.378 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb74fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81aa060 con 0x562cb74f40a0 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81ac8c0 con 0x562cb74f49c0 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7de18af740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81aa6b0 con 0x562cb7500950 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53838/0 (socket says 172.21.15.120:53838) 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.375+0000 7f7ddda77640 1 -- 172.21.15.120:0/2424972919 learned_addr learned my addr 172.21.15.120:0/2424972919 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddca75640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7515260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb74fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 -- 172.21.15.120:0/2424972919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 msgr2=0x562cb74fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb74fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 -- 172.21.15.120:0/2424972919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 msgr2=0x562cb7515260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7515260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 -- 172.21.15.120:0/2424972919 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb74f2bb0 con 0x562cb74f40a0 2024-09-10T23:08:31.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddca75640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7515260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:31.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0004670 tx=0x7f7dd0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855e398a44b0943c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7dc2ffd640 1 -- 172.21.15.120:0/2424972919 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002b070 con 0x562cb74f40a0 2024-09-10T23:08:31.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7dc2ffd640 1 -- 172.21.15.120:0/2424972919 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd0004710 con 0x562cb74f40a0 2024-09-10T23:08:31.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.376+0000 7f7dc2ffd640 1 -- 172.21.15.120:0/2424972919 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0027970 con 0x562cb74f40a0 2024-09-10T23:08:31.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- 172.21.15.120:0/2424972919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 msgr2=0x562cb74f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0004670 tx=0x7f7dd0014040 comp rx=0 tx=0).stop 2024-09-10T23:08:31.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- 172.21.15.120:0/2424972919 shutdown_connections 2024-09-10T23:08:31.382 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb74fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7515260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 --2- 172.21.15.120:0/2424972919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb74f36a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- 172.21.15.120:0/2424972919 >> 172.21.15.120:0/2424972919 conn(0x562cb74f1ed0 msgr2=0x562cb74f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- 172.21.15.120:0/2424972919 shutdown_connections 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- 172.21.15.120:0/2424972919 wait complete. 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.377+0000 7f7de18af740 1 -- start start 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 Processor -- start 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7504510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb7504a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81aad80 con 0x562cb74f40a0 2024-09-10T23:08:31.383 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7ddda77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53844/0 (socket says 172.21.15.120:53844) 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81ac1f0 con 0x562cb74f49c0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7ddda77640 1 -- 172.21.15.120:0/2618151914 learned_addr learned my addr 172.21.15.120:0/2618151914 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81ab450 con 0x562cb7500950 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7ddca75640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7504510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.378+0000 7f7de18af740 10 client.0 _ll_register_callbacks cb 0x7f7dd40eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 -- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 msgr2=0x562cb7504a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb7504a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 -- 172.21.15.120:0/2618151914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 msgr2=0x562cb7504510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7504510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 -- 172.21.15.120:0/2618151914 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb7505280 con 0x562cb74f40a0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd0004710 con 0x562cb74f40a0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7ddda77640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009da0 tx=0x7f7dd0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0878ce7c45875 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:31.384 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7dc0ff9640 10 client.18606 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/2618151914 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002b040 con 0x562cb74f40a0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/2618151914 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7dd0027470 con 0x562cb74f40a0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/2618151914 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7dd00276b0 con 0x562cb74f40a0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7de18af740 10 client.18606 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.379+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=26}) -- 0x562cb758e580 con 0x562cb74f40a0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/2618151914 <== mon.0 v2:172.21.15.120:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7dd0044030 con 0x562cb74f40a0 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 10 client.18606 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 20 client.18606 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb758eb30 0x562cb7590f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:31.385 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 4 client.18606 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:08:31.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cb746cda0 con 0x562cb758eb30 2024-09-10T23:08:31.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7ddca75640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb758eb30 0x562cb7590f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:31.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7ddca75640 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb758eb30 0x562cb7590f20 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31638d19b7fb8bd4 server_cookie=63293bffc3a0c450 in_seq=0 out_seq=0 2024-09-10T23:08:31.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.380+0000 7f7dc0ff9640 10 client.18606 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:31.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7dc0ff9640 1 -- 172.21.15.120:0/2618151914 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8238 (crc 0 0 0) 0x562cb746cda0 con 0x562cb758eb30 2024-09-10T23:08:31.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7dc0ff9640 10 client.18606 handle_command_reply: tid=0 2024-09-10T23:08:31.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 client.18606 shutdown 2024-09-10T23:08:31.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 msgr2=0x562cb7509040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.387 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009da0 tx=0x7f7dd0009500 comp rx=0 tx=0).stop 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 shutdown_connections 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562cb758eb30 0x562cb7590f20 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7dc0ff9640 0 client.18606 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562cb7500950 0x562cb7504a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb74f49c0 0x562cb7504510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 --2- 172.21.15.120:0/2618151914 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562cb74f40a0 0x562cb7509040 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 >> 172.21.15.120:0/2618151914 conn(0x562cb74f1ed0 msgr2=0x562cb74fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.384+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 shutdown_connections 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7de18af740 1 -- 172.21.15.120:0/2618151914 wait complete. 2024-09-10T23:08:31.388 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7de18af740 20 client.18606 trim_cache size 0 max 16384 2024-09-10T23:08:31.389 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:08:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:08:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:08:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "da14d37b-643f-43a0-a252-93f0ac5bb9b1", 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:08:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:08:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:08:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:08:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:08:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:08:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:08:31.408 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 msgr2=0x7f7dc40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 0x7f7dc40b7820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40abc50 tx=0x7f7dd40eb530 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 msgr2=0x7f7dd81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d950 tx=0x7f7dd400de10 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 shutdown_connections 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7dc40b5360 0x7f7dc40b7820 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dd810d0e0 0x7f7dd8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7dd8104530 0x7f7dd8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 --2- 172.21.15.120:0/3497567686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dd8103b30 0x7f7dd81019b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 >> 172.21.15.120:0/3497567686 conn(0x7f7dd80fd700 msgr2=0x7f7dd80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 shutdown_connections 2024-09-10T23:08:31.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:08:31.385+0000 7f7ddf4da640 1 -- 172.21.15.120:0/3497567686 wait complete. 2024-09-10T23:08:31.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 86021 22.07 MB/sec execute 86 sec latency 282.910 ms 2024-09-10T23:08:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 -- 172.21.15.120:0/2715866870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0101860 msgr2=0x7f75f0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 --2- 172.21.15.120:0/2715866870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0101860 0x7f75f0101c60 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009930 tx=0x7f75e002a420 comp rx=0 tx=0).stop 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 -- 172.21.15.120:0/2715866870 shutdown_connections 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 --2- 172.21.15.120:0/2715866870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0103370 0x7f75f0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 --2- 172.21.15.120:0/2715866870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 0x7f75f0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 --2- 172.21.15.120:0/2715866870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0101860 0x7f75f0101c60 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 -- 172.21.15.120:0/2715866870 >> 172.21.15.120:0/2715866870 conn(0x7f75f00fd140 msgr2=0x7f75f00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 -- 172.21.15.120:0/2715866870 shutdown_connections 2024-09-10T23:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.983+0000 7f75f4c79640 1 -- 172.21.15.120:0/2715866870 wait complete. 2024-09-10T23:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 Processor -- start 2024-09-10T23:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 -- start start 2024-09-10T23:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0101860 0x7f75f0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 0x7f75f0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75f010d4b0 con 0x7f75f0103370 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75f010d330 con 0x7f75f0101860 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75f4c79640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f75f010d630 con 0x7f75f01029b0 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53864/0 (socket says 172.21.15.120:53864) 2024-09-10T23:08:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75edd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 0x7f75f0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 -- 172.21.15.120:0/120106981 learned_addr learned my addr 172.21.15.120:0/120106981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75ee59c640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0101860 0x7f75f0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 -- 172.21.15.120:0/120106981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 msgr2=0x7f75f0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 0x7f75f0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 -- 172.21.15.120:0/120106981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0101860 msgr2=0x7f75f0198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0101860 0x7f75f0198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 -- 172.21.15.120:0/120106981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f019a920 con 0x7f75f0103370 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.984+0000 7f75eed9d640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009fd0 tx=0x7f75e800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.985+0000 7f75df7fe640 1 -- 172.21.15.120:0/120106981 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017070 con 0x7f75f0103370 2024-09-10T23:08:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.985+0000 7f75df7fe640 1 -- 172.21.15.120:0/120106981 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e8003e20 con 0x7f75f0103370 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.985+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f019ac10 con 0x7f75f0103370 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.985+0000 7f75df7fe640 1 -- 172.21.15.120:0/120106981 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8010460 con 0x7f75f0103370 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.985+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a12e0 con 0x7f75f0103370 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.987+0000 7f75df7fe640 1 -- 172.21.15.120:0/120106981 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f75e8003970 con 0x7f75f0103370 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.988+0000 7f75df7fe640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 0x7f75d80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.988+0000 7f75ee59c640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 0x7f75d80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.988+0000 7f75f7a49740 1 Processor -- start 2024-09-10T23:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.988+0000 7f75df7fe640 1 -- 172.21.15.120:0/120106981 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f75e80e5840 con 0x7f75f0103370 2024-09-10T23:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 -- start start 2024-09-10T23:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b0986c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0989bd0 0x5605b098c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5605b08f66f0 con 0x5605b0987190 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605b08e5b20 con 0x5605b0984860 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75f7a49740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5605b08e9930 con 0x5605b0989bd0 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75ee59c640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 0x7f75d80b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f75e0000c00 tx=0x7f75e00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0989bd0 0x5605b098c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53874/0 (socket says 172.21.15.120:53874) 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 -- 172.21.15.120:0/2436082598 learned_addr learned my addr 172.21.15.120:0/2436082598 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75ee59c640 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b0986c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 -- 172.21.15.120:0/2436082598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0989bd0 msgr2=0x5605b098c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0989bd0 0x5605b098c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 -- 172.21.15.120:0/2436082598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 msgr2=0x5605b0986c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b0986c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 -- 172.21.15.120:0/2436082598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b08ab530 con 0x5605b0987190 2024-09-10T23:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75edd9b640 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f75e400bd50 tx=0x7f75e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d188e0959cc54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.989+0000 7f75dcff9640 1 -- 172.21.15.120:0/2436082598 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4005220 con 0x5605b0987190 2024-09-10T23:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75dcff9640 1 -- 172.21.15.120:0/2436082598 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e400be40 con 0x5605b0987190 2024-09-10T23:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75dcff9640 1 -- 172.21.15.120:0/2436082598 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40117c0 con 0x5605b0987190 2024-09-10T23:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 -- 172.21.15.120:0/2436082598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 msgr2=0x5605b0989580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f75e400bd50 tx=0x7f75e40107a0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 -- 172.21.15.120:0/2436082598 shutdown_connections 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0989bd0 0x5605b098c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b0989580 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 --2- 172.21.15.120:0/2436082598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b0986c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 -- 172.21.15.120:0/2436082598 >> 172.21.15.120:0/2436082598 conn(0x5605b097f9b0 msgr2=0x5605b0980740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 -- 172.21.15.120:0/2436082598 shutdown_connections 2024-09-10T23:08:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.990+0000 7f75f7a49740 1 -- 172.21.15.120:0/2436082598 wait complete. 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 -- start start 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 Processor -- start 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0984860 0x5605b098ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b098e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5605b081dab0 con 0x5605b0989bd0 2024-09-10T23:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5605b088b730 con 0x5605b0987190 2024-09-10T23:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5605b098cbb0 con 0x5605b0984860 2024-09-10T23:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75f7a49740 10 client.0 _ll_register_callbacks cb 0x5605b08df6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53882/0 (socket says 172.21.15.120:53882) 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0984860 0x5605b098ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 -- 172.21.15.120:0/190455271 learned_addr learned my addr 172.21.15.120:0/190455271 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75edd9b640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b098e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 -- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0984860 msgr2=0x5605b098ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0984860 0x5605b098ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 -- 172.21.15.120:0/190455271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 msgr2=0x5605b098e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b098e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.991+0000 7f75eed9d640 1 -- 172.21.15.120:0/190455271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b0647790 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75eed9d640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f75e80e3370 tx=0x7f75e80aba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4441b385d9dc81b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8020040 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605b08f8640 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f75e80ab490 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75e80ac730 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 10 client.18618 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605b06c0620 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605b08f8640 con 0x5605b0989bd0 2024-09-10T23:08:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f75e80971b0 con 0x5605b0989bd0 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 10 client.18618 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 10 client.18618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 20 client.18618 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b09997b0 0x5605b099bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 4 client.18618 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.992+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b08f40f0 con 0x5605b09997b0 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.993+0000 7f75ee59c640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b09997b0 0x5605b099bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.993+0000 7f75d5ffb640 10 client.18618 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.993+0000 7f75ee59c640 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b09997b0 0x5605b099bba0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aafee3c7dc59914 server_cookie=32a06d8dba700c47 in_seq=0 out_seq=0 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.997+0000 7f75d5ffb640 1 -- 172.21.15.120:0/190455271 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605b08f40f0 con 0x5605b09997b0 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.997+0000 7f75d5ffb640 10 client.18618 handle_command_reply: tid=0 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 client.18618 shutdown 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 msgr2=0x5605b098f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f75e80e3370 tx=0x7f75e80aba40 comp rx=0 tx=0).stop 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 shutdown_connections 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b09997b0 0x5605b099bba0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0989bd0 0x5605b098f140 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b098e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75d5ffb640 0 client.18618 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 --2- 172.21.15.120:0/190455271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b0984860 0x5605b098ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 >> 172.21.15.120:0/190455271 conn(0x5605b097f9b0 msgr2=0x5605b097ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 shutdown_connections 2024-09-10T23:08:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 1 -- 172.21.15.120:0/190455271 wait complete. 2024-09-10T23:08:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:31.998+0000 7f75f7a49740 20 client.18618 trim_cache size 0 max 16384 2024-09-10T23:08:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 Processor -- start 2024-09-10T23:08:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 -- start start 2024-09-10T23:08:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b098e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b098d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01a7ec0 con 0x5605b0987190 2024-09-10T23:08:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01aa740 con 0x5605b0984860 2024-09-10T23:08:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75f7a49740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01a8530 con 0x5605b097f9b0 2024-09-10T23:08:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53890/0 (socket says 172.21.15.120:53890) 2024-09-10T23:08:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 -- 172.21.15.120:0/263098328 learned_addr learned my addr 172.21.15.120:0/263098328 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75ee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b098e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75edd9b640 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b098d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 -- 172.21.15.120:0/263098328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 msgr2=0x5605b098e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b098e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 -- 172.21.15.120:0/263098328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 msgr2=0x5605b098d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b098d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.015+0000 7f75eed9d640 1 -- 172.21.15.120:0/263098328 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b09898f0 con 0x5605b0987190 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.016+0000 7f75eed9d640 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f75e8005c40 tx=0x7f75e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daba9224b2bdb26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.016+0000 7f75d7fff640 1 -- 172.21.15.120:0/263098328 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8020030 con 0x5605b0987190 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.016+0000 7f75d7fff640 1 -- 172.21.15.120:0/263098328 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e8091e50 con 0x5605b0987190 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.016+0000 7f75d7fff640 1 -- 172.21.15.120:0/263098328 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8097410 con 0x5605b0987190 2024-09-10T23:08:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- 172.21.15.120:0/263098328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 msgr2=0x5605b098d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f75e8005c40 tx=0x7f75e800b910 comp rx=0 tx=0).stop 2024-09-10T23:08:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- 172.21.15.120:0/263098328 shutdown_connections 2024-09-10T23:08:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0987190 0x5605b098d670 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0984860 0x5605b098d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 --2- 172.21.15.120:0/263098328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b098e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- 172.21.15.120:0/263098328 >> 172.21.15.120:0/263098328 conn(0x5605b097aea0 msgr2=0x5605b0989fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- 172.21.15.120:0/263098328 shutdown_connections 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- 172.21.15.120:0/263098328 wait complete. 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 -- start start 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.017+0000 7f75f7a49740 1 Processor -- start 2024-09-10T23:08:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b09958a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b0994b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01a8c00 con 0x5605b0984860 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01aa070 con 0x5605b0987190 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01a92d0 con 0x5605b097f9b0 2024-09-10T23:08:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75f7a49740 10 client.0 _ll_register_callbacks cb 0x7f75e80a86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53896/0 (socket says 172.21.15.120:53896) 2024-09-10T23:08:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 -- 172.21.15.120:0/256383623 learned_addr learned my addr 172.21.15.120:0/256383623 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75eed9d640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b0994b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75ee59c640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b09958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 -- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 msgr2=0x5605b09958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b09958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 -- 172.21.15.120:0/256383623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 msgr2=0x5605b0994b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b0994b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 -- 172.21.15.120:0/256383623 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b0995240 con 0x5605b0984860 2024-09-10T23:08:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75eed9d640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b0994b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75edd9b640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f75e400bd50 tx=0x7f75e4005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f8c329a5ca95d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.018+0000 7f75d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e402b060 con 0x5605b0984860 2024-09-10T23:08:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f75e4004860 con 0x5605b0984860 2024-09-10T23:08:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e8091e50 con 0x5605b0984860 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75e4004a60 con 0x5605b0984860 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 10 client.18624 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605b0a06690 con 0x5605b0984860 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e8091e50 con 0x5605b0984860 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f75e4025070 con 0x5605b0984860 2024-09-10T23:08:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 10 client.18624 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 10 client.18624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 20 client.18624 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b0989bd0 0x5605b0a08b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 4 client.18624 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:08:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605b08f1520 con 0x5605b0989bd0 2024-09-10T23:08:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.019+0000 7f75ee59c640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b0989bd0 0x5605b0a08b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.020+0000 7f75d5ffb640 10 client.18624 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.020+0000 7f75ee59c640 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b0989bd0 0x5605b0a08b90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feaad5408885bf27 server_cookie=4b12ea19e1444d1f in_seq=0 out_seq=0 2024-09-10T23:08:32.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:31 smithi120 ceph-mon[31247]: pgmap v829: 97 pgs: 97 active+clean; 243 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 614 op/s 2024-09-10T23:08:32.180 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:31 smithi153 ceph-mon[34951]: pgmap v829: 97 pgs: 97 active+clean; 243 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 614 op/s 2024-09-10T23:08:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:32.233 INFO:tasks.cephfs.filesystem:scrub status for tag:da14d37b-643f-43a0-a252-93f0ac5bb9b1 - {'path': '/', 'tag': 'da14d37b-643f-43a0-a252-93f0ac5bb9b1', 'options': 'recursive,force'} 2024-09-10T23:08:32.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:31 smithi176 ceph-mon[34982]: pgmap v829: 97 pgs: 97 active+clean; 243 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 614 op/s 2024-09-10T23:08:32.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75d5ffb640 1 -- 172.21.15.120:0/256383623 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605b08f1520 con 0x5605b0989bd0 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75d5ffb640 10 client.18624 handle_command_reply: tid=0 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 client.18624 shutdown 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 msgr2=0x5605b09945c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f75e400bd50 tx=0x7f75e4005340 comp rx=0 tx=0).stop 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 shutdown_connections 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5605b0989bd0 0x5605b0a08b90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5605b0987190 0x5605b0994b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5605b0984860 0x5605b09945c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 --2- 172.21.15.120:0/256383623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5605b097f9b0 0x5605b09958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75d5ffb640 0 client.18624 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 >> 172.21.15.120:0/256383623 conn(0x5605b097aea0 msgr2=0x5605b098d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 shutdown_connections 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.819+0000 7f75f7a49740 1 -- 172.21.15.120:0/256383623 wait complete. 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f7a49740 20 client.18624 trim_cache size 0 max 16384 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:08:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:08:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:08:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 msgr2=0x7f75d80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 0x7f75d80b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f75e0000c00 tx=0x7f75e00314a0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 msgr2=0x7f75f019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009fd0 tx=0x7f75e800b260 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 shutdown_connections 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f75d80b5360 0x7f75d80b7820 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75f0103370 0x7f75f019a060 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01029b0 0x7f75f0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 --2- 172.21.15.120:0/120106981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75f0101860 0x7f75f0198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 >> 172.21.15.120:0/120106981 conn(0x7f75f00fd140 msgr2=0x7f75f00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 shutdown_connections 2024-09-10T23:08:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:32.820+0000 7f75f4c79640 1 -- 172.21.15.120:0/120106981 wait complete. 2024-09-10T23:08:32.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 86165 21.90 MB/sec execute 87 sec latency 245.236 ms 2024-09-10T23:08:33.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 86497 21.76 MB/sec execute 88 sec latency 286.063 ms 2024-09-10T23:08:34.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:33 smithi120 ceph-mon[31247]: pgmap v830: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 513 op/s 2024-09-10T23:08:34.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:33 smithi153 ceph-mon[34951]: pgmap v830: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 513 op/s 2024-09-10T23:08:34.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:33 smithi176 ceph-mon[34982]: pgmap v830: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 513 op/s 2024-09-10T23:08:34.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 87042 21.80 MB/sec execute 89 sec latency 62.469 ms 2024-09-10T23:08:35.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 87732 21.98 MB/sec execute 90 sec latency 39.324 ms 2024-09-10T23:08:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:35 smithi120 ceph-mon[31247]: pgmap v831: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 383 op/s 2024-09-10T23:08:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:08:36.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:35 smithi153 ceph-mon[34951]: pgmap v831: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 383 op/s 2024-09-10T23:08:36.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:08:36.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:35 smithi176 ceph-mon[34982]: pgmap v831: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 383 op/s 2024-09-10T23:08:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:08:36.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 88324 22.25 MB/sec execute 91 sec latency 57.469 ms 2024-09-10T23:08:37.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 88682 22.24 MB/sec execute 92 sec latency 179.553 ms 2024-09-10T23:08:38.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:37 smithi120 ceph-mon[31247]: pgmap v832: 97 pgs: 97 active+clean; 236 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-10T23:08:38.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:37 smithi153 ceph-mon[34951]: pgmap v832: 97 pgs: 97 active+clean; 236 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-10T23:08:38.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:37 smithi176 ceph-mon[34982]: pgmap v832: 97 pgs: 97 active+clean; 236 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-10T23:08:38.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 88981 22.23 MB/sec execute 93 sec latency 112.921 ms 2024-09-10T23:08:39.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 89258 22.10 MB/sec execute 94 sec latency 115.486 ms 2024-09-10T23:08:40.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:39 smithi120 ceph-mon[31247]: pgmap v833: 97 pgs: 97 active+clean; 232 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-10T23:08:40.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:39 smithi153 ceph-mon[34951]: pgmap v833: 97 pgs: 97 active+clean; 232 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-10T23:08:40.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:39 smithi176 ceph-mon[34982]: pgmap v833: 97 pgs: 97 active+clean; 232 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-10T23:08:40.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 89612 22.00 MB/sec execute 95 sec latency 72.167 ms 2024-09-10T23:08:41.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 90120 21.97 MB/sec execute 96 sec latency 76.090 ms 2024-09-10T23:08:42.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:41 smithi120 ceph-mon[31247]: pgmap v834: 97 pgs: 97 active+clean; 239 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 723 op/s 2024-09-10T23:08:42.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:41 smithi153 ceph-mon[34951]: pgmap v834: 97 pgs: 97 active+clean; 239 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 723 op/s 2024-09-10T23:08:42.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:41 smithi176 ceph-mon[34982]: pgmap v834: 97 pgs: 97 active+clean; 239 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 723 op/s 2024-09-10T23:08:42.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 90693 22.04 MB/sec execute 97 sec latency 83.037 ms 2024-09-10T23:08:43.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 91170 22.11 MB/sec execute 98 sec latency 271.053 ms 2024-09-10T23:08:44.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:43 smithi120 ceph-mon[31247]: pgmap v835: 97 pgs: 97 active+clean; 242 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2024-09-10T23:08:44.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:43 smithi153 ceph-mon[34951]: pgmap v835: 97 pgs: 97 active+clean; 242 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2024-09-10T23:08:44.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:43 smithi176 ceph-mon[34982]: pgmap v835: 97 pgs: 97 active+clean; 242 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 688 op/s 2024-09-10T23:08:44.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 91721 22.25 MB/sec execute 99 sec latency 74.899 ms 2024-09-10T23:08:45.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92201 22.36 MB/sec execute 100 sec latency 55.978 ms 2024-09-10T23:08:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 -- 172.21.15.120:0/3132093056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 msgr2=0x7f0cb0109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/3132093056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb0109cf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009cd0 tx=0x7f0ca802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 -- 172.21.15.120:0/3132093056 shutdown_connections 2024-09-10T23:08:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/3132093056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb0109cf0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/3132093056 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 0x7f0cb0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/3132093056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 -- 172.21.15.120:0/3132093056 >> 172.21.15.120:0/3132093056 conn(0x7f0cb00fd140 msgr2=0x7f0cb00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 -- 172.21.15.120:0/3132093056 shutdown_connections 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.916+0000 7f0cb6d94640 1 -- 172.21.15.120:0/3132093056 wait complete. 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 Processor -- start 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 -- start start 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb006b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 0x7f0cb019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010d4b0 con 0x7f0cb0103370 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010d330 con 0x7f0cb0101860 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb6d94640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010d630 con 0x7f0cb01029b0 2024-09-10T23:08:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb5331640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb5331640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44542/0 (socket says 172.21.15.120:44542) 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb5331640 1 -- 172.21.15.120:0/1457848638 learned_addr learned my addr 172.21.15.120:0/1457848638 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0caffff640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 0x7f0cb019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.917+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb006b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 -- 172.21.15.120:0/1457848638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 msgr2=0x7f0cb019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 0x7f0cb019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 -- 172.21.15.120:0/1457848638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 msgr2=0x7f0cb006b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb006b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 -- 172.21.15.120:0/1457848638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb01a08d0 con 0x7f0cb0103370 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb5331640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004af0 tx=0x7f0ca8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cadffb640 1 -- 172.21.15.120:0/1457848638 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8004250 con 0x7f0cb0103370 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb01a0b60 con 0x7f0cb0103370 2024-09-10T23:08:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cadffb640 1 -- 172.21.15.120:0/1457848638 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca80305a0 con 0x7f0cb0103370 2024-09-10T23:08:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cadffb640 1 -- 172.21.15.120:0/1457848638 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80466e0 con 0x7f0cb0103370 2024-09-10T23:08:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.918+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01a1100 con 0x7f0cb0103370 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.920+0000 7f0cadffb640 1 -- 172.21.15.120:0/1457848638 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0ca804e070 con 0x7f0cb0103370 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 Processor -- start 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 -- start start 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f626580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f626bd0 0x55cc5f629090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f5936f0 con 0x55cc5f621860 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f582b20 con 0x55cc5f626bd0 2024-09-10T23:08:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f586930 con 0x55cc5f624190 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cadffb640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 0x7f0ca00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44548/0 (socket says 172.21.15.120:44548) 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 -- 172.21.15.120:0/2300934256 learned_addr learned my addr 172.21.15.120:0/2300934256 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f626580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb5331640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 0x7f0ca00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f626bd0 0x55cc5f629090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cadffb640 1 -- 172.21.15.120:0/1457848638 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0ca802e030 con 0x7f0cb0103370 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 -- 172.21.15.120:0/2300934256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 msgr2=0x55cc5f626580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f626580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 -- 172.21.15.120:0/2300934256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f626bd0 msgr2=0x55cc5f629090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f626bd0 0x55cc5f629090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0caffff640 1 -- 172.21.15.120:0/2300934256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5f548530 con 0x55cc5f621860 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.921+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0caffff640 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b620 tx=0x7f0c9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aec3eb52c89a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0cb5331640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 0x7f0ca00b77a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80cab30 tx=0x7f0ca8097990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0c9affd640 1 -- 172.21.15.120:0/2300934256 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004030 con 0x55cc5f621860 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0c9affd640 1 -- 172.21.15.120:0/2300934256 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c0041d0 con 0x55cc5f621860 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0c9affd640 1 -- 172.21.15.120:0/2300934256 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00e770 con 0x55cc5f621860 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/2300934256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 msgr2=0x55cc5f623c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.922+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b620 tx=0x7f0c9c00b8e0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/2300934256 shutdown_connections 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f626bd0 0x55cc5f629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/2300934256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f621860 0x55cc5f623c50 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/2300934256 >> 172.21.15.120:0/2300934256 conn(0x55cc5f61c9b0 msgr2=0x55cc5f61d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/2300934256 shutdown_connections 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/2300934256 wait complete. 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- start start 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 Processor -- start 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f62b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f4baab0 con 0x55cc5f626bd0 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f528730 con 0x55cc5f621860 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cc5f629bb0 con 0x55cc5f624190 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.923+0000 7f0cb9c3a740 10 client.0 _ll_register_callbacks cb 0x55cc5f57c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0cb4b30640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f62b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44554/0 (socket says 172.21.15.120:44554) 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 -- 172.21.15.120:0/3409459073 learned_addr learned my addr 172.21.15.120:0/3409459073 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 -- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 msgr2=0x55cc5f62b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f62b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 -- 172.21.15.120:0/3409459073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 msgr2=0x55cc5f62af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 -- 172.21.15.120:0/3409459073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5f2e4790 con 0x55cc5f626bd0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f62b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0caffff640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b5f0 tx=0x7f0c9c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a343b2f15625c152 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0037f0 con 0x55cc5f626bd0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0c9c005030 con 0x55cc5f626bd0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9c0041d0 con 0x55cc5f626bd0 2024-09-10T23:08:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c9c005230 con 0x55cc5f626bd0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0cb9c3a740 10 client.18636 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc5f35d620 con 0x55cc5f626bd0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.924+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c0041d0 con 0x55cc5f626bd0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0c9c01c020 con 0x55cc5f626bd0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 10 client.18636 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 20 client.18636 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6365b0 0x55cc5f6389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 4 client.18636 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc5f595640 con 0x55cc5f6365b0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6365b0 0x55cc5f6389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0c8ffff640 10 client.18636 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.925+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6365b0 0x55cc5f6389a0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8f6077d5cfa89c server_cookie=eb112bb5bc584638 in_seq=0 out_seq=0 2024-09-10T23:08:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0c8ffff640 1 -- 172.21.15.120:0/3409459073 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc5f595640 con 0x55cc5f6365b0 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0c8ffff640 10 client.18636 handle_command_reply: tid=0 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 client.18636 shutdown 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 msgr2=0x55cc5f62c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b5f0 tx=0x7f0c9c005c50 comp rx=0 tx=0).stop 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 shutdown_connections 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6365b0 0x55cc5f6389a0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f62c200 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f62b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0c8ffff640 0 client.18636 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/3409459073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 >> 172.21.15.120:0/3409459073 conn(0x55cc5f61c9b0 msgr2=0x55cc5f61bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 shutdown_connections 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/3409459073 wait complete. 2024-09-10T23:08:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.927+0000 7f0cb9c3a740 20 client.18636 trim_cache size 0 max 16384 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 Processor -- start 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 -- start start 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f621860 0x55cc5f61c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 0x55cc5f626fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01a7cc0 con 0x55cc5f626bd0 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01aa540 con 0x55cc5f624190 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01a8330 con 0x55cc5f621860 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44562/0 (socket says 172.21.15.120:44562) 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 -- 172.21.15.120:0/419751375 learned_addr learned my addr 172.21.15.120:0/419751375 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f621860 0x55cc5f61c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 0x55cc5f626fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 -- 172.21.15.120:0/419751375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f621860 msgr2=0x55cc5f61c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f621860 0x55cc5f61c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 -- 172.21.15.120:0/419751375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 msgr2=0x55cc5f626fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 0x55cc5f626fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 -- 172.21.15.120:0/419751375 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5f62ad60 con 0x55cc5f626bd0 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 0x55cc5f626fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.943+0000 7f0caffff640 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0121e0 tx=0x7f0c9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271e763de5509359 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0c9a7fc640 1 -- 172.21.15.120:0/419751375 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00e5c0 con 0x55cc5f626bd0 2024-09-10T23:08:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0c9a7fc640 1 -- 172.21.15.120:0/419751375 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c029040 con 0x55cc5f626bd0 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0c9a7fc640 1 -- 172.21.15.120:0/419751375 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0035f0 con 0x55cc5f626bd0 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/419751375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 msgr2=0x55cc5f6274f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0121e0 tx=0x7f0c9c00b8e0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/419751375 shutdown_connections 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f6274f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f624190 0x55cc5f626fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/419751375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f621860 0x55cc5f61c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/419751375 >> 172.21.15.120:0/419751375 conn(0x55cc5f61c9b0 msgr2=0x55cc5f620de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/419751375 shutdown_connections 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.944+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/419751375 wait complete. 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 -- start start 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 Processor -- start 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f6301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01a8a00 con 0x55cc5f626bd0 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01a9e70 con 0x55cc5f621860 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb01a90d0 con 0x55cc5f624190 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb9c3a740 10 client.0 _ll_register_callbacks cb 0x7f0c9c012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0caffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44572/0 (socket says 172.21.15.120:44572) 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0caffff640 1 -- 172.21.15.120:0/414631628 learned_addr learned my addr 172.21.15.120:0/414631628 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f6301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.945+0000 7f0caffff640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 -- 172.21.15.120:0/414631628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 msgr2=0x55cc5f62fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 -- 172.21.15.120:0/414631628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 msgr2=0x55cc5f630ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 -- 172.21.15.120:0/414631628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5f6314d0 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0caffff640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb4b30640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f6301c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0122a0 tx=0x7f0ca4012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1705d211a22e06d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40116c0 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9c029040 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0ca4011860 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ca4011a60 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb9c3a740 10 client.28111 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cc5f632910 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.946+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9c029040 con 0x55cc5f624190 2024-09-10T23:08:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0ca402e430 con 0x55cc5f624190 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 10 client.28111 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 10 client.28111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 20 client.28111 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6a3470 0x55cc5f6a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 4 client.28111 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.947+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc5f5910f0 con 0x55cc5f6a3470 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.948+0000 7f0caffff640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6a3470 0x55cc5f6a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.949+0000 7f0caffff640 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6a3470 0x55cc5f6a5860 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70e4d2efba4c9ab server_cookie=1f3ae649aaded114 in_seq=0 out_seq=0 2024-09-10T23:08:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:45.949+0000 7f0c8ffff640 10 client.28111 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:46.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:45 smithi120 ceph-mon[31247]: pgmap v836: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-09-10T23:08:46.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:45 smithi153 ceph-mon[34951]: pgmap v836: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-09-10T23:08:46.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:45 smithi176 ceph-mon[34982]: pgmap v836: 97 pgs: 97 active+clean; 241 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.5 MiB/s wr, 577 op/s 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0c8ffff640 1 -- 172.21.15.120:0/414631628 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc5f5910f0 con 0x55cc5f6a3470 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0c8ffff640 10 client.28111 handle_command_reply: tid=0 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 client.28111 shutdown 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 msgr2=0x55cc5f6301c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f6301c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0122a0 tx=0x7f0ca4012ad0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 shutdown_connections 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cc5f6a3470 0x55cc5f6a5860 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0c8ffff640 0 client.28111 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cc5f626bd0 0x55cc5f630ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc5f624190 0x55cc5f6301c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 --2- 172.21.15.120:0/414631628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cc5f621860 0x55cc5f62fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 >> 172.21.15.120:0/414631628 conn(0x55cc5f61c9b0 msgr2=0x55cc5f61d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 shutdown_connections 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.604+0000 7f0cb9c3a740 1 -- 172.21.15.120:0/414631628 wait complete. 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb9c3a740 20 client.28111 trim_cache size 0 max 16384 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:08:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 msgr2=0x7f0ca00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 0x7f0ca00b77a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80cab30 tx=0x7f0ca8097990 comp rx=0 tx=0).stop 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 msgr2=0x7f0cb01a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004af0 tx=0x7f0ca8003c10 comp rx=0 tx=0).stop 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 shutdown_connections 2024-09-10T23:08:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0ca00b52e0 0x7f0ca00b77a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cb0103370 0x7f0cb01a0010 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cb01029b0 0x7f0cb019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 --2- 172.21.15.120:0/1457848638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cb0101860 0x7f0cb006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 >> 172.21.15.120:0/1457848638 conn(0x7f0cb00fd140 msgr2=0x7f0cb0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 shutdown_connections 2024-09-10T23:08:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:46.605+0000 7f0cb6d94640 1 -- 172.21.15.120:0/1457848638 wait complete. 2024-09-10T23:08:46.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92378 22.26 MB/sec execute 101 sec latency 386.674 ms 2024-09-10T23:08:47.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92641 22.17 MB/sec execute 102 sec latency 141.140 ms 2024-09-10T23:08:48.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:47 smithi120 ceph-mon[31247]: pgmap v837: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.1 MiB/s wr, 670 op/s 2024-09-10T23:08:48.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:47 smithi153 ceph-mon[34951]: pgmap v837: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.1 MiB/s wr, 670 op/s 2024-09-10T23:08:48.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:47 smithi176 ceph-mon[34982]: pgmap v837: 97 pgs: 97 active+clean; 248 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.1 MiB/s wr, 670 op/s 2024-09-10T23:08:48.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 92837 22.00 MB/sec execute 103 sec latency 182.752 ms 2024-09-10T23:08:49.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 93201 21.94 MB/sec execute 104 sec latency 88.258 ms 2024-09-10T23:08:50.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:49 smithi120 ceph-mon[31247]: pgmap v838: 97 pgs: 97 active+clean; 249 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-09-10T23:08:50.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:49 smithi153 ceph-mon[34951]: pgmap v838: 97 pgs: 97 active+clean; 249 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-09-10T23:08:50.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:49 smithi176 ceph-mon[34982]: pgmap v838: 97 pgs: 97 active+clean; 249 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-09-10T23:08:50.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 93648 21.91 MB/sec execute 105 sec latency 52.490 ms 2024-09-10T23:08:51.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:51.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:51.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:51.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:50 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:08:51.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:51.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:51.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:51.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:50 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:08:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:08:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:08:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:08:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:50 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:08:51.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 94248 21.97 MB/sec execute 106 sec latency 41.267 ms 2024-09-10T23:08:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:51 smithi120 ceph-mon[31247]: pgmap v839: 97 pgs: 97 active+clean; 238 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-09-10T23:08:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:51 smithi153 ceph-mon[34951]: pgmap v839: 97 pgs: 97 active+clean; 238 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-09-10T23:08:52.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:51 smithi176 ceph-mon[34982]: pgmap v839: 97 pgs: 97 active+clean; 238 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-09-10T23:08:52.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 94932 22.13 MB/sec execute 107 sec latency 43.335 ms 2024-09-10T23:08:53.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 95346 22.23 MB/sec execute 108 sec latency 152.322 ms 2024-09-10T23:08:54.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:53 smithi120 ceph-mon[31247]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-09-10T23:08:54.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:53 smithi153 ceph-mon[34951]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-09-10T23:08:54.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:53 smithi176 ceph-mon[34982]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-09-10T23:08:54.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 95764 22.27 MB/sec execute 109 sec latency 142.474 ms 2024-09-10T23:08:55.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96057 22.25 MB/sec execute 110 sec latency 114.187 ms 2024-09-10T23:08:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:55 smithi120 ceph-mon[31247]: pgmap v841: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 577 op/s 2024-09-10T23:08:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:55 smithi153 ceph-mon[34951]: pgmap v841: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 577 op/s 2024-09-10T23:08:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:55 smithi176 ceph-mon[34982]: pgmap v841: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 577 op/s 2024-09-10T23:08:56.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96338 22.13 MB/sec execute 111 sec latency 84.980 ms 2024-09-10T23:08:57.873 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96680 22.06 MB/sec execute 112 sec latency 87.184 ms 2024-09-10T23:08:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:57 smithi120 ceph-mon[31247]: pgmap v842: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 622 op/s 2024-09-10T23:08:58.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:57 smithi153 ceph-mon[34951]: pgmap v842: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 622 op/s 2024-09-10T23:08:58.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:57 smithi176 ceph-mon[34982]: pgmap v842: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 622 op/s 2024-09-10T23:08:58.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 96962 21.98 MB/sec execute 113 sec latency 285.388 ms 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 -- 172.21.15.120:0/3246877175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 msgr2=0x7fade4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3246877175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4101c60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fade0009930 tx=0x7fade002a420 comp rx=0 tx=0).stop 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 -- 172.21.15.120:0/3246877175 shutdown_connections 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3246877175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 0x7fade4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3246877175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 0x7fade4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3246877175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4101c60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 -- 172.21.15.120:0/3246877175 >> 172.21.15.120:0/3246877175 conn(0x7fade40fd140 msgr2=0x7fade40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 -- 172.21.15.120:0/3246877175 shutdown_connections 2024-09-10T23:08:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 -- 172.21.15.120:0/3246877175 wait complete. 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.699+0000 7fadec1ec640 1 Processor -- start 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 -- start start 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 0x7fade4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 0x7fade419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d4b0 con 0x7fade4101860 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d330 con 0x7fade41029b0 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadec1ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade410d630 con 0x7fade4103370 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45720/0 (socket says 172.21.15.120:45720) 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 -- 172.21.15.120:0/3981196594 learned_addr learned my addr 172.21.15.120:0/3981196594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadea789640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 0x7fade419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9787640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 0x7fade4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 -- 172.21.15.120:0/3981196594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 msgr2=0x7fade419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 0x7fade419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 -- 172.21.15.120:0/3981196594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 msgr2=0x7fade4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 0x7fade4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 -- 172.21.15.120:0/3981196594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419a940 con 0x7fade4101860 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fade9f88640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fade0009a50 tx=0x7fade00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadd6ffd640 1 -- 172.21.15.120:0/3981196594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade003f030 con 0x7fade4101860 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.700+0000 7fadd6ffd640 1 -- 172.21.15.120:0/3981196594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0003cb0 con 0x7fade4101860 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.701+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade419abd0 con 0x7fade4101860 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.701+0000 7fadd6ffd640 1 -- 172.21.15.120:0/3981196594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0035070 con 0x7fade4101860 2024-09-10T23:08:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.701+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a1300 con 0x7fade4101860 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 Processor -- start 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadd6ffd640 1 -- 172.21.15.120:0/3981196594 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fade003bb90 con 0x7fade4101860 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 -- start start 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e9546fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95472bd0 0x562e95475090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e953df6f0 con 0x562e95470190 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e953ceb20 con 0x562e95472bd0 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.703+0000 7fadeebfd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562e953d2930 con 0x562e9546d860 2024-09-10T23:08:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadea789640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e9546fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45728/0 (socket says 172.21.15.120:45728) 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 -- 172.21.15.120:0/2975974235 learned_addr learned my addr 172.21.15.120:0/2975974235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9787640 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95472bd0 0x562e95475090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 -- 172.21.15.120:0/2975974235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 msgr2=0x562e9546fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e9546fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 -- 172.21.15.120:0/2975974235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95472bd0 msgr2=0x562e95475090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95472bd0 0x562e95475090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 -- 172.21.15.120:0/2975974235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e95394530 con 0x562e95470190 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadea789640 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e9546fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadd6ffd640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 0x7fadd00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fade9f88640 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fade010cc10 tx=0x7fade010cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c591eb00c7728aca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadea789640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 0x7fadd00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadcffff640 1 -- 172.21.15.120:0/2975974235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00d4e60 con 0x562e95470190 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadd6ffd640 1 -- 172.21.15.120:0/3981196594 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fade0106810 con 0x7fade4101860 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadcffff640 1 -- 172.21.15.120:0/2975974235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0003e60 con 0x562e95470190 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.704+0000 7fadcffff640 1 -- 172.21.15.120:0/2975974235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00c9700 con 0x562e95470190 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 -- 172.21.15.120:0/2975974235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 msgr2=0x562e95472580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadea789640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 0x7fadd00b77a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fade4199b80 tx=0x7faddc0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fade010cc10 tx=0x7fade010cd00 comp rx=0 tx=0).stop 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 -- 172.21.15.120:0/2975974235 shutdown_connections 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95472bd0 0x562e95475090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472580 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2975974235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e9546fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 -- 172.21.15.120:0/2975974235 >> 172.21.15.120:0/2975974235 conn(0x562e954689b0 msgr2=0x562e95469740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 -- 172.21.15.120:0/2975974235 shutdown_connections 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.705+0000 7fadeebfd740 1 -- 172.21.15.120:0/2975974235 wait complete. 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 -- start start 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 Processor -- start 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e95476b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e95477230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562e95306ab0 con 0x562e95472bd0 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562e95374730 con 0x562e95470190 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562e95475970 con 0x562e9546d860 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadeebfd740 10 client.0 _ll_register_callbacks cb 0x562e953c86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadea789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadea789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45738/0 (socket says 172.21.15.120:45738) 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fade9f88640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e95476b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fadea789640 1 -- 172.21.15.120:0/2512731720 learned_addr learned my addr 172.21.15.120:0/2512731720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.706+0000 7fade9787640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e95477230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 -- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 msgr2=0x562e95476b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e95476b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 -- 172.21.15.120:0/2512731720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 msgr2=0x562e95477230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e95477230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 -- 172.21.15.120:0/2512731720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e95130790 con 0x562e95472bd0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fade9f88640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e95476b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadea789640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7faddc009270 tx=0x7faddc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8e1e3cce8fbeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc02c060 con 0x562e95472bd0 2024-09-10T23:08:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7faddc017030 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e953e1640 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faddc011650 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadeebfd740 10 client.18654 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e951a9620 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.707+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e953e1640 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7faddc027850 con 0x562e95472bd0 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 10 client.18654 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 10 client.18654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 20 client.18654 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e95482430 0x562e954848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 4 client.18654 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e953dd0f0 con 0x562e95482430 2024-09-10T23:08:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fade9f88640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e95482430 0x562e954848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fadcd7fa640 10 client.18654 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.708+0000 7fade9f88640 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e95482430 0x562e954848a0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831571545d331048 server_cookie=f281c0387ee244b8 in_seq=0 out_seq=0 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2512731720 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e953dd0f0 con 0x562e95482430 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadcd7fa640 10 client.18654 handle_command_reply: tid=0 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 client.18654 shutdown 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 msgr2=0x562e95477e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7faddc009270 tx=0x7faddc004ab0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 shutdown_connections 2024-09-10T23:08:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e95482430 0x562e954848a0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95472bd0 0x562e95477e00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e95477230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadcd7fa640 0 client.18654 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2512731720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e9546d860 0x562e95476b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 >> 172.21.15.120:0/2512731720 conn(0x562e954689b0 msgr2=0x562e95467bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 shutdown_connections 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.711+0000 7fadeebfd740 1 -- 172.21.15.120:0/2512731720 wait complete. 2024-09-10T23:08:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.712+0000 7fadeebfd740 20 client.18654 trim_cache size 0 max 16384 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 Processor -- start 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 -- start start 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e9546d860 0x562e95477de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e954734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a7ec0 con 0x562e95470190 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aa740 con 0x562e9546d860 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.726+0000 7fadeebfd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a8530 con 0x562e95472bd0 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fadea789640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e954734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45748/0 (socket says 172.21.15.120:45748) 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 -- 172.21.15.120:0/1269254675 learned_addr learned my addr 172.21.15.120:0/1269254675 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9f88640 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e9546d860 0x562e95477de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 -- 172.21.15.120:0/1269254675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 msgr2=0x562e954734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e954734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 -- 172.21.15.120:0/1269254675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e9546d860 msgr2=0x562e95477de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e9546d860 0x562e95477de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 -- 172.21.15.120:0/1269254675 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e95476960 con 0x562e95470190 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fade9787640 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009f10 tx=0x7fadd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2bda67e9732a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1269254675 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8022070 con 0x562e95470190 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1269254675 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80035f0 con 0x562e95470190 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.727+0000 7fadcf7fe640 1 -- 172.21.15.120:0/1269254675 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd801d680 con 0x562e95470190 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- 172.21.15.120:0/1269254675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 msgr2=0x562e95472fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009f10 tx=0x7fadd8005f90 comp rx=0 tx=0).stop 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- 172.21.15.120:0/1269254675 shutdown_connections 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e954734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e95470190 0x562e95472fb0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 --2- 172.21.15.120:0/1269254675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e9546d860 0x562e95477de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- 172.21.15.120:0/1269254675 >> 172.21.15.120:0/1269254675 conn(0x562e954689b0 msgr2=0x562e9546ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- 172.21.15.120:0/1269254675 shutdown_connections 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- 172.21.15.120:0/1269254675 wait complete. 2024-09-10T23:08:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 -- start start 2024-09-10T23:08:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.728+0000 7fadeebfd740 1 Processor -- start 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e9547e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a8c00 con 0x562e9546d860 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aa070 con 0x562e95470190 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a92d0 con 0x562e95472bd0 2024-09-10T23:08:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadeebfd740 10 client.0 _ll_register_callbacks cb 0x7faddc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadea789640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e9547e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45760/0 (socket says 172.21.15.120:45760) 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9787640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36010/0 (socket says 172.21.15.120:36010) 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fadea789640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e9547e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50268/0 (socket says 172.21.15.120:50268) 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 -- 172.21.15.120:0/2876057509 learned_addr learned my addr 172.21.15.120:0/2876057509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 -- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 msgr2=0x562e9547e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e9547e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 -- 172.21.15.120:0/2876057509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 msgr2=0x562e9547de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9f88640 1 -- 172.21.15.120:0/2876057509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e9547c6d0 con 0x562e9546d860 2024-09-10T23:08:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.729+0000 7fade9787640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fade9f88640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fade0104390 tx=0x7fade010c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8205c5318995a5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0049040 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fade003b460 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd80035f0 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fade00d4e60 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 10 client.18660 fetch_fsmap learned FSMap version 26 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e954ef950 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80035f0 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fade00cd310 con 0x562e9546d860 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 10 client.18660 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 10 client.18660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:08:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 20 client.18660 populate_metadata read hostname 'smithi120' 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e954f0140 0x562e954f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 4 client.18660 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.730+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e953da520 con 0x562e954f0140 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.731+0000 7fade9787640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e954f0140 0x562e954f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.731+0000 7fadcd7fa640 10 client.18660 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:08:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:08:59.731+0000 7fade9787640 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e954f0140 0x562e954f2560 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426e633fb0b37998 server_cookie=a0b4e9b307e4d8c7 in_seq=0 out_seq=0 2024-09-10T23:08:59.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 97469 21.97 MB/sec execute 114 sec latency 108.735 ms 2024-09-10T23:09:00.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:08:59 smithi120 ceph-mon[31247]: pgmap v843: 97 pgs: 97 active+clean; 242 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-09-10T23:09:00.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:08:59 smithi153 ceph-mon[34951]: pgmap v843: 97 pgs: 97 active+clean; 242 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-09-10T23:09:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadcd7fa640 1 -- 172.21.15.120:0/2876057509 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e953da520 con 0x562e954f0140 2024-09-10T23:09:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadcd7fa640 10 client.18660 handle_command_reply: tid=0 2024-09-10T23:09:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 client.18660 shutdown 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 msgr2=0x562e9547d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fade0104390 tx=0x7fade010c730 comp rx=0 tx=0).stop 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 shutdown_connections 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562e954f0140 0x562e954f2560 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadcd7fa640 0 client.18660 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562e95472bd0 0x562e9547e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562e95470190 0x562e9547de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 --2- 172.21.15.120:0/2876057509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562e9546d860 0x562e9547d8c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 >> 172.21.15.120:0/2876057509 conn(0x562e954689b0 msgr2=0x562e9546cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 shutdown_connections 2024-09-10T23:09:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 1 -- 172.21.15.120:0/2876057509 wait complete. 2024-09-10T23:09:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.316+0000 7fadeebfd740 20 client.18660 trim_cache size 0 max 16384 2024-09-10T23:09:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:09:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:09:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:09:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:09:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 msgr2=0x7fadd00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 0x7fadd00b77a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fade4199b80 tx=0x7faddc0096a0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 msgr2=0x7fade4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fade0009a50 tx=0x7fade00042f0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 shutdown_connections 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fadd00b52e0 0x7fadd00b77a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fade4103370 0x7fade419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade41029b0 0x7fade4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 --2- 172.21.15.120:0/3981196594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fade4101860 0x7fade4198e00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 >> 172.21.15.120:0/3981196594 conn(0x7fade40fd140 msgr2=0x7fade406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 shutdown_connections 2024-09-10T23:09:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:00.317+0000 7fadec1ec640 1 -- 172.21.15.120:0/3981196594 wait complete. 2024-09-10T23:09:00.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:08:59 smithi176 ceph-mon[34982]: pgmap v843: 97 pgs: 97 active+clean; 242 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-09-10T23:09:00.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 97759 21.91 MB/sec execute 115 sec latency 370.699 ms 2024-09-10T23:09:01.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 98518 22.07 MB/sec execute 116 sec latency 45.101 ms 2024-09-10T23:09:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:01 smithi120 ceph-mon[31247]: pgmap v844: 97 pgs: 97 active+clean; 245 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:09:02.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:01 smithi153 ceph-mon[34951]: pgmap v844: 97 pgs: 97 active+clean; 245 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:09:02.235 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:09:02.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:01 smithi176 ceph-mon[34982]: pgmap v844: 97 pgs: 97 active+clean; 245 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T23:09:02.494 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:02.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 99047 22.24 MB/sec execute 117 sec latency 60.058 ms 2024-09-10T23:09:03.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.090+0000 7fbb312e4640 1 -- 172.21.15.120:0/963140407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 msgr2=0x7fbb2c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.090+0000 7fbb312e4640 1 --2- 172.21.15.120:0/963140407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c107ae0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009900 tx=0x7fbb2002a410 comp rx=0 tx=0).stop 2024-09-10T23:09:03.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 -- 172.21.15.120:0/963140407 shutdown_connections 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 --2- 172.21.15.120:0/963140407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 0x7fbb2c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 --2- 172.21.15.120:0/963140407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c107ae0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 --2- 172.21.15.120:0/963140407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 -- 172.21.15.120:0/963140407 >> 172.21.15.120:0/963140407 conn(0x7fbb2c0fd700 msgr2=0x7fbb2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 -- 172.21.15.120:0/963140407 shutdown_connections 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.091+0000 7fbb312e4640 1 -- 172.21.15.120:0/963140407 wait complete. 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 Processor -- start 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 -- start start 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 0x7fbb2c109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c111ca0 con 0x7fbb2c1076a0 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c111b20 con 0x7fbb2c101c90 2024-09-10T23:09:03.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb312e4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c111e20 con 0x7fbb2c108020 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb2a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb2a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45774/0 (socket says 172.21.15.120:45774) 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb2a7fc640 1 -- 172.21.15.120:0/1610274587 learned_addr learned my addr 172.21.15.120:0/1610274587 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.092+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 0x7fbb2c109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2affd640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 -- 172.21.15.120:0/1610274587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 msgr2=0x7fbb2c109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 0x7fbb2c109290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 -- 172.21.15.120:0/1610274587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 msgr2=0x7fbb2c108810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c108810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 -- 172.21.15.120:0/1610274587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c1abab0 con 0x7fbb2c1076a0 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2affd640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005940 tx=0x7fbb200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb13fff640 1 -- 172.21.15.120:0/1610274587 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2003f030 con 0x7fbb2c1076a0 2024-09-10T23:09:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb13fff640 1 -- 172.21.15.120:0/1610274587 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb200044c0 con 0x7fbb2c1076a0 2024-09-10T23:09:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb13fff640 1 -- 172.21.15.120:0/1610274587 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20035070 con 0x7fbb2c1076a0 2024-09-10T23:09:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.093+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c1abd40 con 0x7fbb2c1076a0 2024-09-10T23:09:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.094+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c1ac160 con 0x7fbb2c1076a0 2024-09-10T23:09:03.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.096+0000 7fbb13fff640 1 -- 172.21.15.120:0/1610274587 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbb2003ba20 con 0x7fbb2c1076a0 2024-09-10T23:09:03.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 Processor -- start 2024-09-10T23:09:03.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 -- start start 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f69c0 0x563621902410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x563621904d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56362186ce50 con 0x5636218f69c0 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56362185c280 con 0x563621902950 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb336b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563621860090 con 0x5636218f60a0 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb13fff640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 0x7fbb140b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.097+0000 7fbb2b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:50298/0 (socket says 172.21.15.120:50298) 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 -- 172.21.15.120:0/963698543 learned_addr learned my addr 172.21.15.120:0/963698543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x563621904d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb13fff640 1 -- 172.21.15.120:0/1610274587 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbb20106180 con 0x7fbb2c1076a0 2024-09-10T23:09:03.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 0x7fbb140b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 -- 172.21.15.120:0/963698543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 msgr2=0x563621904d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x563621904d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 -- 172.21.15.120:0/963698543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f69c0 msgr2=0x563621902410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f69c0 0x563621902410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 -- 172.21.15.120:0/963698543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636215ba650 con 0x5636218f60a0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.098+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005c40 tx=0x7fbb2400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=351fe7e6aee4f791 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.099+0000 7fbb117fa640 1 -- 172.21.15.120:0/963698543 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24009be0 con 0x5636218f60a0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.099+0000 7fbb117fa640 1 -- 172.21.15.120:0/963698543 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb24022030 con 0x5636218f60a0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.099+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 0x7fbb140b78a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbb24013fd0 tx=0x7fbb2401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.099+0000 7fbb117fa640 1 -- 172.21.15.120:0/963698543 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24010670 con 0x5636218f60a0 2024-09-10T23:09:03.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- 172.21.15.120:0/963698543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 msgr2=0x5636218f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005c40 tx=0x7fbb2400f3c0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- 172.21.15.120:0/963698543 shutdown_connections 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x563621904d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f69c0 0x563621902410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 --2- 172.21.15.120:0/963698543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f60a0 0x5636218f6480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- 172.21.15.120:0/963698543 >> 172.21.15.120:0/963698543 conn(0x5636218f3ed0 msgr2=0x5636218f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- 172.21.15.120:0/963698543 shutdown_connections 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- 172.21.15.120:0/963698543 wait complete. 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 -- start start 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.100+0000 7fbb336b9740 1 Processor -- start 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218ff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5636216bd700 con 0x5636218f60a0 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5636217f7110 con 0x563621902950 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563621905570 con 0x5636218f69c0 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 10 client.0 _ll_register_callbacks cb 0x563621810990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45794/0 (socket says 172.21.15.120:45794) 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 -- 172.21.15.120:0/2704819858 learned_addr learned my addr 172.21.15.120:0/2704819858 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 -- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 msgr2=0x5636218ff710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218ff710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 -- 172.21.15.120:0/2704819858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 msgr2=0x5636218ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218ffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.106 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 -- 172.21.15.120:0/2704819858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563621824c40 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0119e0 tx=0x7fbb1c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8beb1fd7c511ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c01e060 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbb1c00fc60 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.101+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb24022030 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c00fe60 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 10 client.18669 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563621910fd0 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb24022030 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbb1c018440 con 0x5636218f60a0 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 10 client.18669 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 10 client.18669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 20 client.18669 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:03.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563621912490 0x5636219148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 4 client.18669 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:03.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56362186eda0 con 0x563621912490 2024-09-10T23:09:03.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.102+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563621912490 0x5636219148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.103+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563621912490 0x5636219148a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b17ca76e1a750b server_cookie=f1a05b0b66f4fdcf in_seq=0 out_seq=0 2024-09-10T23:09:03.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.103+0000 7fbb0eff5640 10 client.18669 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:03.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.107+0000 7fbb0eff5640 1 -- 172.21.15.120:0/2704819858 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56362186eda0 con 0x563621912490 2024-09-10T23:09:03.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.107+0000 7fbb0eff5640 10 client.18669 handle_command_reply: tid=0 2024-09-10T23:09:03.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.107+0000 7fbb336b9740 1 client.18669 shutdown 2024-09-10T23:09:03.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 msgr2=0x5636219022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0119e0 tx=0x7fbb1c011ea0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 shutdown_connections 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563621912490 0x5636219148a0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2704819858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219022c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 >> 172.21.15.120:0/2704819858 conn(0x5636218f3ed0 msgr2=0x5636218f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb0eff5640 0 client.18669 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 shutdown_connections 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 1 -- 172.21.15.120:0/2704819858 wait complete. 2024-09-10T23:09:03.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.108+0000 7fbb336b9740 20 client.18669 trim_cache size 0 max 16384 2024-09-10T23:09:03.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.128+0000 7fbb336b9740 1 Processor -- start 2024-09-10T23:09:03.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.128+0000 7fbb336b9740 1 -- start start 2024-09-10T23:09:03.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b27f0 con 0x5636218f60a0 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b5070 con 0x563621902950 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb336b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b2e60 con 0x5636218f69c0 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53796/0 (socket says 172.21.15.120:53796) 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 -- 172.21.15.120:0/2135407848 learned_addr learned my addr 172.21.15.120:0/2135407848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 -- 172.21.15.120:0/2135407848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 msgr2=0x5636218fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 -- 172.21.15.120:0/2135407848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 msgr2=0x5636218fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218fe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 -- 172.21.15.120:0/2135407848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563621900100 con 0x5636218f60a0 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.129+0000 7fbb2affd640 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c018860 tx=0x7fbb1c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ad9db829e825ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.130+0000 7fbb10ff9640 1 -- 172.21.15.120:0/2135407848 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c029070 con 0x5636218f60a0 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.130+0000 7fbb10ff9640 1 -- 172.21.15.120:0/2135407848 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c018ad0 con 0x5636218f60a0 2024-09-10T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.130+0000 7fbb10ff9640 1 -- 172.21.15.120:0/2135407848 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c02edb0 con 0x5636218f60a0 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- 172.21.15.120:0/2135407848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 msgr2=0x5636219144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c018860 tx=0x7fbb1c005f30 comp rx=0 tx=0).stop 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- 172.21.15.120:0/2135407848 shutdown_connections 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563621902950 0x5636218fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x5636218fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 --2- 172.21.15.120:0/2135407848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5636218f60a0 0x5636219144a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- 172.21.15.120:0/2135407848 >> 172.21.15.120:0/2135407848 conn(0x5636218f3ed0 msgr2=0x5636218f4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- 172.21.15.120:0/2135407848 shutdown_connections 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- 172.21.15.120:0/2135407848 wait complete. 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 -- start start 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.131+0000 7fbb336b9740 1 Processor -- start 2024-09-10T23:09:03.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 0x56362190aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x56362190af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b3530 con 0x563621902950 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b49a0 con 0x5636218f60a0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2a7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x56362190af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1b3c00 con 0x5636218f69c0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53812/0 (socket says 172.21.15.120:53812) 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2b7fe640 1 -- 172.21.15.120:0/3734531143 learned_addr learned my addr 172.21.15.120:0/3734531143 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2affd640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 0x56362190aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb336b9740 10 client.0 _ll_register_callbacks cb 0x7fbb1c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2affd640 1 -- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 msgr2=0x56362190af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.132+0000 7fbb2affd640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x56362190af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb2affd640 1 -- 172.21.15.120:0/3734531143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 msgr2=0x5636218fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb2affd640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb2affd640 1 -- 172.21.15.120:0/3734531143 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636218fc040 con 0x5636218f60a0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb2b7fe640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c018ad0 con 0x5636218f60a0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb2affd640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 0x56362190aa30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0188e0 tx=0x7fbb1c011270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1554f3a8a6ed2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb0eff5640 10 client.27932 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb0eff5640 1 -- 172.21.15.120:0/3734531143 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c01e070 con 0x5636218f60a0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb0eff5640 1 -- 172.21.15.120:0/3734531143 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbb1c02e670 con 0x5636218f60a0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.133+0000 7fbb0eff5640 1 -- 172.21.15.120:0/3734531143 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c029040 con 0x5636218f60a0 2024-09-10T23:09:03.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 10 client.27932 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5636218fd250 con 0x5636218f60a0 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb0eff5640 1 -- 172.21.15.120:0/3734531143 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbb1c02eab0 con 0x5636218f60a0 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 10 client.27932 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 20 client.27932 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636219803d0 0x5636219827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 4 client.27932 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.134+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56362186a850 con 0x5636219803d0 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.135+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636219803d0 0x5636219827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:03.140 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.135+0000 7fbb2a7fc640 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636219803d0 0x5636219827c0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252eb78d276edf32 server_cookie=bdd69b3cb17be68d in_seq=0 out_seq=0 2024-09-10T23:09:03.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.135+0000 7fbb0eff5640 10 client.27932 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb0eff5640 1 -- 172.21.15.120:0/3734531143 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8042 (crc 0 0 0) 0x56362186a850 con 0x5636219803d0 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb0eff5640 10 client.27932 handle_command_reply: tid=0 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 client.27932 shutdown 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 msgr2=0x56362190aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 0x56362190aa30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0188e0 tx=0x7fbb1c011270 comp rx=0 tx=0).stop 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 shutdown_connections 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5636219803d0 0x5636219827c0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563621902950 0x5636218fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb0eff5640 0 client.27932 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5636218f69c0 0x56362190af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 --2- 172.21.15.120:0/3734531143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5636218f60a0 0x56362190aa30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 >> 172.21.15.120:0/3734531143 conn(0x5636218f3ed0 msgr2=0x5636218fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.139+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 shutdown_connections 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb336b9740 1 -- 172.21.15.120:0/3734531143 wait complete. 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb336b9740 20 client.27932 trim_cache size 0 max 16384 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:09:03.143 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:09:03.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.146 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:09:03.148 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:09:03.150 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.151 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:09:03.153 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.154 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:09:03.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:03.160 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 msgr2=0x7fbb140b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 0x7fbb140b78a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbb24013fd0 tx=0x7fbb2401f000 comp rx=0 tx=0).stop 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 msgr2=0x7fbb2c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005940 tx=0x7fbb200040c0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 shutdown_connections 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbb140b53e0 0x7fbb140b78a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.140+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbb2c108020 0x7fbb2c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.141+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb2c1076a0 0x7fbb2c108d50 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.141+0000 7fbb312e4640 1 --2- 172.21.15.120:0/1610274587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb2c101c90 0x7fbb2c108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:03.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.141+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 >> 172.21.15.120:0/1610274587 conn(0x7fbb2c0fd700 msgr2=0x7fbb2c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:03.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.141+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 shutdown_connections 2024-09-10T23:09:03.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:03.141+0000 7fbb312e4640 1 -- 172.21.15.120:0/1610274587 wait complete. 2024-09-10T23:09:03.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 99427 22.24 MB/sec execute 118 sec latency 120.671 ms 2024-09-10T23:09:04.197 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:03 smithi120 ceph-mon[31247]: pgmap v845: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:09:04.390 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:03 smithi176 ceph-mon[34982]: pgmap v845: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:09:04.390 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:03 smithi153 ceph-mon[34951]: pgmap v845: 97 pgs: 97 active+clean; 236 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-09-10T23:09:04.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:09:04.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:09:04.675 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:04.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 99722 22.20 MB/sec execute 119 sec latency 99.957 ms 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 -- 172.21.15.120:0/3321686165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a341076a0 msgr2=0x7f5a34107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 --2- 172.21.15.120:0/3321686165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a341076a0 0x7f5a34107ae0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009900 tx=0x7f5a2802a410 comp rx=0 tx=0).stop 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 -- 172.21.15.120:0/3321686165 shutdown_connections 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 --2- 172.21.15.120:0/3321686165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a34108020 0x7f5a3410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 --2- 172.21.15.120:0/3321686165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a341076a0 0x7f5a34107ae0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 --2- 172.21.15.120:0/3321686165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a34102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 -- 172.21.15.120:0/3321686165 >> 172.21.15.120:0/3321686165 conn(0x7f5a340fd700 msgr2=0x7f5a340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 -- 172.21.15.120:0/3321686165 shutdown_connections 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.205+0000 7f5a39f71640 1 -- 172.21.15.120:0/3321686165 wait complete. 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 Processor -- start 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 -- start start 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a3419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 0x7f5a341a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 0x7f5a341a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111d30 con 0x7f5a34108020 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111bb0 con 0x7f5a341076a0 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a39f71640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111eb0 con 0x7f5a34101c90 2024-09-10T23:09:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a33fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 0x7f5a341a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a33fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 0x7f5a341a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53830/0 (socket says 172.21.15.120:53830) 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a33fff640 1 -- 172.21.15.120:0/766941127 learned_addr learned my addr 172.21.15.120:0/766941127 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a337fe640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a3419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.206+0000 7f5a32ffd640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 0x7f5a341a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 -- 172.21.15.120:0/766941127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 msgr2=0x7f5a341a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 0x7f5a341a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 -- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 msgr2=0x7f5a341a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 0x7f5a341a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 -- 172.21.15.120:0/766941127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a341a2720 con 0x7f5a34101c90 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a32ffd640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 0x7f5a341a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a337fe640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a3419fb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400ccc0 con 0x7f5a34101c90 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a240108e0 con 0x7f5a34101c90 2024-09-10T23:09:05.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.207+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24018880 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.208+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341a2a10 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.208+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a341ac310 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.210+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34102f40 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.211+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5a24010430 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.212+0000 7f5a30ff9640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 0x7f5a1c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.212+0000 7f5a32ffd640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 0x7f5a1c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.212+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5a240e6bc0 con 0x7f5a34101c90 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.212+0000 7f5a32ffd640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 0x7f5a1c0b78f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005ab0 tx=0x7f5a28005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:05.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.218+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5a24027070 con 0x7f5a34101c90 2024-09-10T23:09:05.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.470+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a341ac6b0 con 0x7f5a34101c90 2024-09-10T23:09:05.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.470+0000 7f5a30ff9640 1 -- 172.21.15.120:0/766941127 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f5a240adad0 con 0x7f5a34101c90 2024-09-10T23:09:05.472 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:09:05.473 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:09:05.473 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:09:05.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 msgr2=0x7f5a1c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:05.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 0x7f5a1c0b78f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005ab0 tx=0x7f5a28005a20 comp rx=0 tx=0).stop 2024-09-10T23:09:05.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 msgr2=0x7f5a3419fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:05.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a3419fb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 shutdown_connections 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a1c0b5430 0x7f5a1c0b78f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a34108020 0x7f5a341a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a341076a0 0x7f5a341a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 --2- 172.21.15.120:0/766941127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a34101c90 0x7f5a3419fb80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 >> 172.21.15.120:0/766941127 conn(0x7f5a340fd700 msgr2=0x7f5a3406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:05.475 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 shutdown_connections 2024-09-10T23:09:05.476 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:05.473+0000 7f5a39f71640 1 -- 172.21.15.120:0/766941127 wait complete. 2024-09-10T23:09:05.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 100048 22.14 MB/sec execute 120 sec latency 61.938 ms 2024-09-10T23:09:06.120 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:05 smithi120 ceph-mon[31247]: pgmap v846: 97 pgs: 97 active+clean; 232 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 511 op/s 2024-09-10T23:09:06.121 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:05 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/766941127' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:06.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:09:06.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:05 smithi153 ceph-mon[34951]: pgmap v846: 97 pgs: 97 active+clean; 232 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 511 op/s 2024-09-10T23:09:06.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:05 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/766941127' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:06.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:05 smithi176 ceph-mon[34982]: pgmap v846: 97 pgs: 97 active+clean; 232 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 511 op/s 2024-09-10T23:09:06.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:05 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/766941127' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:06.411 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:06.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 100463 22.11 MB/sec execute 121 sec latency 76.801 ms 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 -- 172.21.15.120:0/1157703227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d141076a0 msgr2=0x7f8d14107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 --2- 172.21.15.120:0/1157703227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d141076a0 0x7f8d14107ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009900 tx=0x7f8d0802a410 comp rx=0 tx=0).stop 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 -- 172.21.15.120:0/1157703227 shutdown_connections 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 --2- 172.21.15.120:0/1157703227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d14108020 0x7f8d1410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 --2- 172.21.15.120:0/1157703227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d141076a0 0x7f8d14107ae0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 --2- 172.21.15.120:0/1157703227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 0x7f8d14102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 -- 172.21.15.120:0/1157703227 >> 172.21.15.120:0/1157703227 conn(0x7f8d140fd700 msgr2=0x7f8d140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 -- 172.21.15.120:0/1157703227 shutdown_connections 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.902+0000 7f8d19394640 1 -- 172.21.15.120:0/1157703227 wait complete. 2024-09-10T23:09:06.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 Processor -- start 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 -- start start 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 0x7f8d141992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d141076a0 0x7f8d14199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111ca0 con 0x7f8d14108020 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111b20 con 0x7f8d14101c90 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d19394640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111e20 con 0x7f8d141076a0 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d127fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d141076a0 0x7f8d14199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 0x7f8d141992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53846/0 (socket says 172.21.15.120:53846) 2024-09-10T23:09:06.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.903+0000 7f8d137fe640 1 -- 172.21.15.120:0/2336374955 learned_addr learned my addr 172.21.15.120:0/2336374955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 -- 172.21.15.120:0/2336374955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d141076a0 msgr2=0x7f8d14199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d141076a0 0x7f8d14199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 -- 172.21.15.120:0/2336374955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 msgr2=0x7f8d141992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 0x7f8d141992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 -- 172.21.15.120:0/2336374955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1419f370 con 0x7f8d14108020 2024-09-10T23:09:06.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d137fe640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00eaa0 tx=0x7f8d0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8cfbfff640 1 -- 172.21.15.120:0/2336374955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00cdb0 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8cfbfff640 1 -- 172.21.15.120:0/2336374955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c005020 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1419f660 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8cfbfff640 1 -- 172.21.15.120:0/2336374955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0116d0 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.904+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a5d20 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.906+0000 7f8cfbfff640 1 -- 172.21.15.120:0/2336374955 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8d0c00f040 con 0x7f8d14108020 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 Processor -- start 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 -- start start 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdced4d0 0x55d4bdced8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdc63ea0 con 0x55d4bdceddf0 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdc532d0 con 0x55d4bdcfe190 2024-09-10T23:09:06.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d1b769740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdc570e0 con 0x55d4bdced4d0 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d127fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdced4d0 0x55d4bdced8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53856/0 (socket says 172.21.15.120:53856) 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 -- 172.21.15.120:0/2873392872 learned_addr learned my addr 172.21.15.120:0/2873392872 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d127fc640 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 -- 172.21.15.120:0/2873392872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdced4d0 msgr2=0x55d4bdced8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdced4d0 0x55d4bdced8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 -- 172.21.15.120:0/2873392872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 msgr2=0x55d4bdd00580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd00580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d12ffd640 1 -- 172.21.15.120:0/2873392872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bd9b1650 con 0x55d4bdceddf0 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.907+0000 7f8d127fc640 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd00580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8cfbfff640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 0x7f8cfc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400bd80 tx=0x7f8d040107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2ad223cc117cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8d127fc640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 0x7f8cfc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8cfbfff640 1 -- 172.21.15.120:0/2336374955 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d0c0e58f0 con 0x7f8d14108020 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8cf97fa640 1 -- 172.21.15.120:0/2873392872 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04005220 con 0x55d4bdceddf0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8cf97fa640 1 -- 172.21.15.120:0/2873392872 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04004c50 con 0x55d4bdceddf0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.908+0000 7f8cf97fa640 1 -- 172.21.15.120:0/2873392872 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011800 con 0x55d4bdceddf0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- 172.21.15.120:0/2873392872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 msgr2=0x55d4bdcfdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400bd80 tx=0x7f8d040107a0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d127fc640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 0x7f8cfc0b77a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d08031440 tx=0x7f8d08005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- 172.21.15.120:0/2873392872 shutdown_connections 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd00580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfdc50 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 --2- 172.21.15.120:0/2873392872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdced4d0 0x55d4bdced8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- 172.21.15.120:0/2873392872 >> 172.21.15.120:0/2873392872 conn(0x55d4bdceb300 msgr2=0x55d4bdcec4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- 172.21.15.120:0/2873392872 shutdown_connections 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- 172.21.15.120:0/2873392872 wait complete. 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 -- start start 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.909+0000 7f8d1b769740 1 Processor -- start 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdcfb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdab4700 con 0x55d4bdced4d0 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdbee160 con 0x55d4bdcfe190 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdd00e40 con 0x55d4bdceddf0 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 10 client.0 _ll_register_callbacks cb 0x55d4bdc079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53866/0 (socket says 172.21.15.120:53866) 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 -- 172.21.15.120:0/1660451551 learned_addr learned my addr 172.21.15.120:0/1660451551 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d127fc640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 -- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdceddf0 msgr2=0x55d4bdcfaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 -- 172.21.15.120:0/1660451551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 msgr2=0x55d4bdcfb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d137fe640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdcfb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdcfb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 -- 172.21.15.120:0/1660451551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bdc1bc90 con 0x55d4bdced4d0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8d04005690 tx=0x7f8d040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ab17fb8ef41b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04005210 con 0x55d4bdced4d0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d04004020 con 0x55d4bdced4d0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.910+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04004c50 con 0x55d4bdced4d0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d0400be70 con 0x55d4bdced4d0 2024-09-10T23:09:06.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 10 client.18693 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4bdd0c7e0 con 0x55d4bdced4d0 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04004c50 con 0x55d4bdced4d0 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8d04023030 con 0x55d4bdced4d0 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 10 client.18693 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 10 client.18693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 20 client.18693 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd0dc40 0x55d4bdd10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 4 client.18693 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4bdc65df0 con 0x55d4bdd0dc40 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.911+0000 7f8d127fc640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd0dc40 0x55d4bdd10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.912+0000 7f8d127fc640 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd0dc40 0x55d4bdd10030 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bcd60c681ff8cb server_cookie=621a3b67c7ffe32 in_seq=0 out_seq=0 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.912+0000 7f8cf6ff5640 10 client.18693 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:06.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/1660451551 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4bdc65df0 con 0x55d4bdd0dc40 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8cf6ff5640 10 client.18693 handle_command_reply: tid=0 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 client.18693 shutdown 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 msgr2=0x55d4bdcfdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8d04005690 tx=0x7f8d040048f0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 shutdown_connections 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd0dc40 0x55d4bdd10030 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdcfb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdceddf0 0x55d4bdcfaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8cf6ff5640 0 client.18693 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 --2- 172.21.15.120:0/1660451551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdced4d0 0x55d4bdcfdac0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.914+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 >> 172.21.15.120:0/1660451551 conn(0x55d4bdceb300 msgr2=0x55d4bdcf0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.915+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 shutdown_connections 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.915+0000 7f8d1b769740 1 -- 172.21.15.120:0/1660451551 wait complete. 2024-09-10T23:09:06.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.915+0000 7f8d1b769740 20 client.18693 trim_cache size 0 max 16384 2024-09-10T23:09:06.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.929+0000 7f8d1b769740 1 Processor -- start 2024-09-10T23:09:06.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.929+0000 7f8d1b769740 1 -- start start 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.929+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.929+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcf8c20 0x55d4bdcec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d1b769740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aa130 con 0x55d4bdceddf0 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d1b769740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ac990 con 0x55d4bdcfe190 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d1b769740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aa780 con 0x55d4bdcf8c20 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53868/0 (socket says 172.21.15.120:53868) 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d12ffd640 1 -- 172.21.15.120:0/763106675 learned_addr learned my addr 172.21.15.120:0/763106675 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d137fe640 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcf8c20 0x55d4bdcec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 -- 172.21.15.120:0/763106675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcf8c20 msgr2=0x55d4bdcec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcf8c20 0x55d4bdcec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 -- 172.21.15.120:0/763106675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 msgr2=0x55d4bdd00620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 -- 172.21.15.120:0/763106675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bdcedc20 con 0x55d4bdcfe190 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8d127fc640 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd0a0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009230 tx=0x7f8d0802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6118ec67d22ef842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8cf8ff9640 1 -- 172.21.15.120:0/763106675 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08035040 con 0x55d4bdcfe190 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8cf8ff9640 1 -- 172.21.15.120:0/763106675 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d080043c0 con 0x55d4bdcfe190 2024-09-10T23:09:06.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.930+0000 7f8cf8ff9640 1 -- 172.21.15.120:0/763106675 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0804a870 con 0x55d4bdcfe190 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 -- 172.21.15.120:0/763106675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 msgr2=0x55d4bdd0a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd0a0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009230 tx=0x7f8d0802a410 comp rx=0 tx=0).stop 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 -- 172.21.15.120:0/763106675 shutdown_connections 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcf8c20 0x55d4bdcec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcfe190 0x55d4bdd0a0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 --2- 172.21.15.120:0/763106675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 -- 172.21.15.120:0/763106675 >> 172.21.15.120:0/763106675 conn(0x55d4bdceb300 msgr2=0x55d4bdcf9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 -- 172.21.15.120:0/763106675 shutdown_connections 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.931+0000 7f8d1b769740 1 -- 172.21.15.120:0/763106675 wait complete. 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 -- start start 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 Processor -- start 2024-09-10T23:09:06.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 0x55d4bdd030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcfe190 0x55d4bdd03610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aae50 con 0x55d4bdceddf0 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ac2c0 con 0x55d4bdcf8c20 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ab520 con 0x55d4bdcfe190 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d1b769740 10 client.0 _ll_register_callbacks cb 0x7f8d080095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53878/0 (socket says 172.21.15.120:53878) 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 -- 172.21.15.120:0/546596653 learned_addr learned my addr 172.21.15.120:0/546596653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d137fe640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcfe190 0x55d4bdd03610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d127fc640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 0x55d4bdd030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 -- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcfe190 msgr2=0x55d4bdd03610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcfe190 0x55d4bdd03610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 -- 172.21.15.120:0/546596653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 msgr2=0x55d4bdd030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 0x55d4bdd030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d12ffd640 1 -- 172.21.15.120:0/546596653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bdd03ba0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.932+0000 7f8d127fc640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 0x55d4bdd030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d12ffd640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006010 tx=0x7f8d0400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f310cf65ddc311 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400b710 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8d040035f0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d080043c0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d040037f0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 10 client.18699 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4bdd7b6c0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d080043c0 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8d04038070 con 0x55d4bdceddf0 2024-09-10T23:09:06.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 10 client.18699 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 10 client.18699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 20 client.18699 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd7be10 0x55d4bdd7e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 4 client.18699 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.933+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4bdc618a0 con 0x55d4bdd7be10 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.934+0000 7f8d127fc640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd7be10 0x55d4bdd7e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.934+0000 7f8d127fc640 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd7be10 0x55d4bdd7e230 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc6c82a47e5facf server_cookie=f8f5c14038f1afa7 in_seq=0 out_seq=0 2024-09-10T23:09:06.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.934+0000 7f8cf6ff5640 10 client.18699 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.935+0000 7f8cf6ff5640 1 -- 172.21.15.120:0/546596653 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4bdc618a0 con 0x55d4bdd7be10 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.935+0000 7f8cf6ff5640 10 client.18699 handle_command_reply: tid=0 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.935+0000 7f8d1b769740 1 client.18699 shutdown 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 msgr2=0x55d4bdd06610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006010 tx=0x7f8d0400b490 comp rx=0 tx=0).stop 2024-09-10T23:09:06.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 shutdown_connections 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d4bdd7be10 0x55d4bdd7e230 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4bdcfe190 0x55d4bdd03610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d4bdcf8c20 0x55d4bdd030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 --2- 172.21.15.120:0/546596653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d4bdceddf0 0x55d4bdd06610 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8cf6ff5640 0 client.18699 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 >> 172.21.15.120:0/546596653 conn(0x55d4bdceb300 msgr2=0x55d4bdcf9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 shutdown_connections 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 1 -- 172.21.15.120:0/546596653 wait complete. 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d1b769740 20 client.18699 trim_cache size 0 max 16384 2024-09-10T23:09:06.941 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 msgr2=0x7f8cfc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 0x7f8cfc0b77a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d08031440 tx=0x7f8d08005780 comp rx=0 tx=0).stop 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 msgr2=0x7f8d1419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00eaa0 tx=0x7f8d0c00ef60 comp rx=0 tx=0).stop 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 shutdown_connections 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cfc0b52e0 0x7f8cfc0b77a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d14108020 0x7f8d1419eba0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d141076a0 0x7f8d14199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 --2- 172.21.15.120:0/2336374955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d14101c90 0x7f8d141992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.936+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 >> 172.21.15.120:0/2336374955 conn(0x7f8d140fd700 msgr2=0x7f8d141047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.937+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 shutdown_connections 2024-09-10T23:09:06.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:06.937+0000 7f8d19394640 1 -- 172.21.15.120:0/2336374955 wait complete. 2024-09-10T23:09:07.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 101039 22.09 MB/sec execute 122 sec latency 56.991 ms 2024-09-10T23:09:08.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:07 smithi120 ceph-mon[31247]: pgmap v847: 97 pgs: 97 active+clean; 233 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-10T23:09:08.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:07 smithi153 ceph-mon[34951]: pgmap v847: 97 pgs: 97 active+clean; 233 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-10T23:09:08.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:07 smithi176 ceph-mon[34982]: pgmap v847: 97 pgs: 97 active+clean; 233 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 586 op/s 2024-09-10T23:09:08.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:09:08.690 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:09:08.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 101358 22.04 MB/sec execute 123 sec latency 295.700 ms 2024-09-10T23:09:08.965 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 -- 172.21.15.120:0/1405596052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 msgr2=0x7fde9c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 --2- 172.21.15.120:0/1405596052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c1040a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fde90009900 tx=0x7fde9002a410 comp rx=0 tx=0).stop 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 -- 172.21.15.120:0/1405596052 shutdown_connections 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 --2- 172.21.15.120:0/1405596052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 0x7fde9c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 --2- 172.21.15.120:0/1405596052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 0x7fde9c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 --2- 172.21.15.120:0/1405596052 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c1040a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.498+0000 7fdea313e640 1 -- 172.21.15.120:0/1405596052 >> 172.21.15.120:0/1405596052 conn(0x7fde9c0fd700 msgr2=0x7fde9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 -- 172.21.15.120:0/1405596052 shutdown_connections 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 -- 172.21.15.120:0/1405596052 wait complete. 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 Processor -- start 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 -- start start 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.499+0000 7fdea313e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 0x7fde9c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea313e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 0x7fde9c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea313e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112d90 con 0x7fde9c103cc0 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea313e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112c10 con 0x7fde9c10d120 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea313e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112f10 con 0x7fde9c1045e0 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53894/0 (socket says 172.21.15.120:53894) 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 -- 172.21.15.120:0/2558669778 learned_addr learned my addr 172.21.15.120:0/2558669778 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:09.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fde9bfff640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 0x7fde9c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea16db640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 0x7fde9c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 -- 172.21.15.120:0/2558669778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 msgr2=0x7fde9c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 0x7fde9c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 -- 172.21.15.120:0/2558669778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 msgr2=0x7fde9c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 0x7fde9c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 -- 172.21.15.120:0/2558669778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c19f3b0 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea0eda640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fde90005940 tx=0x7fde900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fde99ffb640 1 -- 172.21.15.120:0/2558669778 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9003f030 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fde99ffb640 1 -- 172.21.15.120:0/2558669778 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde900044c0 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c19f640 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.500+0000 7fde99ffb640 1 -- 172.21.15.120:0/2558669778 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90035070 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.501+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a5d60 con 0x7fde9c103cc0 2024-09-10T23:09:09.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.503+0000 7fde99ffb640 1 -- 172.21.15.120:0/2558669778 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fde9003ba20 con 0x7fde9c103cc0 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 Processor -- start 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 -- start start 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f93312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f9341660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f1f92a7e60 con 0x55f1f9330ee0 2024-09-10T23:09:09.518 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9297290 con 0x55f1f9341ba0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea5513740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f1f929b0a0 con 0x55f1f9331800 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde99ffb640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 0x7fde8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f9341660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea16db640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f93312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55674/0 (socket says 172.21.15.120:55674) 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937653633 learned_addr learned my addr 172.21.15.120:0/1937653633 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fdea16db640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 0x7fde8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde99ffb640 1 -- 172.21.15.120:0/2558669778 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde90106170 con 0x7fde9c103cc0 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937653633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 msgr2=0x55f1f9341660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f9341660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937653633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 msgr2=0x55f1f93312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f93312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.504+0000 7fde9bfff640 1 -- 172.21.15.120:0/1937653633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f8ff5930 con 0x55f1f9341ba0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.505+0000 7fdea16db640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 0x7fde8c0b77a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde94005c40 tx=0x7fde94009bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.505+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c50 tx=0x7fde88011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e62c3d692f5a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.505+0000 7fde86ffd640 1 -- 172.21.15.120:0/1937653633 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88004020 con 0x55f1f9341ba0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.505+0000 7fde86ffd640 1 -- 172.21.15.120:0/1937653633 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde880041c0 con 0x55f1f9341ba0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.505+0000 7fde86ffd640 1 -- 172.21.15.120:0/1937653633 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88019690 con 0x55f1f9341ba0 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- 172.21.15.120:0/1937653633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 msgr2=0x55f1f9343f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c50 tx=0x7fde88011810 comp rx=0 tx=0).stop 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- 172.21.15.120:0/1937653633 shutdown_connections 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9343f90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f9341660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- 172.21.15.120:0/1937653633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f93312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- 172.21.15.120:0/1937653633 >> 172.21.15.120:0/1937653633 conn(0x55f1f932ed10 msgr2=0x55f1f932feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- 172.21.15.120:0/1937653633 shutdown_connections 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- 172.21.15.120:0/1937653633 wait complete. 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 -- start start 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 Processor -- start 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9330ee0 0x55f1f933eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.506+0000 7fdea5513740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea5513740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f933f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea5513740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f1f90f8890 con 0x55f1f9331800 2024-09-10T23:09:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea5513740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9231d40 con 0x55f1f9341ba0 2024-09-10T23:09:09.522 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea5513740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9344850 con 0x55f1f9330ee0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea5513740 10 client.0 _ll_register_callbacks cb 0x55f1f9295c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9330ee0 0x55f1f933eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53910/0 (socket says 172.21.15.120:53910) 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 -- 172.21.15.120:0/186545804 learned_addr learned my addr 172.21.15.120:0/186545804 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fdea16db640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f933f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 -- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9330ee0 msgr2=0x55f1f933eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9330ee0 0x55f1f933eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 -- 172.21.15.120:0/186545804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 msgr2=0x55f1f933f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f933f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 -- 172.21.15.120:0/186545804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f934dc30 con 0x55f1f9331800 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde9bfff640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c20 tx=0x7fde88003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f39ed76ab80351 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.523 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.507+0000 7fde7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88028070 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde88003800 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde880041c0 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde88034050 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fdea5513740 10 client.18711 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1f93505e0 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.508+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde880041c0 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fde88005040 con 0x55f1f9331800 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 10 client.18711 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 10 client.18711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 20 client.18711 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f9351930 0x55f1f9353d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.524 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 4 client.18711 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f92a9db0 con 0x55f1f9351930 2024-09-10T23:09:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.510+0000 7fdea0eda640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f9351930 0x55f1f9353d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.511+0000 7fdea0eda640 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f9351930 0x55f1f9353d20 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6283317c1c406f23 server_cookie=8b536cd1f6cf4529 in_seq=0 out_seq=0 2024-09-10T23:09:09.525 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.511+0000 7fde7bfff640 10 client.18711 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:09.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fde7bfff640 1 -- 172.21.15.120:0/186545804 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1f92a9db0 con 0x55f1f9351930 2024-09-10T23:09:09.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fde7bfff640 10 client.18711 handle_command_reply: tid=0 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 client.18711 shutdown 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 msgr2=0x55f1f933f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c20 tx=0x7fde88003fe0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 shutdown_connections 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f9351930 0x55f1f9353d20 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f933f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f933f0a0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 --2- 172.21.15.120:0/186545804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9330ee0 0x55f1f933eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fde7bfff640 0 client.18711 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.513+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 >> 172.21.15.120:0/186545804 conn(0x55f1f932ed10 msgr2=0x55f1f932df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.527 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.514+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 shutdown_connections 2024-09-10T23:09:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.514+0000 7fdea5513740 1 -- 172.21.15.120:0/186545804 wait complete. 2024-09-10T23:09:09.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.514+0000 7fdea5513740 20 client.18711 trim_cache size 0 max 16384 2024-09-10T23:09:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 Processor -- start 2024-09-10T23:09:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 -- start start 2024-09-10T23:09:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 0x55f1f9330100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9341ba0 0x55f1f9359a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1aa060 con 0x55f1f9331800 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1ac8e0 con 0x55f1f9330ee0 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.535+0000 7fdea5513740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1aa6d0 con 0x55f1f9341ba0 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53916/0 (socket says 172.21.15.120:53916) 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 -- 172.21.15.120:0/1722109201 learned_addr learned my addr 172.21.15.120:0/1722109201 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:09.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fdea0eda640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 0x55f1f9330100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fdea16db640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9341ba0 0x55f1f9359a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 -- 172.21.15.120:0/1722109201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9341ba0 msgr2=0x55f1f9359a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9341ba0 0x55f1f9359a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 -- 172.21.15.120:0/1722109201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 msgr2=0x55f1f9330100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 0x55f1f9330100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 -- 172.21.15.120:0/1722109201 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f9342710 con 0x55f1f9331800 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fdea0eda640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 0x55f1f9330100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:09.541 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde9bfff640 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fde88012fd0 tx=0x7fde88005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ad545c62481b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.542 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde867fc640 1 -- 172.21.15.120:0/1722109201 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88028070 con 0x55f1f9331800 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde867fc640 1 -- 172.21.15.120:0/1722109201 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde88004680 con 0x55f1f9331800 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.536+0000 7fde867fc640 1 -- 172.21.15.120:0/1722109201 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8802d680 con 0x55f1f9331800 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 -- 172.21.15.120:0/1722109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 msgr2=0x55f1f9359510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fde88012fd0 tx=0x7fde88005dc0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 -- 172.21.15.120:0/1722109201 shutdown_connections 2024-09-10T23:09:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9341ba0 0x55f1f9359a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9331800 0x55f1f9359510 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 --2- 172.21.15.120:0/1722109201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9330ee0 0x55f1f9330100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 -- 172.21.15.120:0/1722109201 >> 172.21.15.120:0/1722109201 conn(0x55f1f932ed10 msgr2=0x55f1f932f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 -- 172.21.15.120:0/1722109201 shutdown_connections 2024-09-10T23:09:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.537+0000 7fdea5513740 1 -- 172.21.15.120:0/1722109201 wait complete. 2024-09-10T23:09:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 -- start start 2024-09-10T23:09:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 Processor -- start 2024-09-10T23:09:09.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f934a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9345d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1aada0 con 0x55f1f9330ee0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1ac210 con 0x55f1f9341ba0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1ab470 con 0x55f1f9331800 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea5513740 10 client.0 _ll_register_callbacks cb 0x7fde90039f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f934a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41804/0 (socket says 172.21.15.120:41804) 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fdea0eda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f934a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53920/0 (socket says 172.21.15.120:53920) 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.538+0000 7fde9bfff640 1 -- 172.21.15.120:0/2454583054 learned_addr learned my addr 172.21.15.120:0/2454583054 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fdea16db640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9345d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 -- 172.21.15.120:0/2454583054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 msgr2=0x55f1f9345d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9345d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 -- 172.21.15.120:0/2454583054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 msgr2=0x55f1f934a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f934a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 -- 172.21.15.120:0/2454583054 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f9346550 con 0x55f1f9331800 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde9bfff640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde88026040 tx=0x7fde88005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d97dfa8857fd1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88045060 con 0x55f1f9331800 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fde88003a00 con 0x55f1f9331800 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde88004680 con 0x55f1f9331800 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde88028040 con 0x55f1f9331800 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fdea5513740 10 client.28186 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.539+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1f93be7f0 con 0x55f1f9331800 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde88004680 con 0x55f1f9331800 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fde8802d780 con 0x55f1f9331800 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 10 client.28186 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 10 client.28186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 20 client.28186 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f93bedf0 0x55f1f93c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 4 client.28186 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:09:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f1f92a5860 con 0x55f1f93bedf0 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea0eda640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f93bedf0 0x55f1f93c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fdea0eda640 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f93bedf0 0x55f1f93c1210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432d5d2d21cc4183 server_cookie=7841d267f1b3cda9 in_seq=0 out_seq=0 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.540+0000 7fde7bfff640 10 client.28186 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fde7bfff640 1 -- 172.21.15.120:0/2454583054 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f1f92a5860 con 0x55f1f93bedf0 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fde7bfff640 10 client.28186 handle_command_reply: tid=0 2024-09-10T23:09:09.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 client.28186 shutdown 2024-09-10T23:09:09.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 msgr2=0x55f1f93457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde88026040 tx=0x7fde88005d50 comp rx=0 tx=0).stop 2024-09-10T23:09:09.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 shutdown_connections 2024-09-10T23:09:09.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f1f93bedf0 0x55f1f93c1210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fde7bfff640 0 client.28186 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f1f9341ba0 0x55f1f9345d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f1f9331800 0x55f1f93457a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 --2- 172.21.15.120:0/2454583054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f1f9330ee0 0x55f1f934a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 >> 172.21.15.120:0/2454583054 conn(0x55f1f932ed10 msgr2=0x55f1f93409b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 shutdown_connections 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 1 -- 172.21.15.120:0/2454583054 wait complete. 2024-09-10T23:09:09.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.543+0000 7fdea5513740 20 client.28186 trim_cache size 0 max 16384 2024-09-10T23:09:09.557 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:09:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "0b9d307e-5656-41f6-b079-f9e7a8dd8606", 2024-09-10T23:09:09.557 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:09:09.557 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:09.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 msgr2=0x7fde8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 0x7fde8c0b77a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde94005c40 tx=0x7fde94009bb0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 msgr2=0x7fde9c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fde90005940 tx=0x7fde900040c0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 shutdown_connections 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fde8c0b52e0 0x7fde8c0b77a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde9c10d120 0x7fde9c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde9c1045e0 0x7fde9c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 --2- 172.21.15.120:0/2558669778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde9c103cc0 0x7fde9c199300 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:09.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.544+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 >> 172.21.15.120:0/2558669778 conn(0x7fde9c0fd700 msgr2=0x7fde9c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:09.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.545+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 shutdown_connections 2024-09-10T23:09:09.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:09.545+0000 7fdea313e640 1 -- 172.21.15.120:0/2558669778 wait complete. 2024-09-10T23:09:09.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 101986 22.17 MB/sec execute 124 sec latency 63.133 ms 2024-09-10T23:09:10.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:09 smithi153 ceph-mon[34951]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-10T23:09:10.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:09 smithi176 ceph-mon[34982]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-10T23:09:10.386 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:09 smithi120 ceph-mon[31247]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-10T23:09:10.415 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:09:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:10.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 102129 22.06 MB/sec execute 125 sec latency 329.826 ms 2024-09-10T23:09:11.140 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:10 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:09:11.140 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:10 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:11.140 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:10 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:09:11.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 -- 172.21.15.120:0/2975206343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce301045e0 msgr2=0x7fce3010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 --2- 172.21.15.120:0/2975206343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce301045e0 0x7fce3010cad0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fce24009900 tx=0x7fce2402a410 comp rx=0 tx=0).stop 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 -- 172.21.15.120:0/2975206343 shutdown_connections 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 --2- 172.21.15.120:0/2975206343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce3010d120 0x7fce3010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 --2- 172.21.15.120:0/2975206343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce301045e0 0x7fce3010cad0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 --2- 172.21.15.120:0/2975206343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce301040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.235+0000 7fce34b67640 1 -- 172.21.15.120:0/2975206343 >> 172.21.15.120:0/2975206343 conn(0x7fce300fd700 msgr2=0x7fce300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.236+0000 7fce34b67640 1 -- 172.21.15.120:0/2975206343 shutdown_connections 2024-09-10T23:09:11.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.236+0000 7fce34b67640 1 -- 172.21.15.120:0/2975206343 wait complete. 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.236+0000 7fce34b67640 1 Processor -- start 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.236+0000 7fce34b67640 1 -- start start 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce30199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce301045e0 0x7fce30199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce30112d90 con 0x7fce3010d120 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce30112c10 con 0x7fce30103cc0 2024-09-10T23:09:11.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce30112f10 con 0x7fce301045e0 2024-09-10T23:09:11.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce301045e0 0x7fce30199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53936/0 (socket says 172.21.15.120:53936) 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 -- 172.21.15.120:0/3206130941 learned_addr learned my addr 172.21.15.120:0/3206130941 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2e59c640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce30199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 -- 172.21.15.120:0/3206130941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce301045e0 msgr2=0x7fce30199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce301045e0 0x7fce30199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 -- 172.21.15.120:0/3206130941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 msgr2=0x7fce30199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce30199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 -- 172.21.15.120:0/3206130941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3019f2d0 con 0x7fce3010d120 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2e59c640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce30199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fce2800eb10 tx=0x7fce2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce1f7fe640 1 -- 172.21.15.120:0/3206130941 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28004b20 con 0x7fce3010d120 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.237+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3019f5c0 con 0x7fce3010d120 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.238+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301a5e50 con 0x7fce3010d120 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.238+0000 7fce1f7fe640 1 -- 172.21.15.120:0/3206130941 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce28004cc0 con 0x7fce3010d120 2024-09-10T23:09:11.246 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.238+0000 7fce1f7fe640 1 -- 172.21.15.120:0/3206130941 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28010690 con 0x7fce3010d120 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 Processor -- start 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce1f7fe640 1 -- 172.21.15.120:0/3206130941 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fce28005020 con 0x7fce3010d120 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 -- start start 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f43d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.240+0000 7fce36f3c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56533f3a4e50 con 0x56533f42eda0 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce36f3c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56533f394280 con 0x56533f42e480 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce36f3c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56533f398090 con 0x56533f43ad30 2024-09-10T23:09:11.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53946/0 (socket says 172.21.15.120:53946) 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 -- 172.21.15.120:0/874836743 learned_addr learned my addr 172.21.15.120:0/874836743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f43d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 -- 172.21.15.120:0/874836743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 msgr2=0x56533f43d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f43d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 -- 172.21.15.120:0/874836743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 msgr2=0x56533f42e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 -- 172.21.15.120:0/874836743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533f0f2650 con 0x56533f42eda0 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2e59c640 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fce20008c60 tx=0x7fce2000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d56e9171467de0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce1f7fe640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 0x7fce180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce1cff9640 1 -- 172.21.15.120:0/874836743 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20014070 con 0x56533f42eda0 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 0x7fce180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.241+0000 7fce1f7fe640 1 -- 172.21.15.120:0/3206130941 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fce280e61d0 con 0x7fce3010d120 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce1cff9640 1 -- 172.21.15.120:0/874836743 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce200047a0 con 0x56533f42eda0 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce1cff9640 1 -- 172.21.15.120:0/874836743 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20010690 con 0x56533f42eda0 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 0x7fce180b77a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 -- 172.21.15.120:0/874836743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 msgr2=0x56533f43a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fce20008c60 tx=0x7fce2000e720 comp rx=0 tx=0).stop 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 -- 172.21.15.120:0/874836743 shutdown_connections 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f43d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f43a7f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 --2- 172.21.15.120:0/874836743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 -- 172.21.15.120:0/874836743 >> 172.21.15.120:0/874836743 conn(0x56533f42c2b0 msgr2=0x56533f42d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 -- 172.21.15.120:0/874836743 shutdown_connections 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.242+0000 7fce36f3c740 1 -- 172.21.15.120:0/874836743 wait complete. 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 -- start start 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 Processor -- start 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f43a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f437a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56533f1f5700 con 0x56533f42eda0 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56533f32f110 con 0x56533f42e480 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56533f43d9b0 con 0x56533f43ad30 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce36f3c740 10 client.0 _ll_register_callbacks cb 0x56533f348990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f437a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41858/0 (socket says 172.21.15.120:41858) 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce2ed9d640 1 -- 172.21.15.120:0/2806682009 learned_addr learned my addr 172.21.15.120:0/2806682009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.243+0000 7fce2e59c640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f43a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 -- 172.21.15.120:0/2806682009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 msgr2=0x56533f43a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f43a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 -- 172.21.15.120:0/2806682009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 msgr2=0x56533f437a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f437a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 -- 172.21.15.120:0/2806682009 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533f35cc40 con 0x56533f43ad30 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce280a8f90 tx=0x7fce280ac850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89840ce6d2c806ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280a8130 con 0x56533f43ad30 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fce280acc20 con 0x56533f43ad30 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce200047a0 con 0x56533f43ad30 2024-09-10T23:09:11.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce28005540 con 0x56533f43ad30 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce36f3c740 10 client.28198 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56533f449310 con 0x56533f43ad30 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.244+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce200047a0 con 0x56533f43ad30 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fce280b5ac0 con 0x56533f43ad30 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 10 client.28198 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 10 client.28198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 20 client.28198 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f44a820 0x56533f44cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 4 client.28198 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56533f3a6da0 con 0x56533f44a820 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce2e59c640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f44a820 0x56533f44cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce15ffb640 10 client.28198 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.245+0000 7fce2e59c640 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f44a820 0x56533f44cc80 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97214d53ba34bcd9 server_cookie=ef4b28f1a1b131e6 in_seq=0 out_seq=0 2024-09-10T23:09:11.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce15ffb640 1 -- 172.21.15.120:0/2806682009 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56533f3a6da0 con 0x56533f44a820 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce15ffb640 10 client.28198 handle_command_reply: tid=0 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 client.28198 shutdown 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 msgr2=0x56533f437fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce280a8f90 tx=0x7fce280ac850 comp rx=0 tx=0).stop 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 shutdown_connections 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f44a820 0x56533f44cc80 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f43ad30 0x56533f437fb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f42eda0 0x56533f437a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce15ffb640 0 client.28198 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 --2- 172.21.15.120:0/2806682009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f43a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 >> 172.21.15.120:0/2806682009 conn(0x56533f42c2b0 msgr2=0x56533f431e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.247+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 shutdown_connections 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.248+0000 7fce36f3c740 1 -- 172.21.15.120:0/2806682009 wait complete. 2024-09-10T23:09:11.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.248+0000 7fce36f3c740 20 client.28198 trim_cache size 0 max 16384 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 Processor -- start 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 -- start start 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42eda0 0x56533f434c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aa020 con 0x56533f43ad30 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ac8a0 con 0x56533f42e480 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce36f3c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aa690 con 0x56533f42eda0 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42eda0 0x56533f434c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:53964/0 (socket says 172.21.15.120:53964) 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.268+0000 7fce2ed9d640 1 -- 172.21.15.120:0/351827818 learned_addr learned my addr 172.21.15.120:0/351827818 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2e59c640 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 -- 172.21.15.120:0/351827818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42eda0 msgr2=0x56533f434c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42eda0 0x56533f434c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 -- 172.21.15.120:0/351827818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 msgr2=0x56533f42bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 -- 172.21.15.120:0/351827818 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533f432730 con 0x56533f43ad30 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fce280049d0 tx=0x7fce2800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c19ada7c3749e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce17fff640 1 -- 172.21.15.120:0/351827818 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280a8130 con 0x56533f43ad30 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce17fff640 1 -- 172.21.15.120:0/351827818 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280ac4d0 con 0x56533f43ad30 2024-09-10T23:09:11.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.269+0000 7fce17fff640 1 -- 172.21.15.120:0/351827818 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280a8310 con 0x56533f43ad30 2024-09-10T23:09:11.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- 172.21.15.120:0/351827818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 msgr2=0x56533f435160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fce280049d0 tx=0x7fce2800efd0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- 172.21.15.120:0/351827818 shutdown_connections 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f435160 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42eda0 0x56533f434c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 --2- 172.21.15.120:0/351827818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42e480 0x56533f42bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- 172.21.15.120:0/351827818 >> 172.21.15.120:0/351827818 conn(0x56533f42c2b0 msgr2=0x56533f439860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- 172.21.15.120:0/351827818 shutdown_connections 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- 172.21.15.120:0/351827818 wait complete. 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.270+0000 7fce36f3c740 1 -- start start 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 Processor -- start 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f43f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aad60 con 0x56533f43ad30 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ac1d0 con 0x56533f42eda0 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce301ab430 con 0x56533f42e480 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce36f3c740 10 client.0 _ll_register_callbacks cb 0x7fce280a9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f43f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:55750/0 (socket says 172.21.15.120:55750) 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2dd9b640 1 -- 172.21.15.120:0/46085410 learned_addr learned my addr 172.21.15.120:0/46085410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41878/0 (socket says 172.21.15.120:41878) 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 -- 172.21.15.120:0/46085410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 msgr2=0x56533f43f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 -- 172.21.15.120:0/46085410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 msgr2=0x56533f43f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:11.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f43f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2e59c640 1 -- 172.21.15.120:0/46085410 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533f4b6a20 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.271+0000 7fce2ed9d640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f43f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce2e59c640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce20008fa0 tx=0x7fce20008230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a24f5efd8b406c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20030060 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fce20005450 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce280ac4d0 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce20014040 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce36f3c740 10 client.28204 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56533f4b7d30 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.272+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280ac4d0 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fce2002a030 con 0x56533f42e480 2024-09-10T23:09:11.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 10 client.28204 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 10 client.28204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 20 client.28204 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f4b8180 0x56533f4ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 4 client.28204 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56533f3a2850 con 0x56533f4b8180 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f4b8180 0x56533f4ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce2dd9b640 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f4b8180 0x56533f4ba5a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab07b8e4e3db5dd2 server_cookie=b39c6cab76d5f180 in_seq=0 out_seq=0 2024-09-10T23:09:11.279 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.273+0000 7fce15ffb640 10 client.28204 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:11.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.274+0000 7fce15ffb640 1 -- 172.21.15.120:0/46085410 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8522 (crc 0 0 0) 0x56533f3a2850 con 0x56533f4b8180 2024-09-10T23:09:11.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.274+0000 7fce15ffb640 10 client.28204 handle_command_reply: tid=0 2024-09-10T23:09:11.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.274+0000 7fce36f3c740 1 client.28204 shutdown 2024-09-10T23:09:11.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 msgr2=0x56533f43eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fce20008fa0 tx=0x7fce20008230 comp rx=0 tx=0).stop 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 shutdown_connections 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56533f4b8180 0x56533f4ba5a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56533f43ad30 0x56533f43f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce15ffb640 0 client.28204 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56533f42eda0 0x56533f43f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 --2- 172.21.15.120:0/46085410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56533f42e480 0x56533f43eef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 >> 172.21.15.120:0/46085410 conn(0x56533f42c2b0 msgr2=0x56533f4314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 shutdown_connections 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 1 -- 172.21.15.120:0/46085410 wait complete. 2024-09-10T23:09:11.281 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.275+0000 7fce36f3c740 20 client.28204 trim_cache size 0 max 16384 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "0b9d307e-5656-41f6-b079-f9e7a8dd8606", 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:09:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:09:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:09:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:09:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:09:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 msgr2=0x7fce180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 0x7fce180b77a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24031400 comp rx=0 tx=0).stop 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 msgr2=0x7fce3019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fce2800eb10 tx=0x7fce2800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 shutdown_connections 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fce180b52e0 0x7fce180b77a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce3010d120 0x7fce3019eb00 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.300 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce301045e0 0x7fce30199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 --2- 172.21.15.120:0/3206130941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce30103cc0 0x7fce30199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 >> 172.21.15.120:0/3206130941 conn(0x7fce300fd700 msgr2=0x7fce300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 shutdown_connections 2024-09-10T23:09:11.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:11.276+0000 7fce34b67640 1 -- 172.21.15.120:0/3206130941 wait complete. 2024-09-10T23:09:11.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:10 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:09:11.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:10 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:11.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:10 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:09:11.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:10 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:09:11.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:10 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:11.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:10 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:09:11.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 102366 22.03 MB/sec execute 126 sec latency 357.305 ms 2024-09-10T23:09:11.942 INFO:tasks.cephfs.filesystem:scrub status for tag:0b9d307e-5656-41f6-b079-f9e7a8dd8606 - {'path': '/', 'tag': '0b9d307e-5656-41f6-b079-f9e7a8dd8606', 'options': 'recursive,force'} 2024-09-10T23:09:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:11 smithi120 ceph-mon[31247]: pgmap v849: 97 pgs: 97 active+clean; 242 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-09-10T23:09:12.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:11 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:12.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:11 smithi176 ceph-mon[34982]: pgmap v849: 97 pgs: 97 active+clean; 242 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-09-10T23:09:12.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:11 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:11 smithi153 ceph-mon[34951]: pgmap v849: 97 pgs: 97 active+clean; 242 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 598 op/s 2024-09-10T23:09:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:11 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:12.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 102832 22.12 MB/sec execute 127 sec latency 47.067 ms 2024-09-10T23:09:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 -- 172.21.15.120:0/2384770789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9564103370 msgr2=0x7f9564109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 --2- 172.21.15.120:0/2384770789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9564103370 0x7f9564109cf0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f955c009cd0 tx=0x7f955c02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 -- 172.21.15.120:0/2384770789 shutdown_connections 2024-09-10T23:09:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 --2- 172.21.15.120:0/2384770789 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9564103370 0x7f9564109cf0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 --2- 172.21.15.120:0/2384770789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95641029b0 0x7f9564102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 --2- 172.21.15.120:0/2384770789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9564101860 0x7f9564101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 -- 172.21.15.120:0/2384770789 >> 172.21.15.120:0/2384770789 conn(0x7f95640fd140 msgr2=0x7f95640ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.408+0000 7f956b9da640 1 -- 172.21.15.120:0/2384770789 shutdown_connections 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 -- 172.21.15.120:0/2384770789 wait complete. 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 Processor -- start 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 -- start start 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9564101860 0x7f9564198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9564103370 0x7f9564199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f956410e600 con 0x7f95641029b0 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f956410e480 con 0x7f9564101860 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f956b9da640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f956410e780 con 0x7f9564103370 2024-09-10T23:09:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f9568f75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f9568f75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38068/0 (socket says 172.21.15.120:38068) 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.409+0000 7f9568f75640 1 -- 172.21.15.120:0/3924731194 learned_addr learned my addr 172.21.15.120:0/3924731194 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 -- 172.21.15.120:0/3924731194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9564103370 msgr2=0x7f9564199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9564103370 0x7f9564199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 -- 172.21.15.120:0/3924731194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9564101860 msgr2=0x7f9564198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9564101860 0x7f9564198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 -- 172.21.15.120:0/3924731194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419a880 con 0x7f95641029b0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f9568f75640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f955400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f955a7fc640 1 -- 172.21.15.120:0/3924731194 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554004220 con 0x7f95641029b0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f955a7fc640 1 -- 172.21.15.120:0/3924731194 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95540043c0 con 0x7f95641029b0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419ab70 con 0x7f95641029b0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f955a7fc640 1 -- 172.21.15.120:0/3924731194 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554004a00 con 0x7f95641029b0 2024-09-10T23:09:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.410+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a1240 con 0x7f95641029b0 2024-09-10T23:09:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f955a7fc640 1 -- 172.21.15.120:0/3924731194 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f95540079b0 con 0x7f95641029b0 2024-09-10T23:09:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 Processor -- start 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 -- start start 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da18060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da1afe0 0x56116da1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56116d9836f0 con 0x56116da185a0 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56116d972b20 con 0x56116da15c70 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f956e884740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56116d976930 con 0x56116da1afe0 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38070/0 (socket says 172.21.15.120:38070) 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9568f75640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da1afe0 0x56116da1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 -- 172.21.15.120:0/2186963813 learned_addr learned my addr 172.21.15.120:0/2186963813 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 -- 172.21.15.120:0/2186963813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da1afe0 msgr2=0x56116da1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da1afe0 0x56116da1d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 -- 172.21.15.120:0/2186963813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 msgr2=0x56116da18060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da18060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.413+0000 7f9569776640 1 -- 172.21.15.120:0/2186963813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116d938530 con 0x56116da185a0 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f9569776640 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9560009c40 tx=0x7f956000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74da21b4664edbc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f955a7fc640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 0x7f95500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f954f7fe640 1 -- 172.21.15.120:0/2186963813 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560005020 con 0x56116da185a0 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f954f7fe640 1 -- 172.21.15.120:0/2186963813 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600047a0 con 0x56116da185a0 2024-09-10T23:09:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f954f7fe640 1 -- 172.21.15.120:0/2186963813 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560010690 con 0x56116da185a0 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f9569f77640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 0x7f95500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f955a7fc640 1 -- 172.21.15.120:0/3924731194 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f95540e59f0 con 0x7f95641029b0 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f956e884740 1 -- 172.21.15.120:0/2186963813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 msgr2=0x56116da1a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f9569f77640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 0x7f95500b77a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c031850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.414+0000 7f956e884740 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9560009c40 tx=0x7f956000e720 comp rx=0 tx=0).stop 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- 172.21.15.120:0/2186963813 shutdown_connections 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da1afe0 0x56116da1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da1a990 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- 172.21.15.120:0/2186963813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da18060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- 172.21.15.120:0/2186963813 >> 172.21.15.120:0/2186963813 conn(0x56116da0c9b0 msgr2=0x56116da0d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- 172.21.15.120:0/2186963813 shutdown_connections 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- 172.21.15.120:0/2186963813 wait complete. 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- start start 2024-09-10T23:09:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 Processor -- start 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da1f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56116d8aaab0 con 0x56116da1afe0 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56116d918730 con 0x56116da15c70 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56116da1dfc0 con 0x56116da185a0 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f956e884740 10 client.0 _ll_register_callbacks cb 0x56116d96c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f9569f77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f9569776640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da1f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f9569f77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38084/0 (socket says 172.21.15.120:38084) 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.415+0000 7f9569f77640 1 -- 172.21.15.120:0/2184230248 learned_addr learned my addr 172.21.15.120:0/2184230248 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9568f75640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 -- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 msgr2=0x56116da1f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da1f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 -- 172.21.15.120:0/2184230248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 msgr2=0x56116da1f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da1f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 -- 172.21.15.120:0/2184230248 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116d6d4790 con 0x56116da1afe0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f9569f77640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f955c035000 tx=0x7f955c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbfb1231d0ebce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c04b030 con 0x56116da1afe0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f955c0047d0 con 0x56116da1afe0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95600047a0 con 0x56116da1afe0 2024-09-10T23:09:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f955c0305a0 con 0x56116da1afe0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f956e884740 10 client.18747 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56116d74d620 con 0x56116da1afe0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.416+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600047a0 con 0x56116da1afe0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f955c030070 con 0x56116da1afe0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 10 client.18747 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 10 client.18747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 20 client.18747 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da2ac10 0x56116da2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 4 client.18747 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56116d985640 con 0x56116da2ac10 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f9569776640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da2ac10 0x56116da2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f954cff9640 10 client.18747 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.417+0000 7f9569776640 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da2ac10 0x56116da2d0b0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7300264851a3e8aa server_cookie=e1c9986f6d4a96d2 in_seq=0 out_seq=0 2024-09-10T23:09:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f954cff9640 1 -- 172.21.15.120:0/2184230248 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56116d985640 con 0x56116da2ac10 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f954cff9640 10 client.18747 handle_command_reply: tid=0 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 client.18747 shutdown 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 msgr2=0x56116da20570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f955c035000 tx=0x7f955c003d40 comp rx=0 tx=0).stop 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 shutdown_connections 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da2ac10 0x56116da2d0b0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da20570 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da1f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f954cff9640 0 client.18747 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 --2- 172.21.15.120:0/2184230248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da1f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.419+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 >> 172.21.15.120:0/2184230248 conn(0x56116da0c9b0 msgr2=0x56116da0bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.420+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 shutdown_connections 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.420+0000 7f956e884740 1 -- 172.21.15.120:0/2184230248 wait complete. 2024-09-10T23:09:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.420+0000 7f956e884740 20 client.18747 trim_cache size 0 max 16384 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 Processor -- start 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 -- start start 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da15c70 0x56116da0fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da1afe0 0x56116da10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a5c30 con 0x56116da185a0 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a84b0 con 0x56116da1afe0 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f956e884740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a62a0 con 0x56116da15c70 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38096/0 (socket says 172.21.15.120:38096) 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 -- 172.21.15.120:0/3671558848 learned_addr learned my addr 172.21.15.120:0/3671558848 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9569776640 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da15c70 0x56116da0fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9569f77640 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da1afe0 0x56116da10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 -- 172.21.15.120:0/3671558848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da15c70 msgr2=0x56116da0fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da15c70 0x56116da0fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 -- 172.21.15.120:0/3671558848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da1afe0 msgr2=0x56116da10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da1afe0 0x56116da10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.435+0000 7f9568f75640 1 -- 172.21.15.120:0/3671558848 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116da1fd10 con 0x56116da185a0 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.436+0000 7f9568f75640 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95540e3520 tx=0x7f95540eb610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b031abac5968a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.436+0000 7f954effd640 1 -- 172.21.15.120:0/3671558848 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955401d040 con 0x56116da185a0 2024-09-10T23:09:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.436+0000 7f954effd640 1 -- 172.21.15.120:0/3671558848 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9554005290 con 0x56116da185a0 2024-09-10T23:09:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.436+0000 7f954effd640 1 -- 172.21.15.120:0/3671558848 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540ac1e0 con 0x56116da185a0 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- 172.21.15.120:0/3671558848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 msgr2=0x56116da0ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95540e3520 tx=0x7f95540eb610 comp rx=0 tx=0).stop 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- 172.21.15.120:0/3671558848 shutdown_connections 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da1afe0 0x56116da10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da185a0 0x56116da0ff40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- 172.21.15.120:0/3671558848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da15c70 0x56116da0fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- 172.21.15.120:0/3671558848 >> 172.21.15.120:0/3671558848 conn(0x56116da0c9b0 msgr2=0x56116da07af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- 172.21.15.120:0/3671558848 shutdown_connections 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- 172.21.15.120:0/3671558848 wait complete. 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 -- start start 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 Processor -- start 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da26c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da25960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.437+0000 7f956e884740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f956e884740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a6970 con 0x56116da1afe0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f956e884740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a7de0 con 0x56116da15c70 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f956e884740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a7040 con 0x56116da185a0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f956e884740 10 client.0 _ll_register_callbacks cb 0x7f955c03f050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38110/0 (socket says 172.21.15.120:38110) 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 -- 172.21.15.120:0/2962737276 learned_addr learned my addr 172.21.15.120:0/2962737276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569776640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da26c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9568f75640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da25960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 -- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 msgr2=0x56116da25960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da25960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 -- 172.21.15.120:0/2962737276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 msgr2=0x56116da26c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da26c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 -- 172.21.15.120:0/2962737276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116da26540 con 0x56116da1afe0 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569776640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da26c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f9569f77640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f955c0476c0 tx=0x7f955c003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b67b953dff4bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c05f070 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f955c030940 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.438+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56116d9810f0 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f955c06b050 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 10 client.18753 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56116da97b80 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56116d9810f0 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f955c030050 con 0x56116da1afe0 2024-09-10T23:09:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 10 client.18753 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 10 client.18753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 20 client.18753 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da984a0 0x56116da9a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 4 client.18753 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56116d97e520 con 0x56116da984a0 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.439+0000 7f9569776640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da984a0 0x56116da9a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.440+0000 7f954cff9640 10 client.18753 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:13.440+0000 7f9569776640 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da984a0 0x56116da9a8c0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05fa048eaad78cd server_cookie=6c333233797e949e in_seq=0 out_seq=0 2024-09-10T23:09:13.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103056 22.03 MB/sec execute 128 sec latency 395.689 ms 2024-09-10T23:09:14.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:13 smithi120 ceph-mon[31247]: pgmap v850: 97 pgs: 97 active+clean; 245 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 547 op/s 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f954cff9640 1 -- 172.21.15.120:0/2962737276 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56116d97e520 con 0x56116da984a0 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f954cff9640 10 client.18753 handle_command_reply: tid=0 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f956e884740 1 client.18753 shutdown 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 msgr2=0x56116da25ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f955c0476c0 tx=0x7f955c003e70 comp rx=0 tx=0).stop 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.235+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 shutdown_connections 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56116da984a0 0x56116da9a8c0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56116da1afe0 0x56116da25ea0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56116da185a0 0x56116da25960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 --2- 172.21.15.120:0/2962737276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56116da15c70 0x56116da26c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 >> 172.21.15.120:0/2962737276 conn(0x56116da0c9b0 msgr2=0x56116da0fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f954cff9640 0 client.18753 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 shutdown_connections 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 1 -- 172.21.15.120:0/2962737276 wait complete. 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.236+0000 7f956e884740 20 client.18753 trim_cache size 0 max 16384 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:09:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:09:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 msgr2=0x7f95500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 0x7f95500b77a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c031850 comp rx=0 tx=0).stop 2024-09-10T23:09:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 msgr2=0x7f9564199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f955400cd50 comp rx=0 tx=0).stop 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 shutdown_connections 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f95500b52e0 0x7f95500b77a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9564103370 0x7f9564199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95641029b0 0x7f9564199390 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 --2- 172.21.15.120:0/3924731194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9564101860 0x7f9564198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 >> 172.21.15.120:0/3924731194 conn(0x7f95640fd140 msgr2=0x7f9564105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 shutdown_connections 2024-09-10T23:09:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:14.237+0000 7f956b9da640 1 -- 172.21.15.120:0/3924731194 wait complete. 2024-09-10T23:09:14.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:13 smithi176 ceph-mon[34982]: pgmap v850: 97 pgs: 97 active+clean; 245 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 547 op/s 2024-09-10T23:09:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:13 smithi153 ceph-mon[34951]: pgmap v850: 97 pgs: 97 active+clean; 245 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 547 op/s 2024-09-10T23:09:14.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103256 21.96 MB/sec execute 129 sec latency 681.288 ms 2024-09-10T23:09:15.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103528 21.87 MB/sec execute 130 sec latency 97.821 ms 2024-09-10T23:09:16.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:15 smithi176 ceph-mon[34982]: pgmap v851: 97 pgs: 97 active+clean; 240 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-09-10T23:09:16.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:09:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:15 smithi120 ceph-mon[31247]: pgmap v851: 97 pgs: 97 active+clean; 240 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-09-10T23:09:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:09:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:15 smithi153 ceph-mon[34951]: pgmap v851: 97 pgs: 97 active+clean; 240 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-09-10T23:09:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:09:16.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 103927 21.87 MB/sec execute 131 sec latency 83.635 ms 2024-09-10T23:09:17.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 104445 21.82 MB/sec execute 132 sec latency 70.903 ms 2024-09-10T23:09:18.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:17 smithi176 ceph-mon[34982]: pgmap v852: 97 pgs: 97 active+clean; 237 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-09-10T23:09:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:17 smithi120 ceph-mon[31247]: pgmap v852: 97 pgs: 97 active+clean; 237 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-09-10T23:09:18.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:17 smithi153 ceph-mon[34951]: pgmap v852: 97 pgs: 97 active+clean; 237 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-09-10T23:09:18.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 104895 21.83 MB/sec execute 133 sec latency 147.842 ms 2024-09-10T23:09:19.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 105553 21.96 MB/sec execute 134 sec latency 137.864 ms 2024-09-10T23:09:20.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:19 smithi176 ceph-mon[34982]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.1 MiB/s wr, 669 op/s 2024-09-10T23:09:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:19 smithi120 ceph-mon[31247]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.1 MiB/s wr, 669 op/s 2024-09-10T23:09:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:19 smithi153 ceph-mon[34951]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.1 MiB/s wr, 669 op/s 2024-09-10T23:09:20.874 INFO:tasks.workunit.client.0.smithi120.stdout: 10 105986 22.03 MB/sec execute 135 sec latency 84.405 ms 2024-09-10T23:09:21.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 106445 22.09 MB/sec execute 136 sec latency 75.102 ms 2024-09-10T23:09:22.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:21 smithi176 ceph-mon[34982]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-09-10T23:09:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:21 smithi120 ceph-mon[31247]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-09-10T23:09:22.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:21 smithi153 ceph-mon[34951]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-09-10T23:09:22.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 106745 22.04 MB/sec execute 137 sec latency 84.591 ms 2024-09-10T23:09:23.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 107043 21.99 MB/sec execute 138 sec latency 144.833 ms 2024-09-10T23:09:24.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:23 smithi176 ceph-mon[34982]: pgmap v855: 97 pgs: 97 active+clean; 241 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:09:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:23 smithi120 ceph-mon[31247]: pgmap v855: 97 pgs: 97 active+clean; 241 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:09:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:23 smithi153 ceph-mon[34951]: pgmap v855: 97 pgs: 97 active+clean; 241 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 677 op/s 2024-09-10T23:09:24.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 107308 21.90 MB/sec execute 139 sec latency 327.931 ms 2024-09-10T23:09:25.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 107651 21.85 MB/sec execute 140 sec latency 121.513 ms 2024-09-10T23:09:26.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:25 smithi176 ceph-mon[34982]: pgmap v856: 97 pgs: 97 active+clean; 243 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-09-10T23:09:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:25 smithi120 ceph-mon[31247]: pgmap v856: 97 pgs: 97 active+clean; 243 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-09-10T23:09:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:25 smithi153 ceph-mon[34951]: pgmap v856: 97 pgs: 97 active+clean; 243 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 593 op/s 2024-09-10T23:09:26.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 108175 21.87 MB/sec execute 141 sec latency 82.201 ms 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- 172.21.15.120:0/1117725033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 msgr2=0x7f9f6c102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- 172.21.15.120:0/1117725033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c102d80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009930 tx=0x7f9f5c02a420 comp rx=0 tx=0).stop 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- 172.21.15.120:0/1117725033 shutdown_connections 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- 172.21.15.120:0/1117725033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c104490 0x7f9f6c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- 172.21.15.120:0/1117725033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c103ad0 0x7f9f6c103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- 172.21.15.120:0/1117725033 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c102d80 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- 172.21.15.120:0/1117725033 >> 172.21.15.120:0/1117725033 conn(0x7f9f6c0fe240 msgr2=0x7f9f6c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- 172.21.15.120:0/1117725033 shutdown_connections 2024-09-10T23:09:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- 172.21.15.120:0/1117725033 wait complete. 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 Processor -- start 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 -- start start 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.327+0000 7f9f7142b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c103ad0 0x7f9f6c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c104490 0x7f9f6c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10e5f0 con 0x7f9f6c102980 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10e470 con 0x7f9f6c104490 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10e770 con 0x7f9f6c103ad0 2024-09-10T23:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48454/0 (socket says 172.21.15.120:48454) 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 -- 172.21.15.120:0/2473444219 learned_addr learned my addr 172.21.15.120:0/2473444219 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c104490 0x7f9f6c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c103ad0 0x7f9f6c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 -- 172.21.15.120:0/2473444219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c103ad0 msgr2=0x7f9f6c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c103ad0 0x7f9f6c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 -- 172.21.15.120:0/2473444219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c104490 msgr2=0x7f9f6c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c104490 0x7f9f6c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 -- 172.21.15.120:0/2473444219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c19a8f0 con 0x7f9f6c102980 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f6affd640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009a50 tx=0x7f9f5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f53fff640 1 -- 172.21.15.120:0/2473444219 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c03f030 con 0x7f9f6c102980 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f53fff640 1 -- 172.21.15.120:0/2473444219 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f5c003cb0 con 0x7f9f6c102980 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c19ab80 con 0x7f9f6c102980 2024-09-10T23:09:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f53fff640 1 -- 172.21.15.120:0/2473444219 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c035070 con 0x7f9f6c102980 2024-09-10T23:09:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.328+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1a12b0 con 0x7f9f6c102980 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 Processor -- start 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 -- start start 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017d190 0x557f0017f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f53fff640 1 -- 172.21.15.120:0/2473444219 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9f5c03bb90 con 0x7f9f6c102980 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017fbd0 0x557f00182090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f000ec6f0 con 0x557f0017a860 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557f000dbb20 con 0x557f0017fbd0 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.331+0000 7f9f741f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f000df930 con 0x557f0017d190 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48458/0 (socket says 172.21.15.120:48458) 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/560998957 learned_addr learned my addr 172.21.15.120:0/560998957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6affd640 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017d190 0x557f0017f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017fbd0 0x557f00182090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/560998957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017d190 msgr2=0x557f0017f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017d190 0x557f0017f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/560998957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017fbd0 msgr2=0x557f00182090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017fbd0 0x557f00182090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/560998957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f000a1530 con 0x557f0017a860 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009f90 tx=0x7f9f640118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ffd49f4fd6544 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f517fa640 1 -- 172.21.15.120:0/560998957 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64004d20 con 0x557f0017a860 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f517fa640 1 -- 172.21.15.120:0/560998957 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64004050 con 0x557f0017a860 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f517fa640 1 -- 172.21.15.120:0/560998957 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64005260 con 0x557f0017a860 2024-09-10T23:09:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f53fff640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 0x7f9f540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.332+0000 7f9f6affd640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 0x7f9f540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f53fff640 1 -- 172.21.15.120:0/2473444219 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9f5c106810 con 0x7f9f6c102980 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- 172.21.15.120:0/560998957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 msgr2=0x557f0017cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f6affd640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 0x7f9f540b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c004650 tx=0x7f9f5c10c550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009f90 tx=0x7f9f640118d0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- 172.21.15.120:0/560998957 shutdown_connections 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017fbd0 0x557f00182090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017d190 0x557f0017f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 --2- 172.21.15.120:0/560998957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017cc50 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- 172.21.15.120:0/560998957 >> 172.21.15.120:0/560998957 conn(0x557f001759b0 msgr2=0x557f00176740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- 172.21.15.120:0/560998957 shutdown_connections 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- 172.21.15.120:0/560998957 wait complete. 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 -- start start 2024-09-10T23:09:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.333+0000 7f9f741f4740 1 Processor -- start 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f00183e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f001850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f00013ab0 con 0x557f0017d190 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557f00081730 con 0x557f0017a860 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f00182bb0 con 0x557f0017fbd0 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 10 client.0 _ll_register_callbacks cb 0x557f000d56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48462/0 (socket says 172.21.15.120:48462) 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/441072666 learned_addr learned my addr 172.21.15.120:0/441072666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f001850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f00183e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 msgr2=0x557f001850b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f001850b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/441072666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 msgr2=0x557f00183e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f00183e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/441072666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557effe3d790 con 0x557f0017d190 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f00183e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000baa0 tx=0x7f9f6000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b0610b9ca74379 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003eb0 con 0x557f0017d190 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9f60011440 con 0x557f0017d190 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f64004050 con 0x557f0017d190 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f60011640 con 0x557f0017d190 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 10 client.18765 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.334+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557effeb6620 con 0x557f0017d190 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64004050 con 0x557f0017d190 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9f60026970 con 0x557f0017d190 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 10 client.18765 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 10 client.18765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 20 client.18765 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f0018f7f0 0x557f00191be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 4 client.18765 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f000ee640 con 0x557f0018f7f0 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f0018f7f0 0x557f00191be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f0018f7f0 0x557f00191be0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9cee1b5338bbb4 server_cookie=937d80b85b2801aa in_seq=0 out_seq=0 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.335+0000 7f9f4eff5640 10 client.18765 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f4eff5640 1 -- 172.21.15.120:0/441072666 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f000ee640 con 0x557f0018f7f0 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f4eff5640 10 client.18765 handle_command_reply: tid=0 2024-09-10T23:09:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 client.18765 shutdown 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 msgr2=0x557f001844e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000baa0 tx=0x7f9f6000bf60 comp rx=0 tx=0).stop 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 shutdown_connections 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f0018f7f0 0x557f00191be0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f001850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f001844e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f4eff5640 0 client.18765 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 --2- 172.21.15.120:0/441072666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f00183e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 >> 172.21.15.120:0/441072666 conn(0x557f001759b0 msgr2=0x557f00174bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 shutdown_connections 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 1 -- 172.21.15.120:0/441072666 wait complete. 2024-09-10T23:09:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.338+0000 7f9f741f4740 20 client.18765 trim_cache size 0 max 16384 2024-09-10T23:09:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.352+0000 7f9f741f4740 1 Processor -- start 2024-09-10T23:09:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 -- start start 2024-09-10T23:09:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017d190 0x557f001804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f00183e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1a7e20 con 0x557f0017a860 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1aa6a0 con 0x557f0017d190 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f741f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1a8490 con 0x557f0017fbd0 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48470/0 (socket says 172.21.15.120:48470) 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/653047836 learned_addr learned my addr 172.21.15.120:0/653047836 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017d190 0x557f001804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f00183e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/653047836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 msgr2=0x557f00183e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f00183e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/653047836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017d190 msgr2=0x557f001804f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017d190 0x557f001804f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 -- 172.21.15.120:0/653047836 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0017cfc0 con 0x557f0017a860 2024-09-10T23:09:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9f64004360 tx=0x7f9f64003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6577fefdbc2d1e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f50ff9640 1 -- 172.21.15.120:0/653047836 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64035070 con 0x557f0017a860 2024-09-10T23:09:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f50ff9640 1 -- 172.21.15.120:0/653047836 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64009b30 con 0x557f0017a860 2024-09-10T23:09:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.353+0000 7f9f50ff9640 1 -- 172.21.15.120:0/653047836 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64005450 con 0x557f0017a860 2024-09-10T23:09:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- 172.21.15.120:0/653047836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 msgr2=0x557f0017ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9f64004360 tx=0x7f9f64003b30 comp rx=0 tx=0).stop 2024-09-10T23:09:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- 172.21.15.120:0/653047836 shutdown_connections 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f00183e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017d190 0x557f001804f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 --2- 172.21.15.120:0/653047836 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017a860 0x557f0017ffb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- 172.21.15.120:0/653047836 >> 172.21.15.120:0/653047836 conn(0x557f001759b0 msgr2=0x557f00179c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- 172.21.15.120:0/653047836 shutdown_connections 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- 172.21.15.120:0/653047836 wait complete. 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 -- start start 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.354+0000 7f9f741f4740 1 Processor -- start 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f0018b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f0018aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1a8b60 con 0x557f0017d190 2024-09-10T23:09:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1a9fd0 con 0x557f0017a860 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1a9230 con 0x557f0017fbd0 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 10 client.0 _ll_register_callbacks cb 0x7f9f600040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48476/0 (socket says 172.21.15.120:48476) 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/2186865439 learned_addr learned my addr 172.21.15.120:0/2186865439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f0018b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f0018aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 msgr2=0x557f0018aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f0018aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/2186865439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 msgr2=0x557f0018b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f0018b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 -- 172.21.15.120:0/2186865439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0017f8f0 con 0x557f0017d190 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f0018b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f6a7fc640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000baa0 tx=0x7f9f60011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615ac0f714fbd881 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60034060 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9f60011440 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f64009b30 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f6002b040 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 10 client.18771 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.355+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f001fc6a0 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f64009b30 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9f6001b420 con 0x557f0017d190 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 10 client.18771 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 10 client.18771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 20 client.18771 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f001fcf10 0x557f001ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 4 client.18771 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:09:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f000ea0f0 con 0x557f001fcf10 2024-09-10T23:09:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f001fcf10 0x557f001ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f4eff5640 10 client.18771 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:27.356+0000 7f9f6b7fe640 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f001fcf10 0x557f001ff330 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282623989f3e0010 server_cookie=41e93025e10c658b in_seq=0 out_seq=0 2024-09-10T23:09:27.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 108496 21.83 MB/sec execute 142 sec latency 386.922 ms 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.012+0000 7f9f4eff5640 1 -- 172.21.15.120:0/2186865439 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f000ea0f0 con 0x557f001fcf10 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.012+0000 7f9f4eff5640 10 client.18771 handle_command_reply: tid=0 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.012+0000 7f9f741f4740 1 client.18771 shutdown 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.012+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 msgr2=0x557f0018a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000baa0 tx=0x7f9f60011a10 comp rx=0 tx=0).stop 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 shutdown_connections 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557f001fcf10 0x557f001ff330 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0017fbd0 0x557f0018aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f0017d190 0x557f0018a580 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 --2- 172.21.15.120:0/2186865439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557f0017a860 0x557f0018b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f4eff5640 0 client.18771 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 >> 172.21.15.120:0/2186865439 conn(0x557f001759b0 msgr2=0x557f00183e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 shutdown_connections 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 1 -- 172.21.15.120:0/2186865439 wait complete. 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f741f4740 20 client.18771 trim_cache size 0 max 16384 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:09:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:09:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.013+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 msgr2=0x7f9f540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 0x7f9f540b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c004650 tx=0x7f9f5c10c550 comp rx=0 tx=0).stop 2024-09-10T23:09:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 msgr2=0x7f9f6c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009a50 tx=0x7f9f5c0042f0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 shutdown_connections 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9f540b52e0 0x7f9f540b77a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c104490 0x7f9f6c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f6c103ad0 0x7f9f6c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 --2- 172.21.15.120:0/2473444219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9f6c102980 0x7f9f6c198db0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 >> 172.21.15.120:0/2473444219 conn(0x7f9f6c0fe240 msgr2=0x7f9f6c106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 shutdown_connections 2024-09-10T23:09:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:28.014+0000 7f9f7142b640 1 -- 172.21.15.120:0/2473444219 wait complete. 2024-09-10T23:09:28.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:27 smithi176 ceph-mon[34982]: pgmap v857: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-09-10T23:09:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:27 smithi120 ceph-mon[31247]: pgmap v857: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-09-10T23:09:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:27 smithi153 ceph-mon[34951]: pgmap v857: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-09-10T23:09:28.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 108909 21.84 MB/sec execute 143 sec latency 403.391 ms 2024-09-10T23:09:29.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 109443 21.96 MB/sec execute 144 sec latency 110.601 ms 2024-09-10T23:09:30.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:29 smithi176 ceph-mon[34982]: pgmap v858: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 408 op/s 2024-09-10T23:09:30.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:29 smithi120 ceph-mon[31247]: pgmap v858: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 408 op/s 2024-09-10T23:09:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:29 smithi153 ceph-mon[34951]: pgmap v858: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 408 op/s 2024-09-10T23:09:30.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 109885 22.00 MB/sec execute 145 sec latency 81.726 ms 2024-09-10T23:09:31.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 110252 21.99 MB/sec execute 146 sec latency 83.883 ms 2024-09-10T23:09:32.250 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:31 smithi153 ceph-mon[34951]: pgmap v859: 97 pgs: 97 active+clean; 242 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-09-10T23:09:32.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:31 smithi176 ceph-mon[34982]: pgmap v859: 97 pgs: 97 active+clean; 242 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-09-10T23:09:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:31 smithi120 ceph-mon[31247]: pgmap v859: 97 pgs: 97 active+clean; 242 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-09-10T23:09:32.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 110578 21.95 MB/sec execute 147 sec latency 111.272 ms 2024-09-10T23:09:33.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 110938 21.90 MB/sec execute 148 sec latency 137.766 ms 2024-09-10T23:09:34.247 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:33 smithi120 ceph-mon[31247]: pgmap v860: 97 pgs: 97 active+clean; 230 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-10T23:09:34.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:33 smithi176 ceph-mon[34982]: pgmap v860: 97 pgs: 97 active+clean; 230 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-10T23:09:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:33 smithi153 ceph-mon[34951]: pgmap v860: 97 pgs: 97 active+clean; 230 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-10T23:09:34.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 111265 21.82 MB/sec execute 149 sec latency 233.314 ms 2024-09-10T23:09:35.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 111684 21.80 MB/sec execute 150 sec latency 99.600 ms 2024-09-10T23:09:36.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:35 smithi176 ceph-mon[34982]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 556 op/s 2024-09-10T23:09:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:35 smithi120 ceph-mon[31247]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 556 op/s 2024-09-10T23:09:36.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:35 smithi153 ceph-mon[34951]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 556 op/s 2024-09-10T23:09:36.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 112344 21.91 MB/sec execute 151 sec latency 39.749 ms 2024-09-10T23:09:37.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 112943 22.00 MB/sec execute 152 sec latency 63.007 ms 2024-09-10T23:09:38.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:37 smithi176 ceph-mon[34982]: pgmap v862: 97 pgs: 97 active+clean; 244 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-10T23:09:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:37 smithi120 ceph-mon[31247]: pgmap v862: 97 pgs: 97 active+clean; 244 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-10T23:09:38.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:37 smithi153 ceph-mon[34951]: pgmap v862: 97 pgs: 97 active+clean; 244 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-10T23:09:38.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 113440 22.09 MB/sec execute 153 sec latency 55.026 ms 2024-09-10T23:09:39.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 113699 22.03 MB/sec execute 154 sec latency 226.259 ms 2024-09-10T23:09:40.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:40 smithi176 ceph-mon[34982]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-09-10T23:09:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:40 smithi120 ceph-mon[31247]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-09-10T23:09:40.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:40 smithi153 ceph-mon[34951]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-09-10T23:09:40.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 113978 21.99 MB/sec execute 155 sec latency 136.338 ms 2024-09-10T23:09:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 -- 172.21.15.120:0/1609940538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff514108680 msgr2=0x7ff51410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1609940538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff514108680 0x7ff51410ab40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 -- 172.21.15.120:0/1609940538 shutdown_connections 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1609940538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff514108680 0x7ff51410ab40 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1609940538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 0x7ff514107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1609940538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51406b410 0x7ff514105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 -- 172.21.15.120:0/1609940538 >> 172.21.15.120:0/1609940538 conn(0x7ff5140fd140 msgr2=0x7ff5140ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.106+0000 7ff51a62a640 1 -- 172.21.15.120:0/1609940538 shutdown_connections 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 -- 172.21.15.120:0/1609940538 wait complete. 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 Processor -- start 2024-09-10T23:09:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 -- start start 2024-09-10T23:09:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 0x7ff51419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514108680 0x7ff51419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410e600 con 0x7ff51406b410 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410e480 con 0x7ff514108680 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff51a62a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410e780 con 0x7ff514105ad0 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff513fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff513fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45108/0 (socket says 172.21.15.120:45108) 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.107+0000 7ff513fff640 1 -- 172.21.15.120:0/1280448512 learned_addr learned my addr 172.21.15.120:0/1280448512 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff5137fe640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 0x7ff51419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514108680 0x7ff51419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 -- 172.21.15.120:0/1280448512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 msgr2=0x7ff51419b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 0x7ff51419b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 -- 172.21.15.120:0/1280448512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514108680 msgr2=0x7ff51419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514108680 0x7ff51419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 -- 172.21.15.120:0/1280448512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419caa0 con 0x7ff51406b410 2024-09-10T23:09:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff513fff640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff50400d950 tx=0x7ff50400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff5117fa640 1 -- 172.21.15.120:0/1280448512 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504017070 con 0x7ff51406b410 2024-09-10T23:09:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff5117fa640 1 -- 172.21.15.120:0/1280448512 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504009c30 con 0x7ff51406b410 2024-09-10T23:09:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419cd90 con 0x7ff51406b410 2024-09-10T23:09:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff5117fa640 1 -- 172.21.15.120:0/1280448512 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400f620 con 0x7ff51406b410 2024-09-10T23:09:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.108+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a3460 con 0x7ff51406b410 2024-09-10T23:09:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.110+0000 7ff5117fa640 1 -- 172.21.15.120:0/1280448512 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff504010460 con 0x7ff51406b410 2024-09-10T23:09:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 Processor -- start 2024-09-10T23:09:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 -- start start 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e599ad86f0 con 0x55e599b66860 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e599ac7b20 con 0x55e599b69190 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e599acb930 con 0x55e599b6bbd0 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff5117fa640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 0x7ff4fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff5137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff518bc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff513fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff5137fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52908/0 (socket says 172.21.15.120:52908) 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff5137fe640 1 -- 172.21.15.120:0/985956926 learned_addr learned my addr 172.21.15.120:0/985956926 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.111+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 0x7ff4fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 -- 172.21.15.120:0/985956926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 msgr2=0x55e599b6b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 -- 172.21.15.120:0/985956926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 msgr2=0x55e599b68c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b68c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 -- 172.21.15.120:0/985956926 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e599a8d530 con 0x55e599b6bbd0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5117fa640 1 -- 172.21.15.120:0/1280448512 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff5040e57c0 con 0x7ff51406b410 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 0x7ff4fc0b7820 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009430 tx=0x7ff50c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff5137fe640 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff508009ca0 tx=0x7ff508011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc16742e9bf514d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff4fa7fc640 1 -- 172.21.15.120:0/985956926 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50801e070 con 0x55e599b6bbd0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff4fa7fc640 1 -- 172.21.15.120:0/985956926 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff508019ae0 con 0x55e599b6bbd0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.112+0000 7ff4fa7fc640 1 -- 172.21.15.120:0/985956926 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800e950 con 0x55e599b6bbd0 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- 172.21.15.120:0/985956926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 msgr2=0x55e599b6e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff508009ca0 tx=0x7ff508011810 comp rx=0 tx=0).stop 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- 172.21.15.120:0/985956926 shutdown_connections 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b6bbd0 0x55e599b6e090 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/985956926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- 172.21.15.120:0/985956926 >> 172.21.15.120:0/985956926 conn(0x55e599b619b0 msgr2=0x55e599b62740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- 172.21.15.120:0/985956926 shutdown_connections 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- 172.21.15.120:0/985956926 wait complete. 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.113+0000 7ff51d3f7740 1 -- start start 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 Processor -- start 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b69190 0x55e599b70560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b6bbd0 0x55e599b71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5999ffab0 con 0x55e599b66860 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e599a6d730 con 0x55e599b6bbd0 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e599b6ebb0 con 0x55e599b69190 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 10 client.0 _ll_register_callbacks cb 0x55e599ac16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45122/0 (socket says 172.21.15.120:45122) 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 -- 172.21.15.120:0/4214711384 learned_addr learned my addr 172.21.15.120:0/4214711384 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff518bc7640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b6bbd0 0x55e599b71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff5137fe640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b69190 0x55e599b70560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 -- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b69190 msgr2=0x55e599b70560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b69190 0x55e599b70560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 -- 172.21.15.120:0/4214711384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b6bbd0 msgr2=0x55e599b71130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b6bbd0 0x55e599b71130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 -- 172.21.15.120:0/4214711384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599829790 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff513fff640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff5040abcb0 tx=0x7ff5040a7f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84f56f84073cc1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff4f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504020040 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff5040dd040 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.114+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff508019ae0 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5040ab720 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 10 client.18783 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5998a2620 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff508019ae0 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff5040b0020 con 0x55e599b66860 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 10 client.18783 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 10 client.18783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 20 client.18783 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599b7b7b0 0x55e599b7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 4 client.18783 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e599ada640 con 0x55e599b7b7b0 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.115+0000 7ff5137fe640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599b7b7b0 0x55e599b7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.116+0000 7ff4f37fe640 10 client.18783 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.116+0000 7ff5137fe640 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599b7b7b0 0x55e599b7dba0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9418890b5d9191 server_cookie=b90e53f7d31ef8b3 in_seq=0 out_seq=0 2024-09-10T23:09:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff4f37fe640 1 -- 172.21.15.120:0/4214711384 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e599ada640 con 0x55e599b7b7b0 2024-09-10T23:09:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff4f37fe640 10 client.18783 handle_command_reply: tid=0 2024-09-10T23:09:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 client.18783 shutdown 2024-09-10T23:09:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 msgr2=0x55e599b6fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff5040abcb0 tx=0x7ff5040a7f80 comp rx=0 tx=0).stop 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 shutdown_connections 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599b7b7b0 0x55e599b7dba0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b6bbd0 0x55e599b71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b69190 0x55e599b70560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff4f37fe640 0 client.18783 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/4214711384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b66860 0x55e599b6fec0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.120+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 >> 172.21.15.120:0/4214711384 conn(0x55e599b619b0 msgr2=0x55e599b60bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.121+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 shutdown_connections 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.121+0000 7ff51d3f7740 1 -- 172.21.15.120:0/4214711384 wait complete. 2024-09-10T23:09:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.121+0000 7ff51d3f7740 20 client.18783 trim_cache size 0 max 16384 2024-09-10T23:09:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 Processor -- start 2024-09-10T23:09:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 -- start start 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aa0b0 con 0x55e599b6bbd0 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ac930 con 0x55e599b69190 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aa720 con 0x55e599b66860 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45132/0 (socket says 172.21.15.120:45132) 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 -- 172.21.15.120:0/379480238 learned_addr learned my addr 172.21.15.120:0/379480238 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff513fff640 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 -- 172.21.15.120:0/379480238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 msgr2=0x55e599b6fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b6fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 -- 172.21.15.120:0/379480238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 msgr2=0x55e599b6bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 -- 172.21.15.120:0/379480238 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e599b7df10 con 0x55e599b6bbd0 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.136+0000 7ff518bc7640 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff50c035000 tx=0x7ff50c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87967907316fe7a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff4f9ffb640 1 -- 172.21.15.120:0/379480238 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c003e40 con 0x55e599b6bbd0 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff4f9ffb640 1 -- 172.21.15.120:0/379480238 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50c036040 con 0x55e599b6bbd0 2024-09-10T23:09:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff4f9ffb640 1 -- 172.21.15.120:0/379480238 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c02fa40 con 0x55e599b6bbd0 2024-09-10T23:09:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 -- 172.21.15.120:0/379480238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 msgr2=0x55e599b6c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff50c035000 tx=0x7ff50c043000 comp rx=0 tx=0).stop 2024-09-10T23:09:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 -- 172.21.15.120:0/379480238 shutdown_connections 2024-09-10T23:09:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b6c4f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b6bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/379480238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 -- 172.21.15.120:0/379480238 >> 172.21.15.120:0/379480238 conn(0x55e599b619b0 msgr2=0x55e599b5caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 -- 172.21.15.120:0/379480238 shutdown_connections 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.137+0000 7ff51d3f7740 1 -- 172.21.15.120:0/379480238 wait complete. 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 -- start start 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 Processor -- start 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aadf0 con 0x55e599b6bbd0 2024-09-10T23:09:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ac260 con 0x55e599b69190 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ab4c0 con 0x55e599b66860 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff51d3f7740 10 client.0 _ll_register_callbacks cb 0x7ff50400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45148/0 (socket says 172.21.15.120:45148) 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff513fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 -- 172.21.15.120:0/1599945261 learned_addr learned my addr 172.21.15.120:0/1599945261 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff5137fe640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 -- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 msgr2=0x55e599b777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 -- 172.21.15.120:0/1599945261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 msgr2=0x55e599b764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff518bc7640 1 -- 172.21.15.120:0/1599945261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599b6b8f0 con 0x55e599b6bbd0 2024-09-10T23:09:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.138+0000 7ff5137fe640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff518bc7640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02ade0 tx=0x7ff50c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea90ae74d71c0f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c05b030 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff50c047070 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50c036040 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff50c003e40 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 10 client.18789 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e599b75ea0 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50c036040 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff50c04a040 con 0x55e599b6bbd0 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 10 client.18789 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 10 client.18789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 20 client.18789 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599be8d30 0x55e599beb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 4 client.18789 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:09:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.139+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e599ad60f0 con 0x55e599be8d30 2024-09-10T23:09:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.140+0000 7ff513fff640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599be8d30 0x55e599beb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.140+0000 7ff513fff640 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599be8d30 0x55e599beb150 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72b79ac31d73a71 server_cookie=128acbfa62807a95 in_seq=0 out_seq=0 2024-09-10T23:09:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.140+0000 7ff4f37fe640 10 client.18789 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff4f37fe640 1 -- 172.21.15.120:0/1599945261 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e599ad60f0 con 0x55e599be8d30 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff4f37fe640 10 client.18789 handle_command_reply: tid=0 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff51d3f7740 1 client.18789 shutdown 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 msgr2=0x55e599b769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02ade0 tx=0x7ff50c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.856+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 shutdown_connections 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e599be8d30 0x55e599beb150 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e599b6bbd0 0x55e599b769f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e599b69190 0x55e599b764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff4f37fe640 0 client.18789 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 --2- 172.21.15.120:0/1599945261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e599b66860 0x55e599b777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 >> 172.21.15.120:0/1599945261 conn(0x55e599b619b0 msgr2=0x55e599b6fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 shutdown_connections 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 1 -- 172.21.15.120:0/1599945261 wait complete. 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51d3f7740 20 client.18789 trim_cache size 0 max 16384 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:09:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:09:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 msgr2=0x7ff4fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 0x7ff4fc0b7820 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009430 tx=0x7ff50c0095c0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 msgr2=0x7ff51419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.857+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff50400d950 tx=0x7ff50400de10 comp rx=0 tx=0).stop 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 shutdown_connections 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff4fc0b5360 0x7ff4fc0b7820 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514108680 0x7ff51419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff514105ad0 0x7ff51419b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 --2- 172.21.15.120:0/1280448512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff51406b410 0x7ff51419af60 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 >> 172.21.15.120:0/1280448512 conn(0x7ff5140fd140 msgr2=0x7ff5140ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 shutdown_connections 2024-09-10T23:09:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:41.858+0000 7ff51a62a640 1 -- 172.21.15.120:0/1280448512 wait complete. 2024-09-10T23:09:41.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 114176 21.92 MB/sec execute 156 sec latency 417.646 ms 2024-09-10T23:09:41.943 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:09:42.227 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:42.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:42 smithi176 ceph-mon[34982]: pgmap v864: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-09-10T23:09:42.350 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:42 smithi120 ceph-mon[31247]: pgmap v864: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-09-10T23:09:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:42 smithi153 ceph-mon[34951]: pgmap v864: 97 pgs: 97 active+clean; 249 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/916808407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 msgr2=0x7f6fa410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/916808407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa410f5a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009ca0 tx=0x7f6f9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/916808407 shutdown_connections 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/916808407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa410f5a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/916808407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4104600 0x7f6fa410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/916808407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4103ce0 0x7f6fa41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/916808407 >> 172.21.15.120:0/916808407 conn(0x7f6fa40fd700 msgr2=0x7f6fa40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/916808407 shutdown_connections 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.735+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/916808407 wait complete. 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 Processor -- start 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 -- start start 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 0x7f6fa4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4104600 0x7f6fa41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4112db0 con 0x7f6fa410d140 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4112c30 con 0x7f6fa4103ce0 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa9ba1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4112f30 con 0x7f6fa4104600 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 0x7f6fa4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45170/0 (socket says 172.21.15.120:45170) 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.736+0000 7f6fa3fff640 1 -- 172.21.15.120:0/491168650 learned_addr learned my addr 172.21.15.120:0/491168650 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4104600 0x7f6fa41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 -- 172.21.15.120:0/491168650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4104600 msgr2=0x7f6fa41997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4104600 0x7f6fa41997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 -- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 msgr2=0x7f6fa419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 -- 172.21.15.120:0/491168650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa419f330 con 0x7f6fa4103ce0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 0x7f6fa4199280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400e9f0 tx=0x7f6f9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa0ff9640 1 -- 172.21.15.120:0/491168650 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400ccc0 con 0x7f6fa4103ce0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa0ff9640 1 -- 172.21.15.120:0/491168650 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f94005020 con 0x7f6fa4103ce0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa419f620 con 0x7f6fa4103ce0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa0ff9640 1 -- 172.21.15.120:0/491168650 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94004a00 con 0x7f6fa4103ce0 2024-09-10T23:09:42.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.737+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a5ce0 con 0x7f6fa4103ce0 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fa0ff9640 1 -- 172.21.15.120:0/491168650 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6f94004ba0 con 0x7f6fa4103ce0 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 Processor -- start 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 -- start start 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 0x55951ab70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55951aae6e50 con 0x55951ab70da0 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55951aad6280 con 0x55951ab7cd30 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fabf76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55951aada090 con 0x55951ab70480 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 0x55951ab70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45172/0 (socket says 172.21.15.120:45172) 2024-09-10T23:09:42.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.740+0000 7f6fa37fe640 1 -- 172.21.15.120:0/3297191377 learned_addr learned my addr 172.21.15.120:0/3297191377 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa0ff9640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 0x7f6f8c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 -- 172.21.15.120:0/3297191377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 msgr2=0x55951ab7f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab7f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 -- 172.21.15.120:0/3297191377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 msgr2=0x55951ab7c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 -- 172.21.15.120:0/3297191377 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951a834650 con 0x55951ab70480 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 0x7f6f8c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa0ff9640 1 -- 172.21.15.120:0/491168650 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6f940e5b30 con 0x7f6fa4103ce0 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 0x55951ab70860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005cc0 tx=0x7f6f9c005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac3f23701978c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 0x7f6f8c0b7820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005e30 tx=0x7f6f9c04e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6f89ffb640 1 -- 172.21.15.120:0/3297191377 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c042040 con 0x55951ab70480 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6f89ffb640 1 -- 172.21.15.120:0/3297191377 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c036070 con 0x55951ab70480 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.741+0000 7f6f89ffb640 1 -- 172.21.15.120:0/3297191377 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c004830 con 0x55951ab70480 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- 172.21.15.120:0/3297191377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 msgr2=0x55951ab70860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 0x55951ab70860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005cc0 tx=0x7f6f9c005f70 comp rx=0 tx=0).stop 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- 172.21.15.120:0/3297191377 shutdown_connections 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab7c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 --2- 172.21.15.120:0/3297191377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70480 0x55951ab70860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- 172.21.15.120:0/3297191377 >> 172.21.15.120:0/3297191377 conn(0x55951ab6e2b0 msgr2=0x55951ab6f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- 172.21.15.120:0/3297191377 shutdown_connections 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- 172.21.15.120:0/3297191377 wait complete. 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.742+0000 7f6fabf76740 1 -- start start 2024-09-10T23:09:42.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 Processor -- start 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 0x55951ab7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55951a937700 con 0x55951ab70da0 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55951aa71110 con 0x55951ab70480 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55951ab7f9b0 con 0x55951ab7cd30 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fabf76740 10 client.0 _ll_register_callbacks cb 0x55951aa8a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45180/0 (socket says 172.21.15.120:45180) 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 0x55951ab7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/431036585 learned_addr learned my addr 172.21.15.120:0/431036585 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 msgr2=0x55951ab7a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/431036585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 msgr2=0x55951ab7c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 0x55951ab7c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/431036585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951aa9ec40 con 0x55951ab70da0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 0x55951ab7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6f98012700 tx=0x7f6f98012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe0704e28853cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.743+0000 7f6f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004030 con 0x55951ab70da0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6f980041d0 con 0x55951ab70da0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c036070 con 0x55951ab70da0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f98023070 con 0x55951ab70da0 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 10 client.18801 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:42.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55951ab8b3f0 con 0x55951ab70da0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c036070 con 0x55951ab70da0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6f980038c0 con 0x55951ab70da0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 10 client.18801 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 20 client.18801 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951ab8c8b0 0x55951ab8eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 4 client.18801 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951aae8da0 con 0x55951ab8c8b0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.744+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951ab8c8b0 0x55951ab8eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.745+0000 7f6f82ffd640 10 client.18801 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.745+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951ab8c8b0 0x55951ab8eca0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c61a72d8ceeeb3 server_cookie=b72340337a0db26e in_seq=0 out_seq=0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.749+0000 7f6f82ffd640 1 -- 172.21.15.120:0/431036585 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55951aae8da0 con 0x55951ab8c8b0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.749+0000 7f6f82ffd640 10 client.18801 handle_command_reply: tid=0 2024-09-10T23:09:42.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.749+0000 7f6fabf76740 1 client.18801 shutdown 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 msgr2=0x55951ab79af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6f98012700 tx=0x7f6f98012bc0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 shutdown_connections 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951ab8c8b0 0x55951ab8eca0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70da0 0x55951ab79af0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 --2- 172.21.15.120:0/431036585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70480 0x55951ab7c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6f82ffd640 0 client.18801 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 >> 172.21.15.120:0/431036585 conn(0x55951ab6e2b0 msgr2=0x55951ab73e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 shutdown_connections 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 1 -- 172.21.15.120:0/431036585 wait complete. 2024-09-10T23:09:42.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.750+0000 7f6fabf76740 20 client.18801 trim_cache size 0 max 16384 2024-09-10T23:09:42.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.767+0000 7f6fabf76740 1 Processor -- start 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.767+0000 7f6fabf76740 1 -- start start 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70da0 0x55951ab78fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41ac210 con 0x55951ab70480 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41aea90 con 0x55951ab7cd30 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fabf76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41ac880 con 0x55951ab70da0 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45192/0 (socket says 172.21.15.120:45192) 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 -- 172.21.15.120:0/1903366141 learned_addr learned my addr 172.21.15.120:0/1903366141 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70da0 0x55951ab78fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa3fff640 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 -- 172.21.15.120:0/1903366141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70da0 msgr2=0x55951ab78fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70da0 0x55951ab78fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 -- 172.21.15.120:0/1903366141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 msgr2=0x55951ab78370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab78370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 -- 172.21.15.120:0/1903366141 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951ab6ef90 con 0x55951ab70480 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.768+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6f940cf010 tx=0x7f6f940e34d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8548876fa44788fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.769+0000 7f6f897fa640 1 -- 172.21.15.120:0/1903366141 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9401c040 con 0x55951ab70480 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.769+0000 7f6f897fa640 1 -- 172.21.15.120:0/1903366141 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f940b5190 con 0x55951ab70480 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.769+0000 7f6f897fa640 1 -- 172.21.15.120:0/1903366141 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f940ac2c0 con 0x55951ab70480 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- 172.21.15.120:0/1903366141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 msgr2=0x55951ab88c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6f940cf010 tx=0x7f6f940e34d0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- 172.21.15.120:0/1903366141 shutdown_connections 2024-09-10T23:09:42.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab7cd30 0x55951ab78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab70da0 0x55951ab78fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 --2- 172.21.15.120:0/1903366141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab88c10 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- 172.21.15.120:0/1903366141 >> 172.21.15.120:0/1903366141 conn(0x55951ab6e2b0 msgr2=0x55951ab74d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- 172.21.15.120:0/1903366141 shutdown_connections 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- 172.21.15.120:0/1903366141 wait complete. 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.770+0000 7f6fabf76740 1 -- start start 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 Processor -- start 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab84c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 0x55951ab851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41acf50 con 0x55951ab70480 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41ae3c0 con 0x55951ab70da0 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa41ad620 con 0x55951ab7cd30 2024-09-10T23:09:42.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab84c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53008/0 (socket says 172.21.15.120:53008) 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa3fff640 1 -- 172.21.15.120:0/2058590875 learned_addr learned my addr 172.21.15.120:0/2058590875 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 0x55951ab851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fabf76740 10 client.0 _ll_register_callbacks cb 0x7f6f98009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 msgr2=0x55951ab7fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/2058590875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 msgr2=0x55951ab84c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.771+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab84c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6fa2ffd640 1 -- 172.21.15.120:0/2058590875 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951ab859c0 con 0x55951ab70da0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6fa2ffd640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 0x55951ab851c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f98000c00 tx=0x7f6f9800fbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3b753b6a302605 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:09:42.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98009c90 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6f980035f0 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.772+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55951aae4850 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f98003810 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 10 client.28046 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55951abfe1b0 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55951aae4850 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6f9801c560 con 0x55951ab70da0 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 10 client.28046 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 10 client.28046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 20 client.28046 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951abfe9b0 0x55951ac00dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 4 client.28046 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:09:42.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55951aae1c80 con 0x55951abfe9b0 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.773+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951abfe9b0 0x55951ac00dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.774+0000 7f6f82ffd640 10 client.28046 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.774+0000 7f6fa37fe640 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951abfe9b0 0x55951ac00dd0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2c0af1425832c server_cookie=ae1474e449bd774e in_seq=0 out_seq=0 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6f82ffd640 1 -- 172.21.15.120:0/2058590875 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8326 (crc 0 0 0) 0x55951aae1c80 con 0x55951abfe9b0 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6f82ffd640 10 client.28046 handle_command_reply: tid=0 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 client.28046 shutdown 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 msgr2=0x55951ab851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 0x55951ab851c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f98000c00 tx=0x7f6f9800fbb0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 shutdown_connections 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55951abfe9b0 0x55951ac00dd0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55951ab7cd30 0x55951ab7fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55951ab70da0 0x55951ab851c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 --2- 172.21.15.120:0/2058590875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55951ab70480 0x55951ab84c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.787 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 >> 172.21.15.120:0/2058590875 conn(0x55951ab6e2b0 msgr2=0x55951ab782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6f82ffd640 0 client.28046 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.776+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 shutdown_connections 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fabf76740 1 -- 172.21.15.120:0/2058590875 wait complete. 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fabf76740 20 client.28046 trim_cache size 0 max 16384 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:09:42.788 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.789 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:09:42.790 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.792 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:09:42.793 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.795 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:09:42.796 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.798 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:09:42.799 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.800 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.801 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.802 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:09:42.803 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:42.805 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:42.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:42.806 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:42.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:42.809 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:42.809 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:42.809 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 msgr2=0x7f6f8c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 0x7f6f8c0b7820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005e30 tx=0x7f6f9c04e000 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 msgr2=0x7f6fa4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 0x7f6fa4199280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400e9f0 tx=0x7f6f9400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 shutdown_connections 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6f8c0b5360 0x7f6f8c0b7820 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6fa410d140 0x7f6fa419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6fa4104600 0x7f6fa41997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 --2- 172.21.15.120:0/491168650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fa4103ce0 0x7f6fa4199280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.777+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 >> 172.21.15.120:0/491168650 conn(0x7f6fa40fd700 msgr2=0x7f6fa40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.778+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 shutdown_connections 2024-09-10T23:09:42.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:42.778+0000 7f6fa9ba1640 1 -- 172.21.15.120:0/491168650 wait complete. 2024-09-10T23:09:42.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 114528 21.88 MB/sec execute 157 sec latency 177.890 ms 2024-09-10T23:09:43.587 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:09:43.587 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:09:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:43.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 115003 21.86 MB/sec execute 158 sec latency 49.428 ms 2024-09-10T23:09:44.286 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:44 smithi120 ceph-mon[31247]: pgmap v865: 97 pgs: 97 active+clean; 250 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 498 op/s 2024-09-10T23:09:44.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:44 smithi176 ceph-mon[34982]: pgmap v865: 97 pgs: 97 active+clean; 250 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 498 op/s 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- 172.21.15.120:0/3705191380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081045c0 msgr2=0x7f210810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 --2- 172.21.15.120:0/3705191380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081045c0 0x7f210810cab0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009900 tx=0x7f20fc02a410 comp rx=0 tx=0).stop 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- 172.21.15.120:0/3705191380 shutdown_connections 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 --2- 172.21.15.120:0/3705191380 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 0x7f210810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 --2- 172.21.15.120:0/3705191380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21081045c0 0x7f210810cab0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 --2- 172.21.15.120:0/3705191380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2108103ca0 0x7f2108104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- 172.21.15.120:0/3705191380 >> 172.21.15.120:0/3705191380 conn(0x7f21080fd700 msgr2=0x7f21080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:44.402 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- 172.21.15.120:0/3705191380 shutdown_connections 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- 172.21.15.120:0/3705191380 wait complete. 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 Processor -- start 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.385+0000 7f210e9f0640 1 -- start start 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 0x7f21081997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 0x7f210819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2108112d70 con 0x7f2108103ca0 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2108112bf0 con 0x7f21081045c0 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2108112ef0 con 0x7f210810d100 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59144/0 (socket says 172.21.15.120:59144) 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 -- 172.21.15.120:0/2603488550 learned_addr learned my addr 172.21.15.120:0/2603488550 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:44.403 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210cf8d640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 0x7f210819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f21077fe640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 0x7f21081997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 -- 172.21.15.120:0/2603488550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 msgr2=0x7f210819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 0x7f210819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 -- 172.21.15.120:0/2603488550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 msgr2=0x7f21081997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 0x7f21081997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 -- 172.21.15.120:0/2603488550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819f350 con 0x7f2108103ca0 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f21077fe640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 0x7f21081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f2107fff640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:44.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8017070 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210819f640 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.386+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f8009c30 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.387+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800f620 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.387+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a5d00 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.388+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2108104080 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.389+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f20f8005030 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.390+0000 7f21057fa640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 0x7f20f00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.390+0000 7f21077fe640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 0x7f20f00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.390+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f20f80e57c0 con 0x7f2108103ca0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.391+0000 7f21077fe640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 0x7f20f00b77a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:44.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.393+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f20f80ac6b0 con 0x7f2108103ca0 2024-09-10T23:09:44.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:44 smithi153 ceph-mon[34951]: pgmap v865: 97 pgs: 97 active+clean; 250 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 498 op/s 2024-09-10T23:09:44.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.631+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210819a3d0 con 0x7f2108103ca0 2024-09-10T23:09:44.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.632+0000 7f21057fa640 1 -- 172.21.15.120:0/2603488550 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f20f8009de0 con 0x7f2108103ca0 2024-09-10T23:09:44.633 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:09:44.633 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:09:44.633 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:09:44.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.634+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 msgr2=0x7f20f00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:44.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.634+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 0x7f20f00b77a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc032d20 comp rx=0 tx=0).stop 2024-09-10T23:09:44.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.634+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 msgr2=0x7f21081992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.634+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).stop 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 shutdown_connections 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f20f00b52e0 0x7f20f00b77a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210810d100 0x7f210819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21081045c0 0x7f21081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 --2- 172.21.15.120:0/2603488550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2108103ca0 0x7f21081992a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 >> 172.21.15.120:0/2603488550 conn(0x7f21080fd700 msgr2=0x7f21080fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 shutdown_connections 2024-09-10T23:09:44.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:44.635+0000 7f210e9f0640 1 -- 172.21.15.120:0/2603488550 wait complete. 2024-09-10T23:09:44.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 115424 21.86 MB/sec execute 159 sec latency 179.044 ms 2024-09-10T23:09:45.283 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:45 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2603488550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:45.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:09:45.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:45 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2603488550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:45.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:45 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2603488550' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:09:45.595 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:45.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 115973 21.94 MB/sec execute 160 sec latency 68.243 ms 2024-09-10T23:09:46.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:46 smithi176 ceph-mon[34982]: pgmap v866: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-10T23:09:46.389 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:46 smithi120 ceph-mon[31247]: pgmap v866: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-10T23:09:46.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:46 smithi153 ceph-mon[34951]: pgmap v866: 97 pgs: 97 active+clean; 246 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 -- 172.21.15.120:0/3266118549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 msgr2=0x7f3ec010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/3266118549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec010f920 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009ca0 tx=0x7f3eb802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 -- 172.21.15.120:0/3266118549 shutdown_connections 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/3266118549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec010f920 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/3266118549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 0x7f3ec0108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/3266118549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 0x7f3ec0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 -- 172.21.15.120:0/3266118549 >> 172.21.15.120:0/3266118549 conn(0x7f3ec00fe820 msgr2=0x7f3ec0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 -- 172.21.15.120:0/3266118549 shutdown_connections 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.621+0000 7f3ec66a3640 1 -- 172.21.15.120:0/3266118549 wait complete. 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 Processor -- start 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 -- start start 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 0x7f3ec00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 0x7f3ec00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0112de0 con 0x7f3ec0109120 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0112c60 con 0x7f3ec0102d90 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec66a3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0112f60 con 0x7f3ec01087a0 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59154/0 (socket says 172.21.15.120:59154) 2024-09-10T23:09:46.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 0x7f3ec00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ec4c40640 1 -- 172.21.15.120:0/2598969732 learned_addr learned my addr 172.21.15.120:0/2598969732 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:46.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.622+0000 7f3ebffff640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 0x7f3ec00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 -- 172.21.15.120:0/2598969732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 msgr2=0x7f3ec00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 0x7f3ec00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 -- 172.21.15.120:0/2598969732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 msgr2=0x7f3ec00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 0x7f3ec00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 -- 172.21.15.120:0/2598969732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec01a7650 con 0x7f3ec0109120 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8005cc0 tx=0x7f3eb8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ebdffb640 1 -- 172.21.15.120:0/2598969732 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8003b70 con 0x7f3ec0109120 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec01a78e0 con 0x7f3ec0109120 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ebdffb640 1 -- 172.21.15.120:0/2598969732 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb80318f0 con 0x7f3ec0109120 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ebdffb640 1 -- 172.21.15.120:0/2598969732 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb802f950 con 0x7f3ec0109120 2024-09-10T23:09:46.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.623+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec01a7e70 con 0x7f3ec0109120 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ebdffb640 1 -- 172.21.15.120:0/2598969732 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3eb802fb10 con 0x7f3ec0109120 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 Processor -- start 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 -- start start 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcafb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637fca71ea0 con 0x5637fcafb4d0 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637fca612d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.626+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5637fca650e0 con 0x5637fcb07d80 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcafb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50044/0 (socket says 172.21.15.120:50044) 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 -- 172.21.15.120:0/1897467944 learned_addr learned my addr 172.21.15.120:0/1897467944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 -- 172.21.15.120:0/1897467944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 msgr2=0x5637fcb0a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 -- 172.21.15.120:0/1897467944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 msgr2=0x5637fcafb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcafb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 -- 172.21.15.120:0/1897467944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fc7bf650 con 0x5637fcafbdf0 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcafb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:46.629 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009cf0 tx=0x7f3eb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b21a91d91455882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.627+0000 7f3eb67fc640 1 -- 172.21.15.120:0/1897467944 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0004230 con 0x5637fcafbdf0 2024-09-10T23:09:46.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3eb67fc640 1 -- 172.21.15.120:0/1897467944 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb00043d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3ebdffb640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 0x7f3eac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 0x7f3eac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3ebdffb640 1 -- 172.21.15.120:0/2598969732 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3eb8106eb0 con 0x7f3ec0109120 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3eb67fc640 1 -- 172.21.15.120:0/1897467944 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0019880 con 0x5637fcafbdf0 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1897467944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 msgr2=0x5637fcb07840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 0x7f3eac0b77a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009710 tx=0x7f3ea8009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.628+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009cf0 tx=0x7f3eb0011810 comp rx=0 tx=0).stop 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1897467944 shutdown_connections 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb0a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb07840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1897467944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcafb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1897467944 >> 172.21.15.120:0/1897467944 conn(0x5637fcaf9300 msgr2=0x5637fcafa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1897467944 shutdown_connections 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1897467944 wait complete. 2024-09-10T23:09:46.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.629+0000 7f3ec8a78740 1 -- start start 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 Processor -- start 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637fc8c2700 con 0x5637fcafb4d0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637fc9fc160 con 0x5637fcafbdf0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5637fcb0aa50 con 0x5637fcb07d80 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec8a78740 10 client.0 _ll_register_callbacks cb 0x5637fca159e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb07590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59174/0 (socket says 172.21.15.120:59174) 2024-09-10T23:09:46.635 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47844/0 (socket says 172.21.15.120:47844) 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50058/0 (socket says 172.21.15.120:50058) 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec4c40640 1 -- 172.21.15.120:0/4110393869 learned_addr learned my addr 172.21.15.120:0/4110393869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 -- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 msgr2=0x5637fcb04f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 -- 172.21.15.120:0/4110393869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 msgr2=0x5637fcb07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3eb7fff640 1 -- 172.21.15.120:0/4110393869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fca29c90 con 0x5637fcafbdf0 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.630+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0076f90 tx=0x7f3ea8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f8bc348e36a168 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8005020 con 0x5637fcafbdf0 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb00043d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.636 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3ea80051c0 con 0x5637fcafbdf0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80053c0 con 0x5637fcafbdf0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3ec8a78740 10 client.28064 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.631+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637fcb163d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb00043d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3ea800f770 con 0x5637fcafbdf0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 10 client.28064 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 10 client.28064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 20 client.28064 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb17990 0x5637fcb19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 4 client.28064 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637fca73df0 con 0x5637fcb17990 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.632+0000 7f3ebffff640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb17990 0x5637fcb19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.633+0000 7f3ebffff640 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb17990 0x5637fcb19dc0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489903b8c923cde5 server_cookie=939ff31105a058e2 in_seq=0 out_seq=0 2024-09-10T23:09:46.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.633+0000 7f3e9f7fe640 10 client.28064 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:46.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.634+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/4110393869 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637fca73df0 con 0x5637fcb17990 2024-09-10T23:09:46.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.634+0000 7f3e9f7fe640 10 client.28064 handle_command_reply: tid=0 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 client.28064 shutdown 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 msgr2=0x5637fcb049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0076f90 tx=0x7f3ea8004730 comp rx=0 tx=0).stop 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 shutdown_connections 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb17990 0x5637fcb19dc0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcb07d80 0x5637fcb04f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafbdf0 0x5637fcb049e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/4110393869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 >> 172.21.15.120:0/4110393869 conn(0x5637fcaf9300 msgr2=0x5637fcafee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3e9f7fe640 0 client.28064 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 shutdown_connections 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 1 -- 172.21.15.120:0/4110393869 wait complete. 2024-09-10T23:09:46.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.635+0000 7f3ec8a78740 20 client.28064 trim_cache size 0 max 16384 2024-09-10T23:09:46.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.650+0000 7f3ec8a78740 1 Processor -- start 2024-09-10T23:09:46.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.650+0000 7f3ec8a78740 1 -- start start 2024-09-10T23:09:46.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.650+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafb4d0 0x5637fcaf4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.650+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0072e80 con 0x5637fcb07d80 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec01a84d0 con 0x5637fcafb4d0 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec8a78740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec00734d0 con 0x5637fcafbdf0 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59180/0 (socket says 172.21.15.120:59180) 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 -- 172.21.15.120:0/3103107314 learned_addr learned my addr 172.21.15.120:0/3103107314 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ebffff640 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafb4d0 0x5637fcaf4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 -- 172.21.15.120:0/3103107314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 msgr2=0x5637fcb01bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb01bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 -- 172.21.15.120:0/3103107314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafb4d0 msgr2=0x5637fcaf4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafb4d0 0x5637fcaf4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 -- 172.21.15.120:0/3103107314 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fcaff780 con 0x5637fcb07d80 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8104d50 tx=0x7f3eb8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb414ce9eb3fbd17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.651+0000 7f3eb5ffb640 1 -- 172.21.15.120:0/3103107314 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb80c9540 con 0x5637fcb07d80 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.652+0000 7f3eb5ffb640 1 -- 172.21.15.120:0/3103107314 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb8036030 con 0x5637fcb07d80 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.652+0000 7f3eb5ffb640 1 -- 172.21.15.120:0/3103107314 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8003e10 con 0x5637fcb07d80 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.652+0000 7f3ec8a78740 1 -- 172.21.15.120:0/3103107314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 msgr2=0x5637fcb02130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.652+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8104d50 tx=0x7f3eb8005f10 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 -- 172.21.15.120:0/3103107314 shutdown_connections 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcb07d80 0x5637fcb02130 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb01bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/3103107314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcafb4d0 0x5637fcaf4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 -- 172.21.15.120:0/3103107314 >> 172.21.15.120:0/3103107314 conn(0x5637fcaf9300 msgr2=0x5637fcafa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 -- 172.21.15.120:0/3103107314 shutdown_connections 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 -- 172.21.15.120:0/3103107314 wait complete. 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.653+0000 7f3ec8a78740 1 -- start start 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 Processor -- start 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb0bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 0x5637fcb0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ebffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59186/0 (socket says 172.21.15.120:59186) 2024-09-10T23:09:46.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0073ba0 con 0x5637fcafb4d0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ebffff640 1 -- 172.21.15.120:0/1098431214 learned_addr learned my addr 172.21.15.120:0/1098431214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0075010 con 0x5637fcb07d80 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0074270 con 0x5637fcafbdf0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3ec8a78740 10 client.0 _ll_register_callbacks cb 0x7f3ea8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.654+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb0bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 0x5637fcb0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 -- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 msgr2=0x5637fcb0bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb0bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 -- 172.21.15.120:0/1098431214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 msgr2=0x5637fcb0c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 0x5637fcb0c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fcb838d0 con 0x5637fcafb4d0 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ec4c40640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 0x5637fcb0c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:46.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3ebffff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0014040 tx=0x7f3eb0019490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc16c068867c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.655+0000 7f3e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb002f070 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3eb0003a00 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb8036030 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3eb0005040 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3ec8a78740 10 client.18831 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637fcb84ad0 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.656+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb8036030 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3eb0019780 con 0x5637fcafb4d0 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 10 client.18831 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 10 client.18831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 20 client.18831 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb850b0 0x5637fcb874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:46.662 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 4 client.18831 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:09:46.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637fca6f8a0 con 0x5637fcb850b0 2024-09-10T23:09:46.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.657+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb850b0 0x5637fcb874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:46.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.658+0000 7f3e9f7fe640 10 client.18831 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:46.663 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.658+0000 7f3eb7fff640 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb850b0 0x5637fcb874d0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d3274006216e57 server_cookie=3d1a1eb60744c495 in_seq=0 out_seq=0 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.662+0000 7f3e9f7fe640 1 -- 172.21.15.120:0/1098431214 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637fca6f8a0 con 0x5637fcb850b0 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.662+0000 7f3e9f7fe640 10 client.18831 handle_command_reply: tid=0 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 client.18831 shutdown 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 msgr2=0x5637fcb106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0014040 tx=0x7f3eb0019490 comp rx=0 tx=0).stop 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 shutdown_connections 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637fcb850b0 0x5637fcb874d0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637fcb07d80 0x5637fcb0c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637fcafbdf0 0x5637fcb0bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 --2- 172.21.15.120:0/1098431214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637fcafb4d0 0x5637fcb106a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 >> 172.21.15.120:0/1098431214 conn(0x5637fcaf9300 msgr2=0x5637fcaf7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3e9f7fe640 0 client.18831 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 shutdown_connections 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 1 -- 172.21.15.120:0/1098431214 wait complete. 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.663+0000 7f3ec8a78740 20 client.18831 trim_cache size 0 max 16384 2024-09-10T23:09:46.665 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 msgr2=0x7f3eac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 0x7f3eac0b77a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009710 tx=0x7f3ea8009640 comp rx=0 tx=0).stop 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 msgr2=0x7f3ec0075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8005cc0 tx=0x7f3eb8031700 comp rx=0 tx=0).stop 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 shutdown_connections 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3eac0b52e0 0x7f3eac0b77a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ec0109120 0x7f3ec0075f20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ec01087a0 0x7f3ec00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 --2- 172.21.15.120:0/2598969732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec0102d90 0x7f3ec00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:46.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 >> 172.21.15.120:0/2598969732 conn(0x7f3ec00fe820 msgr2=0x7f3ec0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:46.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 shutdown_connections 2024-09-10T23:09:46.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:46.664+0000 7f3ec66a3640 1 -- 172.21.15.120:0/2598969732 wait complete. 2024-09-10T23:09:46.875 INFO:tasks.workunit.client.0.smithi120.stdout: 10 116548 21.99 MB/sec execute 161 sec latency 58.748 ms 2024-09-10T23:09:47.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117073 22.07 MB/sec execute 162 sec latency 46.945 ms 2024-09-10T23:09:48.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:48 smithi176 ceph-mon[34982]: pgmap v867: 97 pgs: 97 active+clean; 237 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-10T23:09:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:48 smithi120 ceph-mon[31247]: pgmap v867: 97 pgs: 97 active+clean; 237 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-10T23:09:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:48 smithi153 ceph-mon[34951]: pgmap v867: 97 pgs: 97 active+clean; 237 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-10T23:09:48.483 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:09:48.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:09:48.755 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:48.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117407 22.05 MB/sec execute 163 sec latency 128.399 ms 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 -- 172.21.15.120:0/2025117163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0774101c90 msgr2=0x7f0774102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2025117163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0774101c90 0x7f0774102070 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0770009900 tx=0x7f077002a410 comp rx=0 tx=0).stop 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 -- 172.21.15.120:0/2025117163 shutdown_connections 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2025117163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774108020 0x7f077410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2025117163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07741076a0 0x7f0774107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2025117163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0774101c90 0x7f0774102070 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 -- 172.21.15.120:0/2025117163 >> 172.21.15.120:0/2025117163 conn(0x7f07740fd700 msgr2=0x7f07740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 -- 172.21.15.120:0/2025117163 shutdown_connections 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.320+0000 7f077c4bf640 1 -- 172.21.15.120:0/2025117163 wait complete. 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 Processor -- start 2024-09-10T23:09:49.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 -- start start 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 0x7f077410f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0774108020 0x7f077410fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0774112dd0 con 0x7f07741076a0 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0774112c50 con 0x7f0774101c90 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077c4bf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0774112f50 con 0x7f0774108020 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f0779a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f0779a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59196/0 (socket says 172.21.15.120:59196) 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f0779a5a640 1 -- 172.21.15.120:0/2452575304 learned_addr learned my addr 172.21.15.120:0/2452575304 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077aa5c640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0774108020 0x7f077410fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.321+0000 7f077a25b640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 0x7f077410f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 -- 172.21.15.120:0/2452575304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0774108020 msgr2=0x7f077410fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0774108020 0x7f077410fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 -- 172.21.15.120:0/2452575304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 msgr2=0x7f077410f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 0x7f077410f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 -- 172.21.15.120:0/2452575304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07741a76a0 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f077a25b640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 0x7f077410f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9f0 tx=0x7f076400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f076b7fe640 1 -- 172.21.15.120:0/2452575304 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400cd80 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f076b7fe640 1 -- 172.21.15.120:0/2452575304 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07640108e0 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07741a7990 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f076b7fe640 1 -- 172.21.15.120:0/2452575304 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764018980 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.322+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a7f20 con 0x7f07741076a0 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 Processor -- start 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 -- start start 2024-09-10T23:09:49.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f076b7fe640 1 -- 172.21.15.120:0/2452575304 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0764010430 con 0x7f07741076a0 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55581a04ae60 con 0x55581a0d4800 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55581a03a290 con 0x55581a0e0790 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077e894740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55581a03e0a0 con 0x55581a0d3ee0 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59206/0 (socket says 172.21.15.120:59206) 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077a25b640 1 -- 172.21.15.120:0/807876517 learned_addr learned my addr 172.21.15.120:0/807876517 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f077aa5c640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.325+0000 7f0779a5a640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 -- 172.21.15.120:0/807876517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 msgr2=0x55581a0d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 -- 172.21.15.120:0/807876517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 msgr2=0x55581a0e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 -- 172.21.15.120:0/807876517 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555819d98930 con 0x55581a0d4800 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077aa5c640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f076b7fe640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 0x7f07600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077a25b640 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0770033000 tx=0x7f0770032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279d0555cd1ef725 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f077aa5c640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 0x7f07600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f076b7fe640 1 -- 172.21.15.120:0/2452575304 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f07640e5eb0 con 0x7f07741076a0 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f0768ff9640 1 -- 172.21.15.120:0/807876517 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770035030 con 0x55581a0d4800 2024-09-10T23:09:49.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f0768ff9640 1 -- 172.21.15.120:0/807876517 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0770032e40 con 0x55581a0d4800 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.326+0000 7f0768ff9640 1 -- 172.21.15.120:0/807876517 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700046c0 con 0x55581a0d4800 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 -- 172.21.15.120:0/807876517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 msgr2=0x55581a0e0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0770033000 tx=0x7f0770032c20 comp rx=0 tx=0).stop 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077aa5c640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 0x7f07600b7820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f076c009fd0 tx=0x7f076c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 -- 172.21.15.120:0/807876517 shutdown_connections 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e0250 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 --2- 172.21.15.120:0/807876517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 -- 172.21.15.120:0/807876517 >> 172.21.15.120:0/807876517 conn(0x55581a0d1d10 msgr2=0x55581a0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 -- 172.21.15.120:0/807876517 shutdown_connections 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.327+0000 7f077e894740 1 -- 172.21.15.120:0/807876517 wait complete. 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 -- start start 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 Processor -- start 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d4800 0x55581a0ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555819e9b890 con 0x55581a0d3ee0 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x555819fd4d40 con 0x55581a0e0790 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55581a0e3430 con 0x55581a0d4800 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077e894740 10 client.0 _ll_register_callbacks cb 0x55581a038c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59212/0 (socket says 172.21.15.120:59212) 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077a25b640 1 -- 172.21.15.120:0/881274878 learned_addr learned my addr 172.21.15.120:0/881274878 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f0779a5a640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d4800 0x55581a0ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.328+0000 7f077aa5c640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 -- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d4800 msgr2=0x55581a0ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d4800 0x55581a0ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 -- 172.21.15.120:0/881274878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 msgr2=0x55581a0de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 -- 172.21.15.120:0/881274878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581a0ec7e0 con 0x55581a0d3ee0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077a25b640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0770000c00 tx=0x7f0770031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eb4cca3252edf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f075dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770035040 con 0x55581a0d3ee0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0770004460 con 0x55581a0d3ee0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55581a04cdb0 con 0x55581a0d3ee0 2024-09-10T23:09:49.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07700046c0 con 0x55581a0d3ee0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077e894740 10 client.18843 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55581a0ef190 con 0x55581a0d3ee0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.329+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55581a04cdb0 con 0x55581a0d3ee0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0770003990 con 0x55581a0d3ee0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 10 client.18843 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 20 client.18843 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a0f04e0 0x55581a0f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 4 client.18843 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581a048860 con 0x55581a0f04e0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.330+0000 7f0779a5a640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a0f04e0 0x55581a0f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.331+0000 7f075dffb640 10 client.18843 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:49.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.331+0000 7f0779a5a640 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a0f04e0 0x55581a0f28d0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0e0016641a5b2b server_cookie=5ac342002738b958 in_seq=0 out_seq=0 2024-09-10T23:09:49.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.364+0000 7f075dffb640 1 -- 172.21.15.120:0/881274878 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55581a048860 con 0x55581a0f04e0 2024-09-10T23:09:49.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.364+0000 7f075dffb640 10 client.18843 handle_command_reply: tid=0 2024-09-10T23:09:49.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.364+0000 7f077e894740 1 client.18843 shutdown 2024-09-10T23:09:49.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 msgr2=0x55581a0dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0770000c00 tx=0x7f0770031750 comp rx=0 tx=0).stop 2024-09-10T23:09:49.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 shutdown_connections 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a0f04e0 0x55581a0f28d0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.367+0000 7f075dffb640 0 client.18843 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.368+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d4800 0x55581a0ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.368+0000 7f077e894740 1 --2- 172.21.15.120:0/881274878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d3ee0 0x55581a0dd700 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.369 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.368+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 >> 172.21.15.120:0/881274878 conn(0x55581a0d1d10 msgr2=0x55581a0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.376+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 shutdown_connections 2024-09-10T23:09:49.379 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.376+0000 7f077e894740 1 -- 172.21.15.120:0/881274878 wait complete. 2024-09-10T23:09:49.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.376+0000 7f077e894740 20 client.18843 trim_cache size 0 max 16384 2024-09-10T23:09:49.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.413+0000 7f077e894740 1 Processor -- start 2024-09-10T23:09:49.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.414+0000 7f077e894740 1 -- start start 2024-09-10T23:09:49.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.414+0000 7f077e894740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.414+0000 7f077e894740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0df440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.415+0000 7f077a25b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.415+0000 7f077a25b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47908/0 (socket says 172.21.15.120:47908) 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.415+0000 7f0779a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0df440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.415+0000 7f0779a5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0df440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59216/0 (socket says 172.21.15.120:59216) 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.416+0000 7f077a25b640 1 -- 172.21.15.120:0/2959175086 learned_addr learned my addr 172.21.15.120:0/2959175086 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.416+0000 7f077e894740 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0da930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.416+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07741ae480 con 0x55581a0d4800 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.416+0000 7f077aa5c640 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.416+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07741b0d00 con 0x55581a0e0790 2024-09-10T23:09:49.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07741aeaf0 con 0x55581a0d3ee0 2024-09-10T23:09:49.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077a25b640 1 -- 172.21.15.120:0/2959175086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 msgr2=0x55581a0da930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077a25b640 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0da930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077a25b640 1 -- 172.21.15.120:0/2959175086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 msgr2=0x55581a0df440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077a25b640 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0df440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.419 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.417+0000 7f077a25b640 1 -- 172.21.15.120:0/2959175086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581a0dbc00 con 0x55581a0d3ee0 2024-09-10T23:09:49.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.419+0000 7f077a25b640 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0770006010 tx=0x7f0770005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bdc753efdbad818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.419+0000 7f075ffff640 1 -- 172.21.15.120:0/2959175086 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770035030 con 0x55581a0d3ee0 2024-09-10T23:09:49.420 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.419+0000 7f075ffff640 1 -- 172.21.15.120:0/2959175086 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077004c070 con 0x55581a0d3ee0 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 msgr2=0x55581a0d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0770006010 tx=0x7f0770005380 comp rx=0 tx=0).stop 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 shutdown_connections 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0df440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 --2- 172.21.15.120:0/2959175086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0d3ee0 0x55581a0d3220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.422 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.421+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 >> 172.21.15.120:0/2959175086 conn(0x55581a0d1d10 msgr2=0x55581a0ded30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.422+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 shutdown_connections 2024-09-10T23:09:49.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.422+0000 7f077e894740 1 -- 172.21.15.120:0/2959175086 wait complete. 2024-09-10T23:09:49.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.422+0000 7f077e894740 1 -- start start 2024-09-10T23:09:49.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.422+0000 7f077e894740 1 Processor -- start 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0f04e0 0x55581a0e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07741af1c0 con 0x55581a0d4800 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f07741b0630 con 0x55581a0e0790 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f07741af890 con 0x55581a0f04e0 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077e894740 10 client.0 _ll_register_callbacks cb 0x7f07640e9860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077a25b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59222/0 (socket says 172.21.15.120:59222) 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f077a25b640 1 -- 172.21.15.120:0/2145997387 learned_addr learned my addr 172.21.15.120:0/2145997387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:49.425 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.423+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 -- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0f04e0 msgr2=0x55581a0e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0f04e0 0x55581a0e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 -- 172.21.15.120:0/2145997387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 msgr2=0x55581a0e8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 -- 172.21.15.120:0/2145997387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581a0e6080 con 0x55581a0e0790 2024-09-10T23:09:49.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.424+0000 7f0779a5a640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e9070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f07640e3a30 tx=0x7f0764009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6be1f17fce452a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:49.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.426+0000 7f075dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:09:49.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.426+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640ac470 con 0x55581a0e0790 2024-09-10T23:09:49.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.426+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55581a045c90 con 0x55581a0e0790 2024-09-10T23:09:49.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.426+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f07640ab900 con 0x55581a0e0790 2024-09-10T23:09:49.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.429+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07640abe10 con 0x55581a0e0790 2024-09-10T23:09:49.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 10 client.28088 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:49.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55581a166670 con 0x55581a0e0790 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55581a045c90 con 0x55581a0e0790 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f07640a7ff0 con 0x55581a0e0790 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 10 client.28088 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 10 client.28088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 20 client.28088 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a166da0 0x55581a1691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 4 client.28088 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.431+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555819fbb990 con 0x55581a166da0 2024-09-10T23:09:49.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.432+0000 7f077a25b640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a166da0 0x55581a1691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:49.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.439+0000 7f077a25b640 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a166da0 0x55581a1691c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13c627dc1de077a server_cookie=443ad157522154bb in_seq=0 out_seq=0 2024-09-10T23:09:49.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.439+0000 7f075dffb640 10 client.28088 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:49.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.443+0000 7f075dffb640 1 -- 172.21.15.120:0/2145997387 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555819fbb990 con 0x55581a166da0 2024-09-10T23:09:49.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.443+0000 7f075dffb640 10 client.28088 handle_command_reply: tid=0 2024-09-10T23:09:49.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.443+0000 7f077e894740 1 client.28088 shutdown 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 msgr2=0x55581a0e9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e9070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f07640e3a30 tx=0x7f0764009310 comp rx=0 tx=0).stop 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 shutdown_connections 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55581a166da0 0x55581a1691c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55581a0f04e0 0x55581a0e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55581a0e0790 0x55581a0e9070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 --2- 172.21.15.120:0/2145997387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55581a0d4800 0x55581a0e8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 >> 172.21.15.120:0/2145997387 conn(0x55581a0d1d10 msgr2=0x55581a0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f075dffb640 0 client.28088 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:49.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 shutdown_connections 2024-09-10T23:09:49.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.444+0000 7f077e894740 1 -- 172.21.15.120:0/2145997387 wait complete. 2024-09-10T23:09:49.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.445+0000 7f077e894740 20 client.28088 trim_cache size 0 max 16384 2024-09-10T23:09:49.448 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:09:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "513af67f-ac28-4e63-863d-e11a2008f95c", 2024-09-10T23:09:49.448 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:09:49.448 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:49.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.448+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 msgr2=0x7f07600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.448+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 0x7f07600b7820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f076c009fd0 tx=0x7f076c009500 comp rx=0 tx=0).stop 2024-09-10T23:09:49.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.448+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 msgr2=0x7f077410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:49.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.449+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9f0 tx=0x7f076400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 shutdown_connections 2024-09-10T23:09:49.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f07600b5360 0x7f07600b7820 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0774108020 0x7f077410fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07741076a0 0x7f077410f540 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 --2- 172.21.15.120:0/2452575304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0774101c90 0x7f077410f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:49.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.450+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 >> 172.21.15.120:0/2452575304 conn(0x7f07740fd700 msgr2=0x7f07740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:49.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.451+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 shutdown_connections 2024-09-10T23:09:49.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:49.451+0000 7f077c4bf640 1 -- 172.21.15.120:0/2452575304 wait complete. 2024-09-10T23:09:49.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117604 21.97 MB/sec execute 164 sec latency 366.977 ms 2024-09-10T23:09:50.164 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:50 smithi120 ceph-mon[31247]: pgmap v868: 97 pgs: 97 active+clean; 238 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-09-10T23:09:50.192 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:09:50.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:50 smithi176 ceph-mon[34982]: pgmap v868: 97 pgs: 97 active+clean; 238 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-09-10T23:09:50.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:50 smithi153 ceph-mon[34951]: pgmap v868: 97 pgs: 97 active+clean; 238 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-09-10T23:09:50.510 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:09:50.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117699 21.89 MB/sec execute 165 sec latency 306.451 ms 2024-09-10T23:09:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.051+0000 7fc22a24a640 1 -- 172.21.15.120:0/1392357512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 msgr2=0x7fc224108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.051+0000 7fc22a24a640 1 --2- 172.21.15.120:0/1392357512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224108bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc218009900 tx=0x7fc21802a410 comp rx=0 tx=0).stop 2024-09-10T23:09:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 -- 172.21.15.120:0/1392357512 shutdown_connections 2024-09-10T23:09:51.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 --2- 172.21.15.120:0/1392357512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224109130 0x7fc22410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 --2- 172.21.15.120:0/1392357512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224108bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 --2- 172.21.15.120:0/1392357512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224102da0 0x7fc224103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 -- 172.21.15.120:0/1392357512 >> 172.21.15.120:0/1392357512 conn(0x7fc2240fe850 msgr2=0x7fc224100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 -- 172.21.15.120:0/1392357512 shutdown_connections 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 -- 172.21.15.120:0/1392357512 wait complete. 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.052+0000 7fc22a24a640 1 Processor -- start 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 -- start start 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 0x7fc224197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224109130 0x7fc22419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc224110c00 con 0x7fc224102da0 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc224110a80 con 0x7fc2241087b0 2024-09-10T23:09:51.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc22a24a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc224110d80 con 0x7fc224109130 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 0x7fc224197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 0x7fc224197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59244/0 (socket says 172.21.15.120:59244) 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc2237fe640 1 -- 172.21.15.120:0/378381448 learned_addr learned my addr 172.21.15.120:0/378381448 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc223fff640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224109130 0x7fc22419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 -- 172.21.15.120:0/378381448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224109130 msgr2=0x7fc22419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224109130 0x7fc22419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 -- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 msgr2=0x7fc224197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 0x7fc224197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 -- 172.21.15.120:0/378381448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22419d0e0 con 0x7fc2241087b0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.053+0000 7fc222ffd640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224197570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc21802a930 tx=0x7fc218031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.054+0000 7fc220ff9640 1 -- 172.21.15.120:0/378381448 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21803f030 con 0x7fc2241087b0 2024-09-10T23:09:51.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.054+0000 7fc220ff9640 1 -- 172.21.15.120:0/378381448 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc218004490 con 0x7fc2241087b0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.054+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22419d370 con 0x7fc2241087b0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.054+0000 7fc220ff9640 1 -- 172.21.15.120:0/378381448 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218035070 con 0x7fc2241087b0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.054+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2241a3a90 con 0x7fc2241087b0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.056+0000 7fc22c61f740 1 Processor -- start 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.056+0000 7fc22c61f740 1 -- start start 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.056+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.056+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa3e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa40d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc22c61f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c01f9a8e50 con 0x55c01fa320a0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc22c61f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c01f998280 con 0x55c01fa3e950 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc22c61f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c01f99c090 con 0x55c01fa329c0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc220ff9640 1 -- 172.21.15.120:0/378381448 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc21803ba30 con 0x7fc2241087b0 2024-09-10T23:09:51.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59256/0 (socket says 172.21.15.120:59256) 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 -- 172.21.15.120:0/796689554 learned_addr learned my addr 172.21.15.120:0/796689554 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc2237fe640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa3e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc222ffd640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa40d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 -- 172.21.15.120:0/796689554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 msgr2=0x55c01fa3e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa3e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 -- 172.21.15.120:0/796689554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 msgr2=0x55c01fa40d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa40d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 -- 172.21.15.120:0/796689554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01f6f6650 con 0x55c01fa320a0 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc222ffd640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa40d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc223fff640 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009fd0 tx=0x7fc21c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8644959ee865f23b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.057+0000 7fc209ffb640 1 -- 172.21.15.120:0/796689554 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c004d20 con 0x55c01fa320a0 2024-09-10T23:09:51.083 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc209ffb640 1 -- 172.21.15.120:0/796689554 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c004050 con 0x55c01fa320a0 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc209ffb640 1 -- 172.21.15.120:0/796689554 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c005240 con 0x55c01fa320a0 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc220ff9640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 0x7fc20c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc2237fe640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 0x7fc20c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc220ff9640 1 -- 172.21.15.120:0/378381448 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc218105fa0 con 0x7fc2241087b0 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 -- 172.21.15.120:0/796689554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 msgr2=0x55c01fa32480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc2237fe640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 0x7fc20c0b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55c01fa30db0 tx=0x7fc214005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009fd0 tx=0x7fc21c0118d0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 -- 172.21.15.120:0/796689554 shutdown_connections 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa40d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa3e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 --2- 172.21.15.120:0/796689554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa32480 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.084 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.058+0000 7fc22c61f740 1 -- 172.21.15.120:0/796689554 >> 172.21.15.120:0/796689554 conn(0x55c01fa2fed0 msgr2=0x55c01fa31070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.059+0000 7fc22c61f740 1 -- 172.21.15.120:0/796689554 shutdown_connections 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.059+0000 7fc22c61f740 1 -- 172.21.15.120:0/796689554 wait complete. 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.059+0000 7fc22c61f740 1 -- start start 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.059+0000 7fc22c61f740 1 Processor -- start 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa320a0 0x55c01fa3e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 0x55c01fa3b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c01f7f9700 con 0x55c01fa329c0 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c01f933110 con 0x55c01fa3e950 2024-09-10T23:09:51.085 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c01fa41620 con 0x55c01fa320a0 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc22c61f740 10 client.0 _ll_register_callbacks cb 0x55c01f94c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc223fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 0x55c01fa3b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc223fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50140/0 (socket says 172.21.15.120:50140) 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa320a0 0x55c01fa3e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc223fff640 1 -- 172.21.15.120:0/2923929378 learned_addr learned my addr 172.21.15.120:0/2923929378 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 -- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa320a0 msgr2=0x55c01fa3e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa320a0 0x55c01fa3e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 -- 172.21.15.120:0/2923929378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 msgr2=0x55c01fa3bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 -- 172.21.15.120:0/2923929378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01f960c40 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc223fff640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.060+0000 7fc222ffd640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 0x55c01fa3b620 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc218103bc0 tx=0x7fc2180c8840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a462d85e994d9ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218049040 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc21c004050 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc2180c8b50 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc218049040 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 10 client.18861 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c01fa4ce90 con 0x55c01fa329c0 2024-09-10T23:09:51.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c004050 con 0x55c01fa329c0 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc2181080d0 con 0x55c01fa329c0 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 10 client.18861 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 10 client.18861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 20 client.18861 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fa50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 4 client.18861 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.061+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c01f9aada0 con 0x55c01fa4e280 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.062+0000 7fc2237fe640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fa50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.062+0000 7fc202ffd640 10 client.18861 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:51.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.062+0000 7fc2237fe640 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fa50670 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898ea393c0f21bc7 server_cookie=d4b045a72e9c4393 in_seq=0 out_seq=0 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.064+0000 7fc202ffd640 1 -- 172.21.15.120:0/2923929378 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c01f9aada0 con 0x55c01fa4e280 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.064+0000 7fc202ffd640 10 client.18861 handle_command_reply: tid=0 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.064+0000 7fc22c61f740 1 client.18861 shutdown 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 msgr2=0x55c01fa3b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 0x55c01fa3b620 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc218103bc0 tx=0x7fc2180c8840 comp rx=0 tx=0).stop 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 shutdown_connections 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fa50670 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc202ffd640 0 client.18861 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa3bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa329c0 0x55c01fa3b620 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 --2- 172.21.15.120:0/2923929378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa320a0 0x55c01fa3e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 >> 172.21.15.120:0/2923929378 conn(0x55c01fa2fed0 msgr2=0x55c01fa35a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 shutdown_connections 2024-09-10T23:09:51.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 1 -- 172.21.15.120:0/2923929378 wait complete. 2024-09-10T23:09:51.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.065+0000 7fc22c61f740 20 client.18861 trim_cache size 0 max 16384 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.084+0000 7fc22c61f740 1 Processor -- start 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 -- start start 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa38d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa55ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241a9ff0 con 0x55c01fa320a0 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241ac8d0 con 0x55c01fa3e950 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc22c61f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241aa6c0 con 0x55c01fa329c0 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc222ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa38d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59274/0 (socket says 172.21.15.120:59274) 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc222ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa38d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47966/0 (socket says 172.21.15.120:47966) 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 -- 172.21.15.120:0/870017069 learned_addr learned my addr 172.21.15.120:0/870017069 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc223fff640 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa55ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 -- 172.21.15.120:0/870017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 msgr2=0x55c01fa38d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa38d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 -- 172.21.15.120:0/870017069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 msgr2=0x55c01fa55ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa55ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.085+0000 7fc2237fe640 1 -- 172.21.15.120:0/870017069 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01fa36350 con 0x55c01fa320a0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc2237fe640 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc21400ffa0 tx=0x7fc21401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2666f58dc96b0e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc2097fa640 1 -- 172.21.15.120:0/870017069 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21402a4b0 con 0x55c01fa320a0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc2097fa640 1 -- 172.21.15.120:0/870017069 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc214022050 con 0x55c01fa320a0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc2097fa640 1 -- 172.21.15.120:0/870017069 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214032840 con 0x55c01fa320a0 2024-09-10T23:09:51.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc22c61f740 1 -- 172.21.15.120:0/870017069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 msgr2=0x55c01fa38830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc22c61f740 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc21400ffa0 tx=0x7fc21401c000 comp rx=0 tx=0).stop 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc22c61f740 1 -- 172.21.15.120:0/870017069 shutdown_connections 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc22c61f740 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa3e950 0x55c01fa55ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.086+0000 7fc22c61f740 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa329c0 0x55c01fa38d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 --2- 172.21.15.120:0/870017069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa38830 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- 172.21.15.120:0/870017069 >> 172.21.15.120:0/870017069 conn(0x55c01fa2fed0 msgr2=0x55c01fa2ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- 172.21.15.120:0/870017069 shutdown_connections 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- 172.21.15.120:0/870017069 wait complete. 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- start start 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 Processor -- start 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 0x55c01fa437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa3e950 0x55c01fab40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241aad90 con 0x55c01fa320a0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241ac200 con 0x55c01fa329c0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc2241ab460 con 0x55c01fa3e950 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.087+0000 7fc22c61f740 10 client.0 _ll_register_callbacks cb 0x7fc21803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59280/0 (socket says 172.21.15.120:59280) 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 -- 172.21.15.120:0/436565232 learned_addr learned my addr 172.21.15.120:0/436565232 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc223fff640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa3e950 0x55c01fab40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc222ffd640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 0x55c01fa437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 -- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa3e950 msgr2=0x55c01fab40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa3e950 0x55c01fab40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 -- 172.21.15.120:0/436565232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 msgr2=0x55c01fa437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:51.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 0x55c01fa437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 -- 172.21.15.120:0/436565232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01fab47e0 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc222ffd640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 0x55c01fa437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc2237fe640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc21400fa90 tx=0x7fc214005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ac4819a2b6138 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc202ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21400f080 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fc21400f220 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc214022050 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc21400f420 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc22c61f740 10 client.18867 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.088+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c01fabbbc0 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc214022050 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fc21400c070 con 0x55c01fa320a0 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 10 client.18867 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 10 client.18867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:51.099 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 20 client.18867 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fabe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 4 client.18867 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c01f9a6850 con 0x55c01fa4e280 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc222ffd640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fabe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc222ffd640 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fabe270 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ca5ae259018b92 server_cookie=173fdcb36665ffe3 in_seq=0 out_seq=0 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.089+0000 7fc202ffd640 10 client.18867 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc202ffd640 1 -- 172.21.15.120:0/436565232 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8814 (crc 0 0 0) 0x55c01f9a6850 con 0x55c01fa4e280 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc202ffd640 10 client.18867 handle_command_reply: tid=0 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 client.18867 shutdown 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 msgr2=0x55c01fa43200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc21400fa90 tx=0x7fc214005760 comp rx=0 tx=0).stop 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 shutdown_connections 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c01fa4e280 0x55c01fabe270 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c01fa3e950 0x55c01fab40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c01fa329c0 0x55c01fa437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 --2- 172.21.15.120:0/436565232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c01fa320a0 0x55c01fa43200 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 >> 172.21.15.120:0/436565232 conn(0x55c01fa2fed0 msgr2=0x55c01fa3cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.093+0000 7fc202ffd640 0 client.18867 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 shutdown_connections 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22c61f740 1 -- 172.21.15.120:0/436565232 wait complete. 2024-09-10T23:09:51.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22c61f740 20 client.18867 trim_cache size 0 max 16384 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (231 inodes in the stack)", 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "513af67f-ac28-4e63-863d-e11a2008f95c", 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:09:51.102 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:09:51.103 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.105 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.106 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:09:51.107 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.108 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.109 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:09:51.110 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.111 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.112 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.113 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.114 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.115 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:09:51.116 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.117 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:09:51.119 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:09:51.120 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 msgr2=0x7fc20c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 0x7fc20c0b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55c01fa30db0 tx=0x7fc214005f90 comp rx=0 tx=0).stop 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 msgr2=0x7fc224197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:51.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224197570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc21802a930 tx=0x7fc218031a20 comp rx=0 tx=0).stop 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 shutdown_connections 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fc20c0b5360 0x7fc20c0b7820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc224109130 0x7fc22419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2241087b0 0x7fc224197570 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 --2- 172.21.15.120:0/378381448 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc224102da0 0x7fc224197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 >> 172.21.15.120:0/378381448 conn(0x7fc2240fe850 msgr2=0x7fc224100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.094+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 shutdown_connections 2024-09-10T23:09:51.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:09:51.095+0000 7fc22a24a640 1 -- 172.21.15.120:0/378381448 wait complete. 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:51.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:51 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:09:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:09:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:09:51.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:51.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:51.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:51 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:09:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:09:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:09:51.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:09:51.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:09:51.450 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:51 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:09:51.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 117783 21.77 MB/sec execute 166 sec latency 315.436 ms 2024-09-10T23:09:52.248 INFO:tasks.cephfs.filesystem:scrub status for tag:513af67f-ac28-4e63-863d-e11a2008f95c - {'path': '/', 'tag': '513af67f-ac28-4e63-863d-e11a2008f95c', 'options': 'recursive,force'} 2024-09-10T23:09:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 604 op/s 2024-09-10T23:09:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T23:09:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T23:09:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T23:09:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T23:09:52.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T23:09:52.324 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:52 smithi176 ceph-mon[34982]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 604 op/s 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T23:09:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:52 smithi120 ceph-mon[31247]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T23:09:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: pgmap v869: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 604 op/s 2024-09-10T23:09:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi120 to 176.0M 2024-09-10T23:09:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi153 to 176.0M 2024-09-10T23:09:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi120 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T23:09:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi153 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T23:09:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Adjusting osd_memory_target on smithi176 to 176.0M 2024-09-10T23:09:52.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:52 smithi153 ceph-mon[34951]: Unable to set osd_memory_target on smithi176 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-10T23:09:52.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 118195 21.73 MB/sec execute 167 sec latency 76.880 ms 2024-09-10T23:09:53.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:53 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:53.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:53 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:53.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:53 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:09:53.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 118562 21.69 MB/sec execute 168 sec latency 76.561 ms 2024-09-10T23:09:54.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:54 smithi176 ceph-mon[34982]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-09-10T23:09:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:54 smithi120 ceph-mon[31247]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-09-10T23:09:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:54 smithi153 ceph-mon[34951]: pgmap v870: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.9 MiB/s wr, 589 op/s 2024-09-10T23:09:54.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 118973 21.70 MB/sec execute 169 sec latency 232.846 ms 2024-09-10T23:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 -- 172.21.15.120:0/1246554662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 msgr2=0x7f4778109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 --2- 172.21.15.120:0/1246554662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778109d40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4770009cd0 tx=0x7f477002a8e0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 -- 172.21.15.120:0/1246554662 shutdown_connections 2024-09-10T23:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 --2- 172.21.15.120:0/1246554662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778109d40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 --2- 172.21.15.120:0/1246554662 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 0x7f4778102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 --2- 172.21.15.120:0/1246554662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 0x7f4778101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 -- 172.21.15.120:0/1246554662 >> 172.21.15.120:0/1246554662 conn(0x7f47780fd150 msgr2=0x7f47780ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 -- 172.21.15.120:0/1246554662 shutdown_connections 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.946+0000 7f477d553640 1 -- 172.21.15.120:0/1246554662 wait complete. 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 Processor -- start 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 -- start start 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 0x7f4778197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 0x7f47781982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f477810d500 con 0x7f47781033c0 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f477810d380 con 0x7f47781018b0 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f477d553640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f477810d680 con 0x7f4778102a00 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60010/0 (socket says 172.21.15.120:60010) 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 -- 172.21.15.120:0/983928743 learned_addr learned my addr 172.21.15.120:0/983928743 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f4776ffd640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 0x7f4778197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47767fc640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 0x7f47781982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 -- 172.21.15.120:0/983928743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 msgr2=0x7f47781982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 0x7f47781982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 -- 172.21.15.120:0/983928743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 msgr2=0x7f4778197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 0x7f4778197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 -- 172.21.15.120:0/983928743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781997c0 con 0x7f47781033c0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.947+0000 7f47777fe640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4770033040 tx=0x7f4770003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.948+0000 7f475ffff640 1 -- 172.21.15.120:0/983928743 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004240 con 0x7f47781033c0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.948+0000 7f475ffff640 1 -- 172.21.15.120:0/983928743 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47700305a0 con 0x7f47781033c0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.948+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4778199a50 con 0x7f47781033c0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.948+0000 7f475ffff640 1 -- 172.21.15.120:0/983928743 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700466a0 con 0x7f47781033c0 2024-09-10T23:09:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.948+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a0180 con 0x7f47781033c0 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.950+0000 7f47803f4740 1 Processor -- start 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.950+0000 7f475ffff640 1 -- 172.21.15.120:0/983928743 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f477004e070 con 0x7f47781033c0 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.950+0000 7f47803f4740 1 -- start start 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3c0e6f0 con 0x55bdf3c9ea70 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3bfdb20 con 0x55bdf3ca3de0 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47803f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3c01930 con 0x55bdf3ca13a0 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60014/0 (socket says 172.21.15.120:60014) 2024-09-10T23:09:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 -- 172.21.15.120:0/2004202530 learned_addr learned my addr 172.21.15.120:0/2004202530 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f4776ffd640 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 -- 172.21.15.120:0/2004202530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 msgr2=0x55bdf3ca3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 -- 172.21.15.120:0/2004202530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 msgr2=0x55bdf3ca62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 -- 172.21.15.120:0/2004202530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf3bc3530 con 0x55bdf3c9ea70 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f475ffff640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 0x7f47600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f47767fc640 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f476c0125f0 tx=0x7f476c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbef8c97e84cdec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f4776ffd640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 0x7f47600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f475ffff640 1 -- 172.21.15.120:0/983928743 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f477002e030 con 0x7f47781033c0 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f475d7fa640 1 -- 172.21.15.120:0/2004202530 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c005020 con 0x55bdf3c9ea70 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.951+0000 7f475d7fa640 1 -- 172.21.15.120:0/2004202530 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476c005670 con 0x55bdf3c9ea70 2024-09-10T23:09:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f475d7fa640 1 -- 172.21.15.120:0/2004202530 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c010460 con 0x55bdf3c9ea70 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 -- 172.21.15.120:0/2004202530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 msgr2=0x55bdf3ca0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f476c0125f0 tx=0x7f476c00ba40 comp rx=0 tx=0).stop 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f4776ffd640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 0x7f47600b7820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4768009da0 tx=0x7f4768005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 -- 172.21.15.120:0/2004202530 shutdown_connections 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 --2- 172.21.15.120:0/2004202530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca0e60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 -- 172.21.15.120:0/2004202530 >> 172.21.15.120:0/2004202530 conn(0x55bdf3c979b0 msgr2=0x55bdf3c98740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 -- 172.21.15.120:0/2004202530 shutdown_connections 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.952+0000 7f47803f4740 1 -- 172.21.15.120:0/2004202530 wait complete. 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 -- start start 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 Processor -- start 2024-09-10T23:09:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3b35ab0 con 0x55bdf3ca3de0 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3ba3730 con 0x55bdf3ca13a0 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bdf3ca6dc0 con 0x55bdf3c9ea70 2024-09-10T23:09:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47803f4740 10 client.0 _ll_register_callbacks cb 0x55bdf3bf76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60024/0 (socket says 172.21.15.120:60024) 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 -- 172.21.15.120:0/291013018 learned_addr learned my addr 172.21.15.120:0/291013018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47767fc640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f4776ffd640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 -- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 msgr2=0x55bdf3ca8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 -- 172.21.15.120:0/291013018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca13a0 msgr2=0x55bdf3ca8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 -- 172.21.15.120:0/291013018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf395f790 con 0x55bdf3ca3de0 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f47777fe640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f47700c9c30 tx=0x7f47700ccfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d5ba737633bb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.953+0000 7f475aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770104e60 con 0x55bdf3ca3de0 2024-09-10T23:09:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f47700cc8c0 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476c005670 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47700cd170 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 10 client.18879 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdf39d8620 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476c005670 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4770030070 con 0x55bdf3ca3de0 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 10 client.18879 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 10 client.18879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 20 client.18879 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3cb3940 0x55bdf3cb5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 4 client.18879 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf3c10640 con 0x55bdf3cb3940 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.954+0000 7f4776ffd640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3cb3940 0x55bdf3cb5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.955+0000 7f475aff5640 10 client.18879 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.955+0000 7f4776ffd640 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3cb3940 0x55bdf3cb5d80 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d65d9319d9fbe9 server_cookie=b3a846c88adfd4ee in_seq=0 out_seq=0 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f475aff5640 1 -- 172.21.15.120:0/291013018 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bdf3c10640 con 0x55bdf3cb3940 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f475aff5640 10 client.18879 handle_command_reply: tid=0 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 client.18879 shutdown 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 msgr2=0x55bdf3ca92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f47700c9c30 tx=0x7f47700ccfb0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 shutdown_connections 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3cb3940 0x55bdf3cb5d80 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca92e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca13a0 0x55bdf3ca8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f475aff5640 0 client.18879 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 --2- 172.21.15.120:0/291013018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3ca8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 >> 172.21.15.120:0/291013018 conn(0x55bdf3c979b0 msgr2=0x55bdf3c96bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 shutdown_connections 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 1 -- 172.21.15.120:0/291013018 wait complete. 2024-09-10T23:09:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.958+0000 7f47803f4740 20 client.18879 trim_cache size 0 max 16384 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 Processor -- start 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 -- start start 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3c9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a5bd0 con 0x55bdf3c9ea70 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a8450 con 0x55bdf3ca3de0 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f47803f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a6240 con 0x55bdf3ca13a0 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.973+0000 7f4776ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60030/0 (socket says 172.21.15.120:60030) 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 -- 172.21.15.120:0/1444796475 learned_addr learned my addr 172.21.15.120:0/1444796475 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f47767fc640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3c9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f47777fe640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 -- 172.21.15.120:0/1444796475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 msgr2=0x55bdf3c9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3c9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 -- 172.21.15.120:0/1444796475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 msgr2=0x55bdf3ca72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 -- 172.21.15.120:0/1444796475 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf3ca3b00 con 0x55bdf3c9ea70 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f47767fc640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3c9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f4776ffd640 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x55bdf3cb6630 tx=0x7f476801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34411e7a6dbb76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f475cff9640 1 -- 172.21.15.120:0/1444796475 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476802b420 con 0x55bdf3c9ea70 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f475cff9640 1 -- 172.21.15.120:0/1444796475 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476802ba70 con 0x55bdf3c9ea70 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.974+0000 7f475cff9640 1 -- 172.21.15.120:0/1444796475 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768032980 con 0x55bdf3c9ea70 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- 172.21.15.120:0/1444796475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 msgr2=0x55bdf3c9bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x55bdf3cb6630 tx=0x7f476801b000 comp rx=0 tx=0).stop 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- 172.21.15.120:0/1444796475 shutdown_connections 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3ca72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3ca13a0 0x55bdf3c9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 --2- 172.21.15.120:0/1444796475 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3c9ea70 0x55bdf3c9bc60 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- 172.21.15.120:0/1444796475 >> 172.21.15.120:0/1444796475 conn(0x55bdf3c979b0 msgr2=0x55bdf3c92af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- 172.21.15.120:0/1444796475 shutdown_connections 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- 172.21.15.120:0/1444796475 wait complete. 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 -- start start 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 Processor -- start 2024-09-10T23:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.975+0000 7f47803f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3cafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3caf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a6910 con 0x55bdf3ca13a0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a7d80 con 0x55bdf3ca3de0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f47781a6fe0 con 0x55bdf3c9ea70 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 10 client.0 _ll_register_callbacks cb 0x7f4770009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:60046/0 (socket says 172.21.15.120:60046) 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 -- 172.21.15.120:0/950411954 learned_addr learned my addr 172.21.15.120:0/950411954 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f4776ffd640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3cafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47777fe640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3caf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 -- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 msgr2=0x55bdf3cafd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3cafd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 -- 172.21.15.120:0/950411954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 msgr2=0x55bdf3caf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3caf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 -- 172.21.15.120:0/950411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf3cad590 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47767fc640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f476c0125f0 tx=0x7f476c012370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85bfac35bec57e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f475aff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c004c30 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f476c004dd0 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476802ba70 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f476c005000 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 10 client.18885 fetch_fsmap learned FSMap version 26 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.976+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdf3d20b30 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476802ba70 con 0x55bdf3ca13a0 2024-09-10T23:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f476c010950 con 0x55bdf3ca13a0 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 10 client.18885 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 10 client.18885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 20 client.18885 populate_metadata read hostname 'smithi120' 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3d21370 0x55bdf3d23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 4 client.18885 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdf3c0c0f0 con 0x55bdf3d21370 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f4776ffd640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3d21370 0x55bdf3d23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f475aff5640 10 client.18885 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:54.986 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:54.977+0000 7f4776ffd640 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3d21370 0x55bdf3d23790 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d754e0f2adaef43 server_cookie=6d24eb9f714072ce in_seq=0 out_seq=0 2024-09-10T23:09:55.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 119109 21.60 MB/sec execute 170 sec latency 491.444 ms 2024-09-10T23:09:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f475aff5640 1 -- 172.21.15.120:0/950411954 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdf3c0c0f0 con 0x55bdf3d21370 2024-09-10T23:09:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f475aff5640 10 client.18885 handle_command_reply: tid=0 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 client.18885 shutdown 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 msgr2=0x55bdf3caead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f476c0125f0 tx=0x7f476c012370 comp rx=0 tx=0).stop 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 shutdown_connections 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bdf3d21370 0x55bdf3d23790 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f475aff5640 0 client.18885 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdf3ca3de0 0x55bdf3caf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bdf3ca13a0 0x55bdf3caead0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 --2- 172.21.15.120:0/950411954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bdf3c9ea70 0x55bdf3cafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 >> 172.21.15.120:0/950411954 conn(0x55bdf3c979b0 msgr2=0x55bdf3ca72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 shutdown_connections 2024-09-10T23:09:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 1 -- 172.21.15.120:0/950411954 wait complete. 2024-09-10T23:09:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.974+0000 7f47803f4740 20 client.18885 trim_cache size 0 max 16384 2024-09-10T23:09:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:09:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:09:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:09:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 msgr2=0x7f47600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 0x7f47600b7820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4768009da0 tx=0x7f4768005f90 comp rx=0 tx=0).stop 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 msgr2=0x7f4778198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4770033040 tx=0x7f4770003c00 comp rx=0 tx=0).stop 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 shutdown_connections 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f47600b5360 0x7f47600b7820 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f47781033c0 0x7f4778198f00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4778102a00 0x7f47781982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 --2- 172.21.15.120:0/983928743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47781018b0 0x7f4778197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:09:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 >> 172.21.15.120:0/983928743 conn(0x7f47780fd150 msgr2=0x7f4778105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:09:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 shutdown_connections 2024-09-10T23:09:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:09:55.975+0000 7f477d553640 1 -- 172.21.15.120:0/983928743 wait complete. 2024-09-10T23:09:56.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:56 smithi176 ceph-mon[34982]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-09-10T23:09:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:56 smithi120 ceph-mon[31247]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-09-10T23:09:56.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:56 smithi153 ceph-mon[34951]: pgmap v871: 97 pgs: 97 active+clean; 245 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-09-10T23:09:56.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 119671 21.68 MB/sec execute 171 sec latency 212.896 ms 2024-09-10T23:09:57.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:57 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:09:57.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:57 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:09:57.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:57 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:09:57.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 120239 21.77 MB/sec execute 172 sec latency 86.501 ms 2024-09-10T23:09:58.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:09:58 smithi176 ceph-mon[34982]: pgmap v872: 97 pgs: 97 active+clean; 253 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-09-10T23:09:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:09:58 smithi120 ceph-mon[31247]: pgmap v872: 97 pgs: 97 active+clean; 253 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-09-10T23:09:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:09:58 smithi153 ceph-mon[34951]: pgmap v872: 97 pgs: 97 active+clean; 253 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 719 op/s 2024-09-10T23:09:58.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 120684 21.80 MB/sec execute 173 sec latency 63.345 ms 2024-09-10T23:09:59.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 120944 21.75 MB/sec execute 174 sec latency 234.585 ms 2024-09-10T23:10:00.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:00 smithi176 ceph-mon[34982]: pgmap v873: 97 pgs: 97 active+clean; 250 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-10T23:10:00.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:00 smithi176 ceph-mon[34982]: overall HEALTH_OK 2024-09-10T23:10:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:00 smithi120 ceph-mon[31247]: pgmap v873: 97 pgs: 97 active+clean; 250 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-10T23:10:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:00 smithi120 ceph-mon[31247]: overall HEALTH_OK 2024-09-10T23:10:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:00 smithi153 ceph-mon[34951]: pgmap v873: 97 pgs: 97 active+clean; 250 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 543 op/s 2024-09-10T23:10:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:00 smithi153 ceph-mon[34951]: overall HEALTH_OK 2024-09-10T23:10:00.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 121218 21.69 MB/sec execute 175 sec latency 116.117 ms 2024-09-10T23:10:01.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 121637 21.68 MB/sec execute 176 sec latency 59.998 ms 2024-09-10T23:10:02.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:02 smithi176 ceph-mon[34982]: pgmap v874: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 597 op/s 2024-09-10T23:10:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:02 smithi120 ceph-mon[31247]: pgmap v874: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 597 op/s 2024-09-10T23:10:02.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:02 smithi153 ceph-mon[34951]: pgmap v874: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 597 op/s 2024-09-10T23:10:02.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 122143 21.66 MB/sec execute 177 sec latency 79.256 ms 2024-09-10T23:10:03.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 122742 21.76 MB/sec execute 178 sec latency 57.752 ms 2024-09-10T23:10:04.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:04 smithi120 ceph-mon[31247]: pgmap v875: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 625 op/s 2024-09-10T23:10:04.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:04 smithi153 ceph-mon[34951]: pgmap v875: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 625 op/s 2024-09-10T23:10:04.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:04 smithi176 ceph-mon[34982]: pgmap v875: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.5 MiB/s wr, 625 op/s 2024-09-10T23:10:04.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 123330 21.82 MB/sec execute 179 sec latency 135.989 ms 2024-09-10T23:10:05.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 123722 21.83 MB/sec execute 180 sec latency 161.171 ms 2024-09-10T23:10:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:06 smithi120 ceph-mon[31247]: pgmap v876: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-09-10T23:10:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:06 smithi153 ceph-mon[34951]: pgmap v876: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-09-10T23:10:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:06 smithi176 ceph-mon[34982]: pgmap v876: 97 pgs: 97 active+clean; 238 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 562 op/s 2024-09-10T23:10:06.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 124173 21.86 MB/sec execute 181 sec latency 57.742 ms 2024-09-10T23:10:07.876 INFO:tasks.workunit.client.0.smithi120.stdout: 10 124500 21.85 MB/sec execute 182 sec latency 78.619 ms 2024-09-10T23:10:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:08 smithi120 ceph-mon[31247]: pgmap v877: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 692 op/s 2024-09-10T23:10:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:08 smithi153 ceph-mon[34951]: pgmap v877: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 692 op/s 2024-09-10T23:10:08.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:08 smithi176 ceph-mon[34982]: pgmap v877: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 692 op/s 2024-09-10T23:10:08.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 124901 21.86 MB/sec execute 183 sec latency 59.757 ms 2024-09-10T23:10:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4222879979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 msgr2=0x7f7ff4103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4222879979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4103f60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009930 tx=0x7f7fe002a420 comp rx=0 tx=0).stop 2024-09-10T23:10:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4222879979 shutdown_connections 2024-09-10T23:10:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4222879979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4222879979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4103f60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4222879979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 0x7f7ff4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4222879979 >> 172.21.15.120:0/4222879979 conn(0x7f7ff40fe250 msgr2=0x7f7ff4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4222879979 shutdown_connections 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.068+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4222879979 wait complete. 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 Processor -- start 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 -- start start 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 0x7f7ff4076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410e600 con 0x7f7ff4103ae0 2024-09-10T23:10:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410e480 con 0x7f7ff41044a0 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ffaeed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410e780 con 0x7f7ff4102990 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47026/0 (socket says 172.21.15.120:47026) 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 -- 172.21.15.120:0/4169945798 learned_addr learned my addr 172.21.15.120:0/4169945798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff948a640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 0x7f7ff4076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 -- 172.21.15.120:0/4169945798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 msgr2=0x7f7ff4076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 0x7f7ff4076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 -- 172.21.15.120:0/4169945798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 msgr2=0x7f7ff4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff3fff640 1 -- 172.21.15.120:0/4169945798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff40762f0 con 0x7f7ff4103ae0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.069+0000 7f7ff948a640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009a50 tx=0x7f7fe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ff1ffb640 1 -- 172.21.15.120:0/4169945798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe003f030 con 0x7f7ff4103ae0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ff1ffb640 1 -- 172.21.15.120:0/4169945798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe0031910 con 0x7f7ff4103ae0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff41a2e10 con 0x7f7ff4103ae0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ff1ffb640 1 -- 172.21.15.120:0/4169945798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0035070 con 0x7f7ff4103ae0 2024-09-10T23:10:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.070+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff41a3280 con 0x7f7ff4103ae0 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.072+0000 7f7ff1ffb640 1 -- 172.21.15.120:0/4169945798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7fe003ba60 con 0x7f7ff4103ae0 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 Processor -- start 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 -- start start 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8503860 0x561de8505c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8508bd0 0x561de850b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561de84756f0 con 0x561de8503860 2024-09-10T23:10:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561de8464b20 con 0x561de8508bd0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561de8468930 con 0x561de8506190 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff1ffb640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 0x7f7fe40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8508bd0 0x561de850b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8503860 0x561de8505c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46414/0 (socket says 172.21.15.120:46414) 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 -- 172.21.15.120:0/1294454752 learned_addr learned my addr 172.21.15.120:0/1294454752 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff948a640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 0x7f7fe40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff1ffb640 1 -- 172.21.15.120:0/4169945798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fe0106330 con 0x7f7ff4103ae0 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 -- 172.21.15.120:0/1294454752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8508bd0 msgr2=0x561de850b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8508bd0 0x561de850b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 -- 172.21.15.120:0/1294454752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8503860 msgr2=0x561de8505c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8503860 0x561de8505c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.073+0000 7f7ff8c89640 1 -- 172.21.15.120:0/1294454752 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de842a530 con 0x561de8506190 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.074+0000 7f7ff948a640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 0x7f7fe40b7820 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fec005fd0 tx=0x7f7fec011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.074+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009d20 tx=0x7f7fe8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d49dac3224de53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.074+0000 7f7fdeffd640 1 -- 172.21.15.120:0/1294454752 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe801e070 con 0x561de8506190 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.074+0000 7f7fdeffd640 1 -- 172.21.15.120:0/1294454752 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8019ae0 con 0x561de8506190 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.074+0000 7f7fdeffd640 1 -- 172.21.15.120:0/1294454752 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800ea10 con 0x561de8506190 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1294454752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 msgr2=0x561de8508580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009d20 tx=0x7f7fe8011810 comp rx=0 tx=0).stop 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1294454752 shutdown_connections 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8508bd0 0x561de850b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8508580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1294454752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8503860 0x561de8505c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1294454752 >> 172.21.15.120:0/1294454752 conn(0x561de84fe9b0 msgr2=0x561de84ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1294454752 shutdown_connections 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1294454752 wait complete. 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 -- start start 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.075+0000 7f7ffdcc7740 1 Processor -- start 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de850cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de850d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561de839cab0 con 0x561de8508bd0 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561de840a730 con 0x561de8506190 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561de850bbb0 con 0x561de8503860 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 10 client.0 _ll_register_callbacks cb 0x561de845e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff8c89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de850cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff8c89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de850cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46416/0 (socket says 172.21.15.120:46416) 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47044/0 (socket says 172.21.15.120:47044) 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff8c89640 1 -- 172.21.15.120:0/373286833 learned_addr learned my addr 172.21.15.120:0/373286833 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de850d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 -- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 msgr2=0x561de850cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de850cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 -- 172.21.15.120:0/373286833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 msgr2=0x561de850d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de850d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 -- 172.21.15.120:0/373286833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de81c6790 con 0x561de8508bd0 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de850d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ff948a640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0099e0 tx=0x7f7fec013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12112efc0402dd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec011220 con 0x561de8508bd0 2024-09-10T23:10:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fec004b20 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.076+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe8019ae0 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fec030070 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 10 client.18897 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561de823f620 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8019ae0 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7fec010430 con 0x561de8508bd0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 10 client.18897 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 20 client.18897 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8518810 0x561de851ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 4 client.18897 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de8477640 con 0x561de8518810 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8518810 0x561de851ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7fd3fff640 10 client.18897 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.077+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8518810 0x561de851ac70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8615805542e4eca6 server_cookie=15e2e336eddb2b00 in_seq=0 out_seq=0 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7fd3fff640 1 -- 172.21.15.120:0/373286833 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561de8477640 con 0x561de8518810 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7fd3fff640 10 client.18897 handle_command_reply: tid=0 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 client.18897 shutdown 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 msgr2=0x561de850e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0099e0 tx=0x7f7fec013070 comp rx=0 tx=0).stop 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 shutdown_connections 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8518810 0x561de851ac70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de850e130 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de850d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/373286833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de850cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 >> 172.21.15.120:0/373286833 conn(0x561de84fe9b0 msgr2=0x561de84fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7fd3fff640 0 client.18897 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 shutdown_connections 2024-09-10T23:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/373286833 wait complete. 2024-09-10T23:10:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.080+0000 7f7ffdcc7740 20 client.18897 trim_cache size 0 max 16384 2024-09-10T23:10:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 Processor -- start 2024-09-10T23:10:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 -- start start 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8503860 0x561de8502d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8503290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41a9f10 con 0x561de8508bd0 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41ac790 con 0x561de8503860 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41aa580 con 0x561de8506190 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47052/0 (socket says 172.21.15.120:47052) 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 -- 172.21.15.120:0/304199768 learned_addr learned my addr 172.21.15.120:0/304199768 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8503860 0x561de8502d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff3fff640 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8503290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 -- 172.21.15.120:0/304199768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 msgr2=0x561de8503290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8503290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 -- 172.21.15.120:0/304199768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8503860 msgr2=0x561de8502d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8503860 0x561de8502d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.095+0000 7f7ff948a640 1 -- 172.21.15.120:0/304199768 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de8505fc0 con 0x561de8508bd0 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ff948a640 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0108d0 tx=0x7f7fec003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b682d36db41ed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7fde7fc640 1 -- 172.21.15.120:0/304199768 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec03a070 con 0x561de8508bd0 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7fde7fc640 1 -- 172.21.15.120:0/304199768 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fec010630 con 0x561de8508bd0 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7fde7fc640 1 -- 172.21.15.120:0/304199768 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0110e0 con 0x561de8508bd0 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/304199768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 msgr2=0x561de8508fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0108d0 tx=0x7f7fec003f00 comp rx=0 tx=0).stop 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/304199768 shutdown_connections 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8508fb0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8506190 0x561de8503290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/304199768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8503860 0x561de8502d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.096+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/304199768 >> 172.21.15.120:0/304199768 conn(0x561de84fe9b0 msgr2=0x561de851a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/304199768 shutdown_connections 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/304199768 wait complete. 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- start start 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 Processor -- start 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de8511a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de85120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41aac50 con 0x561de8508bd0 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41ac0c0 con 0x561de8506190 2024-09-10T23:10:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41ab320 con 0x561de8503860 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ffdcc7740 10 client.0 _ll_register_callbacks cb 0x7f7fec003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ff948a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47068/0 (socket says 172.21.15.120:47068) 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.097+0000 7f7ff948a640 1 -- 172.21.15.120:0/1484148333 learned_addr learned my addr 172.21.15.120:0/1484148333 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 -- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 msgr2=0x561de8511a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de8511a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 -- 172.21.15.120:0/1484148333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 msgr2=0x561de85120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de85120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 -- 172.21.15.120:0/1484148333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de85133e0 con 0x561de8508bd0 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ff948a640 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7fec012fd0 tx=0x7f7fec004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3d6418f42da10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec03a070 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7fec010e60 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fec010630 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fec010a40 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ffdcc7740 10 client.18903 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561de8514750 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fec010630 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7fec035480 con 0x561de8508bd0 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ffdcc7740 10 client.18903 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.098+0000 7f7ffdcc7740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.099+0000 7f7ffdcc7740 20 client.18903 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.099+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8585730 0x561de8587b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.099+0000 7f7ffdcc7740 4 client.18903 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:10:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.099+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561de84730f0 con 0x561de8585730 2024-09-10T23:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.100+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8585730 0x561de8587b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.101+0000 7f7ff8c89640 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8585730 0x561de8587b20 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00953732869c098 server_cookie=cb31f65dcd867071 in_seq=0 out_seq=0 2024-09-10T23:10:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.101+0000 7f7fd3fff640 10 client.18903 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:09.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125080 21.77 MB/sec execute 184 sec latency 330.445 ms 2024-09-10T23:10:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7fd3fff640 1 -- 172.21.15.120:0/1484148333 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561de84730f0 con 0x561de8585730 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7fd3fff640 10 client.18903 handle_command_reply: tid=0 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 client.18903 shutdown 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 msgr2=0x561de8512bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7fec012fd0 tx=0x7f7fec004c70 comp rx=0 tx=0).stop 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 shutdown_connections 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561de8585730 0x561de8587b20 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561de8508bd0 0x561de8512bb0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561de8506190 0x561de85120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 --2- 172.21.15.120:0/1484148333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561de8503860 0x561de8511a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 >> 172.21.15.120:0/1484148333 conn(0x561de84fe9b0 msgr2=0x561de8508fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.949+0000 7f7fd3fff640 0 client.18903 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.950+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 shutdown_connections 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.950+0000 7f7ffdcc7740 1 -- 172.21.15.120:0/1484148333 wait complete. 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.950+0000 7f7ffdcc7740 20 client.18903 trim_cache size 0 max 16384 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:10:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:10:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 msgr2=0x7f7fe40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 0x7f7fe40b7820 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fec005fd0 tx=0x7f7fec011810 comp rx=0 tx=0).stop 2024-09-10T23:10:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 msgr2=0x7f7ff4075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009a50 tx=0x7f7fe00042f0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 shutdown_connections 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7fe40b5360 0x7f7fe40b7820 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ff41044a0 0x7f7ff4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ff4103ae0 0x7f7ff4075540 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 --2- 172.21.15.120:0/4169945798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ff4102990 0x7f7ff4076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 >> 172.21.15.120:0/4169945798 conn(0x7f7ff40fe250 msgr2=0x7f7ff4106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 shutdown_connections 2024-09-10T23:10:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:09.951+0000 7f7ffaeed640 1 -- 172.21.15.120:0/4169945798 wait complete. 2024-09-10T23:10:10.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:10 smithi120 ceph-mon[31247]: pgmap v878: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 454 op/s 2024-09-10T23:10:10.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:10 smithi153 ceph-mon[34951]: pgmap v878: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 454 op/s 2024-09-10T23:10:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:10 smithi176 ceph-mon[34982]: pgmap v878: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.2 MiB/s wr, 454 op/s 2024-09-10T23:10:10.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125453 21.72 MB/sec execute 185 sec latency 275.928 ms 2024-09-10T23:10:11.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 125894 21.72 MB/sec execute 186 sec latency 91.726 ms 2024-09-10T23:10:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:12 smithi120 ceph-mon[31247]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-09-10T23:10:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:12 smithi153 ceph-mon[34951]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-09-10T23:10:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:12 smithi176 ceph-mon[34982]: pgmap v879: 97 pgs: 97 active+clean; 249 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-09-10T23:10:12.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 126546 21.81 MB/sec execute 187 sec latency 48.155 ms 2024-09-10T23:10:13.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 127169 21.88 MB/sec execute 188 sec latency 54.669 ms 2024-09-10T23:10:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:14 smithi120 ceph-mon[31247]: pgmap v880: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 622 op/s 2024-09-10T23:10:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:14 smithi153 ceph-mon[34951]: pgmap v880: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 622 op/s 2024-09-10T23:10:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:14 smithi176 ceph-mon[34982]: pgmap v880: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 622 op/s 2024-09-10T23:10:14.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 127670 21.94 MB/sec execute 189 sec latency 48.433 ms 2024-09-10T23:10:15.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 127926 21.91 MB/sec execute 190 sec latency 194.800 ms 2024-09-10T23:10:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:16 smithi120 ceph-mon[31247]: pgmap v881: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 511 op/s 2024-09-10T23:10:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:16 smithi153 ceph-mon[34951]: pgmap v881: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 511 op/s 2024-09-10T23:10:16.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:16 smithi176 ceph-mon[34982]: pgmap v881: 97 pgs: 97 active+clean; 251 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.3 MiB/s wr, 511 op/s 2024-09-10T23:10:16.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 128236 21.87 MB/sec execute 191 sec latency 91.383 ms 2024-09-10T23:10:17.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 128611 21.85 MB/sec execute 192 sec latency 58.614 ms 2024-09-10T23:10:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:18 smithi120 ceph-mon[31247]: pgmap v882: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-09-10T23:10:18.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:18 smithi153 ceph-mon[34951]: pgmap v882: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-09-10T23:10:18.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:18 smithi176 ceph-mon[34982]: pgmap v882: 97 pgs: 97 active+clean; 245 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.2 MiB/s wr, 635 op/s 2024-09-10T23:10:18.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 129026 21.81 MB/sec execute 193 sec latency 77.500 ms 2024-09-10T23:10:19.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 129584 21.84 MB/sec execute 194 sec latency 73.735 ms 2024-09-10T23:10:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:20 smithi120 ceph-mon[31247]: pgmap v883: 97 pgs: 97 active+clean; 246 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-09-10T23:10:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:20 smithi153 ceph-mon[34951]: pgmap v883: 97 pgs: 97 active+clean; 246 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-09-10T23:10:20.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:20 smithi176 ceph-mon[34982]: pgmap v883: 97 pgs: 97 active+clean; 246 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 524 op/s 2024-09-10T23:10:20.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 129956 21.85 MB/sec execute 195 sec latency 197.455 ms 2024-09-10T23:10:21.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 130506 21.89 MB/sec execute 196 sec latency 64.328 ms 2024-09-10T23:10:22.249 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:10:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:22 smithi120 ceph-mon[31247]: pgmap v884: 97 pgs: 97 active+clean; 253 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-10T23:10:22.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:22 smithi153 ceph-mon[34951]: pgmap v884: 97 pgs: 97 active+clean; 253 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-10T23:10:22.522 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:22.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:22 smithi176 ceph-mon[34982]: pgmap v884: 97 pgs: 97 active+clean; 253 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-10T23:10:22.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 131028 21.96 MB/sec execute 197 sec latency 55.906 ms 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 -- 172.21.15.120:0/2667390176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530105b10 msgr2=0x7f9530107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 --2- 172.21.15.120:0/2667390176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530105b10 0x7f9530107f40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f9520009900 tx=0x7f952002a410 comp rx=0 tx=0).stop 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 -- 172.21.15.120:0/2667390176 shutdown_connections 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 --2- 172.21.15.120:0/2667390176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9530108680 0x7f953010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 --2- 172.21.15.120:0/2667390176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530105b10 0x7f9530107f40 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 --2- 172.21.15.120:0/2667390176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f953006b3d0 0x7f95301055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 -- 172.21.15.120:0/2667390176 >> 172.21.15.120:0/2667390176 conn(0x7f95300fd140 msgr2=0x7f95300ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 -- 172.21.15.120:0/2667390176 shutdown_connections 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 -- 172.21.15.120:0/2667390176 wait complete. 2024-09-10T23:10:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.053+0000 7f9537d32640 1 Processor -- start 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 -- start start 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 0x7f9530198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 0x7f9530199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f953010e5d0 con 0x7f9530108680 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f953010e450 con 0x7f9530105b10 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9537d32640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f953010e750 con 0x7f953006b3d0 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9535ace640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 0x7f9530198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35054/0 (socket says 172.21.15.120:35054) 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 -- 172.21.15.120:0/3914249484 learned_addr learned my addr 172.21.15.120:0/3914249484 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95352cd640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 0x7f9530199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 -- 172.21.15.120:0/3914249484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 msgr2=0x7f9530198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 0x7f9530198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 -- 172.21.15.120:0/3914249484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 msgr2=0x7f9530199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 0x7f9530199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 -- 172.21.15.120:0/3914249484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953019a840 con 0x7f9530108680 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f9535ace640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 0x7f9530198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95352cd640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 0x7f9530199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.054+0000 7f95362cf640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f952800eb10 tx=0x7f952800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.055+0000 7f9526ffd640 1 -- 172.21.15.120:0/3914249484 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952800ce20 con 0x7f9530108680 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.055+0000 7f9526ffd640 1 -- 172.21.15.120:0/3914249484 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9528005020 con 0x7f9530108680 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.055+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953019ab30 con 0x7f9530108680 2024-09-10T23:10:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.055+0000 7f9526ffd640 1 -- 172.21.15.120:0/3914249484 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528010690 con 0x7f9530108680 2024-09-10T23:10:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.055+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301a1200 con 0x7f9530108680 2024-09-10T23:10:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.057+0000 7f9526ffd640 1 -- 172.21.15.120:0/3914249484 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9528005510 con 0x7f9530108680 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 Processor -- start 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 -- start start 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642caabc6f0 con 0x5642cab4fbd0 2024-09-10T23:10:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642caaabb20 con 0x5642cab4a860 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f953abdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5642caaaf930 con 0x5642cab4d190 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f9526ffd640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 0x7f951c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f95352cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f9535ace640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f95352cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35062/0 (socket says 172.21.15.120:35062) 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.058+0000 7f95352cd640 1 -- 172.21.15.120:0/462927761 learned_addr learned my addr 172.21.15.120:0/462927761 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f95362cf640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 0x7f951c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f95352cd640 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9526ffd640 1 -- 172.21.15.120:0/3914249484 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f95280e6200 con 0x7f9530108680 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 -- 172.21.15.120:0/462927761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 msgr2=0x5642cab4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab4cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 -- 172.21.15.120:0/462927761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 msgr2=0x5642cab52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 -- 172.21.15.120:0/462927761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642caa71530 con 0x5642cab4d190 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f95352cd640 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f95362cf640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 0x7f951c0b7820 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f95280a8fc0 tx=0x7f95280a90c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f9535ace640 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab4f580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f952c00afc0 tx=0x7f952c00f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65eeb9899ff4af0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f951bfff640 1 -- 172.21.15.120:0/462927761 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c005220 con 0x5642cab4d190 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f951bfff640 1 -- 172.21.15.120:0/462927761 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c0047a0 con 0x5642cab4d190 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.059+0000 7f951bfff640 1 -- 172.21.15.120:0/462927761 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c010a40 con 0x5642cab4d190 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 -- 172.21.15.120:0/462927761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 msgr2=0x5642cab4f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab4f580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f952c00afc0 tx=0x7f952c00f850 comp rx=0 tx=0).stop 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 -- 172.21.15.120:0/462927761 shutdown_connections 2024-09-10T23:10:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab4f580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 --2- 172.21.15.120:0/462927761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 -- 172.21.15.120:0/462927761 >> 172.21.15.120:0/462927761 conn(0x5642cab459b0 msgr2=0x5642cab46740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.060+0000 7f953abdc740 1 -- 172.21.15.120:0/462927761 shutdown_connections 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 -- 172.21.15.120:0/462927761 wait complete. 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 -- start start 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 Processor -- start 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab53ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5642ca9e3ab0 con 0x5642cab4d190 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5642caa51730 con 0x5642cab4fbd0 2024-09-10T23:10:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5642cab52bb0 con 0x5642cab4a860 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.061+0000 7f953abdc740 10 client.0 _ll_register_callbacks cb 0x5642caaa56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95352cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab53ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35064/0 (socket says 172.21.15.120:35064) 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 -- 172.21.15.120:0/2434197995 learned_addr learned my addr 172.21.15.120:0/2434197995 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95352cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab53ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58492/0 (socket says 172.21.15.120:58492) 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95352cd640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 -- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 msgr2=0x5642cab53ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab53ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 -- 172.21.15.120:0/2434197995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 msgr2=0x5642cab55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 -- 172.21.15.120:0/2434197995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ca80d790 con 0x5642cab4d190 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f9535ace640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f952c010a90 tx=0x7f952c010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed0ce9d221767d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c027070 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f952c004ee0 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f952c0047a0 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f952c033050 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f953abdc740 10 client.18915 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.062+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642ca886620 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c0047a0 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f952c018600 con 0x5642cab4d190 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 10 client.18915 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 10 client.18915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 20 client.18915 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cab5f480 0x5642cab61870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 4 client.18915 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642caabe640 con 0x5642cab5f480 2024-09-10T23:10:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.063+0000 7f95352cd640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cab5f480 0x5642cab61870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.064+0000 7f95197fa640 10 client.18915 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.064+0000 7f95352cd640 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cab5f480 0x5642cab61870 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da41875a9f3821ac server_cookie=78d69417bbd0db22 in_seq=0 out_seq=0 2024-09-10T23:10:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f95197fa640 1 -- 172.21.15.120:0/2434197995 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642caabe640 con 0x5642cab5f480 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f95197fa640 10 client.18915 handle_command_reply: tid=0 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 client.18915 shutdown 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 msgr2=0x5642cab54560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f952c010a90 tx=0x7f952c010ac0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 shutdown_connections 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cab5f480 0x5642cab61870 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab54560 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f95197fa640 0 client.18915 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 --2- 172.21.15.120:0/2434197995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab53ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 >> 172.21.15.120:0/2434197995 conn(0x5642cab459b0 msgr2=0x5642cab44bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 shutdown_connections 2024-09-10T23:10:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 1 -- 172.21.15.120:0/2434197995 wait complete. 2024-09-10T23:10:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.067+0000 7f953abdc740 20 client.18915 trim_cache size 0 max 16384 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 Processor -- start 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 -- start start 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab53100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab53640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a7ed0 con 0x5642cab4fbd0 2024-09-10T23:10:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95301aa750 con 0x5642cab4a860 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f953abdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a8540 con 0x5642cab4d190 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35076/0 (socket says 172.21.15.120:35076) 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 -- 172.21.15.120:0/2215479239 learned_addr learned my addr 172.21.15.120:0/2215479239 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab53100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f9535ace640 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab53640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 -- 172.21.15.120:0/2215479239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 msgr2=0x5642cab53640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab53640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 -- 172.21.15.120:0/2215479239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 msgr2=0x5642cab53100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab53100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 -- 172.21.15.120:0/2215479239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cab554a0 con 0x5642cab4fbd0 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f95352cd640 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f952002a930 tx=0x7f952002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8388bb2cfde803 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.082+0000 7f951b7fe640 1 -- 172.21.15.120:0/2215479239 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520035040 con 0x5642cab4fbd0 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f951b7fe640 1 -- 172.21.15.120:0/2215479239 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95200054d0 con 0x5642cab4fbd0 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f951b7fe640 1 -- 172.21.15.120:0/2215479239 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520004b20 con 0x5642cab4fbd0 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 -- 172.21.15.120:0/2215479239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 msgr2=0x5642cab660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f952002a930 tx=0x7f952002a410 comp rx=0 tx=0).stop 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 -- 172.21.15.120:0/2215479239 shutdown_connections 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4fbd0 0x5642cab660e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4d190 0x5642cab53640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 --2- 172.21.15.120:0/2215479239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4a860 0x5642cab53100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 -- 172.21.15.120:0/2215479239 >> 172.21.15.120:0/2215479239 conn(0x5642cab459b0 msgr2=0x5642cab53ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 -- 172.21.15.120:0/2215479239 shutdown_connections 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.083+0000 7f953abdc740 1 -- 172.21.15.120:0/2215479239 wait complete. 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 -- start start 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 Processor -- start 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a8c10 con 0x5642cab4d190 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95301aa080 con 0x5642cab4fbd0 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a92e0 con 0x5642cab4a860 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f953abdc740 10 client.0 _ll_register_callbacks cb 0x7f9520009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f95352cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35088/0 (socket says 172.21.15.120:35088) 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 -- 172.21.15.120:0/1481315717 learned_addr learned my addr 172.21.15.120:0/1481315717 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f95352cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab5bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:58510/0 (socket says 172.21.15.120:58510) 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f95352cd640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 -- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 msgr2=0x5642cab5bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab5bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 -- 172.21.15.120:0/1481315717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 msgr2=0x5642cab5afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab5afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f9535ace640 1 -- 172.21.15.120:0/1481315717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cab593c0 con 0x5642cab4d190 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.084+0000 7f95352cd640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f9535ace640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f952c0189a0 tx=0x7f952c010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b608c1011aab5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c027040 con 0x5642cab4d190 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f952c018440 con 0x5642cab4d190 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95200054d0 con 0x5642cab4d190 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f952c00abe0 con 0x5642cab4d190 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 10 client.18921 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642cabcc580 con 0x5642cab4d190 2024-09-10T23:10:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95200054d0 con 0x5642cab4d190 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f952c040440 con 0x5642cab4d190 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 10 client.18921 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 10 client.18921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 20 client.18921 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cabccf50 0x5642cabcf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 4 client.18921 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:10:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.085+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642caaba0f0 con 0x5642cabccf50 2024-09-10T23:10:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.140+0000 7f95352cd640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cabccf50 0x5642cabcf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.140+0000 7f95352cd640 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cabccf50 0x5642cabcf370 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575a130657c73394 server_cookie=83a52d7b9407aac7 in_seq=0 out_seq=0 2024-09-10T23:10:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.140+0000 7f95197fa640 10 client.18921 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 -- 172.21.15.120:0/4017449017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 msgr2=0x7fbe7410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/4017449017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7410f560 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009ca0 tx=0x7fbe6c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 -- 172.21.15.120:0/4017449017 shutdown_connections 2024-09-10T23:10:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/4017449017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7410f560 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/4017449017 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe741045c0 0x7fbe7410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/4017449017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe74103ca0 0x7fbe74104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 -- 172.21.15.120:0/4017449017 >> 172.21.15.120:0/4017449017 conn(0x7fbe740fd700 msgr2=0x7fbe740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 -- 172.21.15.120:0/4017449017 shutdown_connections 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.205+0000 7fbe7a83f640 1 -- 172.21.15.120:0/4017449017 wait complete. 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 Processor -- start 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 -- start start 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe74103ca0 0x7fbe74199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe741045c0 0x7fbe741997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74112d70 con 0x7fbe7410d100 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74112bf0 con 0x7fbe741045c0 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe7a83f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74112ef0 con 0x7fbe74103ca0 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34804/0 (socket says 172.21.15.120:34804) 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 -- 172.21.15.120:0/1198993147 learned_addr learned my addr 172.21.15.120:0/1198993147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe737fe640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe741045c0 0x7fbe741997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe73fff640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe74103ca0 0x7fbe74199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 -- 172.21.15.120:0/1198993147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe74103ca0 msgr2=0x7fbe74199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe74103ca0 0x7fbe74199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 -- 172.21.15.120:0/1198993147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe741045c0 msgr2=0x7fbe741997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe741045c0 0x7fbe741997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.206+0000 7fbe78ddc640 1 -- 172.21.15.120:0/1198993147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7419f330 con 0x7fbe7410d100 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe78ddc640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c005cc0 tx=0x7fbe6c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe717fa640 1 -- 172.21.15.120:0/1198993147 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003b70 con 0x7fbe7410d100 2024-09-10T23:10:23.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7419f5c0 con 0x7fbe7410d100 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe717fa640 1 -- 172.21.15.120:0/1198993147 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe6c0318f0 con 0x7fbe7410d100 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe717fa640 1 -- 172.21.15.120:0/1198993147 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c02f950 con 0x7fbe7410d100 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.207+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe741a5ce0 con 0x7fbe7410d100 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.209+0000 7fbe717fa640 1 -- 172.21.15.120:0/1198993147 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbe6c02fb50 con 0x7fbe7410d100 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.209+0000 7fbe7cc14740 1 Processor -- start 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.209+0000 7fbe7cc14740 1 -- start start 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x560059996480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x5600599a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56005990ce50 con 0x5600599960a0 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5600598fc280 con 0x5600599a2950 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560059900090 con 0x5600599969c0 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x560059996480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x560059996480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34806/0 (socket says 172.21.15.120:34806) 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe737fe640 1 -- 172.21.15.120:0/533692117 learned_addr learned my addr 172.21.15.120:0/533692117 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe717fa640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 0x7fbe5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe737fe640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x5600599a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 -- 172.21.15.120:0/533692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 msgr2=0x5600599a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x5600599a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 -- 172.21.15.120:0/533692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 msgr2=0x560059996480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x560059996480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 -- 172.21.15.120:0/533692117 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005965a650 con 0x5600599969c0 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe737fe640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x5600599a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe78ddc640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 0x7fbe5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe73fff640 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599a2410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009c20 tx=0x7fbe64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b007a7c4cd2e3e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe717fa640 1 -- 172.21.15.120:0/1198993147 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbe6c106ca0 con 0x7fbe7410d100 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe5a7fc640 1 -- 172.21.15.120:0/533692117 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64004050 con 0x5600599969c0 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.210+0000 7fbe5a7fc640 1 -- 172.21.15.120:0/533692117 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640054d0 con 0x5600599969c0 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe5a7fc640 1 -- 172.21.15.120:0/533692117 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64004d00 con 0x5600599969c0 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe78ddc640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 0x7fbe5c0b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ca700 tx=0x7fbe6c0c1d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 -- 172.21.15.120:0/533692117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 msgr2=0x5600599a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599a2410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009c20 tx=0x7fbe64011810 comp rx=0 tx=0).stop 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 -- 172.21.15.120:0/533692117 shutdown_connections 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x5600599a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599a2410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/533692117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x560059996480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 -- 172.21.15.120:0/533692117 >> 172.21.15.120:0/533692117 conn(0x560059993ed0 msgr2=0x560059995070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 -- 172.21.15.120:0/533692117 shutdown_connections 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.211+0000 7fbe7cc14740 1 -- 172.21.15.120:0/533692117 wait complete. 2024-09-10T23:10:23.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 -- start start 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 Processor -- start 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x5600599a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56005975d700 con 0x5600599960a0 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560059897110 con 0x5600599a2950 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5600599a55d0 con 0x5600599969c0 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe7cc14740 10 client.0 _ll_register_callbacks cb 0x5600598b0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:33208/0 (socket says 172.21.15.120:33208) 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe73fff640 1 -- 172.21.15.120:0/3189233204 learned_addr learned my addr 172.21.15.120:0/3189233204 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe737fe640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x5600599a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.212+0000 7fbe737fe640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 -- 172.21.15.120:0/3189233204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 msgr2=0x56005999fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 -- 172.21.15.120:0/3189233204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 msgr2=0x5600599a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x5600599a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 -- 172.21.15.120:0/3189233204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600598c4c40 con 0x5600599969c0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe73fff640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe64000c00 tx=0x7fbe64004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe36bbe17e316b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe537fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64003a00 con 0x5600599969c0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbe64003ba0 con 0x5600599969c0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.213+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe640054d0 con 0x5600599969c0 2024-09-10T23:10:23.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe64034050 con 0x5600599969c0 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 10 client.28390 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600599b0ee0 con 0x5600599969c0 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640054d0 con 0x5600599969c0 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbe64005190 con 0x5600599969c0 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 10 client.28390 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 10 client.28390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 20 client.28390 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600599b2230 0x5600599b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 4 client.28390 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005990eda0 con 0x5600599b2230 2024-09-10T23:10:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.214+0000 7fbe737fe640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600599b2230 0x5600599b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.215+0000 7fbe537fe640 10 client.28390 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.215+0000 7fbe737fe640 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600599b2230 0x5600599b4620 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8c26244e0928f7 server_cookie=9ad6aeae1d743450 in_seq=0 out_seq=0 2024-09-10T23:10:23.266 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe537fe640 1 -- 172.21.15.120:0/3189233204 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56005990eda0 con 0x5600599b2230 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe537fe640 10 client.28390 handle_command_reply: tid=0 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 client.28390 shutdown 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 msgr2=0x56005999f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe64000c00 tx=0x7fbe64004570 comp rx=0 tx=0).stop 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 shutdown_connections 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5600599b2230 0x5600599b4620 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x56005999f6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe537fe640 0 client.28390 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/3189233204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599960a0 0x5600599a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 >> 172.21.15.120:0/3189233204 conn(0x560059993ed0 msgr2=0x560059999a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.267 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 shutdown_connections 2024-09-10T23:10:23.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.265+0000 7fbe7cc14740 1 -- 172.21.15.120:0/3189233204 wait complete. 2024-09-10T23:10:23.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.266+0000 7fbe7cc14740 20 client.28390 trim_cache size 0 max 16384 2024-09-10T23:10:23.285 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 Processor -- start 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 -- start start 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 0x56005999ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ac310 con 0x5600599969c0 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741aeb90 con 0x5600599a2950 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ac980 con 0x5600599960a0 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34814/0 (socket says 172.21.15.120:34814) 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 -- 172.21.15.120:0/803762068 learned_addr learned my addr 172.21.15.120:0/803762068 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe737fe640 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 0x56005999ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 -- 172.21.15.120:0/803762068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 msgr2=0x56005999ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 0x56005999ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 -- 172.21.15.120:0/803762068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 msgr2=0x56005999e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe73fff640 1 -- 172.21.15.120:0/803762068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600599967f0 con 0x5600599969c0 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.284+0000 7fbe737fe640 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 0x56005999ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.285+0000 7fbe73fff640 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009560 tx=0x7fbe640236d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db63a1cb6aa10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.285+0000 7fbe59ffb640 1 -- 172.21.15.120:0/803762068 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64005b90 con 0x5600599969c0 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.285+0000 7fbe59ffb640 1 -- 172.21.15.120:0/803762068 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640045e0 con 0x5600599969c0 2024-09-10T23:10:23.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.285+0000 7fbe59ffb640 1 -- 172.21.15.120:0/803762068 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64019430 con 0x5600599969c0 2024-09-10T23:10:23.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- 172.21.15.120:0/803762068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 msgr2=0x56005999df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009560 tx=0x7fbe640236d0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- 172.21.15.120:0/803762068 shutdown_connections 2024-09-10T23:10:23.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599a2950 0x56005999e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.288 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599969c0 0x56005999df50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/803762068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599960a0 0x56005999ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- 172.21.15.120:0/803762068 >> 172.21.15.120:0/803762068 conn(0x560059993ed0 msgr2=0x560059995000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- 172.21.15.120:0/803762068 shutdown_connections 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- 172.21.15.120:0/803762068 wait complete. 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.286+0000 7fbe7cc14740 1 -- start start 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 Processor -- start 2024-09-10T23:10:23.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 0x5600599aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 0x5600599a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ad050 con 0x5600599a2950 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ae4c0 con 0x5600599960a0 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741ad720 con 0x5600599969c0 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe7cc14740 10 client.0 _ll_register_callbacks cb 0x7fbe68009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 0x5600599aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe737fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 0x5600599aaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59684/0 (socket says 172.21.15.120:59684) 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe737fe640 1 -- 172.21.15.120:0/1577442720 learned_addr learned my addr 172.21.15.120:0/1577442720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.287+0000 7fbe737fe640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 0x5600599a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.291 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 -- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 msgr2=0x5600599aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 -- 172.21.15.120:0/1577442720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 msgr2=0x5600599aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 0x5600599aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 -- 172.21.15.120:0/1577442720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600599a5ef0 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe737fe640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 0x5600599a5850 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009af0 tx=0x7fbe68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eb25c2d84c16c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe537fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68010870 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.288+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbe68010a10 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.289+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe640045e0 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.289+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe68010c10 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.289+0000 7fbe7cc14740 10 client.18939 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.289+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560059a1f6a0 con 0x5600599a2950 2024-09-10T23:10:23.292 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.289+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe640045e0 con 0x5600599a2950 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbe68004a20 con 0x5600599a2950 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 10 client.18939 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 10 client.18939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 20 client.18939 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560059a1fe00 0x560059a22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:23.293 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 4 client.18939 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:10:23.294 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.290+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56005990a850 con 0x560059a1fe00 2024-09-10T23:10:23.297 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.293+0000 7fbe73fff640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560059a1fe00 0x560059a22220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:23.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.294+0000 7fbe73fff640 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560059a1fe00 0x560059a22220 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb2da54e76cbc161 server_cookie=9396dfe85e3315a4 in_seq=0 out_seq=0 2024-09-10T23:10:23.298 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.294+0000 7fbe537fe640 10 client.18939 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.321 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:10:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:10:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:10:23.321 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.322 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:10:23.323 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.324 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.325 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:10:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:10:23.326 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.327 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.328 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:10:23.330 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.331 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.332 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:10:23.333 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:10:23.337 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.338 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.339 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:10:23.340 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:10:23.341 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:10:23.343 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.344 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:10:23.345 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.314+0000 7fbe537fe640 1 -- 172.21.15.120:0/1577442720 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8610 (crc 0 0 0) 0x56005990a850 con 0x560059a1fe00 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.314+0000 7fbe537fe640 10 client.18939 handle_command_reply: tid=0 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 client.18939 shutdown 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 msgr2=0x5600599a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 0x5600599a5850 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009af0 tx=0x7fbe68005f90 comp rx=0 tx=0).stop 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 shutdown_connections 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560059a1fe00 0x560059a22220 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5600599a2950 0x5600599a5850 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5600599969c0 0x5600599aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 --2- 172.21.15.120:0/1577442720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5600599960a0 0x5600599aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.315+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 >> 172.21.15.120:0/1577442720 conn(0x560059993ed0 msgr2=0x56005999dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.317+0000 7fbe537fe640 0 client.18939 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.319+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 shutdown_connections 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.319+0000 7fbe7cc14740 1 -- 172.21.15.120:0/1577442720 wait complete. 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.319+0000 7fbe7cc14740 20 client.18939 trim_cache size 0 max 16384 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 msgr2=0x7fbe5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 0x7fbe5c0b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ca700 tx=0x7fbe6c0c1d40 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 msgr2=0x7fbe7419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c005cc0 tx=0x7fbe6c031700 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 shutdown_connections 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbe5c0b52e0 0x7fbe5c0b77a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe7410d100 0x7fbe7419eb60 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe741045c0 0x7fbe741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 --2- 172.21.15.120:0/1198993147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe74103ca0 0x7fbe74199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.320+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 >> 172.21.15.120:0/1198993147 conn(0x7fbe740fd700 msgr2=0x7fbe740fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.321+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 shutdown_connections 2024-09-10T23:10:23.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:23.321+0000 7fbe7a83f640 1 -- 172.21.15.120:0/1198993147 wait complete. 2024-09-10T23:10:23.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 131211 21.90 MB/sec execute 198 sec latency 494.313 ms 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.967+0000 7f95197fa640 1 -- 172.21.15.120:0/1481315717 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642caaba0f0 con 0x5642cabccf50 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.967+0000 7f95197fa640 10 client.18921 handle_command_reply: tid=0 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.967+0000 7f953abdc740 1 client.18921 shutdown 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.967+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 msgr2=0x5642cab5a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.967+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f952c0189a0 tx=0x7f952c010ba0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 shutdown_connections 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5642cabccf50 0x5642cabcf370 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f95197fa640 0 client.18921 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5642cab4fbd0 0x5642cab5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5642cab4d190 0x5642cab5a8e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 --2- 172.21.15.120:0/1481315717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642cab4a860 0x5642cab5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 >> 172.21.15.120:0/1481315717 conn(0x5642cab459b0 msgr2=0x5642cab44bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 shutdown_connections 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 1 -- 172.21.15.120:0/1481315717 wait complete. 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f953abdc740 20 client.18921 trim_cache size 0 max 16384 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:10:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 msgr2=0x7f951c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.968+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 0x7f951c0b7820 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f95280a8fc0 tx=0x7f95280a90c0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 msgr2=0x7f9530199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f952800eb10 tx=0x7f952800ef10 comp rx=0 tx=0).stop 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 shutdown_connections 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f951c0b5360 0x7f951c0b7820 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9530108680 0x7f9530199f80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9530105b10 0x7f9530199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 --2- 172.21.15.120:0/3914249484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f953006b3d0 0x7f9530198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 >> 172.21.15.120:0/3914249484 conn(0x7f95300fd140 msgr2=0x7f9530103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 shutdown_connections 2024-09-10T23:10:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:23.969+0000 7f9537d32640 1 -- 172.21.15.120:0/3914249484 wait complete. 2024-09-10T23:10:24.297 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:10:24.297 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:10:24.361 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:24 smithi120 ceph-mon[31247]: pgmap v885: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-09-10T23:10:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:24 smithi153 ceph-mon[34951]: pgmap v885: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-09-10T23:10:24.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:24 smithi176 ceph-mon[34982]: pgmap v885: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-09-10T23:10:24.621 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:24.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 131596 21.92 MB/sec execute 199 sec latency 447.474 ms 2024-09-10T23:10:25.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- 172.21.15.120:0/1828858491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0103ca0 msgr2=0x7f18c0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:25.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 --2- 172.21.15.120:0/1828858491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0103ca0 0x7f18c0104080 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009900 tx=0x7f18bc02a410 comp rx=0 tx=0).stop 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- 172.21.15.120:0/1828858491 shutdown_connections 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 --2- 172.21.15.120:0/1828858491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c010d100 0x7f18c010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 --2- 172.21.15.120:0/1828858491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c01045c0 0x7f18c010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 --2- 172.21.15.120:0/1828858491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c0103ca0 0x7f18c0104080 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- 172.21.15.120:0/1828858491 >> 172.21.15.120:0/1828858491 conn(0x7f18c00fd700 msgr2=0x7f18c00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- 172.21.15.120:0/1828858491 shutdown_connections 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- 172.21.15.120:0/1828858491 wait complete. 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 Processor -- start 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.332+0000 7f18c81ec640 1 -- start start 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 0x7f18c0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 0x7f18c01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:25.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0112d70 con 0x7f18c010d100 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0112bf0 con 0x7f18c01045c0 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c81ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0112ef0 con 0x7f18c0103ca0 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34850/0 (socket says 172.21.15.120:34850) 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 -- 172.21.15.120:0/3785375364 learned_addr learned my addr 172.21.15.120:0/3785375364 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c5f88640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 0x7f18c0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c5787640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 0x7f18c01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 -- 172.21.15.120:0/3785375364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 msgr2=0x7f18c0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 0x7f18c0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 -- 172.21.15.120:0/3785375364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 msgr2=0x7f18c01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 0x7f18c01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 -- 172.21.15.120:0/3785375364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c019f330 con 0x7f18c010d100 2024-09-10T23:10:25.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c5787640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 0x7f18c01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c5f88640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 0x7f18c0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18c6789640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f18b800eaa0 tx=0x7f18b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b800cdb0 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.333+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b8005020 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.334+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c019f620 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.334+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80116d0 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.334+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a5ce0 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.335+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c0104080 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.336+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f18b800f040 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.336+0000 7f18b6ffd640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 0x7f18ac0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.337+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f18b80e5910 con 0x7f18c010d100 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.337+0000 7f18c5f88640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 0x7f18ac0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:25.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.337+0000 7f18c5f88640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 0x7f18ac0b7820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f18bc031440 tx=0x7f18bc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:25.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.341+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18b8016030 con 0x7f18c010d100 2024-09-10T23:10:25.596 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.596+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18c01a60b0 con 0x7f18c010d100 2024-09-10T23:10:25.597 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.596+0000 7f18b6ffd640 1 -- 172.21.15.120:0/3785375364 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f18b80ac1b0 con 0x7f18c010d100 2024-09-10T23:10:25.597 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:10:25.597 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:10:25.598 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:10:25.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 msgr2=0x7f18ac0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:25.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 0x7f18ac0b7820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f18bc031440 tx=0x7f18bc005b50 comp rx=0 tx=0).stop 2024-09-10T23:10:25.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 msgr2=0x7f18c019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:25.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f18b800eaa0 tx=0x7f18b800ef60 comp rx=0 tx=0).stop 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 shutdown_connections 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f18ac0b5360 0x7f18ac0b7820 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18c010d100 0x7f18c019eb60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c01045c0 0x7f18c01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 --2- 172.21.15.120:0/3785375364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c0103ca0 0x7f18c0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.598+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 >> 172.21.15.120:0/3785375364 conn(0x7f18c00fd700 msgr2=0x7f18c00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.599+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 shutdown_connections 2024-09-10T23:10:25.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:25.599+0000 7f18c81ec640 1 -- 172.21.15.120:0/3785375364 wait complete. 2024-09-10T23:10:25.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 131844 21.87 MB/sec execute 200 sec latency 248.998 ms 2024-09-10T23:10:26.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:26 smithi120 ceph-mon[31247]: pgmap v886: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 307 op/s 2024-09-10T23:10:26.291 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3785375364' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:26.319 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:26 smithi153 ceph-mon[34951]: pgmap v886: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 307 op/s 2024-09-10T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3785375364' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:26 smithi176 ceph-mon[34982]: pgmap v886: 97 pgs: 97 active+clean; 260 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.4 MiB/s wr, 307 op/s 2024-09-10T23:10:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3785375364' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:10:26.669 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:26.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 132179 21.82 MB/sec execute 201 sec latency 125.872 ms 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 -- 172.21.15.120:0/2944180630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0108020 msgr2=0x7f24d010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 --2- 172.21.15.120:0/2944180630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0108020 0x7f24d010e820 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009900 tx=0x7f24c802a410 comp rx=0 tx=0).stop 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 -- 172.21.15.120:0/2944180630 shutdown_connections 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 --2- 172.21.15.120:0/2944180630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0108020 0x7f24d010e820 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 --2- 172.21.15.120:0/2944180630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 --2- 172.21.15.120:0/2944180630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0101c90 0x7f24d0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 -- 172.21.15.120:0/2944180630 >> 172.21.15.120:0/2944180630 conn(0x7f24d00fd700 msgr2=0x7f24d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 -- 172.21.15.120:0/2944180630 shutdown_connections 2024-09-10T23:10:27.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.184+0000 7f24d8379640 1 -- 172.21.15.120:0/2944180630 wait complete. 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 Processor -- start 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 -- start start 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 0x7f24d01a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111ca0 con 0x7f24d0101c90 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111b20 con 0x7f24d01076a0 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d8379640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0111e20 con 0x7f24d0108020 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d6115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d6115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34876/0 (socket says 172.21.15.120:34876) 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d6115640 1 -- 172.21.15.120:0/2191110349 learned_addr learned my addr 172.21.15.120:0/2191110349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d5914640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.185+0000 7f24d6916640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 0x7f24d01a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 -- 172.21.15.120:0/2191110349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 msgr2=0x7f24d01a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 0x7f24d01a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 -- 172.21.15.120:0/2191110349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 msgr2=0x7f24d019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 -- 172.21.15.120:0/2191110349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d01a55d0 con 0x7f24d0101c90 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6916640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 0x7f24d01a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d5914640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d6115640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00eb90 tx=0x7f24cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24c77fe640 1 -- 172.21.15.120:0/2191110349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004d20 con 0x7f24d0101c90 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d01a57a0 con 0x7f24d0101c90 2024-09-10T23:10:27.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24c77fe640 1 -- 172.21.15.120:0/2191110349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24cc00ee90 con 0x7f24d0101c90 2024-09-10T23:10:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24c77fe640 1 -- 172.21.15.120:0/2191110349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc010670 con 0x7f24d0101c90 2024-09-10T23:10:27.190 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.186+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d01a5c80 con 0x7f24d0101c90 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 Processor -- start 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 -- start start 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24c77fe640 1 -- 172.21.15.120:0/2191110349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f24cc005020 con 0x7f24d0101c90 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564be0155ea0 con 0x564be01dfa10 2024-09-10T23:10:27.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564be01452d0 con 0x564be01eb9a0 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24da74e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564be01490e0 con 0x564be01df0f0 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24d6115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.189+0000 7f24d6916640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d6916640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:33294/0 (socket says 172.21.15.120:33294) 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d6916640 1 -- 172.21.15.120:0/2850015365 learned_addr learned my addr 172.21.15.120:0/2850015365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 -- 172.21.15.120:0/2850015365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 msgr2=0x564be01df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 -- 172.21.15.120:0/2850015365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 msgr2=0x564be01eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01eb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 -- 172.21.15.120:0/2850015365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bdfea3650 con 0x564be01eb9a0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d6916640 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d5914640 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01edd90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fa0 tx=0x7f24c00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4da10b63d99b37d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24c77fe640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 0x7f24bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24c4de6640 1 -- 172.21.15.120:0/2850015365 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004030 con 0x564be01eb9a0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24c4de6640 1 -- 172.21.15.120:0/2850015365 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00054d0 con 0x564be01eb9a0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24c4de6640 1 -- 172.21.15.120:0/2850015365 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004c40 con 0x564be01eb9a0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.190+0000 7f24d6916640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 0x7f24bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24c77fe640 1 -- 172.21.15.120:0/2191110349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f24cc0e6210 con 0x7f24d0101c90 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 -- 172.21.15.120:0/2850015365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 msgr2=0x564be01edd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24d6916640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 0x7f24bc0b77a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005940 tx=0x7f24c80058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01edd90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fa0 tx=0x7f24c00118d0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 -- 172.21.15.120:0/2850015365 shutdown_connections 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01edd90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 --2- 172.21.15.120:0/2850015365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 -- 172.21.15.120:0/2850015365 >> 172.21.15.120:0/2850015365 conn(0x564be01dcf20 msgr2=0x564be01de0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 -- 172.21.15.120:0/2850015365 shutdown_connections 2024-09-10T23:10:27.195 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.191+0000 7f24da74e740 1 -- 172.21.15.120:0/2850015365 wait complete. 2024-09-10T23:10:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 -- start start 2024-09-10T23:10:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 Processor -- start 2024-09-10T23:10:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01dfa10 0x564be01e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564bdffa6700 con 0x564be01df0f0 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564be00e0160 con 0x564be01dfa10 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564be01ee620 con 0x564be01eb9a0 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24da74e740 10 client.0 _ll_register_callbacks cb 0x564be00f99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34900/0 (socket says 172.21.15.120:34900) 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 -- 172.21.15.120:0/90054881 learned_addr learned my addr 172.21.15.120:0/90054881 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:27.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6916640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d5914640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01dfa10 0x564be01e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 -- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 msgr2=0x564be01e8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 -- 172.21.15.120:0/90054881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01dfa10 msgr2=0x564be01e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01dfa10 0x564be01e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.192+0000 7f24d6115640 1 -- 172.21.15.120:0/90054881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be010dc90 con 0x564be01df0f0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24d6115640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00cba0 tx=0x7f24cc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f50d8512c2babc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0ac7d0 con 0x564be01df0f0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c00054d0 con 0x564be01df0f0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f24cc0b1170 con 0x564be01df0f0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24cc0a8170 con 0x564be01df0f0 2024-09-10T23:10:27.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 10 client.18957 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564be01fa030 con 0x564be01df0f0 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00054d0 con 0x564be01df0f0 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f24cc005340 con 0x564be01df0f0 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 10 client.18957 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 20 client.18957 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be01fb440 0x564be01fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 4 client.18957 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.193+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be0157df0 con 0x564be01fb440 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.194+0000 7f24d5914640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be01fb440 0x564be01fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.202 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.194+0000 7f24d5914640 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be01fb440 0x564be01fd870 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551db2844cf0c5b5 server_cookie=ba8f4ff9435c972d in_seq=0 out_seq=0 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.194+0000 7f24b9ffb640 10 client.18957 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24b9ffb640 1 -- 172.21.15.120:0/90054881 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564be0157df0 con 0x564be01fb440 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24b9ffb640 10 client.18957 handle_command_reply: tid=0 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 client.18957 shutdown 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 msgr2=0x564be01eb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00cba0 tx=0x7f24cc0099b0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 shutdown_connections 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be01fb440 0x564be01fd870 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01dfa10 0x564be01e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24b9ffb640 0 client.18957 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 --2- 172.21.15.120:0/90054881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01df0f0 0x564be01eb260 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 >> 172.21.15.120:0/90054881 conn(0x564be01dcf20 msgr2=0x564be01e2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 shutdown_connections 2024-09-10T23:10:27.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 1 -- 172.21.15.120:0/90054881 wait complete. 2024-09-10T23:10:27.205 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.196+0000 7f24da74e740 20 client.18957 trim_cache size 0 max 16384 2024-09-10T23:10:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 Processor -- start 2024-09-10T23:10:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 -- start start 2024-09-10T23:10:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01df0f0 0x564be01f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ac240 con 0x564be01dfa10 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01aeac0 con 0x564be01df0f0 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24da74e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ac8b0 con 0x564be01eb9a0 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34908/0 (socket says 172.21.15.120:34908) 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 -- 172.21.15.120:0/1803045403 learned_addr learned my addr 172.21.15.120:0/1803045403 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d6115640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01df0f0 0x564be01f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d6916640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 -- 172.21.15.120:0/1803045403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 msgr2=0x564be01e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 -- 172.21.15.120:0/1803045403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01df0f0 msgr2=0x564be01f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01df0f0 0x564be01f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 -- 172.21.15.120:0/1803045403 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be01df840 con 0x564be01dfa10 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d6916640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.210+0000 7f24d5914640 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10410015f5d65fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24bbfff640 1 -- 172.21.15.120:0/1803045403 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c001d040 con 0x564be01dfa10 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24bbfff640 1 -- 172.21.15.120:0/1803045403 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0004660 con 0x564be01dfa10 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24bbfff640 1 -- 172.21.15.120:0/1803045403 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00197d0 con 0x564be01dfa10 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 -- 172.21.15.120:0/1803045403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 msgr2=0x564be01e58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c0009f30 comp rx=0 tx=0).stop 2024-09-10T23:10:27.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 -- 172.21.15.120:0/1803045403 shutdown_connections 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01eb9a0 0x564be01e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01e58c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 --2- 172.21.15.120:0/1803045403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01df0f0 0x564be01f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 -- 172.21.15.120:0/1803045403 >> 172.21.15.120:0/1803045403 conn(0x564be01dcf20 msgr2=0x564be01e6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 -- 172.21.15.120:0/1803045403 shutdown_connections 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.211+0000 7f24da74e740 1 -- 172.21.15.120:0/1803045403 wait complete. 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 -- start start 2024-09-10T23:10:27.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 Processor -- start 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01acf80 con 0x564be01dfa10 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ae3f0 con 0x564be01eb9a0 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01ad650 con 0x564be01df0f0 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d5914640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d6115640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d6916640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24da74e740 10 client.0 _ll_register_callbacks cb 0x7f24cc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d6115640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:33330/0 (socket says 172.21.15.120:33330) 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d6916640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59740/0 (socket says 172.21.15.120:59740) 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d6115640 1 -- 172.21.15.120:0/2480081387 learned_addr learned my addr 172.21.15.120:0/2480081387 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.212+0000 7f24d5914640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34916/0 (socket says 172.21.15.120:34916) 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 -- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 msgr2=0x564be01f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 -- 172.21.15.120:0/2480081387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 msgr2=0x564be01ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 -- 172.21.15.120:0/2480081387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be01ef520 con 0x564be01eb9a0 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24d6916640 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24c802a930 tx=0x7f24c8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5531b06f8da76cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:10:27.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8035040 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f24c80055d0 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c0004660 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f24c805a070 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24da74e740 10 client.28184 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564be0269240 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.213+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0004660 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f24c80038f0 con 0x564be01eb9a0 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 10 client.28184 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 10 client.28184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 20 client.28184 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be0269990 0x564be026bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 4 client.28184 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564be01538a0 con 0x564be0269990 2024-09-10T23:10:27.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24d6115640 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be0269990 0x564be026bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24b9ffb640 10 client.28184 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:27.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.214+0000 7f24d6115640 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be0269990 0x564be026bdb0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b9778dbdcb9273 server_cookie=f866d0aa6a8361b8 in_seq=0 out_seq=0 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24b9ffb640 1 -- 172.21.15.120:0/2480081387 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564be01538a0 con 0x564be0269990 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24b9ffb640 10 client.28184 handle_command_reply: tid=0 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 client.28184 shutdown 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 msgr2=0x564be01eee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24c802a930 tx=0x7f24c8009260 comp rx=0 tx=0).stop 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 shutdown_connections 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564be0269990 0x564be026bdb0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564be01eb9a0 0x564be01eee10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564be01dfa10 0x564be01ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 --2- 172.21.15.120:0/2480081387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564be01df0f0 0x564be01f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 >> 172.21.15.120:0/2480081387 conn(0x564be01dcf20 msgr2=0x564be01e9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24b9ffb640 0 client.28184 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 shutdown_connections 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 1 -- 172.21.15.120:0/2480081387 wait complete. 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.219+0000 7f24da74e740 20 client.28184 trim_cache size 0 max 16384 2024-09-10T23:10:27.222 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:10:27.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 msgr2=0x7f24bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 0x7f24bc0b77a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005940 tx=0x7f24c80058b0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 msgr2=0x7f24d019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:27.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00eb90 tx=0x7f24cc00c6a0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 shutdown_connections 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f24bc0b52e0 0x7f24bc0b77a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24d0108020 0x7f24d01a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24d01076a0 0x7f24d019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 --2- 172.21.15.120:0/2191110349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f24d0101c90 0x7f24d019a010 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 >> 172.21.15.120:0/2191110349 conn(0x7f24d00fd700 msgr2=0x7f24d00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 shutdown_connections 2024-09-10T23:10:27.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:27.220+0000 7f24d8379640 1 -- 172.21.15.120:0/2191110349 wait complete. 2024-09-10T23:10:27.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 132584 21.80 MB/sec execute 202 sec latency 65.064 ms 2024-09-10T23:10:28.189 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:28 smithi120 ceph-mon[31247]: pgmap v887: 97 pgs: 97 active+clean; 238 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 576 op/s 2024-09-10T23:10:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:28 smithi153 ceph-mon[34951]: pgmap v887: 97 pgs: 97 active+clean; 238 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 576 op/s 2024-09-10T23:10:28.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:28 smithi176 ceph-mon[34982]: pgmap v887: 97 pgs: 97 active+clean; 238 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 576 op/s 2024-09-10T23:10:28.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 133128 21.81 MB/sec execute 203 sec latency 72.913 ms 2024-09-10T23:10:29.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:10:29.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:10:29.486 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:29.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 133752 21.91 MB/sec execute 204 sec latency 43.253 ms 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 -- 172.21.15.120:0/4122602862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c1076a0 msgr2=0x7fcb3c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 --2- 172.21.15.120:0/4122602862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c107ae0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009900 tx=0x7fcb3002a410 comp rx=0 tx=0).stop 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 -- 172.21.15.120:0/4122602862 shutdown_connections 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 --2- 172.21.15.120:0/4122602862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c108020 0x7fcb3c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 --2- 172.21.15.120:0/4122602862 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c107ae0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 --2- 172.21.15.120:0/4122602862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 -- 172.21.15.120:0/4122602862 >> 172.21.15.120:0/4122602862 conn(0x7fcb3c0fd700 msgr2=0x7fcb3c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 -- 172.21.15.120:0/4122602862 shutdown_connections 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.024+0000 7fcb40b83640 1 -- 172.21.15.120:0/4122602862 wait complete. 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 Processor -- start 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 -- start start 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c108020 0x7fcb3c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c111ce0 con 0x7fcb3c108020 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c111b60 con 0x7fcb3c101c90 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb40b83640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c111e60 con 0x7fcb3c1076a0 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59748/0 (socket says 172.21.15.120:59748) 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 -- 172.21.15.120:0/3295182338 learned_addr learned my addr 172.21.15.120:0/3295182338 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c108020 0x7fcb3c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 -- 172.21.15.120:0/3295182338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c1076a0 msgr2=0x7fcb3c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 -- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c108020 msgr2=0x7fcb3c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c108020 0x7fcb3c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.025+0000 7fcb3a59c640 1 -- 172.21.15.120:0/3295182338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c19f320 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb3a59c640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00e9f0 tx=0x7fcb2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb2b7fe640 1 -- 172.21.15.120:0/3295182338 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00cd80 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c19f610 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb2b7fe640 1 -- 172.21.15.120:0/3295182338 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2c0108e0 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb2b7fe640 1 -- 172.21.15.120:0/3295182338 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c018980 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.026+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a5cd0 con 0x7fcb3c101c90 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.028+0000 7fcb42f58740 1 Processor -- start 2024-09-10T23:10:30.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 -- start start 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb2b7fe640 1 -- 172.21.15.120:0/3295182338 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcb2c010430 con 0x7fcb3c101c90 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55696aeb3e60 con 0x55696af49790 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55696aea3290 con 0x55696af3cee0 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb42f58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55696aea70a0 con 0x55696af3d800 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb3a59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af4bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34942/0 (socket says 172.21.15.120:34942) 2024-09-10T23:10:30.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb39d9b640 1 -- 172.21.15.120:0/1314490109 learned_addr learned my addr 172.21.15.120:0/1314490109 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59752/0 (socket says 172.21.15.120:59752) 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.029+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/1314490109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 msgr2=0x55696af49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/1314490109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 msgr2=0x55696af4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af4bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/1314490109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696ac01930 con 0x55696af3cee0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009f30 tx=0x7fcb34010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c97fa8c792483f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb28ff9640 1 -- 172.21.15.120:0/1314490109 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34016070 con 0x55696af3cee0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb28ff9640 1 -- 172.21.15.120:0/1314490109 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34004d20 con 0x55696af3cee0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb28ff9640 1 -- 172.21.15.120:0/1314490109 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340127d0 con 0x55696af3cee0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb2b7fe640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 0x7fcb240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.030+0000 7fcb3a59c640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 0x7fcb240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb2b7fe640 1 -- 172.21.15.120:0/3295182338 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb2c0e5f50 con 0x7fcb3c101c90 2024-09-10T23:10:30.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- 172.21.15.120:0/1314490109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 msgr2=0x55696af3d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb3a59c640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 0x7fcb240b77a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0ac490 tx=0x7fcb2c0e9990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009f30 tx=0x7fcb34010800 comp rx=0 tx=0).stop 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- 172.21.15.120:0/1314490109 shutdown_connections 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1314490109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3cee0 0x55696af3d2c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- 172.21.15.120:0/1314490109 >> 172.21.15.120:0/1314490109 conn(0x55696af3ad10 msgr2=0x55696af3beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- 172.21.15.120:0/1314490109 shutdown_connections 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- 172.21.15.120:0/1314490109 wait complete. 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.031+0000 7fcb42f58740 1 -- start start 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 Processor -- start 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af49790 0x55696af47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55696ad04890 con 0x55696af3cee0 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55696ae3dd40 con 0x55696af49790 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55696af4c440 con 0x55696af3d800 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb42f58740 10 client.0 _ll_register_callbacks cb 0x55696aea1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34948/0 (socket says 172.21.15.120:34948) 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/1668758276 learned_addr learned my addr 172.21.15.120:0/1668758276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af49790 0x55696af47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 -- 172.21.15.120:0/1668758276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af49790 msgr2=0x55696af47240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af49790 0x55696af47240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 -- 172.21.15.120:0/1668758276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 msgr2=0x55696af467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.059 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb39d9b640 1 -- 172.21.15.120:0/1668758276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696af558a0 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.032+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af46d00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009230 tx=0x7fcb30032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bccb5c78a336966 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30035040 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb34004d20 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb300317f0 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb30004040 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 10 client.28426 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55696af58280 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34004d20 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb3003d760 con 0x55696af3d800 2024-09-10T23:10:30.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 10 client.28426 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 10 client.28426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 20 client.28426 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696af59690 0x55696af5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 4 client.28426 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.033+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696aeb5db0 con 0x55696af59690 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.034+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696af59690 0x55696af5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.034+0000 7fcb21ffb640 10 client.28426 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.034+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696af59690 0x55696af5ba80 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a063bc1066133e55 server_cookie=d96ae3b30b6dda3f in_seq=0 out_seq=0 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb21ffb640 1 -- 172.21.15.120:0/1668758276 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55696aeb5db0 con 0x55696af59690 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb21ffb640 10 client.28426 handle_command_reply: tid=0 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 client.28426 shutdown 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 msgr2=0x55696af46d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.061 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af46d00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009230 tx=0x7fcb30032f20 comp rx=0 tx=0).stop 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 shutdown_connections 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696af59690 0x55696af5ba80 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af49790 0x55696af47240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3d800 0x55696af46d00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 --2- 172.21.15.120:0/1668758276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb21ffb640 0 client.28426 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.036+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 >> 172.21.15.120:0/1668758276 conn(0x55696af3ad10 msgr2=0x55696af39f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.037+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 shutdown_connections 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.037+0000 7fcb42f58740 1 -- 172.21.15.120:0/1668758276 wait complete. 2024-09-10T23:10:30.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.037+0000 7fcb42f58740 20 client.28426 trim_cache size 0 max 16384 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 Processor -- start 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 -- start start 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af3c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af61370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1aa0b0 con 0x55696af49790 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1ac930 con 0x55696af3d800 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.054+0000 7fcb42f58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1aa720 con 0x55696af3cee0 2024-09-10T23:10:30.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:33378/0 (socket says 172.21.15.120:33378) 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/3249939328 learned_addr learned my addr 172.21.15.120:0/3249939328 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af61370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af3c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/3249939328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 msgr2=0x55696af3c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af3c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/3249939328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 msgr2=0x55696af61370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af61370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 -- 172.21.15.120:0/3249939328 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696af5c0e0 con 0x55696af3cee0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009640 tx=0x7fcb34014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3591a028810dbf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.055+0000 7fcb23fff640 1 -- 172.21.15.120:0/3249939328 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34016040 con 0x55696af3cee0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.056+0000 7fcb23fff640 1 -- 172.21.15.120:0/3249939328 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34012ad0 con 0x55696af3cee0 2024-09-10T23:10:30.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.056+0000 7fcb23fff640 1 -- 172.21.15.120:0/3249939328 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3401aa40 con 0x55696af3cee0 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- 172.21.15.120:0/3249939328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 msgr2=0x55696af44250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009640 tx=0x7fcb34014040 comp rx=0 tx=0).stop 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- 172.21.15.120:0/3249939328 shutdown_connections 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af49790 0x55696af61370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af3c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- 172.21.15.120:0/3249939328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af3cee0 0x55696af44250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- 172.21.15.120:0/3249939328 >> 172.21.15.120:0/3249939328 conn(0x55696af3ad10 msgr2=0x55696af3bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- 172.21.15.120:0/3249939328 shutdown_connections 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- 172.21.15.120:0/3249939328 wait complete. 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- start start 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 Processor -- start 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af51a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 0x55696af4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1aadf0 con 0x55696af3cee0 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1ac260 con 0x55696af3d800 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.057+0000 7fcb42f58740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c1ab4c0 con 0x55696af49790 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb42f58740 10 client.0 _ll_register_callbacks cb 0x7fcb30005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59792/0 (socket says 172.21.15.120:59792) 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 -- 172.21.15.120:0/164953869 learned_addr learned my addr 172.21.15.120:0/164953869 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af51a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 0x55696af4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 -- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 msgr2=0x55696af4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 0x55696af4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 -- 172.21.15.120:0/164953869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 msgr2=0x55696af51a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af51a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 -- 172.21.15.120:0/164953869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696af3ab40 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 0x55696af4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb39d9b640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005ec0 tx=0x7fcb3002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aca685611e7f10e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30035070 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb300043f0 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.058+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb34012ad0 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb300317f0 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 10 client.28202 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55696afc7020 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb34012ad0 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb30003b60 con 0x55696af3d800 2024-09-10T23:10:30.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 10 client.28202 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 10 client.28202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 20 client.28202 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696afc7820 0x55696afc9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 4 client.28202 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55696aeb1860 con 0x55696afc7820 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.059+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696afc7820 0x55696afc9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.060+0000 7fcb21ffb640 10 client.28202 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.060+0000 7fcb3ad9d640 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696afc7820 0x55696afc9c40 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6313bcedbda0581 server_cookie=896ace286060a9b5 in_seq=0 out_seq=0 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb21ffb640 1 -- 172.21.15.120:0/164953869 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55696aeb1860 con 0x55696afc7820 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb21ffb640 10 client.28202 handle_command_reply: tid=0 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 client.28202 shutdown 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 msgr2=0x55696af51fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005ec0 tx=0x7fcb3002a410 comp rx=0 tx=0).stop 2024-09-10T23:10:30.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 shutdown_connections 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55696afc7820 0x55696afc9c40 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55696af49790 0x55696af4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55696af3d800 0x55696af51fa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 --2- 172.21.15.120:0/164953869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55696af3cee0 0x55696af51a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb21ffb640 0 client.28202 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 >> 172.21.15.120:0/164953869 conn(0x55696af3ad10 msgr2=0x55696af3c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 shutdown_connections 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 1 -- 172.21.15.120:0/164953869 wait complete. 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.063+0000 7fcb42f58740 20 client.28202 trim_cache size 0 max 16384 2024-09-10T23:10:30.073 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5", 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 msgr2=0x7fcb240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 0x7fcb240b77a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0ac490 tx=0x7fcb2c0e9990 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 msgr2=0x7fcb3c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00e9f0 tx=0x7fcb2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 shutdown_connections 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb240b52e0 0x7fcb240b77a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb3c108020 0x7fcb3c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb3c1076a0 0x7fcb3c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 --2- 172.21.15.120:0/3295182338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c101c90 0x7fcb3c199270 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00e9f0 tx=0x7fcb2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 >> 172.21.15.120:0/3295182338 conn(0x7fcb3c0fd700 msgr2=0x7fcb3c06acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 shutdown_connections 2024-09-10T23:10:30.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:30.064+0000 7fcb40b83640 1 -- 172.21.15.120:0/3295182338 wait complete. 2024-09-10T23:10:30.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:30 smithi120 ceph-mon[31247]: pgmap v888: 97 pgs: 97 active+clean; 235 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-10T23:10:30.375 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:30 smithi176 ceph-mon[34982]: pgmap v888: 97 pgs: 97 active+clean; 235 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-10T23:10:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:30 smithi153 ceph-mon[34951]: pgmap v888: 97 pgs: 97 active+clean; 235 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-10T23:10:30.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:10:30.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134008 21.84 MB/sec execute 205 sec latency 442.663 ms 2024-09-10T23:10:31.072 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:10:31.367 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:31 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:10:31.368 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:31.368 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:31 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:31 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:31 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:10:31.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:31 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:10:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:31 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 -- 172.21.15.120:0/1539607372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098107800 msgr2=0x7ff09810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 --2- 172.21.15.120:0/1539607372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098107800 0x7ff09810dfe0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff090009be0 tx=0x7ff09002a6c0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 -- 172.21.15.120:0/1539607372 shutdown_connections 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 --2- 172.21.15.120:0/1539607372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098107800 0x7ff09810dfe0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 --2- 172.21.15.120:0/1539607372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff0981072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 --2- 172.21.15.120:0/1539607372 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098101450 0x7ff098101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.605+0000 7ff09eebb640 1 -- 172.21.15.120:0/1539607372 >> 172.21.15.120:0/1539607372 conn(0x7ff098078140 msgr2=0x7ff0980ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.606+0000 7ff09eebb640 1 -- 172.21.15.120:0/1539607372 shutdown_connections 2024-09-10T23:10:31.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.606+0000 7ff09eebb640 1 -- 172.21.15.120:0/1539607372 wait complete. 2024-09-10T23:10:31.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.607+0000 7ff09eebb640 1 Processor -- start 2024-09-10T23:10:31.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.607+0000 7ff09eebb640 1 -- start start 2024-09-10T23:10:31.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.607+0000 7ff09eebb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 0x7ff098196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff09eebb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff098197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff09eebb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 0x7ff09819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff09cc57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 0x7ff098196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff09cc57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 0x7ff098196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34976/0 (socket says 172.21.15.120:34976) 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff09cc57640 1 -- 172.21.15.120:0/3903814231 learned_addr learned my addr 172.21.15.120:0/3903814231 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.608+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff098197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff09d458640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 0x7ff09819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff098110bd0 con 0x7ff098101450 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff098110a50 con 0x7ff098106e60 2024-09-10T23:10:31.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff098110d50 con 0x7ff098107800 2024-09-10T23:10:31.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff097fff640 1 -- 172.21.15.120:0/3903814231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 msgr2=0x7ff09819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 0x7ff09819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff097fff640 1 -- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 msgr2=0x7ff098196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.609+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 0x7ff098196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff097fff640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09819cf70 con 0x7ff098106e60 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff09d458640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 0x7ff09819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff098197400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff08400ca40 tx=0x7ff08400cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff095ffb640 1 -- 172.21.15.120:0/3903814231 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084023070 con 0x7ff098106e60 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff095ffb640 1 -- 172.21.15.120:0/3903814231 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08401e410 con 0x7ff098106e60 2024-09-10T23:10:31.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.610+0000 7ff095ffb640 1 -- 172.21.15.120:0/3903814231 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08401c570 con 0x7ff098106e60 2024-09-10T23:10:31.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.611+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09819d260 con 0x7ff098106e60 2024-09-10T23:10:31.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.611+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981a39a0 con 0x7ff098106e60 2024-09-10T23:10:31.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 Processor -- start 2024-09-10T23:10:31.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 -- start start 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a05a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a05ad30 0x55f25a05d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.614+0000 7ff0a1290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f259fc4e50 con 0x55f25a04e480 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff0a1290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f259fb4280 con 0x55f25a05ad30 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34982/0 (socket says 172.21.15.120:34982) 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 -- 172.21.15.120:0/2448790741 learned_addr learned my addr 172.21.15.120:0/2448790741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09cc57640 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a05a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff0a1290740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f259fb8090 con 0x55f25a04eda0 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff095ffb640 1 -- 172.21.15.120:0/3903814231 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff08400b040 con 0x7ff098106e60 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff097fff640 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a05ad30 0x55f25a05d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 -- 172.21.15.120:0/2448790741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 msgr2=0x55f25a05a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a05a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 -- 172.21.15.120:0/2448790741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a05ad30 msgr2=0x55f25a05d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a05ad30 0x55f25a05d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.615+0000 7ff09d458640 1 -- 172.21.15.120:0/2448790741 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f259d12650 con 0x55f25a04e480 2024-09-10T23:10:31.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.616+0000 7ff09d458640 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff09002acb0 tx=0x7ff090004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20734d1c61ca42ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.616+0000 7ff082ffd640 1 -- 172.21.15.120:0/2448790741 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090041070 con 0x55f25a04e480 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.616+0000 7ff082ffd640 1 -- 172.21.15.120:0/2448790741 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff090030e20 con 0x55f25a04e480 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.616+0000 7ff095ffb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 0x7ff0880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.616+0000 7ff082ffd640 1 -- 172.21.15.120:0/2448790741 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090036040 con 0x55f25a04e480 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.617+0000 7ff09cc57640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 0x7ff0880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.617+0000 7ff095ffb640 1 -- 172.21.15.120:0/3903814231 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff0840edee0 con 0x7ff098106e60 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff09cc57640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 0x7ff0880b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55f25a04d190 tx=0x7ff08c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 -- 172.21.15.120:0/2448790741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 msgr2=0x55f25a04e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff09002acb0 tx=0x7ff090004a60 comp rx=0 tx=0).stop 2024-09-10T23:10:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 -- 172.21.15.120:0/2448790741 shutdown_connections 2024-09-10T23:10:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a05ad30 0x55f25a05d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a05a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 --2- 172.21.15.120:0/2448790741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a04e860 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.618+0000 7ff0a1290740 1 -- 172.21.15.120:0/2448790741 >> 172.21.15.120:0/2448790741 conn(0x55f25a04c2b0 msgr2=0x55f25a04d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.619+0000 7ff0a1290740 1 -- 172.21.15.120:0/2448790741 shutdown_connections 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.619+0000 7ff0a1290740 1 -- 172.21.15.120:0/2448790741 wait complete. 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.619+0000 7ff0a1290740 1 -- start start 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.619+0000 7ff0a1290740 1 Processor -- start 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.620+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04e480 0x55f25a05a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.620+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a057a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.620+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.620+0000 7ff09cc57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04e480 0x55f25a05a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff0a1290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f259e15700 con 0x55f25a05ad30 2024-09-10T23:10:31.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff0a1290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f259f4f110 con 0x55f25a04eda0 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff0a1290740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f25a05d950 con 0x55f25a04e480 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff0a1290740 10 client.0 _ll_register_callbacks cb 0x55f259f68990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff097fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a057a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34996/0 (socket says 172.21.15.120:34996) 2024-09-10T23:10:31.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.621+0000 7ff09d458640 1 -- 172.21.15.120:0/1089717666 learned_addr learned my addr 172.21.15.120:0/1089717666 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.622+0000 7ff09d458640 1 -- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04e480 msgr2=0x55f25a05a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.622+0000 7ff09d458640 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04e480 0x55f25a05a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.622+0000 7ff09d458640 1 -- 172.21.15.120:0/1089717666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 msgr2=0x55f25a057a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.622+0000 7ff09d458640 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a057a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.622+0000 7ff09d458640 1 -- 172.21.15.120:0/1089717666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f259f7cc40 con 0x55f25a05ad30 2024-09-10T23:10:31.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.623+0000 7ff09d458640 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff09002abe0 tx=0x7ff090009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f151065a37afef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.623+0000 7ff077fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:31.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.623+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090036050 con 0x55f25a05ad30 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.623+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff090003b70 con 0x55f25a05ad30 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.624+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090030e20 con 0x55f25a05ad30 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.624+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff090041040 con 0x55f25a05ad30 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.624+0000 7ff0a1290740 10 client.18993 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.624+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f25a0692e0 con 0x55f25a05ad30 2024-09-10T23:10:31.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff090030e20 con 0x55f25a05ad30 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff09003d820 con 0x55f25a05ad30 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 10 client.18993 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 20 client.18993 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 4 client.18993 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:31.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.625+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f259fc6da0 con 0x55f25a06a630 2024-09-10T23:10:31.628 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.627+0000 7ff09cc57640 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.632+0000 7ff077fff640 10 client.18993 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:31.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.632+0000 7ff09cc57640 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a06ca20 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8ec09febdaaed server_cookie=114043c45f4177a6 in_seq=0 out_seq=0 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff077fff640 1 -- 172.21.15.120:0/1089717666 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f259fc6da0 con 0x55f25a06a630 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff077fff640 10 client.18993 handle_command_reply: tid=0 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 client.18993 shutdown 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 msgr2=0x55f25a058020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff09002abe0 tx=0x7ff090009800 comp rx=0 tx=0).stop 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 shutdown_connections 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a06ca20 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a058020 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff077fff640 0 client.18993 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a057a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1089717666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04e480 0x55f25a05a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 >> 172.21.15.120:0/1089717666 conn(0x55f25a04c2b0 msgr2=0x55f25a051e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 shutdown_connections 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 1 -- 172.21.15.120:0/1089717666 wait complete. 2024-09-10T23:10:31.640 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.638+0000 7ff0a1290740 20 client.18993 trim_cache size 0 max 16384 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 Processor -- start 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 -- start start 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04e480 0x55f25a054c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a055190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00c390 con 0x55f25a05ad30 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00ec10 con 0x55f25a04e480 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff0a1290740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00ca00 con 0x55f25a04eda0 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff097fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a055190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35012/0 (socket says 172.21.15.120:35012) 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 -- 172.21.15.120:0/1831231559 learned_addr learned my addr 172.21.15.120:0/1831231559 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09cc57640 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04e480 0x55f25a054c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 -- 172.21.15.120:0/1831231559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 msgr2=0x55f25a055190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a055190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 -- 172.21.15.120:0/1831231559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04e480 msgr2=0x55f25a054c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04e480 0x55f25a054c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.658+0000 7ff09d458640 1 -- 172.21.15.120:0/1831231559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f25a055f40 con 0x55f25a05ad30 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.659+0000 7ff09d458640 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff09003dc40 tx=0x7ff0900464b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf4ff102759aa40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.659+0000 7ff0827fc640 1 -- 172.21.15.120:0/1831231559 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090036040 con 0x55f25a05ad30 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.659+0000 7ff0827fc640 1 -- 172.21.15.120:0/1831231559 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff090003e20 con 0x55f25a05ad30 2024-09-10T23:10:31.661 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.659+0000 7ff0827fc640 1 -- 172.21.15.120:0/1831231559 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003d6c0 con 0x55f25a05ad30 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- 172.21.15.120:0/1831231559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 msgr2=0x55f25a0562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff09003dc40 tx=0x7ff0900464b0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- 172.21.15.120:0/1831231559 shutdown_connections 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a05ad30 0x55f25a0562c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a04eda0 0x55f25a055190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 --2- 172.21.15.120:0/1831231559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04e480 0x55f25a054c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- 172.21.15.120:0/1831231559 >> 172.21.15.120:0/1831231559 conn(0x55f25a04c2b0 msgr2=0x55f25a059890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- 172.21.15.120:0/1831231559 shutdown_connections 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- 172.21.15.120:0/1831231559 wait complete. 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 -- start start 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.660+0000 7ff0a1290740 1 Processor -- start 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a05ad30 0x55f25a06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:35014/0 (socket says 172.21.15.120:35014) 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 -- 172.21.15.120:0/3903756266 learned_addr learned my addr 172.21.15.120:0/3903756266 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00d0d0 con 0x55f25a04e480 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00e540 con 0x55f25a04eda0 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00d7a0 con 0x55f25a05ad30 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff0a1290740 10 client.0 _ll_register_callbacks cb 0x7ff08c0071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09d458640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a05ad30 0x55f25a06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 -- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a05ad30 msgr2=0x55f25a06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a05ad30 0x55f25a06ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 -- 172.21.15.120:0/3903756266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 msgr2=0x55f25a06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a06e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff09cc57640 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f25a06f190 con 0x55f25a04e480 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.661+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff09cc57640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff08c010fd0 tx=0x7ff08c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414ccb4a2c3aeaf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c01c040 con 0x55f25a04e480 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff08c023700 con 0x55f25a04e480 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090003e20 con 0x55f25a04e480 2024-09-10T23:10:31.667 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff08c03a070 con 0x55f25a04e480 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff0a1290740 10 client.18999 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f25a0d80f0 con 0x55f25a04e480 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08c02e070 con 0x55f25a04e480 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.662+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff08c014a00 con 0x55f25a04e480 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 10 client.18999 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 10 client.18999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 20 client.18999 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a0da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 4 client.18999 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff090003e20 con 0x55f25a06a630 2024-09-10T23:10:31.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a0da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:31.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff077fff640 10 client.18999 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:31.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.663+0000 7ff097fff640 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a0da910 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccc03a658d8f979 server_cookie=2bddc1fc285eb723 in_seq=0 out_seq=0 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff077fff640 1 -- 172.21.15.120:0/3903756266 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9098 (crc 0 0 0) 0x7ff090003e20 con 0x55f25a06a630 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff077fff640 10 client.18999 handle_command_reply: tid=0 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 client.18999 shutdown 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 msgr2=0x55f25a070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff08c010fd0 tx=0x7ff08c0047d0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 shutdown_connections 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f25a06a630 0x55f25a0da910 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff077fff640 0 client.18999 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:31.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f25a05ad30 0x55f25a06ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f25a04eda0 0x55f25a06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 --2- 172.21.15.120:0/3903756266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f25a04e480 0x55f25a070090 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 >> 172.21.15.120:0/3903756266 conn(0x55f25a04c2b0 msgr2=0x55f25a04abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 shutdown_connections 2024-09-10T23:10:31.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 1 -- 172.21.15.120:0/3903756266 wait complete. 2024-09-10T23:10:31.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.666+0000 7ff0a1290740 20 client.18999 trim_cache size 0 max 16384 2024-09-10T23:10:31.672 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (118 inodes in the stack)", 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5", 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:10:31.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.674 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:10:31.677 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:10:31.679 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:10:31.688 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:10:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.692 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:10:31.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 msgr2=0x7ff0880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 0x7ff0880b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55f25a04d190 tx=0x7ff08c008040 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 msgr2=0x7ff098197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff098197400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff08400ca40 tx=0x7ff08400cf00 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 shutdown_connections 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0880b5360 0x7ff0880b7820 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff098107800 0x7ff09819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff098106e60 0x7ff098197400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 --2- 172.21.15.120:0/3903814231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff098101450 0x7ff098196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 >> 172.21.15.120:0/3903814231 conn(0x7ff098078140 msgr2=0x7ff0981054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 shutdown_connections 2024-09-10T23:10:31.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:10:31.667+0000 7ff09eebb640 1 -- 172.21.15.120:0/3903814231 wait complete. 2024-09-10T23:10:31.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134071 21.78 MB/sec execute 206 sec latency 520.015 ms 2024-09-10T23:10:32.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:32 smithi120 ceph-mon[31247]: pgmap v889: 97 pgs: 97 active+clean; 245 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:10:32.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:32 smithi153 ceph-mon[34951]: pgmap v889: 97 pgs: 97 active+clean; 245 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:10:32.561 INFO:tasks.cephfs.filesystem:scrub status for tag:a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5 - {'path': '/', 'tag': 'a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5', 'options': 'recursive,force'} 2024-09-10T23:10:32.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:32 smithi176 ceph-mon[34982]: pgmap v889: 97 pgs: 97 active+clean; 245 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:10:32.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134467 21.81 MB/sec execute 207 sec latency 551.432 ms 2024-09-10T23:10:33.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:33 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:33.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:33 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:33.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:33 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:10:33.877 INFO:tasks.workunit.client.0.smithi120.stdout: 10 134912 21.81 MB/sec execute 208 sec latency 55.303 ms 2024-09-10T23:10:34.370 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:34 smithi120 ceph-mon[31247]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-09-10T23:10:34.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:34 smithi153 ceph-mon[34951]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-09-10T23:10:34.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:34 smithi176 ceph-mon[34982]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-09-10T23:10:34.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135211 21.79 MB/sec execute 209 sec latency 97.125 ms 2024-09-10T23:10:35.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135426 21.73 MB/sec execute 210 sec latency 263.878 ms 2024-09-10T23:10:36.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:36 smithi120 ceph-mon[31247]: pgmap v891: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-09-10T23:10:36.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:36 smithi153 ceph-mon[34951]: pgmap v891: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-09-10T23:10:36.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:36 smithi176 ceph-mon[34982]: pgmap v891: 97 pgs: 97 active+clean; 252 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-09-10T23:10:36.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135799 21.70 MB/sec execute 211 sec latency 68.456 ms 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.061+0000 7fcb6bee6640 1 -- 172.21.15.120:0/3265867074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64101860 msgr2=0x7fcb64101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.061+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/3265867074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64101860 0x7fcb64101c60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009930 tx=0x7fcb6002a420 comp rx=0 tx=0).stop 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 -- 172.21.15.120:0/3265867074 shutdown_connections 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/3265867074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 0x7fcb64109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/3265867074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb641029b0 0x7fcb64102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/3265867074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb64101860 0x7fcb64101c60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 -- 172.21.15.120:0/3265867074 >> 172.21.15.120:0/3265867074 conn(0x7fcb640fd140 msgr2=0x7fcb640ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 -- 172.21.15.120:0/3265867074 shutdown_connections 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 -- 172.21.15.120:0/3265867074 wait complete. 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 Processor -- start 2024-09-10T23:10:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 -- start start 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.062+0000 7fcb6bee6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6bee6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 0x7fcb6419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6bee6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d4d0 con 0x7fcb641029b0 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6bee6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d350 con 0x7fcb64103370 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6bee6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410d650 con 0x7fcb64101860 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38168/0 (socket says 172.21.15.120:38168) 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 -- 172.21.15.120:0/1954005720 learned_addr learned my addr 172.21.15.120:0/1954005720 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69c82640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47664/0 (socket says 172.21.15.120:47664) 2024-09-10T23:10:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6a483640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 0x7fcb6419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 -- 172.21.15.120:0/1954005720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 msgr2=0x7fcb64198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 -- 172.21.15.120:0/1954005720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 msgr2=0x7fcb6419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 0x7fcb6419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 -- 172.21.15.120:0/1954005720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419a910 con 0x7fcb641029b0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69c82640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb69481640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400e9f0 tx=0x7fcb5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb5affd640 1 -- 172.21.15.120:0/1954005720 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400cd80 con 0x7fcb641029b0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb5affd640 1 -- 172.21.15.120:0/1954005720 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb540054d0 con 0x7fcb641029b0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419ac00 con 0x7fcb641029b0 2024-09-10T23:10:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.063+0000 7fcb5affd640 1 -- 172.21.15.120:0/1954005720 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54010690 con 0x7fcb641029b0 2024-09-10T23:10:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.064+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a12d0 con 0x7fcb641029b0 2024-09-10T23:10:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.066+0000 7fcb6ecb4740 1 Processor -- start 2024-09-10T23:10:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.066+0000 7fcb6ecb4740 1 -- start start 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.066+0000 7fcb5affd640 1 -- 172.21.15.120:0/1954005720 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcb54004e90 con 0x7fcb641029b0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bfeec70 0x55813bff1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 0x55813bff64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55813bf5c6f0 con 0x55813bff3fe0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55813bf4bb20 con 0x55813bfeec70 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55813bf4f930 con 0x55813bff15a0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 0x55813bff64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:47672/0 (socket says 172.21.15.120:47672) 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 -- 172.21.15.120:0/709480279 learned_addr learned my addr 172.21.15.120:0/709480279 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb6a483640 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bfeec70 0x55813bff1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 -- 172.21.15.120:0/709480279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bfeec70 msgr2=0x55813bff1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bfeec70 0x55813bff1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 -- 172.21.15.120:0/709480279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 msgr2=0x55813bff64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 0x55813bff64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 -- 172.21.15.120:0/709480279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813bf11530 con 0x55813bff15a0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69481640 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 0x55813bff64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb5affd640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 0x7fcb500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.067+0000 7fcb69c82640 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009930 tx=0x7fcb60005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5f19537d76d13b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.068+0000 7fcb6a483640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 0x7fcb500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.068+0000 7fcb4ffff640 1 -- 172.21.15.120:0/709480279 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60035070 con 0x55813bff15a0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.068+0000 7fcb5affd640 1 -- 172.21.15.120:0/1954005720 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb540e5fb0 con 0x7fcb641029b0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.068+0000 7fcb4ffff640 1 -- 172.21.15.120:0/709480279 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb600318a0 con 0x55813bff15a0 2024-09-10T23:10:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.068+0000 7fcb4ffff640 1 -- 172.21.15.120:0/709480279 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60003e40 con 0x55813bff15a0 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/709480279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 msgr2=0x55813bff3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6a483640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 0x7fcb500b77a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcb64199b50 tx=0x7fcb5c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009930 tx=0x7fcb60005e80 comp rx=0 tx=0).stop 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/709480279 shutdown_connections 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff3fe0 0x55813bff64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff15a0 0x55813bff3990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/709480279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bfeec70 0x55813bff1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/709480279 >> 172.21.15.120:0/709480279 conn(0x55813bfe9dc0 msgr2=0x55813bfeab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/709480279 shutdown_connections 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/709480279 wait complete. 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 -- start start 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.069+0000 7fcb6ecb4740 1 Processor -- start 2024-09-10T23:10:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bff8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813bff94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55813be83ab0 con 0x55813bff15a0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55813bef1730 con 0x55813bff3fe0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55813bff6f60 con 0x55813bfeec70 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 10 client.0 _ll_register_callbacks cb 0x55813bf456b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bff8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38180/0 (socket says 172.21.15.120:38180) 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 -- 172.21.15.120:0/534216297 learned_addr learned my addr 172.21.15.120:0/534216297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6a483640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813bff94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 -- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 msgr2=0x55813bff8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bff8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 -- 172.21.15.120:0/534216297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 msgr2=0x55813bff94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813bff94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 -- 172.21.15.120:0/534216297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813bcad790 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69c82640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bff8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb69481640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400e9c0 tx=0x7fcb54009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bf9de58532c6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540ac570 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb540e9c40 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb600318a0 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.070+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb540abf10 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 10 client.19011 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55813bd26620 con 0x55813bff15a0 2024-09-10T23:10:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb600318a0 con 0x55813bff15a0 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb540e80c0 con 0x55813bff15a0 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 10 client.19011 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 10 client.19011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 20 client.19011 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c003af0 0x55813c005ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 4 client.19011 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55813bf5e640 con 0x55813c003af0 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb69c82640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c003af0 0x55813c005ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb4d7fa640 10 client.19011 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.071+0000 7fcb69c82640 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c003af0 0x55813c005ee0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515320eb8be16097 server_cookie=1a4faf39e1b3b168 in_seq=0 out_seq=0 2024-09-10T23:10:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/534216297 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55813bf5e640 con 0x55813c003af0 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb4d7fa640 10 client.19011 handle_command_reply: tid=0 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 client.19011 shutdown 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 msgr2=0x55813bff8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400e9c0 tx=0x7fcb54009310 comp rx=0 tx=0).stop 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 shutdown_connections 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c003af0 0x55813c005ee0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813bff94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bff8920 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/534216297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bff8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb4d7fa640 0 client.19011 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 >> 172.21.15.120:0/534216297 conn(0x55813bfe9dc0 msgr2=0x55813bfe4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 shutdown_connections 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/534216297 wait complete. 2024-09-10T23:10:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.075+0000 7fcb6ecb4740 20 client.19011 trim_cache size 0 max 16384 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 Processor -- start 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 -- start start 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 0x55813bfee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff3fe0 0x55813bfece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a5c80 con 0x55813bfeec70 2024-09-10T23:10:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a8500 con 0x55813bff15a0 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a62f0 con 0x55813bff3fe0 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38184/0 (socket says 172.21.15.120:38184) 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb69c82640 1 -- 172.21.15.120:0/893513535 learned_addr learned my addr 172.21.15.120:0/893513535 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb6a483640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff3fe0 0x55813bfece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.090+0000 7fcb69481640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 0x55813bfee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 -- 172.21.15.120:0/893513535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff3fe0 msgr2=0x55813bfece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff3fe0 0x55813bfece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 -- 172.21.15.120:0/893513535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 msgr2=0x55813bfee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 0x55813bfee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 -- 172.21.15.120:0/893513535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813bff3d00 con 0x55813bfeec70 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69481640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 0x55813bfee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:10:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb69c82640 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006010 tx=0x7fcb60009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245b82a5564edb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb4f7fe640 1 -- 172.21.15.120:0/893513535 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60035040 con 0x55813bfeec70 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb4f7fe640 1 -- 172.21.15.120:0/893513535 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60031d10 con 0x55813bfeec70 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.091+0000 7fcb4f7fe640 1 -- 172.21.15.120:0/893513535 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6004a040 con 0x55813bfeec70 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/893513535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 msgr2=0x55813bfee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcb60006010 tx=0x7fcb60009660 comp rx=0 tx=0).stop 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/893513535 shutdown_connections 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bff3fe0 0x55813bfece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff15a0 0x55813bfee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/893513535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bfeec70 0x55813bfee100 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/893513535 >> 172.21.15.120:0/893513535 conn(0x55813bfe9dc0 msgr2=0x55813bfe0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/893513535 shutdown_connections 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/893513535 wait complete. 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 -- start start 2024-09-10T23:10:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.092+0000 7fcb6ecb4740 1 Processor -- start 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bfff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813c06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a69c0 con 0x55813bff15a0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a7e30 con 0x55813bff3fe0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641a7090 con 0x55813bfeec70 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 10 client.0 _ll_register_callbacks cb 0x7fcb540e9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38186/0 (socket says 172.21.15.120:38186) 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69c82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bfff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 -- 172.21.15.120:0/213812783 learned_addr learned my addr 172.21.15.120:0/213812783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6a483640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813c06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 -- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 msgr2=0x55813bfff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bfff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 -- 172.21.15.120:0/213812783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 msgr2=0x55813c06e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813c06e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 -- 172.21.15.120:0/213812783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813c06f120 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb69481640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcb540e9830 tx=0x7fcb540a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d7bbacf76981d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540ac0f0 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcb540aba00 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb60031d10 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb540ac570 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 10 client.19017 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55813c070430 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.093+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60031d10 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fcb540a80f0 con 0x55813bff15a0 2024-09-10T23:10:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 10 client.19017 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 10 client.19017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 20 client.19017 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c070e10 0x55813c073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 4 client.19017 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55813bf5a0f0 con 0x55813c070e10 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb69c82640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c070e10 0x55813c073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb69c82640 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c070e10 0x55813c073230 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549964b072dec9e5 server_cookie=f2cfad04f5955da5 in_seq=0 out_seq=0 2024-09-10T23:10:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.094+0000 7fcb4d7fa640 10 client.19017 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:37.448 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:37 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:10:37.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:37 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:10:37.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:37 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:10:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb4d7fa640 1 -- 172.21.15.120:0/213812783 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55813bf5a0f0 con 0x55813c070e10 2024-09-10T23:10:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb4d7fa640 10 client.19017 handle_command_reply: tid=0 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 client.19017 shutdown 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 msgr2=0x55813bfffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcb540e9830 tx=0x7fcb540a8c40 comp rx=0 tx=0).stop 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 shutdown_connections 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55813c070e10 0x55813c073230 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55813bff3fe0 0x55813c06e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55813bff15a0 0x55813bfffde0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 --2- 172.21.15.120:0/213812783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55813bfeec70 0x55813bfff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 >> 172.21.15.120:0/213812783 conn(0x55813bfe9dc0 msgr2=0x55813bfece10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb4d7fa640 0 client.19017 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 shutdown_connections 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.760+0000 7fcb6ecb4740 1 -- 172.21.15.120:0/213812783 wait complete. 2024-09-10T23:10:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6ecb4740 20 client.19017 trim_cache size 0 max 16384 2024-09-10T23:10:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:10:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:10:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:10:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 msgr2=0x7fcb500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 0x7fcb500b77a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcb64199b50 tx=0x7fcb5c009600 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 msgr2=0x7fcb64199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400e9f0 tx=0x7fcb5400edf0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 shutdown_connections 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcb500b52e0 0x7fcb500b77a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb64103370 0x7fcb6419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb641029b0 0x7fcb64199420 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 --2- 172.21.15.120:0/1954005720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb64101860 0x7fcb64198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 >> 172.21.15.120:0/1954005720 conn(0x7fcb640fd140 msgr2=0x7fcb640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 shutdown_connections 2024-09-10T23:10:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:37.761+0000 7fcb6bee6640 1 -- 172.21.15.120:0/1954005720 wait complete. 2024-09-10T23:10:37.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 135998 21.65 MB/sec execute 212 sec latency 344.480 ms 2024-09-10T23:10:38.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:38 smithi120 ceph-mon[31247]: pgmap v892: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 689 op/s 2024-09-10T23:10:38.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:38 smithi153 ceph-mon[34951]: pgmap v892: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 689 op/s 2024-09-10T23:10:38.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:38 smithi176 ceph-mon[34982]: pgmap v892: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.1 MiB/s wr, 689 op/s 2024-09-10T23:10:38.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 136471 21.64 MB/sec execute 213 sec latency 139.746 ms 2024-09-10T23:10:39.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 137060 21.70 MB/sec execute 214 sec latency 49.242 ms 2024-09-10T23:10:40.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:40 smithi120 ceph-mon[31247]: pgmap v893: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 444 op/s 2024-09-10T23:10:40.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:40 smithi153 ceph-mon[34951]: pgmap v893: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 444 op/s 2024-09-10T23:10:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:40 smithi176 ceph-mon[34982]: pgmap v893: 97 pgs: 97 active+clean; 254 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 444 op/s 2024-09-10T23:10:40.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 137537 21.71 MB/sec execute 215 sec latency 176.592 ms 2024-09-10T23:10:41.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 138019 21.78 MB/sec execute 216 sec latency 79.425 ms 2024-09-10T23:10:42.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:42 smithi120 ceph-mon[31247]: pgmap v894: 97 pgs: 97 active+clean; 246 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-09-10T23:10:42.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:42 smithi153 ceph-mon[34951]: pgmap v894: 97 pgs: 97 active+clean; 246 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-09-10T23:10:42.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:42 smithi176 ceph-mon[34982]: pgmap v894: 97 pgs: 97 active+clean; 246 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-09-10T23:10:42.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 138449 21.80 MB/sec execute 217 sec latency 103.700 ms 2024-09-10T23:10:43.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 138779 21.78 MB/sec execute 218 sec latency 97.211 ms 2024-09-10T23:10:44.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:44 smithi120 ceph-mon[31247]: pgmap v895: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-09-10T23:10:44.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:44 smithi153 ceph-mon[34951]: pgmap v895: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-09-10T23:10:44.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:44 smithi176 ceph-mon[34982]: pgmap v895: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 668 op/s 2024-09-10T23:10:44.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 139156 21.76 MB/sec execute 219 sec latency 73.397 ms 2024-09-10T23:10:45.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 139462 21.71 MB/sec execute 220 sec latency 221.906 ms 2024-09-10T23:10:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:46 smithi120 ceph-mon[31247]: pgmap v896: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-10T23:10:46.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:46 smithi153 ceph-mon[34951]: pgmap v896: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-10T23:10:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:46 smithi176 ceph-mon[34982]: pgmap v896: 97 pgs: 97 active+clean; 241 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-10T23:10:46.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 139830 21.68 MB/sec execute 221 sec latency 295.203 ms 2024-09-10T23:10:47.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 140441 21.74 MB/sec execute 222 sec latency 57.287 ms 2024-09-10T23:10:48.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:48 smithi120 ceph-mon[31247]: pgmap v897: 97 pgs: 97 active+clean; 247 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 689 op/s 2024-09-10T23:10:48.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:48 smithi153 ceph-mon[34951]: pgmap v897: 97 pgs: 97 active+clean; 247 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 689 op/s 2024-09-10T23:10:48.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:48 smithi176 ceph-mon[34982]: pgmap v897: 97 pgs: 97 active+clean; 247 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 689 op/s 2024-09-10T23:10:48.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 141096 21.81 MB/sec execute 223 sec latency 54.013 ms 2024-09-10T23:10:49.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 141636 21.87 MB/sec execute 224 sec latency 39.733 ms 2024-09-10T23:10:50.447 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:50 smithi120 ceph-mon[31247]: pgmap v898: 97 pgs: 97 active+clean; 254 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-10T23:10:50.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:50 smithi153 ceph-mon[34951]: pgmap v898: 97 pgs: 97 active+clean; 254 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-10T23:10:50.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:50 smithi176 ceph-mon[34982]: pgmap v898: 97 pgs: 97 active+clean; 254 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 549 op/s 2024-09-10T23:10:50.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142008 21.88 MB/sec execute 225 sec latency 114.594 ms 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.910+0000 7ff10e647640 1 -- 172.21.15.120:0/2524147668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 msgr2=0x7ff108103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.910+0000 7ff10e647640 1 --2- 172.21.15.120:0/2524147668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff108103f60 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009930 tx=0x7ff0f802a440 comp rx=0 tx=0).stop 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 -- 172.21.15.120:0/2524147668 shutdown_connections 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 --2- 172.21.15.120:0/2524147668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1081044a0 0x7ff10810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 --2- 172.21.15.120:0/2524147668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff108103f60 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 --2- 172.21.15.120:0/2524147668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff108102990 0x7ff108102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 -- 172.21.15.120:0/2524147668 >> 172.21.15.120:0/2524147668 conn(0x7ff1080fe250 msgr2=0x7ff108100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 -- 172.21.15.120:0/2524147668 shutdown_connections 2024-09-10T23:10:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 -- 172.21.15.120:0/2524147668 wait complete. 2024-09-10T23:10:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 Processor -- start 2024-09-10T23:10:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.911+0000 7ff10e647640 1 -- start start 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff10819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1081044a0 0x7ff1081a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff108076430 con 0x7ff108103ae0 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080762b0 con 0x7ff108102990 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10e647640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080765b0 con 0x7ff1081044a0 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff10819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1081044a0 0x7ff1081a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff10819fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38136/0 (socket says 172.21.15.120:38136) 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff1077fe640 1 -- 172.21.15.120:0/935811359 learned_addr learned my addr 172.21.15.120:0/935811359 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48880/0 (socket says 172.21.15.120:48880) 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 -- 172.21.15.120:0/935811359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1081044a0 msgr2=0x7ff1081a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1081044a0 0x7ff1081a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 -- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 msgr2=0x7ff10819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.912+0000 7ff107fff640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff10819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff107fff640 1 -- 172.21.15.120:0/935811359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1081a0f70 con 0x7ff108102990 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff107fff640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009fd0 tx=0x7ff0fc00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff1057fa640 1 -- 172.21.15.120:0/935811359 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc015020 con 0x7ff108102990 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1081a1260 con 0x7ff108102990 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff1057fa640 1 -- 172.21.15.120:0/935811359 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc011ce0 con 0x7ff108102990 2024-09-10T23:10:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff1057fa640 1 -- 172.21.15.120:0/935811359 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc010e40 con 0x7ff108102990 2024-09-10T23:10:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.913+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a7930 con 0x7ff108102990 2024-09-10T23:10:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 Processor -- start 2024-09-10T23:10:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 -- start start 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8af8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8a656f0 con 0x55ccb8af8bd0 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8a54b20 con 0x55ccb8af6190 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.916+0000 7ff11141b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8a58930 con 0x55ccb8af3860 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53502/0 (socket says 172.21.15.120:53502) 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 -- 172.21.15.120:0/1492374408 learned_addr learned my addr 172.21.15.120:0/1492374408 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff1077fe640 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff1057fa640 1 -- 172.21.15.120:0/935811359 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff0fc011830 con 0x7ff108102990 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 -- 172.21.15.120:0/1492374408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 msgr2=0x55ccb8af8580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8af8580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 -- 172.21.15.120:0/1492374408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 msgr2=0x55ccb8afb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 -- 172.21.15.120:0/1492374408 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb8a1a530 con 0x55ccb8af3860 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.917+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff100009f90 tx=0x7ff1000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341a24887044773 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff0ee7fc640 1 -- 172.21.15.120:0/1492374408 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100004b20 con 0x55ccb8af3860 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff0ee7fc640 1 -- 172.21.15.120:0/1492374408 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff100004cc0 con 0x55ccb8af3860 2024-09-10T23:10:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff0ee7fc640 1 -- 172.21.15.120:0/1492374408 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100005020 con 0x55ccb8af3860 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff1057fa640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 0x7ff0f00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff107fff640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 0x7ff0f00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.918+0000 7ff1057fa640 1 -- 172.21.15.120:0/935811359 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff0fc0e5560 con 0x7ff108102990 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 -- 172.21.15.120:0/1492374408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 msgr2=0x55ccb8af5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff100009f90 tx=0x7ff1000118d0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff107fff640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 0x7ff0f00b77a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006fd0 tx=0x7ff0fc010860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 -- 172.21.15.120:0/1492374408 shutdown_connections 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8af8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 --2- 172.21.15.120:0/1492374408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8af5c50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 -- 172.21.15.120:0/1492374408 >> 172.21.15.120:0/1492374408 conn(0x55ccb8aee9b0 msgr2=0x55ccb8aef740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 -- 172.21.15.120:0/1492374408 shutdown_connections 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.919+0000 7ff11141b740 1 -- 172.21.15.120:0/1492374408 wait complete. 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.920+0000 7ff11141b740 1 -- start start 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.920+0000 7ff11141b740 1 Processor -- start 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.920+0000 7ff11141b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.920+0000 7ff11141b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ccb898cab0 con 0x55ccb8af8bd0 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:48884/0 (socket says 172.21.15.120:48884) 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 -- 172.21.15.120:0/2369157044 learned_addr learned my addr 172.21.15.120:0/2369157044 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ccb89fa730 con 0x55ccb8af6190 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ccb8afbbb0 con 0x55ccb8af3860 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 10 client.0 _ll_register_callbacks cb 0x55ccb8a4e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 -- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 msgr2=0x55ccb8afcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 -- 172.21.15.120:0/2369157044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 msgr2=0x55ccb8afe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb87b6790 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009900 tx=0x7ff0f8004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84af4e338fb34908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff0e77fe640 10 client.28244 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.921+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff100004cc0 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2369157044 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8035040 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2369157044 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff0f8003ac0 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2369157044 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0f8003cc0 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 10 client.28244 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ccb882f620 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2369157044 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff0f80315a0 con 0x55ccb8af6190 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 10 client.28244 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 10 client.28244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 20 client.28244 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b08670 0x55ccb8b0aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 4 client.28244 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.922+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb8a67640 con 0x55ccb8b08670 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.923+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b08670 0x55ccb8b0aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.923+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b08670 0x55ccb8b0aa60 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e57b2aaa5d5a1e server_cookie=74466a78a4d82fd8 in_seq=0 out_seq=0 2024-09-10T23:10:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.923+0000 7ff0e77fe640 10 client.28244 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.924+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2369157044 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccb8a67640 con 0x55ccb8b08670 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.924+0000 7ff0e77fe640 10 client.28244 handle_command_reply: tid=0 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.924+0000 7ff11141b740 1 client.28244 shutdown 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 msgr2=0x55ccb8afd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009900 tx=0x7ff0f8004440 comp rx=0 tx=0).stop 2024-09-10T23:10:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 shutdown_connections 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b08670 0x55ccb8b0aa60 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8afe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af6190 0x55ccb8afd5d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff0e77fe640 0 client.28244 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 --2- 172.21.15.120:0/2369157044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 >> 172.21.15.120:0/2369157044 conn(0x55ccb8aee9b0 msgr2=0x55ccb8aedbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 shutdown_connections 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 1 -- 172.21.15.120:0/2369157044 wait complete. 2024-09-10T23:10:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.925+0000 7ff11141b740 20 client.28244 trim_cache size 0 max 16384 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 Processor -- start 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 -- start start 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 0x55ccb8afd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081ae500 con 0x55ccb8af8bd0 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081b0d80 con 0x55ccb8af3860 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.939+0000 7ff11141b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081aeb70 con 0x55ccb8af6190 2024-09-10T23:10:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 0x55ccb8afd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff10cbe4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38158/0 (socket says 172.21.15.120:38158) 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff10cbe4640 1 -- 172.21.15.120:0/421748804 learned_addr learned my addr 172.21.15.120:0/421748804 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 -- 172.21.15.120:0/421748804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 msgr2=0x55ccb8afcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 -- 172.21.15.120:0/421748804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 msgr2=0x55ccb8af8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 -- 172.21.15.120:0/421748804 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb8afe510 con 0x55ccb8af6190 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff1077fe640 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 0x55ccb8afd470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e76512719d61ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff0edffb640 1 -- 172.21.15.120:0/421748804 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8035030 con 0x55ccb8af6190 2024-09-10T23:10:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff0edffb640 1 -- 172.21.15.120:0/421748804 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8055070 con 0x55ccb8af6190 2024-09-10T23:10:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.940+0000 7ff0edffb640 1 -- 172.21.15.120:0/421748804 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f803c8f0 con 0x55ccb8af6190 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- 172.21.15.120:0/421748804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 msgr2=0x55ccb8afd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 0x55ccb8afd470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8000c00 tx=0x7ff0f8040000 comp rx=0 tx=0).stop 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- 172.21.15.120:0/421748804 shutdown_connections 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af8bd0 0x55ccb8af8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af6190 0x55ccb8afd470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 --2- 172.21.15.120:0/421748804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8afcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- 172.21.15.120:0/421748804 >> 172.21.15.120:0/421748804 conn(0x55ccb8aee9b0 msgr2=0x55ccb8aef390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- 172.21.15.120:0/421748804 shutdown_connections 2024-09-10T23:10:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- 172.21.15.120:0/421748804 wait complete. 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.941+0000 7ff11141b740 1 -- start start 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 Processor -- start 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8b044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af8bd0 0x55ccb8b032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081af240 con 0x55ccb8af6190 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081b06b0 con 0x55ccb8af3860 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081af910 con 0x55ccb8af8bd0 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff11141b740 10 client.0 _ll_register_callbacks cb 0x7ff0f8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38162/0 (socket says 172.21.15.120:38162) 2024-09-10T23:10:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 -- 172.21.15.120:0/2525837192 learned_addr learned my addr 172.21.15.120:0/2525837192 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8b044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af8bd0 0x55ccb8b032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 -- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af8bd0 msgr2=0x55ccb8b032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af8bd0 0x55ccb8b032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 -- 172.21.15.120:0/2525837192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 msgr2=0x55ccb8b044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8b044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 -- 172.21.15.120:0/2525837192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb8b0add0 con 0x55ccb8af6190 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff1077fe640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8031500 tx=0x7ff0f8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1908f21ad27316 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.942+0000 7ff0e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8035070 con 0x55ccb8af6190 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f8055070 con 0x55ccb8af6190 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff0f803c440 con 0x55ccb8af6190 2024-09-10T23:10:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0f80044a0 con 0x55ccb8af6190 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff11141b740 10 client.19035 fetch_fsmap learned FSMap version 26 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccb8b02710 con 0x55ccb8af6190 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8055070 con 0x55ccb8af6190 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.943+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff0f8046550 con 0x55ccb8af6190 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 10 client.19035 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 10 client.19035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 20 client.19035 populate_metadata read hostname 'smithi120' 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b75980 0x55ccb8b77da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 4 client.19035 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccb8a630f0 con 0x55ccb8b75980 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b75980 0x55ccb8b77da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff10cbe4640 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b75980 0x55ccb8b77da0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21028e3459825c30 server_cookie=f5038fc8a59ef6ea in_seq=0 out_seq=0 2024-09-10T23:10:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:50.944+0000 7ff0e77fe640 10 client.19035 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:51.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142058 21.79 MB/sec execute 226 sec latency 450.254 ms 2024-09-10T23:10:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff0e77fe640 1 -- 172.21.15.120:0/2525837192 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccb8a630f0 con 0x55ccb8b75980 2024-09-10T23:10:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff0e77fe640 10 client.19035 handle_command_reply: tid=0 2024-09-10T23:10:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 client.19035 shutdown 2024-09-10T23:10:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 msgr2=0x55ccb8b04b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8031500 tx=0x7ff0f8003e60 comp rx=0 tx=0).stop 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 shutdown_connections 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ccb8b75980 0x55ccb8b77da0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccb8af8bd0 0x55ccb8b032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ccb8af6190 0x55ccb8b04b80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 --2- 172.21.15.120:0/2525837192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ccb8af3860 0x55ccb8b044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 >> 172.21.15.120:0/2525837192 conn(0x55ccb8aee9b0 msgr2=0x55ccb8af8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff0e77fe640 0 client.19035 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:10:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.985+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 shutdown_connections 2024-09-10T23:10:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.986+0000 7ff11141b740 1 -- 172.21.15.120:0/2525837192 wait complete. 2024-09-10T23:10:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.986+0000 7ff11141b740 20 client.19035 trim_cache size 0 max 16384 2024-09-10T23:10:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:10:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:10:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:10:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 msgr2=0x7ff0f00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 0x7ff0f00b77a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006fd0 tx=0x7ff0fc010860 comp rx=0 tx=0).stop 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 msgr2=0x7ff10819f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009fd0 tx=0x7ff0fc00b870 comp rx=0 tx=0).stop 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 shutdown_connections 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff0f00b52e0 0x7ff0f00b77a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1081044a0 0x7ff1081a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff108103ae0 0x7ff10819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 --2- 172.21.15.120:0/935811359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff108102990 0x7ff10819f430 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:10:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 >> 172.21.15.120:0/935811359 conn(0x7ff1080fe250 msgr2=0x7ff1080ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:10:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 shutdown_connections 2024-09-10T23:10:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:10:51.987+0000 7ff10e647640 1 -- 172.21.15.120:0/935811359 wait complete. 2024-09-10T23:10:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:52 smithi120 ceph-mon[31247]: pgmap v899: 97 pgs: 97 active+clean; 259 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-10T23:10:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:52.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:10:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:52 smithi153 ceph-mon[34951]: pgmap v899: 97 pgs: 97 active+clean; 259 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-10T23:10:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:52.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:10:52.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:52 smithi176 ceph-mon[34982]: pgmap v899: 97 pgs: 97 active+clean; 259 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.6 MiB/s wr, 607 op/s 2024-09-10T23:10:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:10:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:10:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:10:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:10:52.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142354 21.76 MB/sec execute 227 sec latency 255.293 ms 2024-09-10T23:10:53.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 142728 21.74 MB/sec execute 228 sec latency 86.559 ms 2024-09-10T23:10:54.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:54 smithi120 ceph-mon[31247]: pgmap v900: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-09-10T23:10:54.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:54 smithi153 ceph-mon[34951]: pgmap v900: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-09-10T23:10:54.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:54 smithi176 ceph-mon[34982]: pgmap v900: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.2 MiB/s wr, 649 op/s 2024-09-10T23:10:54.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 143119 21.71 MB/sec execute 229 sec latency 88.100 ms 2024-09-10T23:10:55.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 143575 21.71 MB/sec execute 230 sec latency 264.053 ms 2024-09-10T23:10:56.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:56 smithi120 ceph-mon[31247]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-10T23:10:56.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:56 smithi153 ceph-mon[34951]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-10T23:10:56.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:56 smithi176 ceph-mon[34982]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-10T23:10:56.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 143979 21.72 MB/sec execute 231 sec latency 205.652 ms 2024-09-10T23:10:57.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 144562 21.77 MB/sec execute 232 sec latency 38.996 ms 2024-09-10T23:10:58.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:10:58 smithi120 ceph-mon[31247]: pgmap v902: 97 pgs: 97 active+clean; 246 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-09-10T23:10:58.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:10:58 smithi153 ceph-mon[34951]: pgmap v902: 97 pgs: 97 active+clean; 246 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-09-10T23:10:58.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:10:58 smithi176 ceph-mon[34982]: pgmap v902: 97 pgs: 97 active+clean; 246 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-09-10T23:10:58.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145105 21.83 MB/sec execute 233 sec latency 49.275 ms 2024-09-10T23:10:59.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145549 21.85 MB/sec execute 234 sec latency 66.359 ms 2024-09-10T23:11:00.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:00 smithi120 ceph-mon[31247]: pgmap v903: 97 pgs: 97 active+clean; 244 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-10T23:11:00.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:00 smithi153 ceph-mon[34951]: pgmap v903: 97 pgs: 97 active+clean; 244 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-10T23:11:00.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:00 smithi176 ceph-mon[34982]: pgmap v903: 97 pgs: 97 active+clean; 244 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-10T23:11:00.878 INFO:tasks.workunit.client.0.smithi120.stdout: 10 145851 21.82 MB/sec execute 235 sec latency 94.178 ms 2024-09-10T23:11:01.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 146085 21.78 MB/sec execute 236 sec latency 262.795 ms 2024-09-10T23:11:02.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:02 smithi120 ceph-mon[31247]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T23:11:02.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:02 smithi153 ceph-mon[34951]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T23:11:02.563 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:11:02.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:02 smithi176 ceph-mon[34982]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-09-10T23:11:02.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 146477 21.75 MB/sec execute 237 sec latency 140.979 ms 2024-09-10T23:11:02.881 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 -- 172.21.15.120:0/3316459540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e010b370 msgr2=0x7f98e0111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 --2- 172.21.15.120:0/3316459540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e010b370 0x7f98e0111b70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009ca0 tx=0x7f98d802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 -- 172.21.15.120:0/3316459540 shutdown_connections 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 --2- 172.21.15.120:0/3316459540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e010b370 0x7f98e0111b70 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 --2- 172.21.15.120:0/3316459540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010a9f0 0x7f98e010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 --2- 172.21.15.120:0/3316459540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e0104fe0 0x7f98e01053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 -- 172.21.15.120:0/3316459540 >> 172.21.15.120:0/3316459540 conn(0x7f98e0100a10 msgr2=0x7f98e0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.404 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 -- 172.21.15.120:0/3316459540 shutdown_connections 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.389+0000 7f98e6807640 1 -- 172.21.15.120:0/3316459540 wait complete. 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.390+0000 7f98e6807640 1 Processor -- start 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.390+0000 7f98e6807640 1 -- start start 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e010a9f0 0x7f98e010eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010b370 0x7f98e010f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0114fa0 con 0x7f98e0104fe0 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0114e20 con 0x7f98e010b370 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e6807640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0115120 con 0x7f98e010a9f0 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43550/0 (socket says 172.21.15.120:43550) 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 -- 172.21.15.120:0/480282819 learned_addr learned my addr 172.21.15.120:0/480282819 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:03.405 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98e4da4640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010b370 0x7f98e010f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98df7fe640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e010a9f0 0x7f98e010eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 -- 172.21.15.120:0/480282819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e010a9f0 msgr2=0x7f98e010eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e010a9f0 0x7f98e010eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 -- 172.21.15.120:0/480282819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010b370 msgr2=0x7f98e010f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010b370 0x7f98e010f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 -- 172.21.15.120:0/480282819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e010f8e0 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dffff640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e9f0 tx=0x7f98d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.391+0000 7f98dd7fa640 1 -- 172.21.15.120:0/480282819 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d000cd80 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.392+0000 7f98dd7fa640 1 -- 172.21.15.120:0/480282819 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d00054d0 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.392+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e01abda0 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.392+0000 7f98dd7fa640 1 -- 172.21.15.120:0/480282819 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0010690 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.392+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01ac250 con 0x7f98e0104fe0 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 Processor -- start 2024-09-10T23:11:03.406 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 -- start start 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98dd7fa640 1 -- 172.21.15.120:0/480282819 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f98d0004e90 con 0x7f98e0104fe0 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc580b4e50 con 0x55dc5814ad30 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc580a4280 con 0x55dc5813e480 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dc580a8090 con 0x55dc5813eda0 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43554/0 (socket says 172.21.15.120:43554) 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98df7fe640 1 -- 172.21.15.120:0/627985629 learned_addr learned my addr 172.21.15.120:0/627985629 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98dffff640 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.394+0000 7f98e4da4640 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.407 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 -- 172.21.15.120:0/627985629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 msgr2=0x55dc5814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5814a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 -- 172.21.15.120:0/627985629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 msgr2=0x55dc5813e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 -- 172.21.15.120:0/627985629 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc57e02650 con 0x55dc5814ad30 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98df7fe640 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c10 tx=0x7f98d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36a4aa5411438a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98c67fc640 1 -- 172.21.15.120:0/627985629 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004230 con 0x55dc5814ad30 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98dd7fa640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 0x7f98c80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98c67fc640 1 -- 172.21.15.120:0/627985629 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d40043d0 con 0x55dc5814ad30 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98c67fc640 1 -- 172.21.15.120:0/627985629 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004d30 con 0x55dc5814ad30 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98e4da4640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 0x7f98c80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.395+0000 7f98dd7fa640 1 -- 172.21.15.120:0/480282819 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98d00e5f60 con 0x7f98e0104fe0 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- 172.21.15.120:0/627985629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 msgr2=0x55dc5814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.408 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e4da4640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 0x7f98c80b7820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006010 tx=0x7f98d80099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c10 tx=0x7f98d4011810 comp rx=0 tx=0).stop 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- 172.21.15.120:0/627985629 shutdown_connections 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814d120 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/627985629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- 172.21.15.120:0/627985629 >> 172.21.15.120:0/627985629 conn(0x55dc5813c2b0 msgr2=0x55dc5813d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- 172.21.15.120:0/627985629 shutdown_connections 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- 172.21.15.120:0/627985629 wait complete. 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.396+0000 7f98e8bdc740 1 -- start start 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 Processor -- start 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813eda0 0x55dc58147a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc58147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc57f05700 con 0x55dc5813e480 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55dc5803f110 con 0x55dc5813eda0 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55dc5814d9b0 con 0x55dc5814ad30 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e8bdc740 10 client.0 _ll_register_callbacks cb 0x55dc58058990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43562/0 (socket says 172.21.15.120:43562) 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 -- 172.21.15.120:0/2197272974 learned_addr learned my addr 172.21.15.120:0/2197272974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98df7fe640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813eda0 0x55dc58147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e4da4640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc58147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 -- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 msgr2=0x55dc58147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc58147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 -- 172.21.15.120:0/2197272974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813eda0 msgr2=0x55dc58147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813eda0 0x55dc58147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98dffff640 1 -- 172.21.15.120:0/2197272974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5806cc40 con 0x55dc5813e480 2024-09-10T23:11:03.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.397+0000 7f98e4da4640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc58147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98dffff640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f98d00e3a90 tx=0x7f98d00e9630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb35895062f3e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00ac0a0 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98d00ac9b0 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d40043d0 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d00abec0 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 10 client.19047 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc58159330 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d40043d0 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f98d00b5850 con 0x55dc5813e480 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 10 client.19047 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:03.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 20 client.19047 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc5815a930 0x55dc5815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 4 client.19047 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.398+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc580b6da0 con 0x55dc5815a930 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.399+0000 7f98df7fe640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc5815a930 0x55dc5815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.399+0000 7f98df7fe640 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc5815a930 0x55dc5815cd20 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0fd17caec59288a server_cookie=da078f66e49498e8 in_seq=0 out_seq=0 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.399+0000 7f98bf7fe640 10 client.19047 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.402+0000 7f98bf7fe640 1 -- 172.21.15.120:0/2197272974 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc580b6da0 con 0x55dc5815a930 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.402+0000 7f98bf7fe640 10 client.19047 handle_command_reply: tid=0 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 client.19047 shutdown 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 msgr2=0x55dc5814a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f98d00e3a90 tx=0x7f98d00e9630 comp rx=0 tx=0).stop 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 shutdown_connections 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc5815a930 0x55dc5815cd20 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc58147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813eda0 0x55dc58147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98bf7fe640 0 client.19047 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/2197272974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813e480 0x55dc5814a5e0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 >> 172.21.15.120:0/2197272974 conn(0x55dc5813c2b0 msgr2=0x55dc58141e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 shutdown_connections 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 1 -- 172.21.15.120:0/2197272974 wait complete. 2024-09-10T23:11:03.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.403+0000 7f98e8bdc740 20 client.19047 trim_cache size 0 max 16384 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 Processor -- start 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 -- start start 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b2870 con 0x55dc5814ad30 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b50f0 con 0x55dc5813e480 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b2ee0 con 0x55dc5813eda0 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e4da4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e4da4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43570/0 (socket says 172.21.15.120:43570) 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98e4da4640 1 -- 172.21.15.120:0/1962060560 learned_addr learned my addr 172.21.15.120:0/1962060560 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.423 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.421+0000 7f98dffff640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 -- 172.21.15.120:0/1962060560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 msgr2=0x55dc5813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 -- 172.21.15.120:0/1962060560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 msgr2=0x55dc5813d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 -- 172.21.15.120:0/1962060560 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5813cd30 con 0x55dc5814ad30 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98dffff640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98df7fe640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5813b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98e4da4640 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f98d8031e40 tx=0x7f98d8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cb027510f35968 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98c5ffb640 1 -- 172.21.15.120:0/1962060560 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8056070 con 0x55dc5814ad30 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98c5ffb640 1 -- 172.21.15.120:0/1962060560 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8036050 con 0x55dc5814ad30 2024-09-10T23:11:03.424 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.422+0000 7f98c5ffb640 1 -- 172.21.15.120:0/1962060560 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8004820 con 0x55dc5814ad30 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- 172.21.15.120:0/1962060560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 msgr2=0x55dc5814c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f98d8031e40 tx=0x7f98d8041000 comp rx=0 tx=0).stop 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- 172.21.15.120:0/1962060560 shutdown_connections 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5814ad30 0x55dc5814c120 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5813eda0 0x55dc5813b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/1962060560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- 172.21.15.120:0/1962060560 >> 172.21.15.120:0/1962060560 conn(0x55dc5813c2b0 msgr2=0x55dc5813abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- 172.21.15.120:0/1962060560 shutdown_connections 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- 172.21.15.120:0/1962060560 wait complete. 2024-09-10T23:11:03.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 -- start start 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.423+0000 7f98e8bdc740 1 Processor -- start 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc581c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b35b0 con 0x55dc5813eda0 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b4a20 con 0x55dc5813e480 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01b3c80 con 0x55dc5814ad30 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 10 client.0 _ll_register_callbacks cb 0x7f98d00e9730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98dffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43572/0 (socket says 172.21.15.120:43572) 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e4da4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc581c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.428 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98dffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52180/0 (socket says 172.21.15.120:52180) 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 -- 172.21.15.120:0/726174519 learned_addr learned my addr 172.21.15.120:0/726174519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 -- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 msgr2=0x55dc581c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc581c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 -- 172.21.15.120:0/726174519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 msgr2=0x55dc5814e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 -- 172.21.15.120:0/726174519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc581c7c90 con 0x55dc5813eda0 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98dffff640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98df7fe640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000c00 tx=0x7f98d4005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f6b4274185307 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d401e040 con 0x55dc5813eda0 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f98d4004260 con 0x55dc5813eda0 2024-09-10T23:11:03.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.424+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d8036050 con 0x55dc5813eda0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d4005c50 con 0x55dc5813eda0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 10 client.19053 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc581c8fd0 con 0x55dc5813eda0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8036050 con 0x55dc5813eda0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f98d00051b0 con 0x55dc5813eda0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 10 client.19053 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 20 client.19053 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc581c95e0 0x55dc581cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 4 client.19053 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc580b2850 con 0x55dc581c95e0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.425+0000 7f98dffff640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc581c95e0 0x55dc581cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:03.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.426+0000 7f98bf7fe640 10 client.19053 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:03.431 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.426+0000 7f98dffff640 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc581c95e0 0x55dc581cba00 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9930967226759c server_cookie=b731bc7de7d7bf86 in_seq=0 out_seq=0 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98bf7fe640 1 -- 172.21.15.120:0/726174519 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8894 (crc 0 0 0) 0x55dc580b2850 con 0x55dc581c95e0 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98bf7fe640 10 client.19053 handle_command_reply: tid=0 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 client.19053 shutdown 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 msgr2=0x55dc5814e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000c00 tx=0x7f98d4005fa0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 shutdown_connections 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55dc581c95e0 0x55dc581cba00 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc5814ad30 0x55dc581c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc5813eda0 0x55dc5814e960 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 --2- 172.21.15.120:0/726174519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dc5813e480 0x55dc5814e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 >> 172.21.15.120:0/726174519 conn(0x55dc5813c2b0 msgr2=0x55dc5813b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.429+0000 7f98bf7fe640 0 client.19053 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 shutdown_connections 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e8bdc740 1 -- 172.21.15.120:0/726174519 wait complete. 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e8bdc740 20 client.19053 trim_cache size 0 max 16384 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:11:03.433 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.434 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:11:03.435 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:11:03.436 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.438 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:11:03.439 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.442 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:11:03.445 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.449 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:03.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 msgr2=0x7f98c80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 0x7f98c80b7820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006010 tx=0x7f98d80099a0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 msgr2=0x7f98e01118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e9f0 tx=0x7f98d000edf0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 shutdown_connections 2024-09-10T23:11:03.452 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f98c80b5360 0x7f98c80b7820 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98e010b370 0x7f98e010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98e010a9f0 0x7f98e010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 --2- 172.21.15.120:0/480282819 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98e0104fe0 0x7f98e01118c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 >> 172.21.15.120:0/480282819 conn(0x7f98e0100a10 msgr2=0x7f98e0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 shutdown_connections 2024-09-10T23:11:03.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:03.430+0000 7f98e6807640 1 -- 172.21.15.120:0/480282819 wait complete. 2024-09-10T23:11:03.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 146976 21.74 MB/sec execute 238 sec latency 59.568 ms 2024-09-10T23:11:04.369 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:04 smithi120 ceph-mon[31247]: pgmap v905: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T23:11:04.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:11:04.399 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:11:04.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:04 smithi153 ceph-mon[34951]: pgmap v905: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T23:11:04.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:04 smithi176 ceph-mon[34982]: pgmap v905: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T23:11:04.689 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:04.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 147546 21.78 MB/sec execute 239 sec latency 50.007 ms 2024-09-10T23:11:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2606320677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 msgr2=0x7fad2810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2606320677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad2810ae10 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fad20009cd0 tx=0x7fad2002a8e0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2606320677 shutdown_connections 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2606320677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad2810ae10 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2606320677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28103ad0 0x7fad28103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2606320677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28102980 0x7fad28102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2606320677 >> 172.21.15.120:0/2606320677 conn(0x7fad280fe240 msgr2=0x7fad28100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2606320677 shutdown_connections 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.098+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2606320677 wait complete. 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 Processor -- start 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 -- start start 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 0x7fad28198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28103ad0 0x7fad28199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e5d0 con 0x7fad28104490 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e450 con 0x7fad28102980 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2e8a3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e750 con 0x7fad28103ad0 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2ce40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad27fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 0x7fad28198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad27fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 0x7fad28198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52194/0 (socket says 172.21.15.120:52194) 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad2ce40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43596/0 (socket says 172.21.15.120:43596) 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad27fff640 1 -- 172.21.15.120:0/2956980803 learned_addr learned my addr 172.21.15.120:0/2956980803 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.099+0000 7fad277fe640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28103ad0 0x7fad28199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 -- 172.21.15.120:0/2956980803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28103ad0 msgr2=0x7fad28199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28103ad0 0x7fad28199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 -- 172.21.15.120:0/2956980803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 msgr2=0x7fad28198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 0x7fad28198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 -- 172.21.15.120:0/2956980803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2819a860 con 0x7fad28104490 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2ce40640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fad20031780 tx=0x7fad20003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad257fa640 1 -- 172.21.15.120:0/2956980803 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20004040 con 0x7fad28104490 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad257fa640 1 -- 172.21.15.120:0/2956980803 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad20004690 con 0x7fad28104490 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2819aaf0 con 0x7fad28104490 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad257fa640 1 -- 172.21.15.120:0/2956980803 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20030ea0 con 0x7fad28104490 2024-09-10T23:11:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.100+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281a1220 con 0x7fad28104490 2024-09-10T23:11:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.102+0000 7fad257fa640 1 -- 172.21.15.120:0/2956980803 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fad20030070 con 0x7fad28104490 2024-09-10T23:11:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 Processor -- start 2024-09-10T23:11:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 -- start start 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5627b715e6f0 con 0x5627b71f1bd0 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627b714db20 con 0x5627b71ef190 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad3166f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5627b7151930 con 0x5627b71ec860 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71eec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55590/0 (socket says 172.21.15.120:55590) 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad277fe640 1 -- 172.21.15.120:0/3620789355 learned_addr learned my addr 172.21.15.120:0/3620789355 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad277fe640 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad257fa640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 0x7fad100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 -- 172.21.15.120:0/3620789355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 msgr2=0x5627b71eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 -- 172.21.15.120:0/3620789355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 msgr2=0x5627b71f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.103+0000 7fad27fff640 1 -- 172.21.15.120:0/3620789355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b7113530 con 0x5627b71ef190 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad2ce40640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 0x7fad100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad27fff640 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71f1580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad18004610 tx=0x7fad1800bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd4864e342cd6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad257fa640 1 -- 172.21.15.120:0/2956980803 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad2002e030 con 0x7fad28104490 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad0e7fc640 1 -- 172.21.15.120:0/3620789355 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18003dd0 con 0x5627b71ef190 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad0e7fc640 1 -- 172.21.15.120:0/3620789355 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad18015070 con 0x5627b71ef190 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.104+0000 7fad0e7fc640 1 -- 172.21.15.120:0/3620789355 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18004d70 con 0x5627b71ef190 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- 172.21.15.120:0/3620789355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 msgr2=0x5627b71f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad2ce40640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 0x7fad100b77a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad20041000 tx=0x7fad20091c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71f1580 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad18004610 tx=0x7fad1800bc70 comp rx=0 tx=0).stop 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- 172.21.15.120:0/3620789355 shutdown_connections 2024-09-10T23:11:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71f1580 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 --2- 172.21.15.120:0/3620789355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- 172.21.15.120:0/3620789355 >> 172.21.15.120:0/3620789355 conn(0x5627b71e79b0 msgr2=0x5627b71e8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- 172.21.15.120:0/3620789355 shutdown_connections 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- 172.21.15.120:0/3620789355 wait complete. 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 -- start start 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.105+0000 7fad3166f740 1 Processor -- start 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 0x5627b71f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 0x5627b71f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5627b7085ab0 con 0x5627b71ef190 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5627b70f3730 con 0x5627b71f1bd0 2024-09-10T23:11:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5627b71f4bb0 con 0x5627b71ec860 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 10 client.0 _ll_register_callbacks cb 0x5627b71476b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 0x5627b71f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55592/0 (socket says 172.21.15.120:55592) 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad277fe640 1 -- 172.21.15.120:0/301246798 learned_addr learned my addr 172.21.15.120:0/301246798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad277fe640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 0x5627b71f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 -- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 msgr2=0x5627b71f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 -- 172.21.15.120:0/301246798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 msgr2=0x5627b71f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 0x5627b71f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 -- 172.21.15.120:0/301246798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b6eaf790 con 0x5627b71ef190 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad277fe640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 0x5627b71f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad27fff640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 0x5627b71f6590 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fad18000c00 tx=0x7fad18003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a668aa39a95f5901 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18028060 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad18011850 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.106+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad18015070 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad180046c0 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 10 client.19065 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627b6f28620 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad18015070 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fad18022070 con 0x5627b71ef190 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 10 client.19065 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 10 client.19065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 20 client.19065 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b72014a0 0x5627b7203890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 4 client.19065 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b7160640 con 0x5627b72014a0 2024-09-10T23:11:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.107+0000 7fad277fe640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b72014a0 0x5627b7203890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.108+0000 7fad277fe640 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b72014a0 0x5627b7203890 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3961f6b4b790ad7b server_cookie=3f2a500fe93418b7 in_seq=0 out_seq=0 2024-09-10T23:11:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.108+0000 7fad077fe640 10 client.19065 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.109+0000 7fad077fe640 1 -- 172.21.15.120:0/301246798 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627b7160640 con 0x5627b72014a0 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.109+0000 7fad077fe640 10 client.19065 handle_command_reply: tid=0 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 client.19065 shutdown 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 msgr2=0x5627b71f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 0x5627b71f6590 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fad18000c00 tx=0x7fad18003e00 comp rx=0 tx=0).stop 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 shutdown_connections 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b72014a0 0x5627b7203890 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71f1bd0 0x5627b71f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ef190 0x5627b71f6590 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad077fe640 0 client.19065 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 --2- 172.21.15.120:0/301246798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ec860 0x5627b71f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 >> 172.21.15.120:0/301246798 conn(0x5627b71e79b0 msgr2=0x5627b71e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 shutdown_connections 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 1 -- 172.21.15.120:0/301246798 wait complete. 2024-09-10T23:11:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.110+0000 7fad3166f740 20 client.19065 trim_cache size 0 max 16384 2024-09-10T23:11:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 Processor -- start 2024-09-10T23:11:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 -- start start 2024-09-10T23:11:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 0x5627b71e8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ef190 0x5627b71f6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad281a7e20 con 0x5627b71f1bd0 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad281aa6a0 con 0x5627b71ec860 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad3166f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad281a8490 con 0x5627b71ef190 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43630/0 (socket says 172.21.15.120:43630) 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 -- 172.21.15.120:0/2758523454 learned_addr learned my addr 172.21.15.120:0/2758523454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 0x5627b71e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad27fff640 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ef190 0x5627b71f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 -- 172.21.15.120:0/2758523454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ef190 msgr2=0x5627b71f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ef190 0x5627b71f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 -- 172.21.15.120:0/2758523454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 msgr2=0x5627b71f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.125+0000 7fad277fe640 1 -- 172.21.15.120:0/2758523454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b71f74d0 con 0x5627b71ec860 2024-09-10T23:11:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad277fe640 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 0x5627b71e8880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad1c010610 tx=0x7fad1c00a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baabd289626a04ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad0dffb640 1 -- 172.21.15.120:0/2758523454 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c01a040 con 0x5627b71ec860 2024-09-10T23:11:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad0dffb640 1 -- 172.21.15.120:0/2758523454 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1c004220 con 0x5627b71ec860 2024-09-10T23:11:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad0dffb640 1 -- 172.21.15.120:0/2758523454 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c02a6a0 con 0x5627b71ec860 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad3166f740 1 -- 172.21.15.120:0/2758523454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 msgr2=0x5627b71e8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.126+0000 7fad3166f740 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 0x5627b71e8880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad1c010610 tx=0x7fad1c00a660 comp rx=0 tx=0).stop 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- 172.21.15.120:0/2758523454 shutdown_connections 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71f1bd0 0x5627b71f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71ef190 0x5627b71f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- 172.21.15.120:0/2758523454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ec860 0x5627b71e8880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- 172.21.15.120:0/2758523454 >> 172.21.15.120:0/2758523454 conn(0x5627b71e79b0 msgr2=0x5627b71f5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- 172.21.15.120:0/2758523454 shutdown_connections 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- 172.21.15.120:0/2758523454 wait complete. 2024-09-10T23:11:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- start start 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 Processor -- start 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71f1bd0 0x5627b71fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad281a8b60 con 0x5627b71ec860 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad281a9fd0 con 0x5627b71ef190 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad281a9230 con 0x5627b71f1bd0 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad3166f740 10 client.0 _ll_register_callbacks cb 0x7fad1c00a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43642/0 (socket says 172.21.15.120:43642) 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.127+0000 7fad277fe640 1 -- 172.21.15.120:0/3669177655 learned_addr learned my addr 172.21.15.120:0/3669177655 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad27fff640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71f1bd0 0x5627b71fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 -- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71f1bd0 msgr2=0x5627b71fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71f1bd0 0x5627b71fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 -- 172.21.15.120:0/3669177655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 msgr2=0x5627b71fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 -- 172.21.15.120:0/3669177655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b71f18f0 con 0x5627b71ec860 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad27fff640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad277fe640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fad1c003fd0 tx=0x7fad1c00a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b992b3a5b94fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c005020 con 0x5627b71ec860 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad1c0051c0 con 0x5627b71ec860 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad1c004220 con 0x5627b71ec860 2024-09-10T23:11:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad1c0053c0 con 0x5627b71ec860 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 10 client.19071 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627b71fd800 con 0x5627b71ec860 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1c004220 con 0x5627b71ec860 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fad1c038940 con 0x5627b71ec860 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 10 client.19071 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 10 client.19071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 20 client.19071 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b726e300 0x5627b72706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 4 client.19071 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.128+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627b715c0f0 con 0x5627b726e300 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.129+0000 7fad27fff640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b726e300 0x5627b72706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.129+0000 7fad077fe640 10 client.19071 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.129+0000 7fad27fff640 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b726e300 0x5627b72706f0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9e2d9c0b120edd server_cookie=448aaa3e45343cd3 in_seq=0 out_seq=0 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- 172.21.15.120:0/250056410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40101cb0 msgr2=0x7ffb40102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 --2- 172.21.15.120:0/250056410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40101cb0 0x7ffb40102090 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009900 tx=0x7ffb3002a410 comp rx=0 tx=0).stop 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- 172.21.15.120:0/250056410 shutdown_connections 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 --2- 172.21.15.120:0/250056410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40108040 0x7ffb4010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 --2- 172.21.15.120:0/250056410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 --2- 172.21.15.120:0/250056410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40101cb0 0x7ffb40102090 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- 172.21.15.120:0/250056410 >> 172.21.15.120:0/250056410 conn(0x7ffb400fd700 msgr2=0x7ffb400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- 172.21.15.120:0/250056410 shutdown_connections 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- 172.21.15.120:0/250056410 wait complete. 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 Processor -- start 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.235+0000 7ffb46658640 1 -- start start 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 0x7ffb40197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111d00 con 0x7ffb40108040 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111b80 con 0x7ffb401076c0 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb46658640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111e80 con 0x7ffb40101cb0 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb44bf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 0x7ffb40197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb44bf5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43660/0 (socket says 172.21.15.120:43660) 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 0x7ffb40197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55632/0 (socket says 172.21.15.120:55632) 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb44bf5640 1 -- 172.21.15.120:0/1575764851 learned_addr learned my addr 172.21.15.120:0/1575764851 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 -- 172.21.15.120:0/1575764851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 msgr2=0x7ffb40197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 0x7ffb40197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 -- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 msgr2=0x7ffb40198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 -- 172.21.15.120:0/1575764851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb401a76a0 con 0x7ffb401076c0 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb44bf5640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.236+0000 7ffb3f7fe640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40198400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9f0 tx=0x7ffb2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.237+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800ccc0 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.237+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb401a7990 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.237+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb280108e0 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.237+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28018880 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.237+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a7f20 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.238+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb40102090 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.239+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ffb28010430 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.240+0000 7ffb3d7fa640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 0x7ffb2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.240+0000 7ffb3ffff640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 0x7ffb2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.240+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ffb280e6380 con 0x7ffb401076c0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.240+0000 7ffb3ffff640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 0x7ffb2c0b77a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffb30031440 tx=0x7ffb30005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:05.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.243+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffb280acbc0 con 0x7ffb401076c0 2024-09-10T23:11:05.623 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.622+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb4006b8a0 con 0x7ffb401076c0 2024-09-10T23:11:05.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.626+0000 7ffb3d7fa640 1 -- 172.21.15.120:0/1575764851 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7ffb4006b8a0 con 0x7ffb401076c0 2024-09-10T23:11:05.627 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:11:05.627 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:11:05.627 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:11:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.632+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 msgr2=0x7ffb2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.632+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 0x7ffb2c0b77a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffb30031440 tx=0x7ffb30005b50 comp rx=0 tx=0).stop 2024-09-10T23:11:05.633 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.632+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 msgr2=0x7ffb40198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.634 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.632+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40198400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9f0 tx=0x7ffb2800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 shutdown_connections 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffb2c0b52e0 0x7ffb2c0b77a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffb40108040 0x7ffb40198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb401076c0 0x7ffb40198400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 --2- 172.21.15.120:0/1575764851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb40101cb0 0x7ffb40197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 >> 172.21.15.120:0/1575764851 conn(0x7ffb400fd700 msgr2=0x7ffb400ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 shutdown_connections 2024-09-10T23:11:05.639 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:05.638+0000 7ffb46658640 1 -- 172.21.15.120:0/1575764851 wait complete. 2024-09-10T23:11:05.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 147811 21.76 MB/sec execute 240 sec latency 449.954 ms 2024-09-10T23:11:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.896+0000 7fad077fe640 1 -- 172.21.15.120:0/3669177655 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627b715c0f0 con 0x5627b726e300 2024-09-10T23:11:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.896+0000 7fad077fe640 10 client.19071 handle_command_reply: tid=0 2024-09-10T23:11:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.896+0000 7fad3166f740 1 client.19071 shutdown 2024-09-10T23:11:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 msgr2=0x5627b71fab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fad1c003fd0 tx=0x7fad1c00a7c0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 shutdown_connections 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5627b726e300 0x5627b72706f0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627b71f1bd0 0x5627b71fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5627b71ef190 0x5627b71fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad077fe640 0 client.19071 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 --2- 172.21.15.120:0/3669177655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5627b71ec860 0x5627b71fab70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 >> 172.21.15.120:0/3669177655 conn(0x5627b71e79b0 msgr2=0x5627b71e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 shutdown_connections 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 1 -- 172.21.15.120:0/3669177655 wait complete. 2024-09-10T23:11:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad3166f740 20 client.19071 trim_cache size 0 max 16384 2024-09-10T23:11:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:11:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:11:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:11:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 msgr2=0x7fad100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 0x7fad100b77a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad20041000 tx=0x7fad20091c20 comp rx=0 tx=0).stop 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 msgr2=0x7fad28199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.897+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fad20031780 tx=0x7fad20003b80 comp rx=0 tx=0).stop 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 shutdown_connections 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad100b52e0 0x7fad100b77a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad28104490 0x7fad28199fa0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad28103ad0 0x7fad28199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 --2- 172.21.15.120:0/2956980803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad28102980 0x7fad28198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 >> 172.21.15.120:0/2956980803 conn(0x7fad280fe240 msgr2=0x7fad28106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 shutdown_connections 2024-09-10T23:11:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:05.898+0000 7fad2e8a3640 1 -- 172.21.15.120:0/2956980803 wait complete. 2024-09-10T23:11:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:06 smithi120 ceph-mon[31247]: pgmap v906: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-09-10T23:11:06.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1575764851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:06 smithi153 ceph-mon[34951]: pgmap v906: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-09-10T23:11:06.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1575764851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:06 smithi176 ceph-mon[34982]: pgmap v906: 97 pgs: 97 active+clean; 249 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-09-10T23:11:06.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1575764851' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:06.649 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:11:06.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 148170 21.75 MB/sec execute 241 sec latency 161.896 ms 2024-09-10T23:11:06.918 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:07.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.456+0000 7f8cc55b8640 1 -- 172.21.15.120:0/1328898972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 msgr2=0x7f8cc0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.456+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/1328898972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0101c60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009900 tx=0x7f8cb002a410 comp rx=0 tx=0).stop 2024-09-10T23:11:07.477 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 -- 172.21.15.120:0/1328898972 shutdown_connections 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/1328898972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 0x7f8cc010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/1328898972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/1328898972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0101c60 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 -- 172.21.15.120:0/1328898972 >> 172.21.15.120:0/1328898972 conn(0x7f8cc0076e60 msgr2=0x7f8cc0077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 -- 172.21.15.120:0/1328898972 shutdown_connections 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 -- 172.21.15.120:0/1328898972 wait complete. 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 Processor -- start 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.457+0000 7f8cc55b8640 1 -- start start 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 0x7f8cc019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112de0 con 0x7f8cc0101880 2024-09-10T23:11:07.478 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112c60 con 0x7f8cc0102230 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cc55b8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0112f60 con 0x7f8cc010d0e0 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43684/0 (socket says 172.21.15.120:43684) 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 -- 172.21.15.120:0/840735122 learned_addr learned my addr 172.21.15.120:0/840735122 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 0x7f8cc019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 -- 172.21.15.120:0/840735122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 msgr2=0x7f8cc019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 0x7f8cc019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 -- 172.21.15.120:0/840735122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 msgr2=0x7f8cc01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.479 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 -- 172.21.15.120:0/840735122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc019f340 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.458+0000 7f8cbeffd640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005ab0 tx=0x7f8cb0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.459+0000 7f8cbcff9640 1 -- 172.21.15.120:0/840735122 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb003f070 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.459+0000 7f8cbcff9640 1 -- 172.21.15.120:0/840735122 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb0031d60 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.459+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc019f5d0 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.459+0000 7f8cbcff9640 1 -- 172.21.15.120:0/840735122 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0035030 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.459+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc0102730 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.462+0000 7f8cbcff9640 1 -- 172.21.15.120:0/840735122 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8cb003b460 con 0x7f8cc0101880 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.462+0000 7f8cc798d740 1 Processor -- start 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.462+0000 7f8cc798d740 1 -- start start 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.480 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cbcff9640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 0x7f8cac0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564795308ea0 con 0x5647953a71b0 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647952f82d0 con 0x564795392a10 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cc798d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5647952fc0e0 con 0x5647953920f0 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52282/0 (socket says 172.21.15.120:52282) 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1185552947 learned_addr learned my addr 172.21.15.120:0/1185552947 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cbeffd640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 0x7f8cac0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cbcff9640 1 -- 172.21.15.120:0/840735122 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8cb01066d0 con 0x7f8cc0101880 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.481 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1185552947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 msgr2=0x5647953924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1185552947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 msgr2=0x5647953a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1185552947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564795056650 con 0x564795392a10 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.463+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.464+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80104d0 tx=0x7f8ca8010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27da595b40d8117b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.464+0000 7f8cbeffd640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 0x7f8cac0b7820 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0000c00 tx=0x7f8cb00c17c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.464+0000 7f8cb57fa640 1 -- 172.21.15.120:0/1185552947 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011760 con 0x564795392a10 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.464+0000 7f8cb57fa640 1 -- 172.21.15.120:0/1185552947 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca8011db0 con 0x564795392a10 2024-09-10T23:11:07.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.464+0000 7f8cb57fa640 1 -- 172.21.15.120:0/1185552947 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800e430 con 0x564795392a10 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- 172.21.15.120:0/1185552947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 msgr2=0x5647953a6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80104d0 tx=0x7f8ca8010990 comp rx=0 tx=0).stop 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- 172.21.15.120:0/1185552947 shutdown_connections 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a6c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1185552947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- 172.21.15.120:0/1185552947 >> 172.21.15.120:0/1185552947 conn(0x56479538ff20 msgr2=0x5647953910c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- 172.21.15.120:0/1185552947 shutdown_connections 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- 172.21.15.120:0/1185552947 wait complete. 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 -- start start 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.465+0000 7f8cc798d740 1 Processor -- start 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564795159700 con 0x5647953a71b0 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x564795293160 con 0x564795392a10 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5647953a9e30 con 0x5647953920f0 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cc798d740 10 client.0 _ll_register_callbacks cb 0x5647952ac9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:07.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43702/0 (socket says 172.21.15.120:43702) 2024-09-10T23:11:07.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/398460466 learned_addr learned my addr 172.21.15.120:0/398460466 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/398460466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 msgr2=0x5647953a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/398460466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 msgr2=0x5647953a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/398460466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647952c0c90 con 0x5647953920f0 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.466+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953a6ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005f40 tx=0x7f8cb8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b732c0bb6d4e71ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb801f420 con 0x5647953920f0 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca8011db0 con 0x5647953920f0 2024-09-10T23:11:07.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8cb801e3c0 con 0x5647953920f0 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8cb8010730 con 0x5647953920f0 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 10 client.28531 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647953b57c0 con 0x5647953920f0 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca8011db0 con 0x5647953920f0 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8cb801e860 con 0x5647953920f0 2024-09-10T23:11:07.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 10 client.28531 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 10 client.28531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 20 client.28531 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647953b6c20 0x5647953b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 4 client.28531 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:07.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56479530adf0 con 0x5647953b6c20 2024-09-10T23:11:07.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.467+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647953b6c20 0x5647953b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.468+0000 7f8ca67fc640 10 client.28531 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:07.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.468+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647953b6c20 0x5647953b9030 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea22e90b0b5bc07b server_cookie=2b77aabd4cf6c184 in_seq=0 out_seq=0 2024-09-10T23:11:07.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8ca67fc640 1 -- 172.21.15.120:0/398460466 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56479530adf0 con 0x5647953b6c20 2024-09-10T23:11:07.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8ca67fc640 10 client.28531 handle_command_reply: tid=0 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 client.28531 shutdown 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 msgr2=0x5647953a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953a6ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005f40 tx=0x7f8cb8004a40 comp rx=0 tx=0).stop 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 shutdown_connections 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647953b6c20 0x5647953b9030 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 --2- 172.21.15.120:0/398460466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953920f0 0x5647953a6ae0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8ca67fc640 0 client.28531 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 >> 172.21.15.120:0/398460466 conn(0x56479538ff20 msgr2=0x564795395ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 shutdown_connections 2024-09-10T23:11:07.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 1 -- 172.21.15.120:0/398460466 wait complete. 2024-09-10T23:11:07.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.471+0000 7f8cc798d740 20 client.28531 trim_cache size 0 max 16384 2024-09-10T23:11:07.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 Processor -- start 2024-09-10T23:11:07.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 -- start start 2024-09-10T23:11:07.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564795392a10 0x564795391540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01aa0e0 con 0x5647953a71b0 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ac940 con 0x5647953920f0 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cc798d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01aa730 con 0x564795392a10 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52302/0 (socket says 172.21.15.120:52302) 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/494016283 learned_addr learned my addr 172.21.15.120:0/494016283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.490+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564795392a10 0x564795391540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/494016283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564795392a10 msgr2=0x564795391540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564795392a10 0x564795391540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.507 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/494016283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 msgr2=0x5647953be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 -- 172.21.15.120:0/494016283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56479538b7e0 con 0x5647953920f0 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8002230 tx=0x7f8cb8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66caea998cb60e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cb4ff9640 1 -- 172.21.15.120:0/494016283 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8003b70 con 0x5647953920f0 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cb4ff9640 1 -- 172.21.15.120:0/494016283 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb8009500 con 0x5647953920f0 2024-09-10T23:11:07.508 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.491+0000 7f8cb4ff9640 1 -- 172.21.15.120:0/494016283 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8004310 con 0x5647953920f0 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- 172.21.15.120:0/494016283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 msgr2=0x5647953b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8002230 tx=0x7f8cb8004a40 comp rx=0 tx=0).stop 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- 172.21.15.120:0/494016283 shutdown_connections 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953a71b0 0x5647953be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564795392a10 0x564795391540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 --2- 172.21.15.120:0/494016283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647953920f0 0x5647953b3060 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.509 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- 172.21.15.120:0/494016283 >> 172.21.15.120:0/494016283 conn(0x56479538ff20 msgr2=0x564795390e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- 172.21.15.120:0/494016283 shutdown_connections 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- 172.21.15.120:0/494016283 wait complete. 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 -- start start 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.492+0000 7f8cc798d740 1 Processor -- start 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953920f0 0x5647953af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 0x5647953ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01aae00 con 0x5647953920f0 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ac270 con 0x564795392a10 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ab4d0 con 0x5647953a71b0 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953920f0 0x5647953af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cc798d740 10 client.0 _ll_register_callbacks cb 0x7f8cb8010600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52316/0 (socket says 172.21.15.120:52316) 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1713200491 learned_addr learned my addr 172.21.15.120:0/1713200491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:07.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 0x5647953ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 msgr2=0x5647953ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 0x5647953ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1713200491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953920f0 msgr2=0x5647953af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953920f0 0x5647953af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 -- 172.21.15.120:0/1713200491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647953ac650 con 0x564795392a10 2024-09-10T23:11:07.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 0x5647953ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8cb7fff640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80104d0 tx=0x7f8ca8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a2d01cc79269fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.493+0000 7f8ca67fc640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80055c0 con 0x564795392a10 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f8ca8004660 con 0x564795392a10 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647953068a0 con 0x564795392a10 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ca8004860 con 0x564795392a10 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 10 client.28307 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:07.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564795424390 con 0x564795392a10 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5647953068a0 con 0x564795392a10 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f8ca8011c10 con 0x564795392a10 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 10 client.28307 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 10 client.28307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 20 client.28307 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564795424820 0x564795426c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 4 client.28307 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:11:07.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.494+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564795303cd0 con 0x564795424820 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.495+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564795424820 0x564795426c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.495+0000 7f8cbf7fe640 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564795424820 0x564795426c40 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407a0665148cd0ee server_cookie=edd182ddf48b3440 in_seq=0 out_seq=0 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.495+0000 7f8ca67fc640 10 client.28307 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.498+0000 7f8ca67fc640 1 -- 172.21.15.120:0/1713200491 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564795303cd0 con 0x564795424820 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.498+0000 7f8ca67fc640 10 client.28307 handle_command_reply: tid=0 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.498+0000 7f8cc798d740 1 client.28307 shutdown 2024-09-10T23:11:07.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 msgr2=0x5647953af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80104d0 tx=0x7f8ca8005040 comp rx=0 tx=0).stop 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 shutdown_connections 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x564795424820 0x564795426c40 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647953a71b0 0x5647953ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564795392a10 0x5647953af600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.516 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8ca67fc640 0 client.28307 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:07.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 --2- 172.21.15.120:0/1713200491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647953920f0 0x5647953af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 >> 172.21.15.120:0/1713200491 conn(0x56479538ff20 msgr2=0x56479538e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 shutdown_connections 2024-09-10T23:11:07.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 1 -- 172.21.15.120:0/1713200491 wait complete. 2024-09-10T23:11:07.517 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc798d740 20 client.28307 trim_cache size 0 max 16384 2024-09-10T23:11:07.518 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:11:07.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 msgr2=0x7f8cac0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 0x7f8cac0b7820 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0000c00 tx=0x7f8cb00c17c0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.519 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 msgr2=0x7f8cc0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:07.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.499+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005ab0 tx=0x7f8cb0003c80 comp rx=0 tx=0).stop 2024-09-10T23:11:07.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 shutdown_connections 2024-09-10T23:11:07.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f8cac0b5360 0x7f8cac0b7820 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.520 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8cc010d0e0 0x7f8cc019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cc0102230 0x7f8cc01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 --2- 172.21.15.120:0/840735122 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cc0101880 0x7f8cc0199290 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:07.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 >> 172.21.15.120:0/840735122 conn(0x7f8cc0076e60 msgr2=0x7f8cc00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:07.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 shutdown_connections 2024-09-10T23:11:07.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:07.500+0000 7f8cc55b8640 1 -- 172.21.15.120:0/840735122 wait complete. 2024-09-10T23:11:07.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 148667 21.81 MB/sec execute 242 sec latency 56.965 ms 2024-09-10T23:11:08.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:08 smithi120 ceph-mon[31247]: pgmap v907: 97 pgs: 97 active+clean; 250 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 653 op/s 2024-09-10T23:11:08.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:08 smithi153 ceph-mon[34951]: pgmap v907: 97 pgs: 97 active+clean; 250 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 653 op/s 2024-09-10T23:11:08.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:08 smithi176 ceph-mon[34982]: pgmap v907: 97 pgs: 97 active+clean; 250 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.6 MiB/s wr, 653 op/s 2024-09-10T23:11:08.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149101 21.83 MB/sec execute 243 sec latency 77.659 ms 2024-09-10T23:11:09.296 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:11:09.296 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:11:09.589 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:09.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149414 21.80 MB/sec execute 244 sec latency 64.489 ms 2024-09-10T23:11:10.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 -- 172.21.15.120:0/66156264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42541087d0 msgr2=0x7f4254108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 --2- 172.21.15.120:0/66156264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42541087d0 0x7f4254108c10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4248009900 tx=0x7f424802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 -- 172.21.15.120:0/66156264 shutdown_connections 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 --2- 172.21.15.120:0/66156264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254109150 0x7f425410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 --2- 172.21.15.120:0/66156264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42541087d0 0x7f4254108c10 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 --2- 172.21.15.120:0/66156264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4254102dc0 0x7f42541031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 -- 172.21.15.120:0/66156264 >> 172.21.15.120:0/66156264 conn(0x7f42540fe830 msgr2=0x7f4254100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 -- 172.21.15.120:0/66156264 shutdown_connections 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.216+0000 7f4259cfb640 1 -- 172.21.15.120:0/66156264 wait complete. 2024-09-10T23:11:10.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.217+0000 7f4259cfb640 1 Processor -- start 2024-09-10T23:11:10.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.217+0000 7f4259cfb640 1 -- start start 2024-09-10T23:11:10.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.217+0000 7f4259cfb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254102dc0 0x7f4254199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.217+0000 7f4259cfb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 0x7f42541997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4259cfb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4259cfb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112da0 con 0x7f4254109150 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4259cfb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112c20 con 0x7f42541087d0 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4259cfb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112f20 con 0x7f4254102dc0 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43738/0 (socket says 172.21.15.120:43738) 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 0x7f42541997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 -- 172.21.15.120:0/3953216428 learned_addr learned my addr 172.21.15.120:0/3953216428 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f42537fe640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254102dc0 0x7f4254199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 -- 172.21.15.120:0/3953216428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254102dc0 msgr2=0x7f4254199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254102dc0 0x7f4254199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 -- 172.21.15.120:0/3953216428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 msgr2=0x7f42541997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 0x7f42541997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 -- 172.21.15.120:0/3953216428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425419f320 con 0x7f4254109150 2024-09-10T23:11:10.221 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:10 smithi120 ceph-mon[31247]: pgmap v908: 97 pgs: 97 active+clean; 240 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4252ffd640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 0x7f42541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.218+0000 7f4253fff640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f424c00eb10 tx=0x7f424c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.219+0000 7f4250ff9640 1 -- 172.21.15.120:0/3953216428 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c004b20 con 0x7f4254109150 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.219+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425419f610 con 0x7f4254109150 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.219+0000 7f4250ff9640 1 -- 172.21.15.120:0/3953216428 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424c004cc0 con 0x7f4254109150 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.219+0000 7f4250ff9640 1 -- 172.21.15.120:0/3953216428 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c018630 con 0x7f4254109150 2024-09-10T23:11:10.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.219+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a5ea0 con 0x7f4254109150 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.222+0000 7f4250ff9640 1 -- 172.21.15.120:0/3953216428 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f424c005000 con 0x7f4254109150 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 Processor -- start 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 -- start start 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb45032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55adb4479e60 con 0x55adb450f790 2024-09-10T23:11:10.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55adb4469290 con 0x55adb4502ee0 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f425c0d0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55adb446d0a0 con 0x55adb4503800 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43752/0 (socket says 172.21.15.120:43752) 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f4252ffd640 1 -- 172.21.15.120:0/2197929166 learned_addr learned my addr 172.21.15.120:0/2197929166 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f42537fe640 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb45032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.223+0000 7f4250ff9640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 0x7f423c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4253fff640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 0x7f423c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 -- 172.21.15.120:0/2197929166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 msgr2=0x55adb450f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 -- 172.21.15.120:0/2197929166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 msgr2=0x55adb45032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb45032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 -- 172.21.15.120:0/2197929166 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb41c7930 con 0x55adb450f790 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4250ff9640 1 -- 172.21.15.120:0/3953216428 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f424c0e6250 con 0x7f4254109150 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4248032fd0 tx=0x7f4248005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125e01ffea32e87e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4253fff640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 0x7f423c0b77a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f424c0ac9d0 tx=0x7f424c01b8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4239ffb640 1 -- 172.21.15.120:0/2197929166 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248035070 con 0x55adb450f790 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4239ffb640 1 -- 172.21.15.120:0/2197929166 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4248004230 con 0x55adb450f790 2024-09-10T23:11:10.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.224+0000 7f4239ffb640 1 -- 172.21.15.120:0/2197929166 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424803d650 con 0x55adb450f790 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- 172.21.15.120:0/2197929166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 msgr2=0x55adb4511b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4248032fd0 tx=0x7f4248005780 comp rx=0 tx=0).stop 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- 172.21.15.120:0/2197929166 shutdown_connections 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb4511b80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2197929166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb45032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- 172.21.15.120:0/2197929166 >> 172.21.15.120:0/2197929166 conn(0x55adb4500d10 msgr2=0x55adb4501eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- 172.21.15.120:0/2197929166 shutdown_connections 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- 172.21.15.120:0/2197929166 wait complete. 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.225+0000 7f425c0d0740 1 -- start start 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 Processor -- start 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb450d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55adb42ca890 con 0x55adb450f790 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55adb4403d40 con 0x55adb4502ee0 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55adb45123e0 con 0x55adb4503800 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f425c0d0740 10 client.0 _ll_register_callbacks cb 0x55adb4467c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52354/0 (socket says 172.21.15.120:52354) 2024-09-10T23:11:10.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f42537fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 -- 172.21.15.120:0/2798623974 learned_addr learned my addr 172.21.15.120:0/2798623974 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb450d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 -- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 msgr2=0x55adb450cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 -- 172.21.15.120:0/2798623974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 msgr2=0x55adb450d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb450d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.226+0000 7f4252ffd640 1 -- 172.21.15.120:0/2798623974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb451b840 con 0x55adb4502ee0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4252ffd640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4248001eb0 tx=0x7f4248041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eec2c386ae66543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424804e030 con 0x55adb4502ee0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4248004230 con 0x55adb4502ee0 2024-09-10T23:11:10.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4248003a00 con 0x55adb4502ee0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4248035070 con 0x55adb4502ee0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 10 client.28319 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adb451e1f0 con 0x55adb4502ee0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4248004230 con 0x55adb4502ee0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f424803dbe0 con 0x55adb4502ee0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 10 client.28319 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 10 client.28319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 20 client.28319 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.227+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb451f7e0 0x55adb4521c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.228+0000 7f425c0d0740 4 client.28319 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.228+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adb447bdb0 con 0x55adb451f7e0 2024-09-10T23:11:10.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.228+0000 7f42537fe640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb451f7e0 0x55adb4521c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.228+0000 7f4232ffd640 10 client.28319 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.228+0000 7f42537fe640 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb451f7e0 0x55adb4521c00 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd82576eba6e86ca server_cookie=aedbb1abf76f2045 in_seq=0 out_seq=0 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f4232ffd640 1 -- 172.21.15.120:0/2798623974 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adb447bdb0 con 0x55adb451f7e0 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f4232ffd640 10 client.28319 handle_command_reply: tid=0 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 client.28319 shutdown 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 msgr2=0x55adb450c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4248001eb0 tx=0x7f4248041040 comp rx=0 tx=0).stop 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 shutdown_connections 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb451f7e0 0x55adb4521c00 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb450f790 0x55adb450d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f4232ffd640 0 client.28319 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb450cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 --2- 172.21.15.120:0/2798623974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb4502ee0 0x55adb450c7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 >> 172.21.15.120:0/2798623974 conn(0x55adb4500d10 msgr2=0x55adb44fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 shutdown_connections 2024-09-10T23:11:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 1 -- 172.21.15.120:0/2798623974 wait complete. 2024-09-10T23:11:10.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.230+0000 7f425c0d0740 20 client.28319 trim_cache size 0 max 16384 2024-09-10T23:11:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 Processor -- start 2024-09-10T23:11:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 -- start start 2024-09-10T23:11:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 0x55adb450ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4509f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ac360 con 0x55adb4502ee0 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42541aebe0 con 0x55adb450f790 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.248+0000 7f425c0d0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ac9d0 con 0x55adb4503800 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42537fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 0x55adb450ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42537fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55730/0 (socket says 172.21.15.120:55730) 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42537fe640 1 -- 172.21.15.120:0/1094680712 learned_addr learned my addr 172.21.15.120:0/1094680712 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 -- 172.21.15.120:0/1094680712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 msgr2=0x55adb45099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 -- 172.21.15.120:0/1094680712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 msgr2=0x55adb4509f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4509f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 -- 172.21.15.120:0/1094680712 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb4500b40 con 0x55adb4502ee0 2024-09-10T23:11:10.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42537fe640 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f4252ffd640 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 0x55adb450ce20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4248032af0 tx=0x7f4248003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2c0a71ae14c019 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42397fa640 1 -- 172.21.15.120:0/1094680712 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424804e070 con 0x55adb4502ee0 2024-09-10T23:11:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42397fa640 1 -- 172.21.15.120:0/1094680712 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42480042e0 con 0x55adb4502ee0 2024-09-10T23:11:10.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.249+0000 7f42397fa640 1 -- 172.21.15.120:0/1094680712 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424803cc30 con 0x55adb4502ee0 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 -- 172.21.15.120:0/1094680712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 msgr2=0x55adb450ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 0x55adb450ce20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4248032af0 tx=0x7f4248003e60 comp rx=0 tx=0).stop 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 -- 172.21.15.120:0/1094680712 shutdown_connections 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4509f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4503800 0x55adb45099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 --2- 172.21.15.120:0/1094680712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4502ee0 0x55adb450ce20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 -- 172.21.15.120:0/1094680712 >> 172.21.15.120:0/1094680712 conn(0x55adb4500d10 msgr2=0x55adb450c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 -- 172.21.15.120:0/1094680712 shutdown_connections 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.250+0000 7f425c0d0740 1 -- 172.21.15.120:0/1094680712 wait complete. 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 -- start start 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 Processor -- start 2024-09-10T23:11:10.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4502ee0 0x55adb45176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4514670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ad0a0 con 0x55adb4503800 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ae510 con 0x55adb450f790 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ad770 con 0x55adb4502ee0 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f425c0d0740 10 client.0 _ll_register_callbacks cb 0x7f4248009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f42537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4502ee0 0x55adb45176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f42537fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43774/0 (socket says 172.21.15.120:43774) 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f42537fe640 1 -- 172.21.15.120:0/3345385347 learned_addr learned my addr 172.21.15.120:0/3345385347 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.251+0000 7f4252ffd640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4514670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 -- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4502ee0 msgr2=0x55adb45176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4502ee0 0x55adb45176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 -- 172.21.15.120:0/3345385347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 msgr2=0x55adb4514670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4514670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 -- 172.21.15.120:0/3345385347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb4518410 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f42537fe640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x55adb4522450 tx=0x7f4244010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bf995572dbdde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f4232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244004230 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42440043d0 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42480042e0 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4244010e50 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f425c0d0740 10 client.19110 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adb458cd60 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.252+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42480042e0 con 0x55adb4503800 2024-09-10T23:11:10.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4244003ae0 con 0x55adb4503800 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 10 client.19110 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 10 client.19110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 20 client.19110 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb458d4f0 0x55adb458f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 4 client.19110 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55adb4477860 con 0x55adb458d4f0 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f4252ffd640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb458d4f0 0x55adb458f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f4252ffd640 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb458d4f0 0x55adb458f910 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582abd03db151031 server_cookie=1c8b8daba9d656a7 in_seq=0 out_seq=0 2024-09-10T23:11:10.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.253+0000 7f4232ffd640 10 client.19110 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:10.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.259+0000 7f4232ffd640 1 -- 172.21.15.120:0/3345385347 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55adb4477860 con 0x55adb458d4f0 2024-09-10T23:11:10.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.259+0000 7f4232ffd640 10 client.19110 handle_command_reply: tid=0 2024-09-10T23:11:10.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.259+0000 7f425c0d0740 1 client.19110 shutdown 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.259+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 msgr2=0x55adb4517be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.259+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x55adb4522450 tx=0x7f4244010ba0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 shutdown_connections 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55adb458d4f0 0x55adb458f910 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55adb450f790 0x55adb4514670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55adb4503800 0x55adb4517be0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 --2- 172.21.15.120:0/3345385347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55adb4502ee0 0x55adb45176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 >> 172.21.15.120:0/3345385347 conn(0x55adb4500d10 msgr2=0x55adb45068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f4232ffd640 0 client.19110 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 shutdown_connections 2024-09-10T23:11:10.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 1 -- 172.21.15.120:0/3345385347 wait complete. 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.260+0000 7f425c0d0740 20 client.19110 trim_cache size 0 max 16384 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "f966826f-4efc-4867-9127-7696da6e341a", 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:11:10.262 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 msgr2=0x7f423c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 0x7f423c0b77a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f424c0ac9d0 tx=0x7f424c01b8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 msgr2=0x7f425419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f424c00eb10 tx=0x7f424c00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 shutdown_connections 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f423c0b52e0 0x7f423c0b77a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4254109150 0x7f425419eb50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42541087d0 0x7f42541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 --2- 172.21.15.120:0/3953216428 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4254102dc0 0x7f4254199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 >> 172.21.15.120:0/3953216428 conn(0x7f42540fe830 msgr2=0x7f4254100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:10.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 shutdown_connections 2024-09-10T23:11:10.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:10.261+0000 7f4259cfb640 1 -- 172.21.15.120:0/3953216428 wait complete. 2024-09-10T23:11:10.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:10 smithi153 ceph-mon[34951]: pgmap v908: 97 pgs: 97 active+clean; 240 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-09-10T23:11:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:10 smithi176 ceph-mon[34982]: pgmap v908: 97 pgs: 97 active+clean; 240 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-09-10T23:11:10.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149685 21.76 MB/sec execute 245 sec latency 83.660 ms 2024-09-10T23:11:11.352 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:11:11.654 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:11.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149720 21.68 MB/sec execute 246 sec latency 511.009 ms 2024-09-10T23:11:12.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 -- 172.21.15.120:0/1696800892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 msgr2=0x7f6034102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 --2- 172.21.15.120:0/1696800892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f6034102070 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f6030009900 tx=0x7f603002a410 comp rx=0 tx=0).stop 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 -- 172.21.15.120:0/1696800892 shutdown_connections 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 --2- 172.21.15.120:0/1696800892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 --2- 172.21.15.120:0/1696800892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 0x7f6034107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 --2- 172.21.15.120:0/1696800892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f6034102070 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.208+0000 7f603b59b640 1 -- 172.21.15.120:0/1696800892 >> 172.21.15.120:0/1696800892 conn(0x7f60340fd700 msgr2=0x7f60340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.209+0000 7f603b59b640 1 -- 172.21.15.120:0/1696800892 shutdown_connections 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.209+0000 7f603b59b640 1 -- 172.21.15.120:0/1696800892 wait complete. 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.209+0000 7f603b59b640 1 Processor -- start 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.209+0000 7f603b59b640 1 -- start start 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 0x7f6034198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111ce0 con 0x7f6034101c90 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111b60 con 0x7f6034108020 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111e60 con 0x7f60341076a0 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43800/0 (socket says 172.21.15.120:43800) 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 -- 172.21.15.120:0/2553936018 learned_addr learned my addr 172.21.15.120:0/2553936018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039b38640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6038b36640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 0x7f6034198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 -- 172.21.15.120:0/2553936018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 msgr2=0x7f6034198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 0x7f6034198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 -- 172.21.15.120:0/2553936018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 msgr2=0x7f603419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 -- 172.21.15.120:0/2553936018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419e2a0 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039b38640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f6039337640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6030005940 tx=0x7f60300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f602a7fc640 1 -- 172.21.15.120:0/2553936018 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603003f030 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f602a7fc640 1 -- 172.21.15.120:0/2553936018 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60300044c0 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.210+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603419e530 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.211+0000 7f602a7fc640 1 -- 172.21.15.120:0/2553936018 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030035070 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.211+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a4c50 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f602a7fc640 1 -- 172.21.15.120:0/2553936018 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f603004c060 con 0x7f6034101c90 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 Processor -- start 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 -- start start 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e90d406e50 con 0x55e90d49ef40 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e90d3f6280 con 0x55e90d490480 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.213+0000 7f603d970740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e90d3fa090 con 0x55e90d490da0 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039b38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039b38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52404/0 (socket says 172.21.15.120:52404) 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43802/0 (socket says 172.21.15.120:43802) 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039b38640 1 -- 172.21.15.120:0/410959614 learned_addr learned my addr 172.21.15.120:0/410959614 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039337640 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f602a7fc640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 0x7f60200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.237 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 -- 172.21.15.120:0/410959614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 msgr2=0x55e90d49ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 -- 172.21.15.120:0/410959614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 msgr2=0x55e90d490860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 -- 172.21.15.120:0/410959614 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90d154650 con 0x55e90d49ef40 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039b38640 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6039b38640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 0x7f60200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f602a7fc640 1 -- 172.21.15.120:0/2553936018 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6030105b40 con 0x7f6034101c90 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f6038b36640 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6024009bd0 tx=0x7f6024011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdeb0da4ce261b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f601f7fe640 1 -- 172.21.15.120:0/410959614 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024004230 con 0x55e90d49ef40 2024-09-10T23:11:12.238 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f601f7fe640 1 -- 172.21.15.120:0/410959614 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6024009e40 con 0x55e90d49ef40 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.214+0000 7f601f7fe640 1 -- 172.21.15.120:0/410959614 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024004c60 con 0x55e90d49ef40 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 -- 172.21.15.120:0/410959614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 msgr2=0x55e90d4a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6024009bd0 tx=0x7f6024011810 comp rx=0 tx=0).stop 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f6039b38640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 0x7f60200b77a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f602c009fd0 tx=0x7f602c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 -- 172.21.15.120:0/410959614 shutdown_connections 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a1330 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 --2- 172.21.15.120:0/410959614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d490860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 -- 172.21.15.120:0/410959614 >> 172.21.15.120:0/410959614 conn(0x55e90d48e2b0 msgr2=0x55e90d48f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 -- 172.21.15.120:0/410959614 shutdown_connections 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.215+0000 7f603d970740 1 -- 172.21.15.120:0/410959614 wait complete. 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 -- start start 2024-09-10T23:11:12.239 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 Processor -- start 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d49e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e90d257700 con 0x55e90d49ef40 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e90d391110 con 0x55e90d490480 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e90d4a1bf0 con 0x55e90d490da0 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f603d970740 10 client.0 _ll_register_callbacks cb 0x55e90d3aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f6039b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f6039b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43812/0 (socket says 172.21.15.120:43812) 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f6039b38640 1 -- 172.21.15.120:0/1675058941 learned_addr learned my addr 172.21.15.120:0/1675058941 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f6038b36640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.240 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.216+0000 7f6039337640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d49e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 -- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 msgr2=0x55e90d49bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 -- 172.21.15.120:0/1675058941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 msgr2=0x55e90d49e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d49e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 -- 172.21.15.120:0/1675058941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90d3bec40 con 0x55e90d49ef40 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039337640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d49e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6038b36640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f6039b38640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f602c0115f0 tx=0x7f602c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa070556ddc6e2c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c02d060 con 0x55e90d49ef40 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f602c028600 con 0x55e90d49ef40 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e90d408da0 con 0x55e90d49ef40 2024-09-10T23:11:12.241 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f602c004b20 con 0x55e90d49ef40 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f603d970740 10 client.19122 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e90d4ad550 con 0x55e90d49ef40 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.217+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e90d408da0 con 0x55e90d49ef40 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f602c005260 con 0x55e90d49ef40 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 10 client.19122 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 10 client.19122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 20 client.19122 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d4aea00 0x55e90d4b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 4 client.19122 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e90d404850 con 0x55e90d4aea00 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f6039337640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d4aea00 0x55e90d4b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f601cff9640 10 client.19122 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:12.242 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.218+0000 7f6039337640 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d4aea00 0x55e90d4b0e30 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da54042c1a8536f server_cookie=5dfab532755959 in_seq=0 out_seq=0 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.220+0000 7f601cff9640 1 -- 172.21.15.120:0/1675058941 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e90d404850 con 0x55e90d4aea00 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.220+0000 7f601cff9640 10 client.19122 handle_command_reply: tid=0 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.220+0000 7f603d970740 1 client.19122 shutdown 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 msgr2=0x55e90d49c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f602c0115f0 tx=0x7f602c004ab0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 shutdown_connections 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d4aea00 0x55e90d4b0e30 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d49c230 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.243 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490da0 0x55e90d49bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 --2- 172.21.15.120:0/1675058941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490480 0x55e90d49e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f601cff9640 0 client.19122 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 >> 172.21.15.120:0/1675058941 conn(0x55e90d48e2b0 msgr2=0x55e90d493de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 shutdown_connections 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 1 -- 172.21.15.120:0/1675058941 wait complete. 2024-09-10T23:11:12.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.221+0000 7f603d970740 20 client.19122 trim_cache size 0 max 16384 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 Processor -- start 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 -- start start 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aa100 con 0x55e90d49ef40 2024-09-10T23:11:12.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ac960 con 0x55e90d490da0 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f603d970740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aa750 con 0x55e90d490480 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:43828/0 (socket says 172.21.15.120:43828) 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 -- 172.21.15.120:0/1070128931 learned_addr learned my addr 172.21.15.120:0/1070128931 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6038b36640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039337640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 -- 172.21.15.120:0/1070128931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 msgr2=0x55e90d4aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 -- 172.21.15.120:0/1070128931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 msgr2=0x55e90d4997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6039b38640 1 -- 172.21.15.120:0/1070128931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90d48e0e0 con 0x55e90d49ef40 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.239+0000 7f6038b36640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.249 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.240+0000 7f6039b38640 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f602c005630 tx=0x7f602c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f6e21b451380bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.240+0000 7f601effd640 1 -- 172.21.15.120:0/1070128931 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c004cc0 con 0x55e90d49ef40 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.240+0000 7f601effd640 1 -- 172.21.15.120:0/1070128931 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c034480 con 0x55e90d49ef40 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.240+0000 7f601effd640 1 -- 172.21.15.120:0/1070128931 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c028660 con 0x55e90d49ef40 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 -- 172.21.15.120:0/1070128931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 msgr2=0x55e90d48ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f602c005630 tx=0x7f602c009520 comp rx=0 tx=0).stop 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 -- 172.21.15.120:0/1070128931 shutdown_connections 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d48ef90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 --2- 172.21.15.120:0/1070128931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.241+0000 7f603d970740 1 -- 172.21.15.120:0/1070128931 >> 172.21.15.120:0/1070128931 conn(0x55e90d48e2b0 msgr2=0x55e90d49fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.242+0000 7f603d970740 1 -- 172.21.15.120:0/1070128931 shutdown_connections 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.242+0000 7f603d970740 1 -- 172.21.15.120:0/1070128931 wait complete. 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.242+0000 7f603d970740 1 -- start start 2024-09-10T23:11:12.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.242+0000 7f603d970740 1 Processor -- start 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aae20 con 0x55e90d49ef40 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ac290 con 0x55e90d490da0 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ab4f0 con 0x55e90d490480 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f603d970740 10 client.0 _ll_register_callbacks cb 0x7f602c011510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6039b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:52444/0 (socket says 172.21.15.120:52444) 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 -- 172.21.15.120:0/782546582 learned_addr learned my addr 172.21.15.120:0/782546582 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6039337640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 -- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 msgr2=0x55e90d4b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 -- 172.21.15.120:0/782546582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 msgr2=0x55e90d4a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6038b36640 1 -- 172.21.15.120:0/782546582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90d4a64c0 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.243+0000 7f6039b38640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f6038b36640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f60240096e0 tx=0x7f6024005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5604f39d664ae11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602401e040 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f60240038b0 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c034480 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6024034050 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f603d970740 10 client.28340 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e90d4a77d0 con 0x55e90d490da0 2024-09-10T23:11:12.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.244+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c034480 con 0x55e90d490da0 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6024018070 con 0x55e90d490da0 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 10 client.28340 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 10 client.28340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 20 client.28340 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d51dd40 0x55e90d520130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 4 client.28340 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e90d401c80 con 0x55e90d51dd40 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f6039337640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d51dd40 0x55e90d520130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f6039337640 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d51dd40 0x55e90d520130 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4306d46fdeafc214 server_cookie=7c20f8b940d099b1 in_seq=0 out_seq=0 2024-09-10T23:11:12.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.245+0000 7f601cff9640 10 client.28340 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:12.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.248+0000 7f601cff9640 1 -- 172.21.15.120:0/782546582 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9374 (crc 0 0 0) 0x55e90d401c80 con 0x55e90d51dd40 2024-09-10T23:11:12.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.248+0000 7f601cff9640 10 client.28340 handle_command_reply: tid=0 2024-09-10T23:11:12.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.248+0000 7f603d970740 1 client.28340 shutdown 2024-09-10T23:11:12.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 msgr2=0x55e90d4b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f60240096e0 tx=0x7f6024005c50 comp rx=0 tx=0).stop 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 shutdown_connections 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e90d51dd40 0x55e90d520130 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e90d49ef40 0x55e90d4a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e90d490da0 0x55e90d4b6460 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f601cff9640 0 client.28340 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 --2- 172.21.15.120:0/782546582 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e90d490480 0x55e90d4b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 >> 172.21.15.120:0/782546582 conn(0x55e90d48e2b0 msgr2=0x55e90d49d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 shutdown_connections 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 1 -- 172.21.15.120:0/782546582 wait complete. 2024-09-10T23:11:12.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603d970740 20 client.28340 trim_cache size 0 max 16384 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "f966826f-4efc-4867-9127-7696da6e341a", 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:11:12.256 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:11:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:11:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:11:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 msgr2=0x7f60200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 0x7f60200b77a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f602c009fd0 tx=0x7f602c004a40 comp rx=0 tx=0).stop 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 msgr2=0x7f60341981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.249+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6030005940 tx=0x7f60300040c0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 shutdown_connections 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f60200b52e0 0x7f60200b77a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6034108020 0x7f603419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60341076a0 0x7f6034198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 --2- 172.21.15.120:0/2553936018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6034101c90 0x7f60341981f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 >> 172.21.15.120:0/2553936018 conn(0x7f60340fd700 msgr2=0x7f60340ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 shutdown_connections 2024-09-10T23:11:12.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:12.250+0000 7f603b59b640 1 -- 172.21.15.120:0/2553936018 wait complete. 2024-09-10T23:11:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:12 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:11:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:12 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:12 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:11:12.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:12 smithi120 ceph-mon[31247]: pgmap v909: 97 pgs: 97 active+clean; 233 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 528 op/s 2024-09-10T23:11:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:12 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:11:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:12 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:12 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:11:12.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:12 smithi153 ceph-mon[34951]: pgmap v909: 97 pgs: 97 active+clean; 233 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 528 op/s 2024-09-10T23:11:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:12 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:11:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:12 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:12 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:11:12.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:12 smithi176 ceph-mon[34982]: pgmap v909: 97 pgs: 97 active+clean; 233 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 528 op/s 2024-09-10T23:11:12.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 149842 21.62 MB/sec execute 247 sec latency 650.530 ms 2024-09-10T23:11:13.017 INFO:tasks.cephfs.filesystem:scrub status for tag:f966826f-4efc-4867-9127-7696da6e341a - {'path': '/', 'tag': 'f966826f-4efc-4867-9127-7696da6e341a', 'options': 'recursive,force'} 2024-09-10T23:11:13.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:13 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:13.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:13 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:13.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:13 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:13.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 150300 21.60 MB/sec execute 248 sec latency 51.333 ms 2024-09-10T23:11:14.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:14 smithi120 ceph-mon[31247]: pgmap v910: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-10T23:11:14.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:14 smithi153 ceph-mon[34951]: pgmap v910: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-10T23:11:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:14 smithi176 ceph-mon[34982]: pgmap v910: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-10T23:11:14.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 150881 21.64 MB/sec execute 249 sec latency 52.792 ms 2024-09-10T23:11:15.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:11:15.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:11:15.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:11:15.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 151492 21.68 MB/sec execute 250 sec latency 49.865 ms 2024-09-10T23:11:16.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:16 smithi120 ceph-mon[31247]: pgmap v911: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-09-10T23:11:16.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:16 smithi153 ceph-mon[34951]: pgmap v911: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-09-10T23:11:16.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:16 smithi176 ceph-mon[34982]: pgmap v911: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-09-10T23:11:16.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 151901 21.72 MB/sec execute 251 sec latency 272.627 ms 2024-09-10T23:11:17.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 152354 21.74 MB/sec execute 252 sec latency 71.390 ms 2024-09-10T23:11:18.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:18 smithi120 ceph-mon[31247]: pgmap v912: 97 pgs: 97 active+clean; 239 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2024-09-10T23:11:18.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:18 smithi153 ceph-mon[34951]: pgmap v912: 97 pgs: 97 active+clean; 239 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2024-09-10T23:11:18.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:18 smithi176 ceph-mon[34982]: pgmap v912: 97 pgs: 97 active+clean; 239 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.1 MiB/s wr, 682 op/s 2024-09-10T23:11:18.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 152735 21.74 MB/sec execute 253 sec latency 73.079 ms 2024-09-10T23:11:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 -- 172.21.15.120:0/1354606961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 msgr2=0x7fca3410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 --2- 172.21.15.120:0/1354606961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca3410ae20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009cd0 tx=0x7fca2c02a8f0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 -- 172.21.15.120:0/1354606961 shutdown_connections 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 --2- 172.21.15.120:0/1354606961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca3410ae20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 --2- 172.21.15.120:0/1354606961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34103ae0 0x7fca34103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.004+0000 7fca3c582640 1 --2- 172.21.15.120:0/1354606961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34102990 0x7fca34102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.005+0000 7fca3c582640 1 -- 172.21.15.120:0/1354606961 >> 172.21.15.120:0/1354606961 conn(0x7fca340fe250 msgr2=0x7fca34100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.005+0000 7fca3c582640 1 -- 172.21.15.120:0/1354606961 shutdown_connections 2024-09-10T23:11:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.005+0000 7fca3c582640 1 -- 172.21.15.120:0/1354606961 wait complete. 2024-09-10T23:11:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.005+0000 7fca3c582640 1 Processor -- start 2024-09-10T23:11:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.005+0000 7fca3c582640 1 -- start start 2024-09-10T23:11:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34102990 0x7fca34076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca34075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410e600 con 0x7fca341044a0 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410e480 con 0x7fca34103ae0 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3c582640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410e780 con 0x7fca34102990 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3ab1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca34075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40008/0 (socket says 172.21.15.120:40008) 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3ab1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca34075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36382/0 (socket says 172.21.15.120:36382) 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 -- 172.21.15.120:0/3568888887 learned_addr learned my addr 172.21.15.120:0/3568888887 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca3a31e640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34102990 0x7fca34076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 -- 172.21.15.120:0/3568888887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34102990 msgr2=0x7fca34076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34102990 0x7fca34076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 -- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 msgr2=0x7fca34075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca34075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.006+0000 7fca39b1d640 1 -- 172.21.15.120:0/3568888887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca340762f0 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca39b1d640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9f0 tx=0x7fca2400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca2b7fe640 1 -- 172.21.15.120:0/3568888887 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cd80 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341a2e40 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca2b7fe640 1 -- 172.21.15.120:0/3568888887 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca240054d0 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a32b0 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.007+0000 7fca2b7fe640 1 -- 172.21.15.120:0/3568888887 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24010690 con 0x7fca34103ae0 2024-09-10T23:11:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.010+0000 7fca2b7fe640 1 -- 172.21.15.120:0/3568888887 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fca24004e90 con 0x7fca34103ae0 2024-09-10T23:11:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.010+0000 7fca3f429740 1 Processor -- start 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.010+0000 7fca3f429740 1 -- start start 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d363dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 0x5606d3640580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606d35ad6f0 con 0x5606d3640bd0 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606d359cb20 con 0x5606d363b860 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3f429740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606d35a0930 con 0x5606d363e190 2024-09-10T23:11:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d363dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3a31e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 0x5606d3640580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3a31e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 0x5606d3640580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44568/0 (socket says 172.21.15.120:44568) 2024-09-10T23:11:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3a31e640 1 -- 172.21.15.120:0/1597258750 learned_addr learned my addr 172.21.15.120:0/1597258750 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 -- 172.21.15.120:0/1597258750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 msgr2=0x5606d3640580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 0x5606d3640580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 -- 172.21.15.120:0/1597258750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 msgr2=0x5606d3643090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca3ab1f640 1 -- 172.21.15.120:0/1597258750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d3562530 con 0x5606d363b860 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.011+0000 7fca2b7fe640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 0x7fca200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca3a31e640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 0x7fca200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca2b7fe640 1 -- 172.21.15.120:0/3568888887 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fca240e5f80 con 0x7fca34103ae0 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d363dc50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005e30 tx=0x7fca2c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94e26a24e5fc1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca28bfe640 1 -- 172.21.15.120:0/1597258750 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c030900 con 0x5606d363b860 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca28bfe640 1 -- 172.21.15.120:0/1597258750 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c036030 con 0x5606d363b860 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.012+0000 7fca28bfe640 1 -- 172.21.15.120:0/1597258750 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004540 con 0x5606d363b860 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 -- 172.21.15.120:0/1597258750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 msgr2=0x5606d363dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d363dc50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005e30 tx=0x7fca2c009a20 comp rx=0 tx=0).stop 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3a31e640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 0x7fca200b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca30005bc0 tx=0x7fca30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 -- 172.21.15.120:0/1597258750 shutdown_connections 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363e190 0x5606d3640580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 --2- 172.21.15.120:0/1597258750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d363dc50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.013+0000 7fca3f429740 1 -- 172.21.15.120:0/1597258750 >> 172.21.15.120:0/1597258750 conn(0x5606d36369b0 msgr2=0x5606d3637740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.014+0000 7fca3f429740 1 -- 172.21.15.120:0/1597258750 shutdown_connections 2024-09-10T23:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.014+0000 7fca3f429740 1 -- 172.21.15.120:0/1597258750 wait complete. 2024-09-10T23:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.014+0000 7fca3f429740 1 -- start start 2024-09-10T23:11:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.014+0000 7fca3f429740 1 Processor -- start 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.014+0000 7fca3f429740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d36455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44576/0 (socket says 172.21.15.120:44576) 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 -- 172.21.15.120:0/2481997324 learned_addr learned my addr 172.21.15.120:0/2481997324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d3640bd0 0x5606d36461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca39b1d640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d36455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5606d34d4ab0 con 0x5606d363e190 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5606d3542730 con 0x5606d3640bd0 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5606d3643b50 con 0x5606d363b860 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d3640bd0 0x5606d36461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 -- 172.21.15.120:0/2481997324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d3640bd0 msgr2=0x5606d36461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d3640bd0 0x5606d36461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 -- 172.21.15.120:0/2481997324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 msgr2=0x5606d36455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d36455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3a31e640 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d32fe790 con 0x5606d363b860 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.015+0000 7fca3f429740 10 client.0 _ll_register_callbacks cb 0x5606d35966b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3a31e640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca3001e9d0 tx=0x7fca30004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30834dc436c07bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c036030 con 0x5606d363b860 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca1dffb640 1 -- 172.21.15.120:0/2481997324 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3002e060 con 0x5606d363b860 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca1dffb640 1 -- 172.21.15.120:0/2481997324 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fca30018030 con 0x5606d363b860 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca1dffb640 1 -- 172.21.15.120:0/2481997324 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca30009b30 con 0x5606d363b860 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 10 client.28585 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5606d3377620 con 0x5606d363b860 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca1dffb640 1 -- 172.21.15.120:0/2481997324 <== mon.1 v2:172.21.15.176:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fca30029420 con 0x5606d363b860 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 10 client.28585 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 10 client.28585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 20 client.28585 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d3650670 0x5606d3652a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 4 client.28585 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.016+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca2c036030 con 0x5606d3650670 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.017+0000 7fca39b1d640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d3650670 0x5606d3652a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.017+0000 7fca39b1d640 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d3650670 0x5606d3652a60 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892230e822eb3345 server_cookie=4b4000a980d84148 in_seq=0 out_seq=0 2024-09-10T23:11:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.017+0000 7fca1dffb640 10 client.28585 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca1dffb640 1 -- 172.21.15.120:0/2481997324 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fca2c036030 con 0x5606d3650670 2024-09-10T23:11:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca1dffb640 10 client.28585 handle_command_reply: tid=0 2024-09-10T23:11:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 client.28585 shutdown 2024-09-10T23:11:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 msgr2=0x5606d3644f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca3001e9d0 tx=0x7fca30004730 comp rx=0 tx=0).stop 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 shutdown_connections 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d3650670 0x5606d3652a60 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d3640bd0 0x5606d36461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d36455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 --2- 172.21.15.120:0/2481997324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d3644f50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 >> 172.21.15.120:0/2481997324 conn(0x5606d36369b0 msgr2=0x5606d3635bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca1dffb640 0 client.28585 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 shutdown_connections 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 1 -- 172.21.15.120:0/2481997324 wait complete. 2024-09-10T23:11:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.020+0000 7fca3f429740 20 client.28585 trim_cache size 0 max 16384 2024-09-10T23:11:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 Processor -- start 2024-09-10T23:11:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 -- start start 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d36372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363e190 0x5606d3637830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a5b90 con 0x5606d3640bd0 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a8410 con 0x5606d363e190 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3f429740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a6200 con 0x5606d363b860 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36404/0 (socket says 172.21.15.120:36404) 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 -- 172.21.15.120:0/3432849508 learned_addr learned my addr 172.21.15.120:0/3432849508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3a31e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d36372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca39b1d640 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363e190 0x5606d3637830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 -- 172.21.15.120:0/3432849508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 msgr2=0x5606d36372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d36372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 -- 172.21.15.120:0/3432849508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363e190 msgr2=0x5606d3637830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363e190 0x5606d3637830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.038+0000 7fca3ab1f640 1 -- 172.21.15.120:0/3432849508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d363dfc0 con 0x5606d3640bd0 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.039+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009cd0 tx=0x7fca2c030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228bae9776bc7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.039+0000 7fca1ffff640 1 -- 172.21.15.120:0/3432849508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c030460 con 0x5606d3640bd0 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.039+0000 7fca1ffff640 1 -- 172.21.15.120:0/3432849508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c036040 con 0x5606d3640bd0 2024-09-10T23:11:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.039+0000 7fca1ffff640 1 -- 172.21.15.120:0/3432849508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004310 con 0x5606d3640bd0 2024-09-10T23:11:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- 172.21.15.120:0/3432849508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 msgr2=0x5606d3643ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009cd0 tx=0x7fca2c030800 comp rx=0 tx=0).stop 2024-09-10T23:11:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- 172.21.15.120:0/3432849508 shutdown_connections 2024-09-10T23:11:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d3640bd0 0x5606d3643ef0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363e190 0x5606d3637830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 --2- 172.21.15.120:0/3432849508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d363b860 0x5606d36372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- 172.21.15.120:0/3432849508 >> 172.21.15.120:0/3432849508 conn(0x5606d36369b0 msgr2=0x5606d363ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- 172.21.15.120:0/3432849508 shutdown_connections 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- 172.21.15.120:0/3432849508 wait complete. 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 -- start start 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.040+0000 7fca3f429740 1 Processor -- start 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d3649e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d3640bd0 0x5606d364b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a68d0 con 0x5606d363e190 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a7d40 con 0x5606d363b860 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a6fa0 con 0x5606d3640bd0 2024-09-10T23:11:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 10 client.0 _ll_register_callbacks cb 0x7fca30009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36412/0 (socket says 172.21.15.120:36412) 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 -- 172.21.15.120:0/60538973 learned_addr learned my addr 172.21.15.120:0/60538973 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3ab1f640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d3640bd0 0x5606d364b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3a31e640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d3649e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 -- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d3640bd0 msgr2=0x5606d364b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d3640bd0 0x5606d364b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 -- 172.21.15.120:0/60538973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 msgr2=0x5606d3649e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d3649e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 -- 172.21.15.120:0/60538973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d3646530 con 0x5606d363e190 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca39b1d640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca240ac700 tx=0x7fca24005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea95ae40679604 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240c2030 con 0x5606d363e190 2024-09-10T23:11:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fca240a89e0 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.041+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c036040 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca240ace70 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 10 client.19143 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5606d364cb80 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c036040 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fca240e80b0 con 0x5606d363e190 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 10 client.19143 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 10 client.19143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 20 client.19143 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d36bd6f0 0x5606d36bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 4 client.19143 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606d35af640 con 0x5606d36bd6f0 2024-09-10T23:11:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3a31e640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d36bd6f0 0x5606d36bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca3a31e640 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d36bd6f0 0x5606d36bfae0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=600bd5fb51ecf651 server_cookie=cd2846f869f91043 in_seq=0 out_seq=0 2024-09-10T23:11:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.042+0000 7fca1dffb640 10 client.19143 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca1dffb640 1 -- 172.21.15.120:0/60538973 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606d35af640 con 0x5606d36bd6f0 2024-09-10T23:11:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca1dffb640 10 client.19143 handle_command_reply: tid=0 2024-09-10T23:11:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 client.19143 shutdown 2024-09-10T23:11:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 msgr2=0x5606d364a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca240ac700 tx=0x7fca24005700 comp rx=0 tx=0).stop 2024-09-10T23:11:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 shutdown_connections 2024-09-10T23:11:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5606d36bd6f0 0x5606d36bfae0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca1dffb640 0 client.19143 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606d3640bd0 0x5606d364b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5606d363e190 0x5606d364a530 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 --2- 172.21.15.120:0/60538973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5606d363b860 0x5606d3649e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 >> 172.21.15.120:0/60538973 conn(0x5606d36369b0 msgr2=0x5606d363a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 shutdown_connections 2024-09-10T23:11:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 1 -- 172.21.15.120:0/60538973 wait complete. 2024-09-10T23:11:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.687+0000 7fca3f429740 20 client.19143 trim_cache size 0 max 16384 2024-09-10T23:11:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:11:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:11:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:11:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:11:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 msgr2=0x7fca200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 0x7fca200b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca30005bc0 tx=0x7fca30005f90 comp rx=0 tx=0).stop 2024-09-10T23:11:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 msgr2=0x7fca34075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9f0 tx=0x7fca2400edf0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 shutdown_connections 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fca200b52e0 0x7fca200b77a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca341044a0 0x7fca34075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca34103ae0 0x7fca34075540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 --2- 172.21.15.120:0/3568888887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca34102990 0x7fca34076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 >> 172.21.15.120:0/3568888887 conn(0x7fca340fe250 msgr2=0x7fca34106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 shutdown_connections 2024-09-10T23:11:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:19.688+0000 7fca3c582640 1 -- 172.21.15.120:0/3568888887 wait complete. 2024-09-10T23:11:19.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 152902 21.67 MB/sec execute 254 sec latency 353.822 ms 2024-09-10T23:11:20.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:20 smithi120 ceph-mon[31247]: pgmap v913: 97 pgs: 97 active+clean; 245 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 587 op/s 2024-09-10T23:11:20.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:20 smithi153 ceph-mon[34951]: pgmap v913: 97 pgs: 97 active+clean; 245 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 587 op/s 2024-09-10T23:11:20.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:20 smithi176 ceph-mon[34982]: pgmap v913: 97 pgs: 97 active+clean; 245 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 587 op/s 2024-09-10T23:11:20.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 153291 21.67 MB/sec execute 255 sec latency 66.792 ms 2024-09-10T23:11:21.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 153632 21.65 MB/sec execute 256 sec latency 296.694 ms 2024-09-10T23:11:22.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:22 smithi120 ceph-mon[31247]: pgmap v914: 97 pgs: 97 active+clean; 240 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-10T23:11:22.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:22 smithi153 ceph-mon[34951]: pgmap v914: 97 pgs: 97 active+clean; 240 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-10T23:11:22.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:22 smithi176 ceph-mon[34982]: pgmap v914: 97 pgs: 97 active+clean; 240 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-10T23:11:22.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 154159 21.63 MB/sec execute 257 sec latency 93.848 ms 2024-09-10T23:11:23.879 INFO:tasks.workunit.client.0.smithi120.stdout: 10 154715 21.68 MB/sec execute 258 sec latency 69.942 ms 2024-09-10T23:11:24.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:24 smithi120 ceph-mon[31247]: pgmap v915: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 731 op/s 2024-09-10T23:11:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:24 smithi153 ceph-mon[34951]: pgmap v915: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 731 op/s 2024-09-10T23:11:24.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:24 smithi176 ceph-mon[34982]: pgmap v915: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.4 MiB/s wr, 731 op/s 2024-09-10T23:11:24.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 155306 21.73 MB/sec execute 259 sec latency 89.845 ms 2024-09-10T23:11:25.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 155788 21.78 MB/sec execute 260 sec latency 65.540 ms 2024-09-10T23:11:26.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:26 smithi120 ceph-mon[31247]: pgmap v916: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-09-10T23:11:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:26 smithi153 ceph-mon[34951]: pgmap v916: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-09-10T23:11:26.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:26 smithi176 ceph-mon[34982]: pgmap v916: 97 pgs: 97 active+clean; 231 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-09-10T23:11:26.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156133 21.78 MB/sec execute 261 sec latency 262.547 ms 2024-09-10T23:11:27.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156452 21.75 MB/sec execute 262 sec latency 106.037 ms 2024-09-10T23:11:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:28 smithi120 ceph-mon[31247]: pgmap v917: 97 pgs: 97 active+clean; 235 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.0 MiB/s wr, 700 op/s 2024-09-10T23:11:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:28 smithi153 ceph-mon[34951]: pgmap v917: 97 pgs: 97 active+clean; 235 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.0 MiB/s wr, 700 op/s 2024-09-10T23:11:28.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:28 smithi176 ceph-mon[34982]: pgmap v917: 97 pgs: 97 active+clean; 235 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.0 MiB/s wr, 700 op/s 2024-09-10T23:11:28.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 156853 21.74 MB/sec execute 263 sec latency 67.020 ms 2024-09-10T23:11:29.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 157305 21.72 MB/sec execute 264 sec latency 62.103 ms 2024-09-10T23:11:30.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:30 smithi153 ceph-mon[34951]: pgmap v918: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:11:30.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:30 smithi176 ceph-mon[34982]: pgmap v918: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:11:30.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:30 smithi120 ceph-mon[31247]: pgmap v918: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:11:30.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 157840 21.74 MB/sec execute 265 sec latency 78.053 ms 2024-09-10T23:11:31.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 158440 21.79 MB/sec execute 266 sec latency 144.943 ms 2024-09-10T23:11:32.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:32 smithi176 ceph-mon[34982]: pgmap v919: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2024-09-10T23:11:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:32 smithi120 ceph-mon[31247]: pgmap v919: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2024-09-10T23:11:32.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:32 smithi153 ceph-mon[34951]: pgmap v919: 97 pgs: 97 active+clean; 236 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2024-09-10T23:11:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.801+0000 7ff6b8260640 1 -- 172.21.15.120:0/291243086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 msgr2=0x7ff6b0074870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.801+0000 7ff6b8260640 1 --2- 172.21.15.120:0/291243086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b0074870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009ca0 tx=0x7ff6a802a8d0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.805+0000 7ff6b8260640 1 -- 172.21.15.120:0/291243086 shutdown_connections 2024-09-10T23:11:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.805+0000 7ff6b8260640 1 --2- 172.21.15.120:0/291243086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 0x7ff6b01136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.805+0000 7ff6b8260640 1 --2- 172.21.15.120:0/291243086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6b010a150 0x7ff6b010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.805+0000 7ff6b8260640 1 --2- 172.21.15.120:0/291243086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b0074870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.805+0000 7ff6b8260640 1 -- 172.21.15.120:0/291243086 >> 172.21.15.120:0/291243086 conn(0x7ff6b006fa40 msgr2=0x7ff6b0071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.809+0000 7ff6b8260640 1 -- 172.21.15.120:0/291243086 shutdown_connections 2024-09-10T23:11:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.810+0000 7ff6b8260640 1 -- 172.21.15.120:0/291243086 wait complete. 2024-09-10T23:11:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 Processor -- start 2024-09-10T23:11:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 -- start start 2024-09-10T23:11:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6b010a150 0x7ff6b019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 0x7ff6b01a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0114c40 con 0x7ff6b010ab10 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0114ac0 con 0x7ff6b0074470 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b8260640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0114dc0 con 0x7ff6b010a150 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b5ffc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b5ffc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46488/0 (socket says 172.21.15.120:46488) 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b5ffc640 1 -- 172.21.15.120:0/2181719794 learned_addr learned my addr 172.21.15.120:0/2181719794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.811+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 0x7ff6b01a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 -- 172.21.15.120:0/2181719794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6b010a150 msgr2=0x7ff6b019f990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6b010a150 0x7ff6b019f990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 -- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 msgr2=0x7ff6b01a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 0x7ff6b01a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 -- 172.21.15.120:0/2181719794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b01a0e80 con 0x7ff6b0074470 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.812+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6a802aec0 tx=0x7ff6a8030800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.813+0000 7ff6a6ffd640 1 -- 172.21.15.120:0/2181719794 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8030bf0 con 0x7ff6b0074470 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.813+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b01a1110 con 0x7ff6b0074470 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.813+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b007b590 con 0x7ff6b0074470 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.813+0000 7ff6a6ffd640 1 -- 172.21.15.120:0/2181719794 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a8030d90 con 0x7ff6b0074470 2024-09-10T23:11:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.813+0000 7ff6a6ffd640 1 -- 172.21.15.120:0/2181719794 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a803a710 con 0x7ff6b0074470 2024-09-10T23:11:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.817+0000 7ff6a6ffd640 1 -- 172.21.15.120:0/2181719794 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff6a803b9b0 con 0x7ff6b0074470 2024-09-10T23:11:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.820+0000 7ff6a6ffd640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 0x7ff69c0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.821+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 0x7ff69c0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.821+0000 7ff6a6ffd640 1 -- 172.21.15.120:0/2181719794 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff6a8106f40 con 0x7ff6b0074470 2024-09-10T23:11:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.821+0000 7ff6bb02c740 1 Processor -- start 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.821+0000 7ff6bb02c740 1 -- start start 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d485860 0x55cf5d487c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 0x7ff69c0b79b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b990 tx=0x7ff6a000b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d3f76f0 con 0x55cf5d48abd0 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d3e6b20 con 0x55cf5d485860 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d3ea930 con 0x55cf5d488190 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41570/0 (socket says 172.21.15.120:41570) 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 -- 172.21.15.120:0/3425912499 learned_addr learned my addr 172.21.15.120:0/3425912499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 -- 172.21.15.120:0/3425912499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 msgr2=0x55cf5d48a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 -- 172.21.15.120:0/3425912499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d485860 msgr2=0x55cf5d487c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d485860 0x55cf5d487c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.822+0000 7ff6b67fd640 1 -- 172.21.15.120:0/3425912499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5d3ac530 con 0x55cf5d48abd0 2024-09-10T23:11:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.823+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009660 tx=0x7ff6ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e88641a9ebae0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.823+0000 7ff69bfff640 1 -- 172.21.15.120:0/3425912499 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac024070 con 0x55cf5d48abd0 2024-09-10T23:11:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.823+0000 7ff69bfff640 1 -- 172.21.15.120:0/3425912499 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac00bcf0 con 0x55cf5d48abd0 2024-09-10T23:11:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.823+0000 7ff69bfff640 1 -- 172.21.15.120:0/3425912499 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac01fd30 con 0x55cf5d48abd0 2024-09-10T23:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.824+0000 7ff6bb02c740 1 -- 172.21.15.120:0/3425912499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 msgr2=0x55cf5d48d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.824+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009660 tx=0x7ff6ac009b20 comp rx=0 tx=0).stop 2024-09-10T23:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.824+0000 7ff6bb02c740 1 -- 172.21.15.120:0/3425912499 shutdown_connections 2024-09-10T23:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.825+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d48abd0 0x55cf5d48d090 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.825+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.825+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/3425912499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d485860 0x55cf5d487c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.825+0000 7ff6bb02c740 1 -- 172.21.15.120:0/3425912499 >> 172.21.15.120:0/3425912499 conn(0x55cf5d4809b0 msgr2=0x55cf5d481740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.825+0000 7ff6bb02c740 1 -- 172.21.15.120:0/3425912499 shutdown_connections 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.826+0000 7ff6bb02c740 1 -- 172.21.15.120:0/3425912499 wait complete. 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.826+0000 7ff6bb02c740 1 -- start start 2024-09-10T23:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.826+0000 7ff6bb02c740 1 Processor -- start 2024-09-10T23:11:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d48abd0 0x55cf5d4901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d31eab0 con 0x55cf5d485860 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d38c730 con 0x55cf5d48abd0 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d48dbb0 con 0x55cf5d488190 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.827+0000 7ff6bb02c740 10 client.0 _ll_register_callbacks cb 0x55cf5d3e06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:45628/0 (socket says 172.21.15.120:45628) 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1280899191 learned_addr learned my addr 172.21.15.120:0/1280899191 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d48abd0 0x55cf5d4901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1280899191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d48abd0 msgr2=0x55cf5d4901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d48abd0 0x55cf5d4901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1280899191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 msgr2=0x55cf5d48ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1280899191 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5d148790 con 0x55cf5d488190 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.828+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000df40 tx=0x7ff6a0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350ddf2e6f3808a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0022070 con 0x55cf5d488190 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff6a001e830 con 0x55cf5d488190 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac00bcf0 con 0x55cf5d488190 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6a002e050 con 0x55cf5d488190 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.829+0000 7ff6bb02c740 10 client.28600 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.830+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf5d1c1620 con 0x55cf5d488190 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a0016040 con 0x55cf5d488190 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff6a000fbe0 con 0x55cf5d488190 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 10 client.28600 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 10 client.28600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 20 client.28600 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d49cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 4 client.28600 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.832+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff6ac00bcf0 con 0x55cf5d49a930 2024-09-10T23:11:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.833+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d49cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.838+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d49cd40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c715ab0b22cd6ee1 server_cookie=611198bd10408972 in_seq=0 out_seq=0 2024-09-10T23:11:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.838+0000 7ff6997fa640 10 client.28600 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.845+0000 7ff6997fa640 1 -- 172.21.15.120:0/1280899191 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff6ac00bcf0 con 0x55cf5d49a930 2024-09-10T23:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.845+0000 7ff6997fa640 10 client.28600 handle_command_reply: tid=0 2024-09-10T23:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.846+0000 7ff6bb02c740 1 client.28600 shutdown 2024-09-10T23:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.846+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 msgr2=0x55cf5d48f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.846+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000df40 tx=0x7ff6a0004730 comp rx=0 tx=0).stop 2024-09-10T23:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 shutdown_connections 2024-09-10T23:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d49cd40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d48abd0 0x55cf5d4901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6997fa640 0 client.28600 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d488190 0x55cf5d48f620 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1280899191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 >> 172.21.15.120:0/1280899191 conn(0x55cf5d4809b0 msgr2=0x55cf5d47fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 shutdown_connections 2024-09-10T23:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1280899191 wait complete. 2024-09-10T23:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.847+0000 7ff6bb02c740 20 client.28600 trim_cache size 0 max 16384 2024-09-10T23:11:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 Processor -- start 2024-09-10T23:11:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 -- start start 2024-09-10T23:11:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d48afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d48b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400c380 con 0x55cf5d485860 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400ec00 con 0x55cf5d488190 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400c9f0 con 0x55cf5d48abd0 2024-09-10T23:11:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6b5ffc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6b5ffc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41584/0 (socket says 172.21.15.120:41584) 2024-09-10T23:11:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6b5ffc640 1 -- 172.21.15.120:0/722494886 learned_addr learned my addr 172.21.15.120:0/722494886 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.868+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d48b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 -- 172.21.15.120:0/722494886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 msgr2=0x55cf5d48afb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d48afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 -- 172.21.15.120:0/722494886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 msgr2=0x55cf5d48ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 -- 172.21.15.120:0/722494886 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5d48a8f0 con 0x55cf5d48abd0 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff6b67fd640 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d48b4f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55cf5d48fcd0 tx=0x7ff6ac009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f737e0060dcda374 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff69b7fe640 1 -- 172.21.15.120:0/722494886 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac024040 con 0x55cf5d48abd0 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff69b7fe640 1 -- 172.21.15.120:0/722494886 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac0057c0 con 0x55cf5d48abd0 2024-09-10T23:11:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.869+0000 7ff69b7fe640 1 -- 172.21.15.120:0/722494886 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac01fe10 con 0x55cf5d48abd0 2024-09-10T23:11:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.870+0000 7ff6bb02c740 1 -- 172.21.15.120:0/722494886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 msgr2=0x55cf5d48b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.870+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d48b4f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55cf5d48fcd0 tx=0x7ff6ac009e20 comp rx=0 tx=0).stop 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 -- 172.21.15.120:0/722494886 shutdown_connections 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d48b4f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d48afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/722494886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d48ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 -- 172.21.15.120:0/722494886 >> 172.21.15.120:0/722494886 conn(0x55cf5d4809b0 msgr2=0x55cf5d48dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 -- 172.21.15.120:0/722494886 shutdown_connections 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.871+0000 7ff6bb02c740 1 -- 172.21.15.120:0/722494886 wait complete. 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 -- start start 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 Processor -- start 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d496b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d495e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400d0c0 con 0x55cf5d485860 2024-09-10T23:11:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400e530 con 0x55cf5d488190 2024-09-10T23:11:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff69400d790 con 0x55cf5d48abd0 2024-09-10T23:11:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.872+0000 7ff6bb02c740 10 client.0 _ll_register_callbacks cb 0x7ff6a0005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b67fd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d495e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46516/0 (socket says 172.21.15.120:46516) 2024-09-10T23:11:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1211697423 learned_addr learned my addr 172.21.15.120:0/1211697423 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d496b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 msgr2=0x55cf5d495e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d495e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1211697423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 msgr2=0x55cf5d496b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d496b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.873+0000 7ff6b57fb640 1 -- 172.21.15.120:0/1211697423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5d4943a0 con 0x55cf5d488190 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.874+0000 7ff6b57fb640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005bf0 tx=0x7ff6a00267f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9fb03ff35b798d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.874+0000 7ff6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.874+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0036020 con 0x55cf5d488190 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.874+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac0057c0 con 0x55cf5d488190 2024-09-10T23:11:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.875+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff6a0026d20 con 0x55cf5d488190 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.875+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6a0022040 con 0x55cf5d488190 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.876+0000 7ff6bb02c740 10 client.28370 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.876+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf5d507a30 con 0x55cf5d488190 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.876+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac0057c0 con 0x55cf5d488190 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.876+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff6a000f450 con 0x55cf5d488190 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 10 client.28370 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 10 client.28370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 20 client.28370 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d50a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 4 client.28370 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:11:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.877+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf5d3f9640 con 0x55cf5d49a930 2024-09-10T23:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.878+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d50a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:32.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 158910 21.81 MB/sec execute 267 sec latency 152.351 ms 2024-09-10T23:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.880+0000 7ff6b5ffc640 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d50a450 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b885f348623936ec server_cookie=64c8211d0acd1901 in_seq=0 out_seq=0 2024-09-10T23:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:32.880+0000 7ff6997fa640 10 client.28370 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6997fa640 1 -- 172.21.15.120:0/1211697423 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf5d3f9640 con 0x55cf5d49a930 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6997fa640 10 client.28370 handle_command_reply: tid=0 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 client.28370 shutdown 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 msgr2=0x55cf5d495830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005bf0 tx=0x7ff6a00267f0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 shutdown_connections 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55cf5d49a930 0x55cf5d50a450 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cf5d48abd0 0x55cf5d495e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf5d488190 0x55cf5d495830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 --2- 172.21.15.120:0/1211697423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cf5d485860 0x55cf5d496b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 >> 172.21.15.120:0/1211697423 conn(0x55cf5d4809b0 msgr2=0x55cf5d48dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6997fa640 0 client.28370 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 shutdown_connections 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.724+0000 7ff6bb02c740 1 -- 172.21.15.120:0/1211697423 wait complete. 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6bb02c740 20 client.28370 trim_cache size 0 max 16384 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:11:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:11:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:11:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 msgr2=0x7ff69c0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 0x7ff69c0b79b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b990 tx=0x7ff6a000b370 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 msgr2=0x7ff6b019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6a802aec0 tx=0x7ff6a8030800 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 shutdown_connections 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff69c0b54f0 0x7ff69c0b79b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6b010ab10 0x7ff6b01a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6b010a150 0x7ff6b019f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 --2- 172.21.15.120:0/2181719794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6b0074470 0x7ff6b019f340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 >> 172.21.15.120:0/2181719794 conn(0x7ff6b006fa40 msgr2=0x7ff6b010f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 shutdown_connections 2024-09-10T23:11:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:33.725+0000 7ff6b8260640 1 -- 172.21.15.120:0/2181719794 wait complete. 2024-09-10T23:11:33.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159022 21.75 MB/sec execute 268 sec latency 454.541 ms 2024-09-10T23:11:34.497 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:34 smithi120 ceph-mon[31247]: pgmap v920: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 585 op/s 2024-09-10T23:11:34.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:34 smithi176 ceph-mon[34982]: pgmap v920: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 585 op/s 2024-09-10T23:11:34.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:34 smithi153 ceph-mon[34951]: pgmap v920: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 585 op/s 2024-09-10T23:11:34.880 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159507 21.81 MB/sec execute 269 sec latency 64.149 ms 2024-09-10T23:11:35.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 159864 21.80 MB/sec execute 270 sec latency 60.817 ms 2024-09-10T23:11:36.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:36 smithi176 ceph-mon[34982]: pgmap v921: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-09-10T23:11:36.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:36 smithi120 ceph-mon[31247]: pgmap v921: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-09-10T23:11:36.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:36 smithi153 ceph-mon[34951]: pgmap v921: 97 pgs: 97 active+clean; 244 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 316 op/s 2024-09-10T23:11:36.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 160158 21.76 MB/sec execute 271 sec latency 74.737 ms 2024-09-10T23:11:37.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 160444 21.73 MB/sec execute 272 sec latency 241.670 ms 2024-09-10T23:11:38.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:38 smithi176 ceph-mon[34982]: pgmap v922: 97 pgs: 97 active+clean; 238 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 601 op/s 2024-09-10T23:11:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:38 smithi120 ceph-mon[31247]: pgmap v922: 97 pgs: 97 active+clean; 238 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 601 op/s 2024-09-10T23:11:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:38 smithi153 ceph-mon[34951]: pgmap v922: 97 pgs: 97 active+clean; 238 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 601 op/s 2024-09-10T23:11:38.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 160858 21.71 MB/sec execute 273 sec latency 142.669 ms 2024-09-10T23:11:39.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 161483 21.73 MB/sec execute 274 sec latency 43.249 ms 2024-09-10T23:11:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:40 smithi176 ceph-mon[34982]: pgmap v923: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-09-10T23:11:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:40 smithi120 ceph-mon[31247]: pgmap v923: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-09-10T23:11:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:40 smithi153 ceph-mon[34951]: pgmap v923: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 585 op/s 2024-09-10T23:11:40.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 162198 21.81 MB/sec execute 275 sec latency 74.543 ms 2024-09-10T23:11:41.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 162724 21.87 MB/sec execute 276 sec latency 66.585 ms 2024-09-10T23:11:42.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:42 smithi176 ceph-mon[34982]: pgmap v924: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-09-10T23:11:42.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:42 smithi120 ceph-mon[31247]: pgmap v924: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-09-10T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:42 smithi153 ceph-mon[34951]: pgmap v924: 97 pgs: 97 active+clean; 228 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 546 op/s 2024-09-10T23:11:42.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 163114 21.89 MB/sec execute 277 sec latency 162.930 ms 2024-09-10T23:11:43.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:11:43.301 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.831+0000 7f9ef6657640 1 -- 172.21.15.120:0/3500377939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 msgr2=0x7f9ef010f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.831+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3500377939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef010f650 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40074e0 tx=0x7f9ee402b060 comp rx=0 tx=0).stop 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 -- 172.21.15.120:0/3500377939 shutdown_connections 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3500377939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010fb90 0x7f9ef0116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3500377939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef010f650 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3500377939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 -- 172.21.15.120:0/3500377939 >> 172.21.15.120:0/3500377939 conn(0x7f9ef006e2e0 msgr2=0x7f9ef006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.833+0000 7f9ef6657640 1 -- 172.21.15.120:0/3500377939 shutdown_connections 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.834+0000 7f9ef6657640 1 -- 172.21.15.120:0/3500377939 wait complete. 2024-09-10T23:11:43.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.835+0000 7f9ef6657640 1 Processor -- start 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 -- start start 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef0110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010fb90 0x7f9ef0110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0119380 con 0x7f9ef010f210 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0119200 con 0x7f9ef010fb90 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.836+0000 7f9ef6657640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0119500 con 0x7f9ef010a3b0 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37612/0 (socket says 172.21.15.120:37612) 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 -- 172.21.15.120:0/3152557599 learned_addr learned my addr 172.21.15.120:0/3152557599 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef0110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 -- 172.21.15.120:0/3152557599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010fb90 msgr2=0x7f9ef0110a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:11:43.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010fb90 0x7f9ef0110a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 -- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 msgr2=0x7f9ef0110460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef0110460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 -- 172.21.15.120:0/3152557599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef01a5af0 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.837+0000 7f9eeffff640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c890 tx=0x7f9ee000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.838+0000 7f9eed7fa640 1 -- 172.21.15.120:0/3152557599 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0004220 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.838+0000 7f9eed7fa640 1 -- 172.21.15.120:0/3152557599 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee00043c0 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.838+0000 7f9eed7fa640 1 -- 172.21.15.120:0/3152557599 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0004a00 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.838+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01a5de0 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.838+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01a62d0 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9eed7fa640 1 -- 172.21.15.120:0/3152557599 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9ee00079b0 con 0x7f9ef010a3b0 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 Processor -- start 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 -- start start 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.854 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55563a487e50 con 0x55563a5110a0 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55563a477280 con 0x55563a5119c0 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.842+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55563a47b090 con 0x55563a51d950 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.843+0000 7f9ef4bf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.843+0000 7f9ef4bf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55838/0 (socket says 172.21.15.120:55838) 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.843+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/1869981724 learned_addr learned my addr 172.21.15.120:0/1869981724 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.843+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.843+0000 7f9eeffff640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/1869981724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 msgr2=0x55563a51fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/1869981724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 msgr2=0x55563a51d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.855 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/1869981724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563a1d5650 con 0x55563a5110a0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9eeffff640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.844+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8007bd0 tx=0x7f9ee800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510e7c3c57c438bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.845+0000 7f9ed67fc640 1 -- 172.21.15.120:0/1869981724 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8014070 con 0x55563a5110a0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.845+0000 7f9eed7fa640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 0x7f9ed80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.845+0000 7f9ed67fc640 1 -- 172.21.15.120:0/1869981724 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee800fb00 con 0x55563a5110a0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.845+0000 7f9ed67fc640 1 -- 172.21.15.120:0/1869981724 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800e9f0 con 0x55563a5110a0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.845+0000 7f9eed7fa640 1 -- 172.21.15.120:0/3152557599 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9ee00e5920 con 0x7f9ef010a3b0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9eeffff640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 0x7f9ed80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/1869981724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 msgr2=0x55563a511480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8007bd0 tx=0x7f9ee800c6a0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/1869981724 shutdown_connections 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.856 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/1869981724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a511480 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/1869981724 >> 172.21.15.120:0/1869981724 conn(0x55563a50eed0 msgr2=0x55563a510070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9eeffff640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 0x7f9ed80b78d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00e5c00 tx=0x7f9ee0004570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/1869981724 shutdown_connections 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.846+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/1869981724 wait complete. 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 -- start start 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 Processor -- start 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a51d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55563a2d8700 con 0x55563a5110a0 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55563a412110 con 0x55563a5119c0 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55563a5205d0 con 0x55563a51d950 2024-09-10T23:11:43.857 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.847+0000 7f9ef8a2c740 10 client.0 _ll_register_callbacks cb 0x55563a42b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50918/0 (socket says 172.21.15.120:50918) 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 -- 172.21.15.120:0/944020699 learned_addr learned my addr 172.21.15.120:0/944020699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a51d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 -- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 msgr2=0x55563a51ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 -- 172.21.15.120:0/944020699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 msgr2=0x55563a51d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a51d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 -- 172.21.15.120:0/944020699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563a43fc40 con 0x55563a5119c0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40074e0 tx=0x7f9ee400ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc38fa7a30b4031 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.848+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4042070 con 0x55563a5119c0 2024-09-10T23:11:43.858 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee800fb00 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9ee4007da0 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ee4004040 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 10 client.28373 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55563a52bf30 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee800fb00 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9ee4045030 con 0x55563a5119c0 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 10 client.28373 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 10 client.28373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 20 client.28373 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a52d8a0 0x55563a52fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 4 client.28373 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:43.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.849+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55563a489da0 con 0x55563a52d8a0 2024-09-10T23:11:43.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.853+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a52d8a0 0x55563a52fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.861+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a52d8a0 0x55563a52fd00 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a54cc3741053445 server_cookie=ed9074bb97817684 in_seq=0 out_seq=0 2024-09-10T23:11:43.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.861+0000 7f9ecf7fe640 10 client.28373 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:43.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.868+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/944020699 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55563a489da0 con 0x55563a52d8a0 2024-09-10T23:11:43.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.868+0000 7f9ecf7fe640 10 client.28373 handle_command_reply: tid=0 2024-09-10T23:11:43.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.868+0000 7f9ef8a2c740 1 client.28373 shutdown 2024-09-10T23:11:43.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 msgr2=0x55563a51a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40074e0 tx=0x7f9ee400ac70 comp rx=0 tx=0).stop 2024-09-10T23:11:43.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 shutdown_connections 2024-09-10T23:11:43.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a52d8a0 0x55563a52fd00 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a51d950 0x55563a51ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a5119c0 0x55563a51a640 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/944020699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a51d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.870+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 >> 172.21.15.120:0/944020699 conn(0x55563a50eed0 msgr2=0x55563a514a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.871+0000 7f9ecf7fe640 0 client.28373 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:43.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.872+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 shutdown_connections 2024-09-10T23:11:43.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.872+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/944020699 wait complete. 2024-09-10T23:11:43.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.872+0000 7f9ef8a2c740 20 client.28373 trim_cache size 0 max 16384 2024-09-10T23:11:43.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 163423 21.86 MB/sec execute 278 sec latency 97.716 ms 2024-09-10T23:11:43.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.904+0000 7f9ef8a2c740 1 Processor -- start 2024-09-10T23:11:43.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.904+0000 7f9ef8a2c740 1 -- start start 2024-09-10T23:11:43.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a5298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a518ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010a790 con 0x55563a5110a0 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01c61c0 con 0x55563a51d950 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef8a2c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01acf70 con 0x55563a5119c0 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9eef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9eef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37654/0 (socket says 172.21.15.120:37654) 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9eef7fe640 1 -- 172.21.15.120:0/2346331299 learned_addr learned my addr 172.21.15.120:0/2346331299 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.905+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a5298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a518ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 -- 172.21.15.120:0/2346331299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 msgr2=0x55563a518ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a518ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 -- 172.21.15.120:0/2346331299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 msgr2=0x55563a5298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a5298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 -- 172.21.15.120:0/2346331299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563a5117f0 con 0x55563a5119c0 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.906+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4007320 tx=0x7f9ee402b060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8622e746a2acbae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.907+0000 7f9ed5ffb640 1 -- 172.21.15.120:0/2346331299 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4042040 con 0x55563a5119c0 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.907+0000 7f9ed5ffb640 1 -- 172.21.15.120:0/2346331299 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee402be50 con 0x55563a5119c0 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.907+0000 7f9ed5ffb640 1 -- 172.21.15.120:0/2346331299 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4004440 con 0x55563a5119c0 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.908+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/2346331299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 msgr2=0x55563a510000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.908+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4007320 tx=0x7f9ee402b060 comp rx=0 tx=0).stop 2024-09-10T23:11:43.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.910+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/2346331299 shutdown_connections 2024-09-10T23:11:43.912 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.911+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a518ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.911+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5119c0 0x55563a510000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.911+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/2346331299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5110a0 0x55563a5298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.911+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/2346331299 >> 172.21.15.120:0/2346331299 conn(0x55563a50eed0 msgr2=0x55563a5158c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.912+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/2346331299 shutdown_connections 2024-09-10T23:11:43.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.912+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/2346331299 wait complete. 2024-09-10T23:11:43.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.912+0000 7f9ef8a2c740 1 -- start start 2024-09-10T23:11:43.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.913+0000 7f9ef8a2c740 1 Processor -- start 2024-09-10T23:11:43.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.913+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 0x55563a526160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.913+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5119c0 0x55563a535280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef8a2c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a535840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef4bf4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 0x55563a526160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef4bf4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 0x55563a526160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37658/0 (socket says 172.21.15.120:37658) 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/4200479996 learned_addr learned my addr 172.21.15.120:0/4200479996 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01ad640 con 0x55563a5119c0 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01c5af0 con 0x55563a51d950 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01add10 con 0x55563a5110a0 2024-09-10T23:11:43.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.914+0000 7f9ef8a2c740 10 client.0 _ll_register_callbacks cb 0x7f9ee40047a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:43.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.915+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a535840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.915+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5119c0 0x55563a535280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.916+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 msgr2=0x55563a526160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.916+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 0x55563a526160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.916+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/4200479996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5119c0 msgr2=0x55563a535280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.916+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5119c0 0x55563a535280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.916+0000 7f9ef4bf4640 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563a536070 con 0x55563a51d950 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.917+0000 7f9ef4bf4640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a535840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8007af0 tx=0x7f9ee4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea39d9914f48fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.917+0000 7f9ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.917+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8014040 con 0x55563a51d950 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.917+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee402be50 con 0x55563a51d950 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.917+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9ee8029f90 con 0x55563a51d950 2024-09-10T23:11:43.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.918+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ee800e700 con 0x55563a51d950 2024-09-10T23:11:43.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.918+0000 7f9ef8a2c740 10 client.28379 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:43.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.918+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55563a59a900 con 0x55563a51d950 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee402be50 con 0x55563a51d950 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9ee80198a0 con 0x55563a51d950 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 10 client.28379 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 10 client.28379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 20 client.28379 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a59ad70 0x55563a59d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 4 client.28379 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.919+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55563a485850 con 0x55563a59ad70 2024-09-10T23:11:43.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.920+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a59ad70 0x55563a59d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:43.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.926+0000 7f9eef7fe640 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a59ad70 0x55563a59d190 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f0300ed8074e74 server_cookie=f75ce07787b9d8e9 in_seq=0 out_seq=0 2024-09-10T23:11:43.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.926+0000 7f9ecf7fe640 10 client.28379 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:43.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.930+0000 7f9ecf7fe640 1 -- 172.21.15.120:0/4200479996 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9178 (crc 0 0 0) 0x55563a485850 con 0x55563a59ad70 2024-09-10T23:11:43.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.930+0000 7f9ecf7fe640 10 client.28379 handle_command_reply: tid=0 2024-09-10T23:11:43.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.930+0000 7f9ef8a2c740 1 client.28379 shutdown 2024-09-10T23:11:43.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.930+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 msgr2=0x55563a535840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.930+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a535840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8007af0 tx=0x7f9ee4005fd0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 shutdown_connections 2024-09-10T23:11:43.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55563a59ad70 0x55563a59d190 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55563a51d950 0x55563a535840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55563a5119c0 0x55563a535280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 --2- 172.21.15.120:0/4200479996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55563a5110a0 0x55563a526160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 >> 172.21.15.120:0/4200479996 conn(0x55563a50eed0 msgr2=0x55563a5158c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ecf7fe640 0 client.28379 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 shutdown_connections 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 1 -- 172.21.15.120:0/4200479996 wait complete. 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.931+0000 7f9ef8a2c740 20 client.28379 trim_cache size 0 max 16384 2024-09-10T23:11:43.934 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:11:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:11:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:11:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:11:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:11:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:43.953 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:43.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 msgr2=0x7f9ed80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 0x7f9ed80b78d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00e5c00 tx=0x7f9ee0004570 comp rx=0 tx=0).stop 2024-09-10T23:11:43.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 msgr2=0x7f9ef0111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c890 tx=0x7f9ee000cd50 comp rx=0 tx=0).stop 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 shutdown_connections 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.933+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f9ed80b5410 0x7f9ed80b78d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010fb90 0x7f9ef0110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9ef010f210 0x7f9ef0110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 --2- 172.21.15.120:0/3152557599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ef010a3b0 0x7f9ef0111f10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 >> 172.21.15.120:0/3152557599 conn(0x7f9ef006e2e0 msgr2=0x7f9ef0073280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 shutdown_connections 2024-09-10T23:11:43.955 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:43.934+0000 7f9ef6657640 1 -- 172.21.15.120:0/3152557599 wait complete. 2024-09-10T23:11:44.514 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:44 smithi120 ceph-mon[31247]: pgmap v925: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-10T23:11:44.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:44 smithi176 ceph-mon[34982]: pgmap v925: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-10T23:11:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:44 smithi153 ceph-mon[34951]: pgmap v925: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.2 MiB/s wr, 669 op/s 2024-09-10T23:11:44.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 163743 21.82 MB/sec execute 279 sec latency 133.538 ms 2024-09-10T23:11:44.969 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:11:44.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:11:45.239 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:45.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 164131 21.82 MB/sec execute 280 sec latency 94.052 ms 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 -- 172.21.15.120:0/1773171308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d041076a0 msgr2=0x7f2d04107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/1773171308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d041076a0 0x7f2d04107ae0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009900 tx=0x7f2cf002a410 comp rx=0 tx=0).stop 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 -- 172.21.15.120:0/1773171308 shutdown_connections 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/1773171308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d04108020 0x7f2d0410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/1773171308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d041076a0 0x7f2d04107ae0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/1773171308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04101c90 0x7f2d04102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 -- 172.21.15.120:0/1773171308 >> 172.21.15.120:0/1773171308 conn(0x7f2d040fd700 msgr2=0x7f2d040ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:45.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 -- 172.21.15.120:0/1773171308 shutdown_connections 2024-09-10T23:11:45.892 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.890+0000 7f2d0b0df640 1 -- 172.21.15.120:0/1773171308 wait complete. 2024-09-10T23:11:45.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 Processor -- start 2024-09-10T23:11:45.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 -- start start 2024-09-10T23:11:45.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 0x7f2d0419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:45.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:45.893 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04108020 0x7f2d041a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111ce0 con 0x7f2d04101c90 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111b60 con 0x7f2d041076a0 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d0b0df640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04111e60 con 0x7f2d04108020 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d08e7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 0x7f2d0419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50962/0 (socket says 172.21.15.120:50962) 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d03fff640 1 -- 172.21.15.120:0/2050195276 learned_addr learned my addr 172.21.15.120:0/2050195276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.891+0000 7f2d08e7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 0x7f2d0419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55882/0 (socket says 172.21.15.120:55882) 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d0967c640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04108020 0x7f2d041a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 -- 172.21.15.120:0/2050195276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04108020 msgr2=0x7f2d041a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04108020 0x7f2d041a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 -- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 msgr2=0x7f2d0419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 0x7f2d0419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:45.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 -- 172.21.15.120:0/2050195276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d041a1570 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d03fff640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0005940 tx=0x7f2cf00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf003f030 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf00044c0 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d041a1800 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.892+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0035070 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.893+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d041a7f20 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.893+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d04102070 con 0x7f2d041076a0 2024-09-10T23:11:45.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.895+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2cf003ba20 con 0x7f2d041076a0 2024-09-10T23:11:45.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.896+0000 7f2d01ffb640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 0x7f2cf40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:45.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.896+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f2cf01063d0 con 0x7f2d041076a0 2024-09-10T23:11:45.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.896+0000 7f2d08e7b640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 0x7f2cf40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:45.897 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.896+0000 7f2d08e7b640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 0x7f2cf40b7820 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009bd0 tx=0x7f2cf8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:45.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:45.899+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2cf00ccc10 con 0x7f2d041076a0 2024-09-10T23:11:46.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.131+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d041a8200 con 0x7f2d041076a0 2024-09-10T23:11:46.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.132+0000 7f2d01ffb640 1 -- 172.21.15.120:0/2050195276 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f2cf00d1640 con 0x7f2d041076a0 2024-09-10T23:11:46.133 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:11:46.133 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:11:46.133 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 msgr2=0x7f2cf40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 0x7f2cf40b7820 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009bd0 tx=0x7f2cf8005f90 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 msgr2=0x7f2d0419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0005940 tx=0x7f2cf00040c0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 shutdown_connections 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2cf40b5360 0x7f2cf40b7820 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d04108020 0x7f2d041a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d041076a0 0x7f2d0419ba00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 --2- 172.21.15.120:0/2050195276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d04101c90 0x7f2d0419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 >> 172.21.15.120:0/2050195276 conn(0x7f2d040fd700 msgr2=0x7f2d0406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 shutdown_connections 2024-09-10T23:11:46.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:46.135+0000 7f2d0b0df640 1 -- 172.21.15.120:0/2050195276 wait complete. 2024-09-10T23:11:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:46 smithi120 ceph-mon[31247]: pgmap v926: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 539 op/s 2024-09-10T23:11:46.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2050195276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:46.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:46 smithi176 ceph-mon[34982]: pgmap v926: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 539 op/s 2024-09-10T23:11:46.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:46 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2050195276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:46 smithi153 ceph-mon[34951]: pgmap v926: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 539 op/s 2024-09-10T23:11:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:46 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2050195276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:11:46.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:11:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 -- 172.21.15.120:0/876979171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 msgr2=0x7f808c10d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 --2- 172.21.15.120:0/876979171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c10d820 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f807c009930 tx=0x7f807c02a420 comp rx=0 tx=0).stop 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 -- 172.21.15.120:0/876979171 shutdown_connections 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 --2- 172.21.15.120:0/876979171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10eea0 0x7f808c115800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 --2- 172.21.15.120:0/876979171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10e4e0 0x7f808c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 --2- 172.21.15.120:0/876979171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c10d820 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 -- 172.21.15.120:0/876979171 >> 172.21.15.120:0/876979171 conn(0x7f808c06d2e0 msgr2=0x7f808c06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 -- 172.21.15.120:0/876979171 shutdown_connections 2024-09-10T23:11:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.834+0000 7f8091201640 1 -- 172.21.15.120:0/876979171 wait complete. 2024-09-10T23:11:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 Processor -- start 2024-09-10T23:11:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 -- start start 2024-09-10T23:11:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10e4e0 0x7f808c1a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10eea0 0x7f808c1a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f808c119000 con 0x7f808c10d420 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f808c118e80 con 0x7f808c10eea0 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f8091201640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f808c119180 con 0x7f808c10e4e0 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55888/0 (socket says 172.21.15.120:55888) 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f808ad9d640 1 -- 172.21.15.120:0/1037908125 learned_addr learned my addr 172.21.15.120:0/1037908125 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f808a59c640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10e4e0 0x7f808c1a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.835+0000 7f808b59e640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10eea0 0x7f808c1a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 -- 172.21.15.120:0/1037908125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10e4e0 msgr2=0x7f808c1a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10e4e0 0x7f808c1a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 -- 172.21.15.120:0/1037908125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10eea0 msgr2=0x7f808c1a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10eea0 0x7f808c1a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 -- 172.21.15.120:0/1037908125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c1a52d0 con 0x7f808c10d420 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f807c009a50 tx=0x7f807c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f8073fff640 1 -- 172.21.15.120:0/1037908125 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c03f030 con 0x7f808c10d420 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1a5560 con 0x7f808c10d420 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f8073fff640 1 -- 172.21.15.120:0/1037908125 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c003cb0 con 0x7f808c10d420 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f8073fff640 1 -- 172.21.15.120:0/1037908125 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c035070 con 0x7f808c10d420 2024-09-10T23:11:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.836+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c1abc90 con 0x7f808c10d420 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8073fff640 1 -- 172.21.15.120:0/1037908125 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f807c03bb90 con 0x7f808c10d420 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 Processor -- start 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 -- start start 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec44ca46f0 con 0x55ec44d32860 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ec44c93b20 con 0x55ec44d35190 2024-09-10T23:11:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f8093fd3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ec44c97930 con 0x55ec44d37bd0 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50994/0 (socket says 172.21.15.120:50994) 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808b59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55892/0 (socket says 172.21.15.120:55892) 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 -- 172.21.15.120:0/1170819218 learned_addr learned my addr 172.21.15.120:0/1170819218 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808a59c640 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 -- 172.21.15.120:0/1170819218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 msgr2=0x55ec44d3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 -- 172.21.15.120:0/1170819218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 msgr2=0x55ec44d34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808ad9d640 1 -- 172.21.15.120:0/1170819218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec44c59530 con 0x55ec44d35190 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.840+0000 7f808b59e640 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f807c10ca80 tx=0x7f807c10cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22d4d76afbecbb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f8073fff640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 0x7f80740b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f80717fa640 1 -- 172.21.15.120:0/1170819218 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0cc7e0 con 0x55ec44d35190 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f808b59e640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 0x7f80740b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f80717fa640 1 -- 172.21.15.120:0/1170819218 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c116020 con 0x55ec44d35190 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f80717fa640 1 -- 172.21.15.120:0/1170819218 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0cc9c0 con 0x55ec44d35190 2024-09-10T23:11:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.841+0000 7f8073fff640 1 -- 172.21.15.120:0/1037908125 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f807c106880 con 0x7f808c10d420 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- 172.21.15.120:0/1170819218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 msgr2=0x55ec44d37580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f807c10ca80 tx=0x7f807c10cb70 comp rx=0 tx=0).stop 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f808b59e640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 0x7f80740b77a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f808c1a4510 tx=0x7f8084009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- 172.21.15.120:0/1170819218 shutdown_connections 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d37580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1170819218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- 172.21.15.120:0/1170819218 >> 172.21.15.120:0/1170819218 conn(0x55ec44d2d9b0 msgr2=0x55ec44d2e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- 172.21.15.120:0/1170819218 shutdown_connections 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- 172.21.15.120:0/1170819218 wait complete. 2024-09-10T23:11:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.842+0000 7f8093fd3740 1 -- start start 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 Processor -- start 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d37bd0 0x55ec44d3cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec44bcbab0 con 0x55ec44d35190 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ec44c39730 con 0x55ec44d37bd0 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ec44d3a970 con 0x55ec44d32860 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f8093fd3740 10 client.0 _ll_register_callbacks cb 0x55ec44c8d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55906/0 (socket says 172.21.15.120:55906) 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808a59c640 1 -- 172.21.15.120:0/246846839 learned_addr learned my addr 172.21.15.120:0/246846839 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37734/0 (socket says 172.21.15.120:37734) 2024-09-10T23:11:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808b59e640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d37bd0 0x55ec44d3cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 -- 172.21.15.120:0/246846839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d37bd0 msgr2=0x55ec44d3cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d37bd0 0x55ec44d3cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 -- 172.21.15.120:0/246846839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 msgr2=0x55ec44d3c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808ad9d640 1 -- 172.21.15.120:0/246846839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec449f5790 con 0x55ec44d32860 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.843+0000 7f808a59c640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f808ad9d640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f807c03af90 tx=0x7f807c03afc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5414c4d2eee2bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c049040 con 0x55ec44d32860 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807c116020 con 0x55ec44d32860 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f807c004460 con 0x55ec44d32860 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f807c0ccbb0 con 0x55ec44d32860 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 10 client.28639 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec44a6e620 con 0x55ec44d32860 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c116020 con 0x55ec44d32860 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f807c0c92e0 con 0x55ec44d32860 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 10 client.28639 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 10 client.28639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 20 client.28639 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.844+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44d471a0 0x55ec44d49590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.845+0000 7f8093fd3740 4 client.28639 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.845+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec44ca6640 con 0x55ec44d471a0 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.845+0000 7f808a59c640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44d471a0 0x55ec44d49590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.845+0000 7f808a59c640 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44d471a0 0x55ec44d49590 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df04ae9bdccdd02 server_cookie=28d1e32433e7c193 in_seq=0 out_seq=0 2024-09-10T23:11:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.845+0000 7f806eff5640 10 client.28639 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f806eff5640 1 -- 172.21.15.120:0/246846839 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec44ca6640 con 0x55ec44d471a0 2024-09-10T23:11:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f806eff5640 10 client.28639 handle_command_reply: tid=0 2024-09-10T23:11:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 client.28639 shutdown 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 msgr2=0x55ec44d3bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f807c03af90 tx=0x7f807c03afc0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 shutdown_connections 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44d471a0 0x55ec44d49590 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d37bd0 0x55ec44d3cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d3c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f806eff5640 0 client.28639 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 --2- 172.21.15.120:0/246846839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d32860 0x55ec44d3bb10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.850+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 >> 172.21.15.120:0/246846839 conn(0x55ec44d2d9b0 msgr2=0x55ec44d2cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.851+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 shutdown_connections 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.851+0000 7f8093fd3740 1 -- 172.21.15.120:0/246846839 wait complete. 2024-09-10T23:11:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.851+0000 7f8093fd3740 20 client.28639 trim_cache size 0 max 16384 2024-09-10T23:11:46.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 164741 21.82 MB/sec execute 281 sec latency 75.590 ms 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 Processor -- start 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 -- start start 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d32860 0x55ec44d3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b2820 con 0x55ec44d35190 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b50a0 con 0x55ec44d32860 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f8093fd3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b2e90 con 0x55ec44d37bd0 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f808a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f808a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55914/0 (socket says 172.21.15.120:55914) 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f808a59c640 1 -- 172.21.15.120:0/2203711725 learned_addr learned my addr 172.21.15.120:0/2203711725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f808ad9d640 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d32860 0x55ec44d3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.881+0000 7f808b59e640 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 -- 172.21.15.120:0/2203711725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 msgr2=0x55ec44d37fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d37fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 -- 172.21.15.120:0/2203711725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d32860 msgr2=0x55ec44d3acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d32860 0x55ec44d3acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 -- 172.21.15.120:0/2203711725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec44d34fc0 con 0x55ec44d35190 2024-09-10T23:11:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f808a59c640 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f80800105d0 tx=0x7f808000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad21f3eaeba481b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f8070ff9640 1 -- 172.21.15.120:0/2203711725 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080004220 con 0x55ec44d35190 2024-09-10T23:11:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f8070ff9640 1 -- 172.21.15.120:0/2203711725 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80800043c0 con 0x55ec44d35190 2024-09-10T23:11:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.882+0000 7f8070ff9640 1 -- 172.21.15.120:0/2203711725 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000f6e0 con 0x55ec44d35190 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- 172.21.15.120:0/2203711725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 msgr2=0x55ec44d43e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f80800105d0 tx=0x7f808000a2d0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- 172.21.15.120:0/2203711725 shutdown_connections 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d37fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d35190 0x55ec44d43e00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 --2- 172.21.15.120:0/2203711725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d32860 0x55ec44d3acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- 172.21.15.120:0/2203711725 >> 172.21.15.120:0/2203711725 conn(0x55ec44d2d9b0 msgr2=0x55ec44d3b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- 172.21.15.120:0/2203711725 shutdown_connections 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- 172.21.15.120:0/2203711725 wait complete. 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 -- start start 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.883+0000 7f8093fd3740 1 Processor -- start 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d43db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b3560 con 0x55ec44d32860 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b49d0 con 0x55ec44d35190 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1b3c30 con 0x55ec44d37bd0 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 10 client.0 _ll_register_callbacks cb 0x7f80840099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55928/0 (socket says 172.21.15.120:55928) 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808b59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 -- 172.21.15.120:0/1035917808 learned_addr learned my addr 172.21.15.120:0/1035917808 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808a59c640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d43db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 -- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 msgr2=0x55ec44d425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 -- 172.21.15.120:0/1035917808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 msgr2=0x55ec44d43db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d43db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 -- 172.21.15.120:0/1035917808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec44d42dd0 con 0x55ec44d32860 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f808ad9d640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c0372d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43eae6eed64a349 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f806eff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c049040 con 0x55ec44d32860 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80800043c0 con 0x55ec44d32860 2024-09-10T23:11:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.884+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f807c0d6170 con 0x55ec44d32860 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f807c0cce40 con 0x55ec44d32860 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 10 client.19200 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec44d42dd0 con 0x55ec44d32860 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80800043c0 con 0x55ec44d32860 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f807c0cd2d0 con 0x55ec44d32860 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 10 client.19200 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 10 client.19200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 20 client.19200 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44db4270 0x55ec44db6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 4 client.19200 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec44ca20f0 con 0x55ec44db4270 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.885+0000 7f808a59c640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44db4270 0x55ec44db6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.886+0000 7f808a59c640 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44db4270 0x55ec44db6660 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d970a4677ecc489 server_cookie=fcd163e33aa81fba in_seq=0 out_seq=0 2024-09-10T23:11:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:46.886+0000 7f806eff5640 10 client.19200 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.308 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:47.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 164807 21.76 MB/sec execute 282 sec latency 692.600 ms 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f806eff5640 1 -- 172.21.15.120:0/1035917808 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec44ca20f0 con 0x55ec44db4270 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f806eff5640 10 client.19200 handle_command_reply: tid=0 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 client.19200 shutdown 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 msgr2=0x55ec44d43710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c0372d0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 shutdown_connections 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ec44db4270 0x55ec44db6660 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ec44d37bd0 0x55ec44d425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ec44d35190 0x55ec44d43db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 --2- 172.21.15.120:0/1035917808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec44d32860 0x55ec44d43710 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 >> 172.21.15.120:0/1035917808 conn(0x55ec44d2d9b0 msgr2=0x55ec44d3b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.889+0000 7f806eff5640 0 client.19200 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.890+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 shutdown_connections 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.890+0000 7f8093fd3740 1 -- 172.21.15.120:0/1035917808 wait complete. 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.890+0000 7f8093fd3740 20 client.19200 trim_cache size 0 max 16384 2024-09-10T23:11:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:11:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:11:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:11:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 msgr2=0x7f80740b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 0x7f80740b77a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f808c1a4510 tx=0x7f8084009500 comp rx=0 tx=0).stop 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 msgr2=0x7f808c1a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f807c009a50 tx=0x7f807c0042f0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 shutdown_connections 2024-09-10T23:11:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f80740b52e0 0x7f80740b77a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f808c10eea0 0x7f808c1a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f808c10e4e0 0x7f808c1a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 --2- 172.21.15.120:0/1037908125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f808c10d420 0x7f808c1a3790 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 >> 172.21.15.120:0/1037908125 conn(0x7f808c06d2e0 msgr2=0x7f808c10f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.891+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 shutdown_connections 2024-09-10T23:11:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:11:47.892+0000 7f8091201640 1 -- 172.21.15.120:0/1037908125 wait complete. 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 -- 172.21.15.120:0/2641204828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa4108060 msgr2=0x7f5fa410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/2641204828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa4108060 0x7f5fa410e860 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ca0 tx=0x7f5f9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 -- 172.21.15.120:0/2641204828 shutdown_connections 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/2641204828 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa4108060 0x7f5fa410e860 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/2641204828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa41076e0 0x7f5fa4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.909+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/2641204828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4101cd0 0x7f5fa41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.910+0000 7f5faa9f4640 1 -- 172.21.15.120:0/2641204828 >> 172.21.15.120:0/2641204828 conn(0x7f5fa40fd780 msgr2=0x7f5fa40ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.910+0000 7f5faa9f4640 1 -- 172.21.15.120:0/2641204828 shutdown_connections 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.910+0000 7f5faa9f4640 1 -- 172.21.15.120:0/2641204828 wait complete. 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.910+0000 7f5faa9f4640 1 Processor -- start 2024-09-10T23:11:47.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.910+0000 7f5faa9f4640 1 -- start start 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 0x7f5fa4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4108060 0x7f5fa4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa4111d30 con 0x7f5fa41076e0 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa4111bb0 con 0x7f5fa4101cd0 2024-09-10T23:11:47.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5faa9f4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa4111eb0 con 0x7f5fa4108060 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55952/0 (socket says 172.21.15.120:55952) 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 -- 172.21.15.120:0/3147229214 learned_addr learned my addr 172.21.15.120:0/3147229214 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4108060 0x7f5fa4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 0x7f5fa4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 -- 172.21.15.120:0/3147229214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4108060 msgr2=0x7f5fa4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4108060 0x7f5fa4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 -- 172.21.15.120:0/3147229214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 msgr2=0x7f5fa4197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 0x7f5fa4197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 -- 172.21.15.120:0/3147229214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa41a7750 con 0x7f5fa41076e0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 0x7f5fa4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800e9f0 tx=0x7f5f9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.911+0000 7f5fa17fa640 1 -- 172.21.15.120:0/3147229214 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800cd80 con 0x7f5fa41076e0 2024-09-10T23:11:47.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.912+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa41a7a40 con 0x7f5fa41076e0 2024-09-10T23:11:47.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.912+0000 7f5fa17fa640 1 -- 172.21.15.120:0/3147229214 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f980108e0 con 0x7f5fa41076e0 2024-09-10T23:11:47.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.912+0000 7f5fa17fa640 1 -- 172.21.15.120:0/3147229214 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98018980 con 0x7f5fa41076e0 2024-09-10T23:11:47.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.912+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa41a7fd0 con 0x7f5fa41076e0 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 Processor -- start 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 -- start start 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5fa17fa640 1 -- 172.21.15.120:0/3147229214 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5f98018b20 con 0x7f5fa41076e0 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e84e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84edf0 0x55e13e85a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e13e7c4ea0 con 0x55e13e85ad80 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e13e7b42d0 con 0x55e13e84edf0 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5facdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e13e7b80e0 con 0x55e13e84e4d0 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.914+0000 7f5fa8f91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e84e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55956/0 (socket says 172.21.15.120:55956) 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa37fe640 1 -- 172.21.15.120:0/625491295 learned_addr learned my addr 172.21.15.120:0/625491295 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84edf0 0x55e13e85a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 -- 172.21.15.120:0/625491295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84edf0 msgr2=0x55e13e85a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84edf0 0x55e13e85a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 -- 172.21.15.120:0/625491295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 msgr2=0x55e13e85d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 -- 172.21.15.120:0/625491295 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13e512650 con 0x55e13e84e4d0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e84e8b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005e30 tx=0x7f5f9c0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ce2c917869ac61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa17fa640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 0x7f5f8c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/625491295 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c031cf0 con 0x55e13e84e4d0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/625491295 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c036030 con 0x55e13e84e4d0 2024-09-10T23:11:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5f8a7fc640 1 -- 172.21.15.120:0/625491295 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0042a0 con 0x55e13e84e4d0 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.915+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 0x7f5f8c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5fa17fa640 1 -- 172.21.15.120:0/3147229214 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5f980e5e00 con 0x7f5fa41076e0 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 -- 172.21.15.120:0/625491295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 msgr2=0x55e13e84e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 0x7f5f8c0b7820 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55e13e84d1e0 tx=0x7f5f94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e84e8b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005e30 tx=0x7f5f9c0093e0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 -- 172.21.15.120:0/625491295 shutdown_connections 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84edf0 0x55e13e85a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 --2- 172.21.15.120:0/625491295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e84e8b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 -- 172.21.15.120:0/625491295 >> 172.21.15.120:0/625491295 conn(0x55e13e84c300 msgr2=0x55e13e84d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 -- 172.21.15.120:0/625491295 shutdown_connections 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.916+0000 7f5facdc9740 1 -- 172.21.15.120:0/625491295 wait complete. 2024-09-10T23:11:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 -- start start 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 Processor -- start 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e857ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 0x55e13e8580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e13e615700 con 0x55e13e84edf0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e13e74f160 con 0x55e13e84e4d0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e13e85d820 con 0x55e13e85ad80 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5facdc9740 10 client.0 _ll_register_callbacks cb 0x55e13e7689e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e857ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5fa8f91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 0x55e13e8580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51058/0 (socket says 172.21.15.120:51058) 2024-09-10T23:11:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.917+0000 7f5fa3fff640 1 -- 172.21.15.120:0/4150249147 learned_addr learned my addr 172.21.15.120:0/4150249147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 -- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 msgr2=0x55e13e8580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 0x55e13e8580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 -- 172.21.15.120:0/4150249147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 msgr2=0x55e13e857ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e857ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 -- 172.21.15.120:0/4150249147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13e77cc90 con 0x55e13e84e4d0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 0x55e13e8580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e857ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009710 tx=0x7f5f94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ebc7cb4d64d0c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5f837fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400f770 con 0x55e13e84e4d0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c036030 con 0x55e13e84e4d0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5f9400f910 con 0x55e13e84e4d0 2024-09-10T23:11:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f9400fb50 con 0x55e13e84e4d0 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5facdc9740 10 client.28412 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.918+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e13e8691b0 con 0x55e13e84e4d0 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c036030 con 0x55e13e84e4d0 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5f94027070 con 0x55e13e84e4d0 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 10 client.28412 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 10 client.28412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 20 client.28412 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e86a710 0x55e13e86cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 4 client.28412 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e13e7c6df0 con 0x55e13e86a710 2024-09-10T23:11:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e86a710 0x55e13e86cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e86a710 0x55e13e86cb50 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e5d24259094576 server_cookie=c735102dbca1527a in_seq=0 out_seq=0 2024-09-10T23:11:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.919+0000 7f5f837fe640 10 client.28412 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5f837fe640 1 -- 172.21.15.120:0/4150249147 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e13e7c6df0 con 0x55e13e86a710 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5f837fe640 10 client.28412 handle_command_reply: tid=0 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 client.28412 shutdown 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 msgr2=0x55e13e85a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009710 tx=0x7f5f94004730 comp rx=0 tx=0).stop 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 shutdown_connections 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e86a710 0x55e13e86cb50 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e85ad80 0x55e13e8580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e857ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5f837fe640 0 client.28412 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 --2- 172.21.15.120:0/4150249147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e85a670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.920+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 >> 172.21.15.120:0/4150249147 conn(0x55e13e84c300 msgr2=0x55e13e851ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.921+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 shutdown_connections 2024-09-10T23:11:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.921+0000 7f5facdc9740 1 -- 172.21.15.120:0/4150249147 wait complete. 2024-09-10T23:11:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.921+0000 7f5facdc9740 20 client.28412 trim_cache size 0 max 16384 2024-09-10T23:11:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 Processor -- start 2024-09-10T23:11:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 -- start start 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e856fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84edf0 0x55e13e84bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41ae480 con 0x55e13e85ad80 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41b0d00 con 0x55e13e84e4d0 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.934+0000 7f5facdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41aeaf0 con 0x55e13e84edf0 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa8f91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e856fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa8f91640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55972/0 (socket says 172.21.15.120:55972) 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa8f91640 1 -- 172.21.15.120:0/3906083064 learned_addr learned my addr 172.21.15.120:0/3906083064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84edf0 0x55e13e84bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 -- 172.21.15.120:0/3906083064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84edf0 msgr2=0x55e13e84bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84edf0 0x55e13e84bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 -- 172.21.15.120:0/3906083064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 msgr2=0x55e13e85c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 -- 172.21.15.120:0/3906083064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13e85a2a0 con 0x55e13e84e4d0 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e856fb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f94029f20 tx=0x7f5f94004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5024f70e151cecda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5f89ffb640 1 -- 172.21.15.120:0/3906083064 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400fcf0 con 0x55e13e84e4d0 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5f89ffb640 1 -- 172.21.15.120:0/3906083064 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f94038020 con 0x55e13e84e4d0 2024-09-10T23:11:47.937 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.935+0000 7f5f89ffb640 1 -- 172.21.15.120:0/3906083064 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9401fac0 con 0x55e13e84e4d0 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- 172.21.15.120:0/3906083064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 msgr2=0x55e13e856fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e856fb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f94029f20 tx=0x7f5f94004620 comp rx=0 tx=0).stop 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- 172.21.15.120:0/3906083064 shutdown_connections 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e85ad80 0x55e13e85c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84edf0 0x55e13e84bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 --2- 172.21.15.120:0/3906083064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e84e4d0 0x55e13e856fb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- 172.21.15.120:0/3906083064 >> 172.21.15.120:0/3906083064 conn(0x55e13e84c300 msgr2=0x55e13e856340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- 172.21.15.120:0/3906083064 shutdown_connections 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- 172.21.15.120:0/3906083064 wait complete. 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.936+0000 7f5facdc9740 1 -- start start 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 Processor -- start 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e862ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e85ad80 0x55e13e85da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41af1c0 con 0x55e13e84edf0 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41b0630 con 0x55e13e85ad80 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa41af890 con 0x55e13e84e4d0 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5facdc9740 10 client.0 _ll_register_callbacks cb 0x7f5f94009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55982/0 (socket says 172.21.15.120:55982) 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 -- 172.21.15.120:0/1680672357 learned_addr learned my addr 172.21.15.120:0/1680672357 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:47.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e862ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa8f91640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e85ad80 0x55e13e85da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 -- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 msgr2=0x55e13e862ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e862ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 -- 172.21.15.120:0/1680672357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e85ad80 msgr2=0x55e13e85da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e85ad80 0x55e13e85da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.937+0000 7f5fa37fe640 1 -- 172.21.15.120:0/1680672357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13e85dfb0 con 0x55e13e84edf0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5fa37fe640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6a0 tx=0x7f5f980e98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5624e9f1cc2f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f980abe10 con 0x55e13e84edf0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5f98010430 con 0x55e13e84edf0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f94038020 con 0x55e13e84edf0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f980c3020 con 0x55e13e84edf0 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 10 client.19218 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:47.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e13e8d7ea0 con 0x55e13e84edf0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f94038020 con 0x55e13e84edf0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5f980a7ff0 con 0x55e13e84edf0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 10 client.19218 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 10 client.19218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 20 client.19218 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e8d8600 0x55e13e8daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 4 client.19218 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.938+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e13e7c28a0 con 0x55e13e8d8600 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e8d8600 0x55e13e8daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5fa3fff640 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e8d8600 0x55e13e8daa20 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7331053fe1dd02e7 server_cookie=9d7bcfcee1236beb in_seq=0 out_seq=0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5f837fe640 10 client.19218 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5f837fe640 1 -- 172.21.15.120:0/1680672357 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e13e7c28a0 con 0x55e13e8d8600 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5f837fe640 10 client.19218 handle_command_reply: tid=0 2024-09-10T23:11:47.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 client.19218 shutdown 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 msgr2=0x55e13e863000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6a0 tx=0x7f5f980e98e0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 shutdown_connections 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e13e8d8600 0x55e13e8daa20 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e13e85ad80 0x55e13e85da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5f837fe640 0 client.19218 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e13e84edf0 0x55e13e863000 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 --2- 172.21.15.120:0/1680672357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e13e84e4d0 0x55e13e862ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.939+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 >> 172.21.15.120:0/1680672357 conn(0x55e13e84c300 msgr2=0x55e13e8590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 shutdown_connections 2024-09-10T23:11:47.945 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5facdc9740 1 -- 172.21.15.120:0/1680672357 wait complete. 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5facdc9740 20 client.19218 trim_cache size 0 max 16384 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 msgr2=0x7f5f8c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 0x7f5f8c0b7820 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55e13e84d1e0 tx=0x7f5f94005f90 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 msgr2=0x7f5fa4198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800e9f0 tx=0x7f5f9800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 shutdown_connections 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5f8c0b5360 0x7f5f8c0b7820 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fa4108060 0x7f5fa4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fa41076e0 0x7f5fa4198450 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 --2- 172.21.15.120:0/3147229214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa4101cd0 0x7f5fa4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 >> 172.21.15.120:0/3147229214 conn(0x7f5fa40fd780 msgr2=0x7f5fa40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:47.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 shutdown_connections 2024-09-10T23:11:47.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:47.940+0000 7f5faa9f4640 1 -- 172.21.15.120:0/3147229214 wait complete. 2024-09-10T23:11:48.517 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:48 smithi120 ceph-mon[31247]: pgmap v927: 97 pgs: 97 active+clean; 239 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-09-10T23:11:48.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:48 smithi176 ceph-mon[34982]: pgmap v927: 97 pgs: 97 active+clean; 239 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-09-10T23:11:48.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:48 smithi153 ceph-mon[34951]: pgmap v927: 97 pgs: 97 active+clean; 239 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.5 MiB/s wr, 603 op/s 2024-09-10T23:11:48.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 165432 21.80 MB/sec execute 283 sec latency 116.799 ms 2024-09-10T23:11:49.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:11:49.841 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:11:49.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 165939 21.85 MB/sec execute 284 sec latency 81.765 ms 2024-09-10T23:11:50.109 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:50.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:50 smithi120 ceph-mon[31247]: pgmap v928: 97 pgs: 97 active+clean; 243 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-09-10T23:11:50.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:50 smithi176 ceph-mon[34982]: pgmap v928: 97 pgs: 97 active+clean; 243 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-09-10T23:11:50.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:50 smithi153 ceph-mon[34951]: pgmap v928: 97 pgs: 97 active+clean; 243 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 371 op/s 2024-09-10T23:11:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 -- 172.21.15.120:0/800475601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1681021b0 msgr2=0x7fe16810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 --2- 172.21.15.120:0/800475601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1681021b0 0x7fe16810ef00 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe158009900 tx=0x7fe15802a480 comp rx=0 tx=0).stop 2024-09-10T23:11:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 -- 172.21.15.120:0/800475601 shutdown_connections 2024-09-10T23:11:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 --2- 172.21.15.120:0/800475601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16810f440 0x7fe168111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 --2- 172.21.15.120:0/800475601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1681021b0 0x7fe16810ef00 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 --2- 172.21.15.120:0/800475601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe168101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 -- 172.21.15.120:0/800475601 >> 172.21.15.120:0/800475601 conn(0x7fe1680fd700 msgr2=0x7fe1680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 -- 172.21.15.120:0/800475601 shutdown_connections 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.775+0000 7fe16ef58640 1 -- 172.21.15.120:0/800475601 wait complete. 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.776+0000 7fe16ef58640 1 Processor -- start 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.776+0000 7fe16ef58640 1 -- start start 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.776+0000 7fe16ef58640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe16819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.776+0000 7fe16ef58640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 0x7fe16819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.776+0000 7fe16ef58640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16ef58640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe168114f60 con 0x7fe16810f440 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16ef58640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe168114de0 con 0x7fe1681021b0 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16ef58640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681150e0 con 0x7fe168101890 2024-09-10T23:11:50.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16ccf4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe16819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56000/0 (socket says 172.21.15.120:56000) 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 -- 172.21.15.120:0/195870523 learned_addr learned my addr 172.21.15.120:0/195870523 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe15ffff640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 0x7fe16819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 -- 172.21.15.120:0/195870523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 msgr2=0x7fe16819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe16819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 -- 172.21.15.120:0/195870523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 msgr2=0x7fe16819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 0x7fe16819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 -- 172.21.15.120:0/195870523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a15a0 con 0x7fe16810f440 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe15ffff640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 0x7fe16819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe16819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe16d4f5640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ccb0 tx=0x7fe16000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.777+0000 7fe15dffb640 1 -- 172.21.15.120:0/195870523 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160003970 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.778+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a1890 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.778+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a7f50 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.778+0000 7fe15dffb640 1 -- 172.21.15.120:0/195870523 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe160004030 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.778+0000 7fe15dffb640 1 -- 172.21.15.120:0/195870523 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16001d060 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.781+0000 7fe17132d740 1 Processor -- start 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.781+0000 7fe15dffb640 1 -- 172.21.15.120:0/195870523 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe1600044e0 con 0x7fe16810f440 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.781+0000 7fe17132d740 1 -- start start 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d6ec0 0x5613eb3d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3e3770 0x5613eb3e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5613eb34de60 con 0x5613eb3d77e0 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613eb33d290 con 0x5613eb3e3770 2024-09-10T23:11:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe17132d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3410a0 con 0x5613eb3d6ec0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56016/0 (socket says 172.21.15.120:56016) 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3e3770 0x5613eb3e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 -- 172.21.15.120:0/105142252 learned_addr learned my addr 172.21.15.120:0/105142252 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe15ffff640 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d6ec0 0x5613eb3d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe15dffb640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 0x7fe1500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 -- 172.21.15.120:0/105142252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d6ec0 msgr2=0x5613eb3d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d6ec0 0x5613eb3d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.782+0000 7fe16ccf4640 1 -- 172.21.15.120:0/105142252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3e3770 msgr2=0x5613eb3e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3e3770 0x5613eb3e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe16ccf4640 1 -- 172.21.15.120:0/105142252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613eb09b930 con 0x5613eb3d77e0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe16d4f5640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 0x7fe1500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe15dffb640 1 -- 172.21.15.120:0/195870523 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe1600e5c10 con 0x7fe16810f440 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe1640115f0 tx=0x7fe164011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e41a3a755f56df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe14effd640 1 -- 172.21.15.120:0/105142252 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16401e070 con 0x5613eb3d77e0 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe14effd640 1 -- 172.21.15.120:0/105142252 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16400faf0 con 0x5613eb3d77e0 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.783+0000 7fe14effd640 1 -- 172.21.15.120:0/105142252 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164019b30 con 0x5613eb3d77e0 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 -- 172.21.15.120:0/105142252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 msgr2=0x5613eb3e3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe1640115f0 tx=0x7fe164011ab0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe16d4f5640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 0x7fe1500b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe16004b4a0 tx=0x7fe160004880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 -- 172.21.15.120:0/105142252 shutdown_connections 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3e3770 0x5613eb3e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3e3230 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 --2- 172.21.15.120:0/105142252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d6ec0 0x5613eb3d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 -- 172.21.15.120:0/105142252 >> 172.21.15.120:0/105142252 conn(0x5613eb3d4cf0 msgr2=0x5613eb3d5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 -- 172.21.15.120:0/105142252 shutdown_connections 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.784+0000 7fe17132d740 1 -- 172.21.15.120:0/105142252 wait complete. 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 -- start start 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 Processor -- start 2024-09-10T23:11:50.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d77e0 0x5613eb3e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5613eb19e890 con 0x5613eb3d6ec0 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5613eb2d7d40 con 0x5613eb3d77e0 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3e6420 con 0x5613eb3e3770 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.785+0000 7fe17132d740 10 client.0 _ll_register_callbacks cb 0x5613eb33bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56028/0 (socket says 172.21.15.120:56028) 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 -- 172.21.15.120:0/3527710376 learned_addr learned my addr 172.21.15.120:0/3527710376 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d77e0 0x5613eb3e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 -- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 msgr2=0x5613eb3e11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3e11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 -- 172.21.15.120:0/3527710376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d77e0 msgr2=0x5613eb3e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d77e0 0x5613eb3e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 -- 172.21.15.120:0/3527710376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613eb3ef800 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe147fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe15ffff640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe15802a9a0 tx=0x7fe158032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1708be2ea1d91e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.786+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158035040 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16400faf0 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe158004390 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe158052050 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe17132d740 10 client.19230 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613eb3f2200 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.787+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16400faf0 con 0x5613eb3d6ec0 2024-09-10T23:11:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe15803a040 con 0x5613eb3d6ec0 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 10 client.19230 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 10 client.19230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 20 client.19230 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb3f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 4 client.19230 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613eb34fdb0 con 0x5613eb3f3660 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb3f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb3f5a70 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da22b2b64f7e7c7f server_cookie=240ceef88a06e59c in_seq=0 out_seq=0 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.788+0000 7fe147fff640 10 client.19230 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.790+0000 7fe147fff640 1 -- 172.21.15.120:0/3527710376 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613eb34fdb0 con 0x5613eb3f3660 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.790+0000 7fe147fff640 10 client.19230 handle_command_reply: tid=0 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.790+0000 7fe17132d740 1 client.19230 shutdown 2024-09-10T23:11:50.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.790+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 msgr2=0x5613eb3e0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.790+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe15802a9a0 tx=0x7fe158032ba0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 shutdown_connections 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb3f5a70 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3e11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe147fff640 0 client.19230 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d77e0 0x5613eb3e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 --2- 172.21.15.120:0/3527710376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d6ec0 0x5613eb3e0730 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 >> 172.21.15.120:0/3527710376 conn(0x5613eb3d4cf0 msgr2=0x5613eb3d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 shutdown_connections 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.791+0000 7fe17132d740 1 -- 172.21.15.120:0/3527710376 wait complete. 2024-09-10T23:11:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.792+0000 7fe17132d740 20 client.19230 trim_cache size 0 max 16384 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 Processor -- start 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 -- start start 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681ae500 con 0x5613eb3d77e0 2024-09-10T23:11:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b0d80 con 0x5613eb3d6ec0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe17132d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681aeb70 con 0x5613eb3e3770 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe16ccf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe16ccf4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56038/0 (socket says 172.21.15.120:56038) 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.811+0000 7fe16ccf4640 1 -- 172.21.15.120:0/1625712064 learned_addr learned my addr 172.21.15.120:0/1625712064 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe15ffff640 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe15ffff640 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 -- 172.21.15.120:0/1625712064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 msgr2=0x5613eb3fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 -- 172.21.15.120:0/1625712064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 msgr2=0x5613eb3dd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3dd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 -- 172.21.15.120:0/1625712064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613eb3d7610 con 0x5613eb3d77e0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe164019780 tx=0x7fe164005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbc7653f50aa22a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe14e7fc640 1 -- 172.21.15.120:0/1625712064 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164029070 con 0x5613eb3d77e0 2024-09-10T23:11:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe14e7fc640 1 -- 172.21.15.120:0/1625712064 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe164005560 con 0x5613eb3d77e0 2024-09-10T23:11:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.812+0000 7fe14e7fc640 1 -- 172.21.15.120:0/1625712064 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16401e040 con 0x5613eb3d77e0 2024-09-10T23:11:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- 172.21.15.120:0/1625712064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 msgr2=0x5613eb3dde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe164019780 tx=0x7fe164005850 comp rx=0 tx=0).stop 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- 172.21.15.120:0/1625712064 shutdown_connections 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3e3770 0x5613eb3fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3d77e0 0x5613eb3dde80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 --2- 172.21.15.120:0/1625712064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3dd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- 172.21.15.120:0/1625712064 >> 172.21.15.120:0/1625712064 conn(0x5613eb3d4cf0 msgr2=0x5613eb3e2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- 172.21.15.120:0/1625712064 shutdown_connections 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- 172.21.15.120:0/1625712064 wait complete. 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.813+0000 7fe17132d740 1 -- start start 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 Processor -- start 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d77e0 0x5613eb3e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681af240 con 0x5613eb3e3770 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681b06b0 con 0x5613eb3d6ec0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681af910 con 0x5613eb3d77e0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe17132d740 10 client.0 _ll_register_callbacks cb 0x7fe1580316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56040/0 (socket says 172.21.15.120:56040) 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe15ffff640 1 -- 172.21.15.120:0/198005059 learned_addr learned my addr 172.21.15.120:0/198005059 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d77e0 0x5613eb3e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.814+0000 7fe15ffff640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 -- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d77e0 msgr2=0x5613eb3e8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d77e0 0x5613eb3e8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 -- 172.21.15.120:0/198005059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 msgr2=0x5613eb3ebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3ebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 -- 172.21.15.120:0/198005059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613eb3e8d30 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe147fff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe15ffff640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe158001eb0 tx=0x7fe158032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82770a2aa09f0127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158035050 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe158047040 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe164005560 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe158004390 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe17132d740 10 client.19236 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613eb4613b0 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.815+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe164005560 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe158003710 con 0x5613eb3e3770 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 10 client.19236 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 10 client.19236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 20 client.19236 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb463a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 4 client.19236 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5613eb34b860 con 0x5613eb3f3660 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb463a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe147fff640 10 client.19236 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.816+0000 7fe16ccf4640 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb463a80 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ebc659b4aa79c6 server_cookie=645c14c7214185b6 in_seq=0 out_seq=0 2024-09-10T23:11:50.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe147fff640 1 -- 172.21.15.120:0/198005059 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5613eb34b860 con 0x5613eb3f3660 2024-09-10T23:11:50.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe147fff640 10 client.19236 handle_command_reply: tid=0 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 client.19236 shutdown 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 msgr2=0x5613eb3e86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe158001eb0 tx=0x7fe158032ba0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 shutdown_connections 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5613eb3f3660 0x5613eb463a80 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe147fff640 0 client.19236 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5613eb3e3770 0x5613eb3e86c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5613eb3d77e0 0x5613eb3e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 --2- 172.21.15.120:0/198005059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613eb3d6ec0 0x5613eb3ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.838+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 >> 172.21.15.120:0/198005059 conn(0x5613eb3d4cf0 msgr2=0x5613eb3e0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 shutdown_connections 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe17132d740 1 -- 172.21.15.120:0/198005059 wait complete. 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe17132d740 20 client.19236 trim_cache size 0 max 16384 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "7b4feb37-814c-4b4e-8d92-89df57f17c70", 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:11:50.841 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:50.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 msgr2=0x7fe1500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 0x7fe1500b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe16004b4a0 tx=0x7fe160004880 comp rx=0 tx=0).stop 2024-09-10T23:11:50.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 msgr2=0x7fe1681a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:50.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe16000ccb0 tx=0x7fe16000a620 comp rx=0 tx=0).stop 2024-09-10T23:11:50.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 shutdown_connections 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe1500b52e0 0x7fe1500b77a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe16810f440 0x7fe1681a0dd0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1681021b0 0x7fe16819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.839+0000 7fe16ef58640 1 --2- 172.21.15.120:0/195870523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe168101890 0x7fe16819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.840+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 >> 172.21.15.120:0/195870523 conn(0x7fe1680fd700 msgr2=0x7fe168110860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.840+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 shutdown_connections 2024-09-10T23:11:50.843 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:50.840+0000 7fe16ef58640 1 -- 172.21.15.120:0/195870523 wait complete. 2024-09-10T23:11:50.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 166367 21.87 MB/sec execute 285 sec latency 96.187 ms 2024-09-10T23:11:51.563 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:51 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:11:51.563 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:51 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:51.563 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:51 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:11:51.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:51 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:11:51.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:51 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:51.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:51 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:11:51.594 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:11:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:51 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:11:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:51 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:51.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:51 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:11:51.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 166551 21.84 MB/sec execute 286 sec latency 240.399 ms 2024-09-10T23:11:51.945 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:11:52.342 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:52 smithi120 ceph-mon[31247]: pgmap v929: 97 pgs: 97 active+clean; 251 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 424 op/s 2024-09-10T23:11:52.342 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:52 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:52.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.449+0000 7f62f53de640 1 -- 172.21.15.120:0/2973758479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 msgr2=0x7f62f010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.453 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 --2- 172.21.15.120:0/2973758479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f010e840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009be0 tx=0x7f62e802a6c0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 -- 172.21.15.120:0/2973758479 shutdown_connections 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 --2- 172.21.15.120:0/2973758479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f010e840 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 --2- 172.21.15.120:0/2973758479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f01076c0 0x7f62f0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 --2- 172.21.15.120:0/2973758479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f0101cb0 0x7f62f0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 -- 172.21.15.120:0/2973758479 >> 172.21.15.120:0/2973758479 conn(0x7f62f00fd740 msgr2=0x7f62f00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 -- 172.21.15.120:0/2973758479 shutdown_connections 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 -- 172.21.15.120:0/2973758479 wait complete. 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.450+0000 7f62f53de640 1 Processor -- start 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 -- start start 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f0101cb0 0x7f62f019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.454 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0111cc0 con 0x7f62f0108040 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0111b40 con 0x7f62f01076c0 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62f53de640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0111e40 con 0x7f62f0101cb0 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51146/0 (socket says 172.21.15.120:51146) 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 -- 172.21.15.120:0/2359985173 learned_addr learned my addr 172.21.15.120:0/2359985173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62eeffd640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f0101cb0 0x7f62f019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.455 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 -- 172.21.15.120:0/2359985173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f0101cb0 msgr2=0x7f62f019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f0101cb0 0x7f62f019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 -- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 msgr2=0x7f62f01a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f01a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.451+0000 7f62ee7fc640 1 -- 172.21.15.120:0/2359985173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f01a14c0 con 0x7f62f01076c0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9f0 tx=0x7f62e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62d7fff640 1 -- 172.21.15.120:0/2359985173 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400cd80 con 0x7f62f01076c0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62d7fff640 1 -- 172.21.15.120:0/2359985173 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e40108e0 con 0x7f62f01076c0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f01a17b0 con 0x7f62f01076c0 2024-09-10T23:11:52.456 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62d7fff640 1 -- 172.21.15.120:0/2359985173 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018980 con 0x7f62f01076c0 2024-09-10T23:11:52.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.452+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a7e70 con 0x7f62f01076c0 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.455+0000 7f62d7fff640 1 -- 172.21.15.120:0/2359985173 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f62e4010430 con 0x7f62f01076c0 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.455+0000 7f62f77b3740 1 Processor -- start 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.455+0000 7f62f77b3740 1 -- start start 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.455+0000 7f62d7fff640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 0x7f62d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62d7fff640 1 -- 172.21.15.120:0/2359985173 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f62e40e5f00 con 0x7f62f01076c0 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5484410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5486d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53eee50 con 0x55e5c5484950 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 0x7f62d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53de280 con 0x55e5c54780a0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62f77b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53e2090 con 0x55e5c54789c0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51162/0 (socket says 172.21.15.120:51162) 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 -- 172.21.15.120:0/744994152 learned_addr learned my addr 172.21.15.120:0/744994152 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5484410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5486d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 -- 172.21.15.120:0/744994152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 msgr2=0x55e5c5484410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5484410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 -- 172.21.15.120:0/744994152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 msgr2=0x55e5c5486d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5486d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 -- 172.21.15.120:0/744994152 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c513c650 con 0x55e5c54780a0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 0x7f62d80b77a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009370 tx=0x7f62e8030700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.456+0000 7f62eeffd640 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009cc0 tx=0x7f62e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751f8b8aff883790 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.457+0000 7f62d57fa640 1 -- 172.21.15.120:0/744994152 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0005220 con 0x55e5c54780a0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.457+0000 7f62d57fa640 1 -- 172.21.15.120:0/744994152 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e0004c50 con 0x55e5c54780a0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.457+0000 7f62d57fa640 1 -- 172.21.15.120:0/744994152 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0019720 con 0x55e5c54780a0 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- 172.21.15.120:0/744994152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 msgr2=0x55e5c5478480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009cc0 tx=0x7f62e0011810 comp rx=0 tx=0).stop 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- 172.21.15.120:0/744994152 shutdown_connections 2024-09-10T23:11:52.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5486d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5484410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 --2- 172.21.15.120:0/744994152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5478480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- 172.21.15.120:0/744994152 >> 172.21.15.120:0/744994152 conn(0x55e5c5475ed0 msgr2=0x55e5c5477070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- 172.21.15.120:0/744994152 shutdown_connections 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- 172.21.15.120:0/744994152 wait complete. 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 -- start start 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.458+0000 7f62f77b3740 1 Processor -- start 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5481650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5481c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5c523f700 con 0x55e5c5484950 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e5c5379110 con 0x55e5c54780a0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5c54875d0 con 0x55e5c54789c0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62f77b3740 10 client.0 _ll_register_callbacks cb 0x55e5c5392990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51164/0 (socket says 172.21.15.120:51164) 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62eeffd640 1 -- 172.21.15.120:0/3601117583 learned_addr learned my addr 172.21.15.120:0/3601117583 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5481c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5481650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 -- 172.21.15.120:0/3601117583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 msgr2=0x55e5c5484200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 -- 172.21.15.120:0/3601117583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 msgr2=0x55e5c5481c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5481c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ee7fc640 1 -- 172.21.15.120:0/3601117583 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c53a6c40 con 0x55e5c54789c0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62eeffd640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.459+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5481c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5481650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9c0 tx=0x7f62e40ac310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500bbc3824b668e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40abba0 con 0x55e5c54789c0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e0004c50 con 0x55e5c54789c0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f62e4018b20 con 0x55e5c54789c0 2024-09-10T23:11:52.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62e40b5c20 con 0x55e5c54789c0 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62f77b3740 10 client.28693 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5c5492ec0 con 0x55e5c54789c0 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e0004c50 con 0x55e5c54789c0 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.460+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f62e40ab510 con 0x55e5c54789c0 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 10 client.28693 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 10 client.28693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 20 client.28693 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c5496770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 4 client.28693 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c53f0da0 con 0x55e5c5494320 2024-09-10T23:11:52.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.461+0000 7f62eeffd640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c5496770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.462+0000 7f62d2ff5640 10 client.28693 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.462+0000 7f62eeffd640 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c5496770 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9142c477a7d85196 server_cookie=a7e4938b7f95c23b in_seq=0 out_seq=0 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62d2ff5640 1 -- 172.21.15.120:0/3601117583 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5c53f0da0 con 0x55e5c5494320 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62d2ff5640 10 client.28693 handle_command_reply: tid=0 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 client.28693 shutdown 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 msgr2=0x55e5c5481650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5481650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9c0 tx=0x7f62e40ac310 comp rx=0 tx=0).stop 2024-09-10T23:11:52.470 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 shutdown_connections 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c5496770 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c5484950 0x55e5c5481c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62d2ff5640 0 client.28693 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c54789c0 0x55e5c5481650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 --2- 172.21.15.120:0/3601117583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54780a0 0x55e5c5484200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.465+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 >> 172.21.15.120:0/3601117583 conn(0x55e5c5475ed0 msgr2=0x55e5c547ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.466+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 shutdown_connections 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.466+0000 7f62f77b3740 1 -- 172.21.15.120:0/3601117583 wait complete. 2024-09-10T23:11:52.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.466+0000 7f62f77b3740 20 client.28693 trim_cache size 0 max 16384 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 Processor -- start 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 -- start start 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5475710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5483390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01ae530 con 0x55e5c54780a0 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b0db0 con 0x55e5c54789c0 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62f77b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01aeba0 con 0x55e5c5484950 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62eeffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56098/0 (socket says 172.21.15.120:56098) 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62eeffd640 1 -- 172.21.15.120:0/2739495624 learned_addr learned my addr 172.21.15.120:0/2739495624 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.487+0000 7f62ef7fe640 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5483390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5475710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 -- 172.21.15.120:0/2739495624 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 msgr2=0x55e5c5483390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5483390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 -- 172.21.15.120:0/2739495624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 msgr2=0x55e5c5475710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5475710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 -- 172.21.15.120:0/2739495624 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c5475c50 con 0x55e5c54780a0 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62eeffd640 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009cc0 tx=0x7f62e0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462aa07a9872617b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62d4ff9640 1 -- 172.21.15.120:0/2739495624 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0031030 con 0x55e5c54780a0 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62d4ff9640 1 -- 172.21.15.120:0/2739495624 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e002d890 con 0x55e5c54780a0 2024-09-10T23:11:52.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.488+0000 7f62d4ff9640 1 -- 172.21.15.120:0/2739495624 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0019670 con 0x55e5c54780a0 2024-09-10T23:11:52.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 -- 172.21.15.120:0/2739495624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 msgr2=0x55e5c5477600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009cc0 tx=0x7f62e0009510 comp rx=0 tx=0).stop 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 -- 172.21.15.120:0/2739495624 shutdown_connections 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5483390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5475710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 --2- 172.21.15.120:0/2739495624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5477600 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 -- 172.21.15.120:0/2739495624 >> 172.21.15.120:0/2739495624 conn(0x55e5c5475ed0 msgr2=0x55e5c5482c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 -- 172.21.15.120:0/2739495624 shutdown_connections 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.489+0000 7f62f77b3740 1 -- 172.21.15.120:0/2739495624 wait complete. 2024-09-10T23:11:52.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 -- start start 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 Processor -- start 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5488b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5500240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01af270 con 0x55e5c54780a0 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b06e0 con 0x55e5c54789c0 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01af940 con 0x55e5c5484950 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62f77b3740 10 client.0 _ll_register_callbacks cb 0x7f62e40a8670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5500240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51186/0 (socket says 172.21.15.120:51186) 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5500240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37926/0 (socket says 172.21.15.120:37926) 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 -- 172.21.15.120:0/671614896 learned_addr learned my addr 172.21.15.120:0/671614896 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62eeffd640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5488b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 -- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 msgr2=0x55e5c5500240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5500240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 -- 172.21.15.120:0/671614896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 msgr2=0x55e5c5488b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.490+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5488b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62ee7fc640 1 -- 172.21.15.120:0/671614896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c5500780 con 0x55e5c54789c0 2024-09-10T23:11:52.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62eeffd640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5488b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62ee7fc640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c6a0 tx=0x7f62e40abeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80edf7350a88278 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40a8d90 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f62e40a8090 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e002d890 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62e40ac090 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62f77b3740 10 client.28451 fetch_fsmap learned FSMap version 26 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5c55018e0 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e002d890 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f62e40ad9c0 con 0x55e5c54789c0 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.491+0000 7f62f77b3740 10 client.28451 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:11:52.498 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62f77b3740 10 client.28451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62f77b3740 20 client.28451 populate_metadata read hostname 'smithi120' 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c55040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62f77b3740 4 client.28451 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5c53ec850 con 0x55e5c5494320 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62eeffd640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c55040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62d2ff5640 10 client.28451 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:52.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.492+0000 7f62eeffd640 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c55040d0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf9d7902f8b15df server_cookie=dd78a7f0c2d45254 in_seq=0 out_seq=0 2024-09-10T23:11:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62d2ff5640 1 -- 172.21.15.120:0/671614896 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9666 (crc 0 0 0) 0x55e5c53ec850 con 0x55e5c5494320 2024-09-10T23:11:52.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62d2ff5640 10 client.28451 handle_command_reply: tid=0 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 client.28451 shutdown 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 msgr2=0x55e5c5489040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c6a0 tx=0x7f62e40abeb0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 shutdown_connections 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5c5494320 0x55e5c55040d0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5c5484950 0x55e5c5500240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5c54789c0 0x55e5c5489040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62d2ff5640 0 client.28451 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 --2- 172.21.15.120:0/671614896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5c54780a0 0x55e5c5488b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.495+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 >> 172.21.15.120:0/671614896 conn(0x55e5c5475ed0 msgr2=0x55e5c54834a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 shutdown_connections 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f77b3740 1 -- 172.21.15.120:0/671614896 wait complete. 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f77b3740 20 client.28451 trim_cache size 0 max 16384 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (152 inodes in the stack)", 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:11:52.502 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "7b4feb37-814c-4b4e-8d92-89df57f17c70", 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.503 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:11:52.504 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.505 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:11:52.506 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:11:52.507 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.508 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:11:52.509 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.510 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.512 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:11:52.514 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.515 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:11:52.517 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.519 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.523 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:11:52.527 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 msgr2=0x7f62d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 0x7f62d80b77a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009370 tx=0x7f62e8030700 comp rx=0 tx=0).stop 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 msgr2=0x7f62f019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9f0 tx=0x7f62e400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 shutdown_connections 2024-09-10T23:11:52.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f62d80b52e0 0x7f62d80b77a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62f0108040 0x7f62f01a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62f01076c0 0x7f62f019b950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 --2- 172.21.15.120:0/2359985173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f62f0101cb0 0x7f62f019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.496+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 >> 172.21.15.120:0/2359985173 conn(0x7f62f00fd740 msgr2=0x7f62f006b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.497+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 shutdown_connections 2024-09-10T23:11:52.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:11:52.497+0000 7f62f53de640 1 -- 172.21.15.120:0/2359985173 wait complete. 2024-09-10T23:11:52.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:52 smithi176 ceph-mon[34982]: pgmap v929: 97 pgs: 97 active+clean; 251 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 424 op/s 2024-09-10T23:11:52.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:52 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:52.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:52 smithi153 ceph-mon[34951]: pgmap v929: 97 pgs: 97 active+clean; 251 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 424 op/s 2024-09-10T23:11:52.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:52 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:11:52.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 166566 21.77 MB/sec execute 287 sec latency 437.122 ms 2024-09-10T23:11:53.237 INFO:tasks.cephfs.filesystem:scrub status for tag:7b4feb37-814c-4b4e-8d92-89df57f17c70 - {'path': '/', 'tag': '7b4feb37-814c-4b4e-8d92-89df57f17c70', 'options': 'recursive,force'} 2024-09-10T23:11:53.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:53.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:53.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:11:53.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:53 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:11:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:53 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:11:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:11:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:11:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:53 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:11:53.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 166818 21.74 MB/sec execute 288 sec latency 512.084 ms 2024-09-10T23:11:54.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:54 smithi176 ceph-mon[34982]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-09-10T23:11:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:54 smithi120 ceph-mon[31247]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-09-10T23:11:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:54 smithi153 ceph-mon[34951]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-09-10T23:11:54.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 167158 21.71 MB/sec execute 289 sec latency 80.309 ms 2024-09-10T23:11:55.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:11:55.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:11:55.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:11:55.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 167542 21.70 MB/sec execute 290 sec latency 90.838 ms 2024-09-10T23:11:56.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:56 smithi176 ceph-mon[34982]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T23:11:56.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:56 smithi120 ceph-mon[31247]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T23:11:56.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:56 smithi153 ceph-mon[34951]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-09-10T23:11:56.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 168106 21.71 MB/sec execute 291 sec latency 75.665 ms 2024-09-10T23:11:57.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 168576 21.72 MB/sec execute 292 sec latency 268.397 ms 2024-09-10T23:11:58.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:11:58 smithi176 ceph-mon[34982]: pgmap v932: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T23:11:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:11:58 smithi120 ceph-mon[31247]: pgmap v932: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T23:11:58.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:11:58 smithi153 ceph-mon[34951]: pgmap v932: 97 pgs: 97 active+clean; 237 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-10T23:11:58.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 169177 21.76 MB/sec execute 293 sec latency 60.464 ms 2024-09-10T23:11:59.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 169718 21.82 MB/sec execute 294 sec latency 42.429 ms 2024-09-10T23:12:00.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:00 smithi176 ceph-mon[34982]: pgmap v933: 97 pgs: 97 active+clean; 241 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 670 op/s 2024-09-10T23:12:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:00 smithi120 ceph-mon[31247]: pgmap v933: 97 pgs: 97 active+clean; 241 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 670 op/s 2024-09-10T23:12:00.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:00 smithi153 ceph-mon[34951]: pgmap v933: 97 pgs: 97 active+clean; 241 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.5 MiB/s wr, 670 op/s 2024-09-10T23:12:00.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170137 21.84 MB/sec execute 295 sec latency 56.266 ms 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 -- 172.21.15.120:0/3283548397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 msgr2=0x7efce4109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3283548397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce4109d00 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009cd0 tx=0x7efcdc02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 -- 172.21.15.120:0/3283548397 shutdown_connections 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3283548397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce4109d00 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3283548397 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 0x7efce4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3283548397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 -- 172.21.15.120:0/3283548397 >> 172.21.15.120:0/3283548397 conn(0x7efce40fd130 msgr2=0x7efce40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 -- 172.21.15.120:0/3283548397 shutdown_connections 2024-09-10T23:12:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.987+0000 7efcea4fe640 1 -- 172.21.15.120:0/3283548397 wait complete. 2024-09-10T23:12:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 Processor -- start 2024-09-10T23:12:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 -- start start 2024-09-10T23:12:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 0x7efce419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce41a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efce410d4e0 con 0x7efce4103380 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efce410d360 con 0x7efce4101870 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efcea4fe640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce410d660 con 0x7efce41029c0 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce41a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47066/0 (socket says 172.21.15.120:47066) 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efce37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 0x7efce419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.988+0000 7efce3fff640 1 -- 172.21.15.120:0/3823691200 learned_addr learned my addr 172.21.15.120:0/3823691200 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 -- 172.21.15.120:0/3823691200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 msgr2=0x7efce419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 0x7efce419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 -- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 msgr2=0x7efce41a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce41a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 -- 172.21.15.120:0/3823691200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce41a0960 con 0x7efce4101870 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce3fff640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efcd400e9f0 tx=0x7efcd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce17fa640 1 -- 172.21.15.120:0/3823691200 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd400cd80 con 0x7efce4101870 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce17fa640 1 -- 172.21.15.120:0/3823691200 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd40108e0 con 0x7efce4101870 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce41a0bf0 con 0x7efce4101870 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efce17fa640 1 -- 172.21.15.120:0/3823691200 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4018980 con 0x7efce4101870 2024-09-10T23:12:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.989+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41a1140 con 0x7efce4101870 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce17fa640 1 -- 172.21.15.120:0/3823691200 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efcd4010430 con 0x7efce4101870 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 Processor -- start 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 -- start start 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f63b8c6f0 con 0x562f63c215a0 2024-09-10T23:12:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f63b7bb20 con 0x562f63c1ec70 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efced2d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562f63b7f930 con 0x562f63c23fe0 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47068/0 (socket says 172.21.15.120:47068) 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54870/0 (socket says 172.21.15.120:54870) 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.992+0000 7efce3fff640 1 -- 172.21.15.120:0/1290763278 learned_addr learned my addr 172.21.15.120:0/1290763278 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 -- 172.21.15.120:0/1290763278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 msgr2=0x562f63c264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce17fa640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 0x7efccc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 -- 172.21.15.120:0/1290763278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 msgr2=0x562f63c21060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 -- 172.21.15.120:0/1290763278 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f63b41530 con 0x562f63c215a0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce8a9b640 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce8a9b640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 0x7efccc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce3fff640 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b8450 tx=0x7efcd40e9f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b51e7e0cf9147d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efce17fa640 1 -- 172.21.15.120:0/3823691200 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efcd40e5ef0 con 0x7efce4101870 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efcca7fc640 1 -- 172.21.15.120:0/1290763278 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40b5e60 con 0x562f63c215a0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efcca7fc640 1 -- 172.21.15.120:0/1290763278 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd40b7ea0 con 0x562f63c215a0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.993+0000 7efcca7fc640 1 -- 172.21.15.120:0/1290763278 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40abea0 con 0x562f63c215a0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- 172.21.15.120:0/1290763278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 msgr2=0x562f63c23990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efce8a9b640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 0x7efccc0b77a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005e30 tx=0x7efcdc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b8450 tx=0x7efcd40e9f60 comp rx=0 tx=0).stop 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- 172.21.15.120:0/1290763278 shutdown_connections 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c23990 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 --2- 172.21.15.120:0/1290763278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c21060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- 172.21.15.120:0/1290763278 >> 172.21.15.120:0/1290763278 conn(0x562f63c159b0 msgr2=0x562f63c16740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- 172.21.15.120:0/1290763278 shutdown_connections 2024-09-10T23:12:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- 172.21.15.120:0/1290763278 wait complete. 2024-09-10T23:12:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.994+0000 7efced2d3740 1 -- start start 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 Processor -- start 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c28940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f63ab3ab0 con 0x562f63c23fe0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x562f63b21730 con 0x562f63c1ec70 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562f63c26fc0 con 0x562f63c215a0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efced2d3740 10 client.0 _ll_register_callbacks cb 0x562f63b756b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c28940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:47082/0 (socket says 172.21.15.120:47082) 2024-09-10T23:12:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c29510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54882/0 (socket says 172.21.15.120:54882) 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 -- 172.21.15.120:0/1659319037 learned_addr learned my addr 172.21.15.120:0/1659319037 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 -- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 msgr2=0x562f63c28940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c28940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 -- 172.21.15.120:0/1659319037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 msgr2=0x562f63c29510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c29510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.995+0000 7efce3fff640 1 -- 172.21.15.120:0/1659319037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f638dd790 con 0x562f63c1ec70 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efce3fff640 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efcd40a8010 tx=0x7efcd40e9460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c7e975ce40eb38a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efcc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40b5e60 con 0x562f63c1ec70 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efcd40a89a0 con 0x562f63c1ec70 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f63b8e640 con 0x562f63c1ec70 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcd40b8090 con 0x562f63c1ec70 2024-09-10T23:12:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efced2d3740 10 client.28463 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f63956620 con 0x562f63c1ec70 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.996+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f63b8e640 con 0x562f63c1ec70 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efcd40c1d30 con 0x562f63c1ec70 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 10 client.28463 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 10 client.28463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 20 client.28463 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63c33740 0x562f63c35b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 4 client.28463 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f63b8a0f0 con 0x562f63c33740 2024-09-10T23:12:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.997+0000 7efce37fe640 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63c33740 0x562f63c35b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.998+0000 7efce37fe640 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63c33740 0x562f63c35b30 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368a3d568159b9bf server_cookie=ab6a43d5ed70dae1 in_seq=0 out_seq=0 2024-09-10T23:12:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:00.998+0000 7efcc37fe640 10 client.28463 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efcc37fe640 1 -- 172.21.15.120:0/1659319037 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f63b8a0f0 con 0x562f63c33740 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efcc37fe640 10 client.28463 handle_command_reply: tid=0 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 client.28463 shutdown 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 msgr2=0x562f63c282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efcd40a8010 tx=0x7efcd40e9460 comp rx=0 tx=0).stop 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 shutdown_connections 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63c33740 0x562f63c35b30 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c29510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c28940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 --2- 172.21.15.120:0/1659319037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c282a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efcc37fe640 0 client.28463 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 >> 172.21.15.120:0/1659319037 conn(0x562f63c159b0 msgr2=0x562f63c14bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 shutdown_connections 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 1 -- 172.21.15.120:0/1659319037 wait complete. 2024-09-10T23:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.002+0000 7efced2d3740 20 client.28463 trim_cache size 0 max 16384 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 Processor -- start 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 -- start start 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c24900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a5c70 con 0x562f63c23fe0 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a84f0 con 0x562f63c1ec70 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efced2d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a62e0 con 0x562f63c215a0 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54896/0 (socket says 172.21.15.120:54896) 2024-09-10T23:12:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 -- 172.21.15.120:0/4291236594 learned_addr learned my addr 172.21.15.120:0/4291236594 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c24900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 -- 172.21.15.120:0/4291236594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 msgr2=0x562f63c24900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c24900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 -- 172.21.15.120:0/4291236594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 msgr2=0x562f63c243c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c243c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce8a9b640 1 -- 172.21.15.120:0/4291236594 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f63c213d0 con 0x562f63c23fe0 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.017+0000 7efce3fff640 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.018+0000 7efce8a9b640 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7efcdc035000 tx=0x7efcdc03e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b3316126129479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.018+0000 7efcc9ffb640 1 -- 172.21.15.120:0/4291236594 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0043c0 con 0x562f63c23fe0 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.018+0000 7efcc9ffb640 1 -- 172.21.15.120:0/4291236594 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc036040 con 0x562f63c23fe0 2024-09-10T23:12:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.018+0000 7efcc9ffb640 1 -- 172.21.15.120:0/4291236594 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc046660 con 0x562f63c23fe0 2024-09-10T23:12:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- 172.21.15.120:0/4291236594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 msgr2=0x562f63c143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7efcdc035000 tx=0x7efcdc03e030 comp rx=0 tx=0).stop 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- 172.21.15.120:0/4291236594 shutdown_connections 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c23fe0 0x562f63c143d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c215a0 0x562f63c24900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 --2- 172.21.15.120:0/4291236594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- 172.21.15.120:0/4291236594 >> 172.21.15.120:0/4291236594 conn(0x562f63c159b0 msgr2=0x562f63c19c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- 172.21.15.120:0/4291236594 shutdown_connections 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- 172.21.15.120:0/4291236594 wait complete. 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 -- start start 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.019+0000 7efced2d3740 1 Processor -- start 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c2e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a69b0 con 0x562f63c215a0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a7e20 con 0x562f63c1ec70 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efce41a7080 con 0x562f63c23fe0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efced2d3740 10 client.0 _ll_register_callbacks cb 0x7efcd8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:54904/0 (socket says 172.21.15.120:54904) 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce37fe640 1 -- 172.21.15.120:0/289484741 learned_addr learned my addr 172.21.15.120:0/289484741 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce8a9b640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c2e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 -- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 msgr2=0x562f63c2e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c2e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 -- 172.21.15.120:0/289484741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 msgr2=0x562f63c2d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 -- 172.21.15.120:0/289484741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f63c2eb30 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce37fe640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce8a9b640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efce3fff640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c2d090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b1320 tx=0x7efcd4022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b7707c059745f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.020+0000 7efcc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40b5e60 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efcd40acb90 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcdc036040 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcd40ac500 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 10 client.28469 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f63c2eb30 con 0x562f63c1ec70 2024-09-10T23:12:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc036040 con 0x562f63c1ec70 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efcd40b5780 con 0x562f63c1ec70 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 10 client.28469 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 10 client.28469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 20 client.28469 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63ca04e0 0x562f63ca28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 4 client.28469 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f63b87520 con 0x562f63ca04e0 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.021+0000 7efce37fe640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63ca04e0 0x562f63ca28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.022+0000 7efce37fe640 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63ca04e0 0x562f63ca28d0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8c1e2c49f3f9df server_cookie=b505a80f38a549db in_seq=0 out_seq=0 2024-09-10T23:12:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.022+0000 7efcc37fe640 10 client.28469 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efcc37fe640 1 -- 172.21.15.120:0/289484741 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f63b87520 con 0x562f63ca04e0 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efcc37fe640 10 client.28469 handle_command_reply: tid=0 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 client.28469 shutdown 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 msgr2=0x562f63c2d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c2d090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b1320 tx=0x7efcd4022000 comp rx=0 tx=0).stop 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 shutdown_connections 2024-09-10T23:12:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x562f63ca04e0 0x562f63ca28d0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562f63c23fe0 0x562f63c2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f63c215a0 0x562f63c2d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 --2- 172.21.15.120:0/289484741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f63c1ec70 0x562f63c2d090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 >> 172.21.15.120:0/289484741 conn(0x562f63c159b0 msgr2=0x562f63c143d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efcc37fe640 0 client.28469 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 shutdown_connections 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 1 -- 172.21.15.120:0/289484741 wait complete. 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.739+0000 7efced2d3740 20 client.28469 trim_cache size 0 max 16384 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:12:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:12:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:12:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:12:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 msgr2=0x7efccc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 0x7efccc0b77a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005e30 tx=0x7efcdc0095c0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 msgr2=0x7efce406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efcd400e9f0 tx=0x7efcd400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 shutdown_connections 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efccc0b52e0 0x7efccc0b77a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efce4103380 0x7efce41a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efce41029c0 0x7efce419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 --2- 172.21.15.120:0/3823691200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efce4101870 0x7efce406b9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 >> 172.21.15.120:0/3823691200 conn(0x7efce40fd130 msgr2=0x7efce4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 shutdown_connections 2024-09-10T23:12:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:01.740+0000 7efcea4fe640 1 -- 172.21.15.120:0/3823691200 wait complete. 2024-09-10T23:12:01.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170254 21.79 MB/sec execute 296 sec latency 411.372 ms 2024-09-10T23:12:02.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:02 smithi176 ceph-mon[34982]: pgmap v934: 97 pgs: 97 active+clean; 246 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 678 op/s 2024-09-10T23:12:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:02 smithi120 ceph-mon[31247]: pgmap v934: 97 pgs: 97 active+clean; 246 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 678 op/s 2024-09-10T23:12:02.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:02 smithi153 ceph-mon[34951]: pgmap v934: 97 pgs: 97 active+clean; 246 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 678 op/s 2024-09-10T23:12:02.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170471 21.75 MB/sec execute 297 sec latency 202.063 ms 2024-09-10T23:12:03.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 170744 21.70 MB/sec execute 298 sec latency 93.933 ms 2024-09-10T23:12:04.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:04 smithi176 ceph-mon[34982]: pgmap v935: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 704 op/s 2024-09-10T23:12:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:04 smithi120 ceph-mon[31247]: pgmap v935: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 704 op/s 2024-09-10T23:12:04.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:04 smithi153 ceph-mon[34951]: pgmap v935: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 704 op/s 2024-09-10T23:12:04.881 INFO:tasks.workunit.client.0.smithi120.stdout: 10 171131 21.69 MB/sec execute 299 sec latency 77.511 ms 2024-09-10T23:12:05.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 171688 21.69 MB/sec execute 300 sec latency 68.603 ms 2024-09-10T23:12:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:06 smithi176 ceph-mon[34982]: pgmap v936: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 459 op/s 2024-09-10T23:12:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:06 smithi120 ceph-mon[31247]: pgmap v936: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 459 op/s 2024-09-10T23:12:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:06 smithi153 ceph-mon[34951]: pgmap v936: 97 pgs: 97 active+clean; 249 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 459 op/s 2024-09-10T23:12:06.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 172402 21.73 MB/sec execute 301 sec latency 43.778 ms 2024-09-10T23:12:07.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 172910 21.78 MB/sec execute 302 sec latency 136.521 ms 2024-09-10T23:12:08.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:08 smithi176 ceph-mon[34982]: pgmap v937: 97 pgs: 97 active+clean; 232 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-10T23:12:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:08 smithi120 ceph-mon[31247]: pgmap v937: 97 pgs: 97 active+clean; 232 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-10T23:12:08.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:08 smithi153 ceph-mon[34951]: pgmap v937: 97 pgs: 97 active+clean; 232 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-10T23:12:08.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 173394 21.82 MB/sec execute 303 sec latency 97.935 ms 2024-09-10T23:12:09.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 173786 21.84 MB/sec execute 304 sec latency 79.043 ms 2024-09-10T23:12:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:10 smithi176 ceph-mon[34982]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-09-10T23:12:10.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:10 smithi120 ceph-mon[31247]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-09-10T23:12:10.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:10 smithi153 ceph-mon[34951]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-09-10T23:12:10.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174104 21.81 MB/sec execute 305 sec latency 98.640 ms 2024-09-10T23:12:11.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174443 21.78 MB/sec execute 306 sec latency 87.110 ms 2024-09-10T23:12:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:12 smithi176 ceph-mon[34982]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 546 op/s 2024-09-10T23:12:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:12 smithi120 ceph-mon[31247]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 546 op/s 2024-09-10T23:12:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:12 smithi153 ceph-mon[34951]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 546 op/s 2024-09-10T23:12:12.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 174846 21.76 MB/sec execute 307 sec latency 196.423 ms 2024-09-10T23:12:13.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 175263 21.75 MB/sec execute 308 sec latency 374.311 ms 2024-09-10T23:12:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:14 smithi176 ceph-mon[34982]: pgmap v940: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-09-10T23:12:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:14 smithi120 ceph-mon[31247]: pgmap v940: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-09-10T23:12:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:14 smithi153 ceph-mon[34951]: pgmap v940: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.5 MiB/s wr, 617 op/s 2024-09-10T23:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 -- 172.21.15.120:0/349781250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed681029b0 msgr2=0x7fed68102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 --2- 172.21.15.120:0/349781250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed681029b0 0x7fed68102e30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009930 tx=0x7fed5c02a420 comp rx=0 tx=0).stop 2024-09-10T23:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 -- 172.21.15.120:0/349781250 shutdown_connections 2024-09-10T23:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 --2- 172.21.15.120:0/349781250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68103370 0x7fed68109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 --2- 172.21.15.120:0/349781250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed681029b0 0x7fed68102e30 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 --2- 172.21.15.120:0/349781250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 -- 172.21.15.120:0/349781250 >> 172.21.15.120:0/349781250 conn(0x7fed680fd140 msgr2=0x7fed680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 -- 172.21.15.120:0/349781250 shutdown_connections 2024-09-10T23:12:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.833+0000 7fed6c80c640 1 -- 172.21.15.120:0/349781250 wait complete. 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 Processor -- start 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 -- start start 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed681029b0 0x7fed681982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810d4b0 con 0x7fed68103370 2024-09-10T23:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810d330 con 0x7fed68101860 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6c80c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed6810d630 con 0x7fed681029b0 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed681029b0 0x7fed681982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed66d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58610/0 (socket says 172.21.15.120:58610) 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37198/0 (socket says 172.21.15.120:37198) 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.834+0000 7fed66d9d640 1 -- 172.21.15.120:0/2751275050 learned_addr learned my addr 172.21.15.120:0/2751275050 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 -- 172.21.15.120:0/2751275050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed681029b0 msgr2=0x7fed681982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed681029b0 0x7fed681982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 -- 172.21.15.120:0/2751275050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 msgr2=0x7fed68197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 -- 172.21.15.120:0/2751275050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed681997d0 con 0x7fed68103370 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed66d9d640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fed60009fd0 tx=0x7fed6000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed577fe640 1 -- 172.21.15.120:0/2751275050 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60017070 con 0x7fed68103370 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed68199ac0 con 0x7fed68103370 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed577fe640 1 -- 172.21.15.120:0/2751275050 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed60003e20 con 0x7fed68103370 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed577fe640 1 -- 172.21.15.120:0/2751275050 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60010460 con 0x7fed68103370 2024-09-10T23:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.835+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681a0190 con 0x7fed68103370 2024-09-10T23:12:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 Processor -- start 2024-09-10T23:12:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed577fe640 1 -- 172.21.15.120:0/2751275050 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fed60003970 con 0x7fed68103370 2024-09-10T23:12:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 -- start start 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d75a6f0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d749b20 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6f5da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d74d930 con 0x55fc3d7efde0 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7ece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58616/0 (socket says 172.21.15.120:58616) 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed65d9b640 1 -- 172.21.15.120:0/3356165838 learned_addr learned my addr 172.21.15.120:0/3356165838 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed6659c640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.838+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 -- 172.21.15.120:0/3356165838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 msgr2=0x55fc3d7f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 -- 172.21.15.120:0/3356165838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 msgr2=0x55fc3d7ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7ece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 -- 172.21.15.120:0/3356165838 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3d70f530 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed577fe640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 0x7fed500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed6659c640 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7ef790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed58009c20 tx=0x7fed58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d425be5ca5dd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed65d9b640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 0x7fed500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed54ff9640 1 -- 172.21.15.120:0/3356165838 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58004050 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed577fe640 1 -- 172.21.15.120:0/2751275050 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed600e5880 con 0x7fed68103370 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed54ff9640 1 -- 172.21.15.120:0/3356165838 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed580054d0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.839+0000 7fed54ff9640 1 -- 172.21.15.120:0/3356165838 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58004c40 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed65d9b640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 0x7fed500b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000c00 tx=0x7fed5c03f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- 172.21.15.120:0/3356165838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 msgr2=0x55fc3d7ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7ef790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed58009c20 tx=0x7fed58011810 comp rx=0 tx=0).stop 2024-09-10T23:12:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- 172.21.15.120:0/3356165838 shutdown_connections 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7ef790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3356165838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- 172.21.15.120:0/3356165838 >> 172.21.15.120:0/3356165838 conn(0x55fc3d7e39b0 msgr2=0x55fc3d7e4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- 172.21.15.120:0/3356165838 shutdown_connections 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- 172.21.15.120:0/3356165838 wait complete. 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 -- start start 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.840+0000 7fed6f5da740 1 Processor -- start 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d681ab0 con 0x55fc3d7efde0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d6ef730 con 0x55fc3d7eaa70 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d7f2dc0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6f5da740 10 client.0 _ll_register_callbacks cb 0x55fc3d7436b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37220/0 (socket says 172.21.15.120:37220) 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 -- 172.21.15.120:0/2303641016 learned_addr learned my addr 172.21.15.120:0/2303641016 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed66d9d640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 -- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 msgr2=0x55fc3d7f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 -- 172.21.15.120:0/2303641016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 msgr2=0x55fc3d7f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 -- 172.21.15.120:0/2303641016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3d4ab790 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed66d9d640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed6659c640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed5801b040 tx=0x7fed580038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf7ea8dbcb3ea32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.841+0000 7fed4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58029070 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed580054d0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed58003ca0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed58035050 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 10 client.28481 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc3d524620 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed580054d0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fed58023430 con 0x55fc3d7eaa70 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 10 client.28481 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 10 client.28481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 20 client.28481 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d7ff670 0x55fc3d801a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 4 client.28481 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc3d75c640 con 0x55fc3d7ff670 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.842+0000 7fed65d9b640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d7ff670 0x55fc3d801a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.843+0000 7fed4dffb640 10 client.28481 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.843+0000 7fed65d9b640 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d7ff670 0x55fc3d801a60 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4490d5c1ef436a4 server_cookie=3d71131e1f78e4a4 in_seq=0 out_seq=0 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.846+0000 7fed4dffb640 1 -- 172.21.15.120:0/2303641016 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc3d75c640 con 0x55fc3d7ff670 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.846+0000 7fed4dffb640 10 client.28481 handle_command_reply: tid=0 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 client.28481 shutdown 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 msgr2=0x55fc3d7f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed5801b040 tx=0x7fed580038a0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 shutdown_connections 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d7ff670 0x55fc3d801a60 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed4dffb640 0 client.28481 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 --2- 172.21.15.120:0/2303641016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f4070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 >> 172.21.15.120:0/2303641016 conn(0x55fc3d7e39b0 msgr2=0x55fc3d7e2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 shutdown_connections 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 1 -- 172.21.15.120:0/2303641016 wait complete. 2024-09-10T23:12:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.847+0000 7fed6f5da740 20 client.28481 trim_cache size 0 max 16384 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 Processor -- start 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 -- start start 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a5b80 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a8400 con 0x55fc3d7eaa70 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6f5da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a61f0 con 0x55fc3d7efde0 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:37228/0 (socket says 172.21.15.120:37228) 2024-09-10T23:12:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed65d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58634/0 (socket says 172.21.15.120:58634) 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 -- 172.21.15.120:0/1155129708 learned_addr learned my addr 172.21.15.120:0/1155129708 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed66d9d640 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 -- 172.21.15.120:0/1155129708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 msgr2=0x55fc3d7f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 -- 172.21.15.120:0/1155129708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 msgr2=0x55fc3d7f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed6659c640 1 -- 172.21.15.120:0/1155129708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3d7ed1d0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.862+0000 7fed65d9b640 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.863+0000 7fed6659c640 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed58023880 tx=0x7fed58011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43448174800d54c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.863+0000 7fed4ffff640 1 -- 172.21.15.120:0/1155129708 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58029070 con 0x55fc3d7eaa70 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.863+0000 7fed4ffff640 1 -- 172.21.15.120:0/1155129708 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed58023430 con 0x55fc3d7eaa70 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.863+0000 7fed4ffff640 1 -- 172.21.15.120:0/1155129708 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5802e8f0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- 172.21.15.120:0/1155129708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 msgr2=0x55fc3d7f0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed58023880 tx=0x7fed58011d10 comp rx=0 tx=0).stop 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- 172.21.15.120:0/1155129708 shutdown_connections 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7efde0 0x55fc3d7f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d7f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- 172.21.15.120:0/1155129708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7eaa70 0x55fc3d7f0d70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- 172.21.15.120:0/1155129708 >> 172.21.15.120:0/1155129708 conn(0x55fc3d7e39b0 msgr2=0x55fc3d7e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- 172.21.15.120:0/1155129708 shutdown_connections 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- 172.21.15.120:0/1155129708 wait complete. 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- start start 2024-09-10T23:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 Processor -- start 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d807a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7efde0 0x55fc3d866870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a68c0 con 0x55fc3d7eaa70 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a7d30 con 0x55fc3d7efde0 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.864+0000 7fed6f5da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed681a6f90 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6f5da740 10 client.0 _ll_register_callbacks cb 0x7fed5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58646/0 (socket says 172.21.15.120:58646) 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6659c640 1 -- 172.21.15.120:0/3412309798 learned_addr learned my addr 172.21.15.120:0/3412309798 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d807a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed66d9d640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7efde0 0x55fc3d866870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 -- 172.21.15.120:0/3412309798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7efde0 msgr2=0x55fc3d866870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7efde0 0x55fc3d866870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 -- 172.21.15.120:0/3412309798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 msgr2=0x55fc3d8073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 -- 172.21.15.120:0/3412309798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3d8670a0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6659c640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed65d9b640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d807a80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005ec0 tx=0x7fed5c0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d4933ef4cbeee33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c035040 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed5c0037f0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.865+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed58023430 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed5c0039f0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 10 client.28735 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc3d86e4b0 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed58023430 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fed5c054070 con 0x55fc3d7ed3a0 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 10 client.28735 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 10 client.28735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 20 client.28735 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d86ede0 0x55fc3d871200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 4 client.28735 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc3d7580f0 con 0x55fc3d86ede0 2024-09-10T23:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.866+0000 7fed6659c640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d86ede0 0x55fc3d871200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.867+0000 7fed4dffb640 10 client.28735 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:14.867+0000 7fed6659c640 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d86ede0 0x55fc3d871200 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3145a4146118a144 server_cookie=f7a3d8a56d3d207e in_seq=0 out_seq=0 2024-09-10T23:12:14.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 175759 21.77 MB/sec execute 309 sec latency 92.760 ms 2024-09-10T23:12:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.577+0000 7fed4dffb640 1 -- 172.21.15.120:0/3412309798 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc3d7580f0 con 0x55fc3d86ede0 2024-09-10T23:12:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.577+0000 7fed4dffb640 10 client.28735 handle_command_reply: tid=0 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.577+0000 7fed6f5da740 1 client.28735 shutdown 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.577+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 msgr2=0x55fc3d807a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.577+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d807a80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005ec0 tx=0x7fed5c0314a0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 shutdown_connections 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc3d86ede0 0x55fc3d871200 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc3d7efde0 0x55fc3d866870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed4dffb640 0 client.28735 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc3d7ed3a0 0x55fc3d807a80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 --2- 172.21.15.120:0/3412309798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc3d7eaa70 0x55fc3d8073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 >> 172.21.15.120:0/3412309798 conn(0x55fc3d7e39b0 msgr2=0x55fc3d7e9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 shutdown_connections 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 1 -- 172.21.15.120:0/3412309798 wait complete. 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6f5da740 20 client.28735 trim_cache size 0 max 16384 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:12:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:12:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 msgr2=0x7fed500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 0x7fed500b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000c00 tx=0x7fed5c03f040 comp rx=0 tx=0).stop 2024-09-10T23:12:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.578+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 msgr2=0x7fed68198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fed60009fd0 tx=0x7fed6000b260 comp rx=0 tx=0).stop 2024-09-10T23:12:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 shutdown_connections 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed500b52e0 0x7fed500b77a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed68103370 0x7fed68198f10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed681029b0 0x7fed681982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 --2- 172.21.15.120:0/2751275050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed68101860 0x7fed68197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 >> 172.21.15.120:0/2751275050 conn(0x7fed680fd140 msgr2=0x7fed68103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 shutdown_connections 2024-09-10T23:12:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:15.579+0000 7fed6c80c640 1 -- 172.21.15.120:0/2751275050 wait complete. 2024-09-10T23:12:15.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 176011 21.74 MB/sec execute 310 sec latency 411.169 ms 2024-09-10T23:12:16.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:16 smithi176 ceph-mon[34982]: pgmap v941: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-09-10T23:12:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:16 smithi120 ceph-mon[31247]: pgmap v941: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-09-10T23:12:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:16 smithi153 ceph-mon[34951]: pgmap v941: 97 pgs: 97 active+clean; 243 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-09-10T23:12:16.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 176634 21.81 MB/sec execute 311 sec latency 40.578 ms 2024-09-10T23:12:17.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 177061 21.83 MB/sec execute 312 sec latency 84.345 ms 2024-09-10T23:12:18.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:18 smithi176 ceph-mon[34982]: pgmap v942: 97 pgs: 97 active+clean; 240 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.2 MiB/s wr, 674 op/s 2024-09-10T23:12:18.700 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:18 smithi120 ceph-mon[31247]: pgmap v942: 97 pgs: 97 active+clean; 240 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.2 MiB/s wr, 674 op/s 2024-09-10T23:12:18.702 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:18 smithi153 ceph-mon[34951]: pgmap v942: 97 pgs: 97 active+clean; 240 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.2 MiB/s wr, 674 op/s 2024-09-10T23:12:18.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 177333 21.83 MB/sec execute 313 sec latency 219.854 ms 2024-09-10T23:12:19.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 177665 21.79 MB/sec execute 314 sec latency 86.732 ms 2024-09-10T23:12:20.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:20 smithi176 ceph-mon[34982]: pgmap v943: 97 pgs: 97 active+clean; 227 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.5 MiB/s wr, 559 op/s 2024-09-10T23:12:20.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:20 smithi120 ceph-mon[31247]: pgmap v943: 97 pgs: 97 active+clean; 227 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.5 MiB/s wr, 559 op/s 2024-09-10T23:12:20.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:20 smithi153 ceph-mon[34951]: pgmap v943: 97 pgs: 97 active+clean; 227 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.5 MiB/s wr, 559 op/s 2024-09-10T23:12:20.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 177959 21.77 MB/sec execute 315 sec latency 69.618 ms 2024-09-10T23:12:21.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 178407 21.75 MB/sec execute 316 sec latency 70.993 ms 2024-09-10T23:12:22.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:22 smithi176 ceph-mon[34982]: pgmap v944: 97 pgs: 97 active+clean; 229 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-09-10T23:12:22.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:22 smithi120 ceph-mon[31247]: pgmap v944: 97 pgs: 97 active+clean; 229 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-09-10T23:12:22.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:22 smithi153 ceph-mon[34951]: pgmap v944: 97 pgs: 97 active+clean; 229 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-09-10T23:12:22.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 178907 21.77 MB/sec execute 317 sec latency 75.238 ms 2024-09-10T23:12:23.338 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:12:23.618 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:12:23.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 179389 21.76 MB/sec execute 318 sec latency 266.992 ms 2024-09-10T23:12:24.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 -- 172.21.15.120:0/1034457697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf0108060 msgr2=0x7facf010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 --2- 172.21.15.120:0/1034457697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf0108060 0x7facf010e860 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7face8009ca0 tx=0x7face802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 -- 172.21.15.120:0/1034457697 shutdown_connections 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 --2- 172.21.15.120:0/1034457697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf0108060 0x7facf010e860 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 --2- 172.21.15.120:0/1034457697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf01076e0 0x7facf0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 --2- 172.21.15.120:0/1034457697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0101cd0 0x7facf01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 -- 172.21.15.120:0/1034457697 >> 172.21.15.120:0/1034457697 conn(0x7facf00fd740 msgr2=0x7facf00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 -- 172.21.15.120:0/1034457697 shutdown_connections 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.145+0000 7facf4cad640 1 -- 172.21.15.120:0/1034457697 wait complete. 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 Processor -- start 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 -- start start 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 0x7facf0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0108060 0x7facf019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112e20 con 0x7facf01076e0 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112ca0 con 0x7facf0108060 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.146+0000 7facf4cad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112fa0 con 0x7facf0101cd0 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58846/0 (socket says 172.21.15.120:58846) 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 -- 172.21.15.120:0/263651807 learned_addr learned my addr 172.21.15.120:0/263651807 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facee59c640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 0x7facf0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7faceed9d640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0108060 0x7facf019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 -- 172.21.15.120:0/263651807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 msgr2=0x7facf0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 0x7facf0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 -- 172.21.15.120:0/263651807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0108060 msgr2=0x7facf019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0108060 0x7facf019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 -- 172.21.15.120:0/263651807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf019f320 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facee59c640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 0x7facf0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facedd9b640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7face400e9f0 tx=0x7face400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.147+0000 7facdf7fe640 1 -- 172.21.15.120:0/263651807 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face400cd80 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.148+0000 7facdf7fe640 1 -- 172.21.15.120:0/263651807 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face40054d0 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.148+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf019f610 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.148+0000 7facdf7fe640 1 -- 172.21.15.120:0/263651807 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4010690 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.148+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf010f0a0 con 0x7facf01076e0 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.150+0000 7facf7082740 1 Processor -- start 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 -- start start 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef678e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef6784fe50 con 0x55ef678d9da0 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ef6783f280 con 0x55ef678d9480 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facf7082740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ef67843090 con 0x55ef678e5d30 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45008/0 (socket says 172.21.15.120:45008) 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 -- 172.21.15.120:0/1417194747 learned_addr learned my addr 172.21.15.120:0/1417194747 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7facee59c640 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef678e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 -- 172.21.15.120:0/1417194747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 msgr2=0x55ef678e8120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678e8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 -- 172.21.15.120:0/1417194747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 msgr2=0x55ef678e57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef678e57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.151+0000 7faceed9d640 1 -- 172.21.15.120:0/1417194747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6759d650 con 0x55ef678d9480 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.152+0000 7faceed9d640 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7face8005fa0 tx=0x7face8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fb105b31f4fc02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.152+0000 7facdcff9640 1 -- 172.21.15.120:0/1417194747 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8031cf0 con 0x55ef678d9480 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.152+0000 7facdf7fe640 1 -- 172.21.15.120:0/263651807 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7face4004e90 con 0x7facf01076e0 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.152+0000 7facdcff9640 1 -- 172.21.15.120:0/1417194747 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face8036030 con 0x55ef678d9480 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facdcff9640 1 -- 172.21.15.120:0/1417194747 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80042a0 con 0x55ef678d9480 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 -- 172.21.15.120:0/1417194747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 msgr2=0x55ef678d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7face8005fa0 tx=0x7face8009800 comp rx=0 tx=0).stop 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 -- 172.21.15.120:0/1417194747 shutdown_connections 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef678e57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 --2- 172.21.15.120:0/1417194747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678d9860 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 -- 172.21.15.120:0/1417194747 >> 172.21.15.120:0/1417194747 conn(0x55ef678d72b0 msgr2=0x55ef678d8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facdf7fe640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 0x7facd80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 -- 172.21.15.120:0/1417194747 shutdown_connections 2024-09-10T23:12:24.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facf7082740 1 -- 172.21.15.120:0/1417194747 wait complete. 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facee59c640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 0x7facd80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.153+0000 7facdf7fe640 1 -- 172.21.15.120:0/263651807 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7face40e5fe0 con 0x7facf01076e0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facf7082740 1 -- start start 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facf7082740 1 Processor -- start 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facee59c640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 0x7facd80b7820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ef678d8190 tx=0x7face000f7a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facf7082740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facf7082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facf7082740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.154+0000 7facee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48280/0 (socket says 172.21.15.120:48280) 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7faceed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facedd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7faceed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58862/0 (socket says 172.21.15.120:58862) 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facedd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45012/0 (socket says 172.21.15.120:45012) 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 -- 172.21.15.120:0/3812828813 learned_addr learned my addr 172.21.15.120:0/3812828813 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef676a0700 con 0x55ef678e5d30 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ef677da110 con 0x55ef678d9da0 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ef678e8820 con 0x55ef678d9480 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facf7082740 10 client.0 _ll_register_callbacks cb 0x55ef677f3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 -- 172.21.15.120:0/3812828813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 msgr2=0x55ef678e2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 -- 172.21.15.120:0/3812828813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 msgr2=0x55ef678e2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef67807c40 con 0x55ef678d9480 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7faceed9d640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facedd9b640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facee59c640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7face0017d50 tx=0x7face00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cc88a72f104c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.155+0000 7facd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0023050 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7face0028640 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face8036030 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7face0028840 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 10 client.28744 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef678f4170 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face002d070 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7face0034ae0 con 0x55ef678d9480 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 10 client.28744 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 10 client.28744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.156+0000 7facf7082740 20 client.28744 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.157+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef678f5630 0x55ef678f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.157+0000 7facf7082740 4 client.28744 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:24.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.157+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7face8036030 con 0x55ef678f5630 2024-09-10T23:12:24.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.157+0000 7facedd9b640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef678f5630 0x55ef678f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.158+0000 7facedd9b640 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef678f5630 0x55ef678f7a40 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f3eb4e75ee89ae server_cookie=fcd0ca14ebc6fe88 in_seq=0 out_seq=0 2024-09-10T23:12:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.158+0000 7facd5ffb640 10 client.28744 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:24.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facd5ffb640 1 -- 172.21.15.120:0/3812828813 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7face8036030 con 0x55ef678f5630 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facd5ffb640 10 client.28744 handle_command_reply: tid=0 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 client.28744 shutdown 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 msgr2=0x55ef678e5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7face0017d50 tx=0x7face00046c0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 shutdown_connections 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef678f5630 0x55ef678f7a40 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9da0 0x55ef678e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facd5ffb640 0 client.28744 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 --2- 172.21.15.120:0/3812828813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9480 0x55ef678e5500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.159+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 >> 172.21.15.120:0/3812828813 conn(0x55ef678d72b0 msgr2=0x55ef678dce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.160+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 shutdown_connections 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.160+0000 7facf7082740 1 -- 172.21.15.120:0/3812828813 wait complete. 2024-09-10T23:12:24.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.160+0000 7facf7082740 20 client.28744 trim_cache size 0 max 16384 2024-09-10T23:12:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.181+0000 7facf7082740 1 Processor -- start 2024-09-10T23:12:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 -- start start 2024-09-10T23:12:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.185 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facee59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facee59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678e2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45020/0 (socket says 172.21.15.120:45020) 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facedd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facedd9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48292/0 (socket says 172.21.15.120:48292) 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facee59c640 1 -- 172.21.15.120:0/2502501085 learned_addr learned my addr 172.21.15.120:0/2502501085 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facf01aa040 con 0x55ef678e5d30 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ac8c0 con 0x55ef678d9480 2024-09-10T23:12:24.186 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.182+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7facf01aa6b0 con 0x55ef678d9da0 2024-09-10T23:12:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.185+0000 7facedd9b640 1 -- 172.21.15.120:0/2502501085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 msgr2=0x55ef678e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.185+0000 7facedd9b640 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.185+0000 7facedd9b640 1 -- 172.21.15.120:0/2502501085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 msgr2=0x55ef678d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.185+0000 7facedd9b640 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.188 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.185+0000 7facedd9b640 1 -- 172.21.15.120:0/2502501085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef678e18b0 con 0x55ef678d9da0 2024-09-10T23:12:24.189 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.187+0000 7facedd9b640 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7face40ac470 tx=0x7face40e3980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd1ce127b196916 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.189+0000 7facd7fff640 1 -- 172.21.15.120:0/2502501085 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4005680 con 0x55ef678d9da0 2024-09-10T23:12:24.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.190+0000 7facd7fff640 1 -- 172.21.15.120:0/2502501085 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face40ace70 con 0x55ef678d9da0 2024-09-10T23:12:24.192 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.190+0000 7facd7fff640 1 -- 172.21.15.120:0/2502501085 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face40a89e0 con 0x55ef678d9da0 2024-09-10T23:12:24.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 msgr2=0x55ef678d6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.193 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7face40ac470 tx=0x7face40e3980 comp rx=0 tx=0).stop 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 shutdown_connections 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678e5d30 0x55ef678d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678d9da0 0x55ef678d6bc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 --2- 172.21.15.120:0/2502501085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 >> 172.21.15.120:0/2502501085 conn(0x55ef678d72b0 msgr2=0x55ef678e2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 shutdown_connections 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.191+0000 7facf7082740 1 -- 172.21.15.120:0/2502501085 wait complete. 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 -- start start 2024-09-10T23:12:24.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 Processor -- start 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef67960540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7facf01aad80 con 0x55ef678d9da0 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ac1f0 con 0x55ef678d9480 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ab450 con 0x55ef678e5d30 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.192+0000 7facf7082740 10 client.0 _ll_register_callbacks cb 0x7face00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.193+0000 7facee59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.193+0000 7faceed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.193+0000 7facedd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef67960540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.193+0000 7facedd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef67960540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58878/0 (socket says 172.21.15.120:58878) 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.193+0000 7facedd9b640 1 -- 172.21.15.120:0/3899758281 learned_addr learned my addr 172.21.15.120:0/3899758281 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.194+0000 7facee59c640 1 -- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 msgr2=0x55ef678fc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.194+0000 7facee59c640 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678fc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.194+0000 7facee59c640 1 -- 172.21.15.120:0/3899758281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 msgr2=0x55ef67960540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.194+0000 7facee59c640 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef67960540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.194+0000 7facee59c640 1 -- 172.21.15.120:0/3899758281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6795b470 con 0x55ef678d9480 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facee59c640 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678fbad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7face0034e80 tx=0x7face00286f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c88f6b235d4b3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0023040 con 0x55ef678d9480 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7face0028df0 con 0x55ef678d9480 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face40ace70 con 0x55ef678d9480 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7face0017930 con 0x55ef678d9480 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facf7082740 10 client.28505 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:24.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.195+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef67973f00 con 0x55ef678d9480 2024-09-10T23:12:24.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.196+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face40ace70 con 0x55ef678d9480 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.197+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7face000f2a0 con 0x55ef678d9480 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.198+0000 7facf7082740 10 client.28505 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.198+0000 7facf7082740 10 client.28505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.198+0000 7facf7082740 20 client.28505 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.198+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef679746d0 0x55ef67976af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.198+0000 7facf7082740 4 client.28505 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:12:24.201 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.199+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef67851da0 con 0x55ef679746d0 2024-09-10T23:12:24.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.201+0000 7facedd9b640 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef679746d0 0x55ef67976af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.202+0000 7facedd9b640 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef679746d0 0x55ef67976af0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436996e004bb887f server_cookie=3cbdfdf047de3aa0 in_seq=0 out_seq=0 2024-09-10T23:12:24.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.202+0000 7facd5ffb640 10 client.28505 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:24.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.205+0000 7facd5ffb640 1 -- 172.21.15.120:0/3899758281 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9462 (crc 0 0 0) 0x55ef67851da0 con 0x55ef679746d0 2024-09-10T23:12:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.205+0000 7facd5ffb640 10 client.28505 handle_command_reply: tid=0 2024-09-10T23:12:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.206+0000 7facf7082740 1 client.28505 shutdown 2024-09-10T23:12:24.208 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 msgr2=0x55ef678fbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678fbad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7face0034e80 tx=0x7face00286f0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 shutdown_connections 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ef679746d0 0x55ef67976af0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef678e5d30 0x55ef678fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef678d9da0 0x55ef67960540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 --2- 172.21.15.120:0/3899758281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ef678d9480 0x55ef678fbad0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7face0034e80 tx=0x7face00286f0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.209 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 >> 172.21.15.120:0/3899758281 conn(0x55ef678d72b0 msgr2=0x55ef678e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.207+0000 7facd5ffb640 0 client.28505 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:24.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.208+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 shutdown_connections 2024-09-10T23:12:24.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.208+0000 7facf7082740 1 -- 172.21.15.120:0/3899758281 wait complete. 2024-09-10T23:12:24.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.208+0000 7facf7082740 20 client.28505 trim_cache size 0 max 16384 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.212 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.213 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:12:24.214 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.216 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:12:24.217 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.218 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:12:24.220 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.222 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.223 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:12:24.224 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.225 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.226 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.227 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.228 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:12:24.229 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.230 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:24.231 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:12:24.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:12:24.232 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 msgr2=0x7facd80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 0x7facd80b7820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ef678d8190 tx=0x7face000f7a0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 msgr2=0x7facf01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7face400e9f0 tx=0x7face400edf0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 shutdown_connections 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7facd80b5360 0x7facd80b7820 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7facf0108060 0x7facf019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facf01076e0 0x7facf01997b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 --2- 172.21.15.120:0/263651807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facf0101cd0 0x7facf0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:24.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 >> 172.21.15.120:0/263651807 conn(0x7facf00fd740 msgr2=0x7facf00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:24.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.211+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 shutdown_connections 2024-09-10T23:12:24.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:24.212+0000 7facf4cad640 1 -- 172.21.15.120:0/263651807 wait complete. 2024-09-10T23:12:24.412 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:24 smithi120 ceph-mon[31247]: pgmap v945: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-09-10T23:12:24.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:24 smithi176 ceph-mon[34982]: pgmap v945: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-09-10T23:12:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:24 smithi153 ceph-mon[34951]: pgmap v945: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.5 MiB/s wr, 644 op/s 2024-09-10T23:12:24.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 179893 21.79 MB/sec execute 319 sec latency 64.771 ms 2024-09-10T23:12:25.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:12:25.083 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:12:25.372 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:12:25.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 180380 21.83 MB/sec execute 320 sec latency 59.685 ms 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2149874972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d981076a0 msgr2=0x7f1d98107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2149874972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d981076a0 0x7f1d98107ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009900 tx=0x7f1d8c02a410 comp rx=0 tx=0).stop 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2149874972 shutdown_connections 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2149874972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2149874972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d981076a0 0x7f1d98107ae0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:25.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2149874972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d98101c90 0x7f1d98102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2149874972 >> 172.21.15.120:0/2149874972 conn(0x7f1d980fd700 msgr2=0x7f1d980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2149874972 shutdown_connections 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.959+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2149874972 wait complete. 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.960+0000 7f1d9d55a640 1 Processor -- start 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.960+0000 7f1d9d55a640 1 -- start start 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 0x7f1d9819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 0x7f1d9819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98111c70 con 0x7f1d98101c90 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98111af0 con 0x7f1d981076a0 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d9d55a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98111df0 con 0x7f1d98108020 2024-09-10T23:12:25.979 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d977fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d96ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 0x7f1d9819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 0x7f1d9819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d977fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48314/0 (socket says 172.21.15.120:48314) 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d977fe640 1 -- 172.21.15.120:0/2181205476 learned_addr learned my addr 172.21.15.120:0/2181205476 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 -- 172.21.15.120:0/2181205476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 msgr2=0x7f1d9819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 -- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 msgr2=0x7f1d9819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 0x7f1d9819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d967fc640 1 -- 172.21.15.120:0/2181205476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981a5480 con 0x7f1d981076a0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.961+0000 7f1d96ffd640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 0x7f1d9819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d967fc640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 0x7f1d9819a5c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c02a930 tx=0x7f1d8c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c048070 con 0x7f1d981076a0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d8c004410 con 0x7f1d981076a0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98197e70 con 0x7f1d981076a0 2024-09-10T23:12:25.980 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c035050 con 0x7f1d981076a0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.962+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d98198350 con 0x7f1d981076a0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.963+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98102070 con 0x7f1d981076a0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.964+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1d8c003ba0 con 0x7f1d981076a0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.965+0000 7f1d7ffff640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 0x7f1d800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.965+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1d8c109c10 con 0x7f1d981076a0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.965+0000 7f1d96ffd640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 0x7f1d800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.965+0000 7f1d96ffd640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 0x7f1d800b77a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009fc0 tx=0x7f1d88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:25.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:25.968+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1d8c0d0450 con 0x7f1d981076a0 2024-09-10T23:12:26.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.266+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9802e5f0 con 0x7f1d981076a0 2024-09-10T23:12:26.269 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.267+0000 7f1d7ffff640 1 -- 172.21.15.120:0/2181205476 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f1d8c0d9250 con 0x7f1d981076a0 2024-09-10T23:12:26.269 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:12:26.269 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:12:26.270 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:12:26.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 msgr2=0x7f1d800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:26.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 0x7f1d800b77a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009fc0 tx=0x7f1d88005f90 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 msgr2=0x7f1d9819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 0x7f1d9819a5c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c02a930 tx=0x7f1d8c031a20 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 shutdown_connections 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1d800b52e0 0x7f1d800b77a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d98108020 0x7f1d9819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d981076a0 0x7f1d9819a5c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 --2- 172.21.15.120:0/2181205476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d98101c90 0x7f1d9819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 >> 172.21.15.120:0/2181205476 conn(0x7f1d980fd700 msgr2=0x7f1d980ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 shutdown_connections 2024-09-10T23:12:26.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:26.270+0000 7f1d9d55a640 1 -- 172.21.15.120:0/2181205476 wait complete. 2024-09-10T23:12:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:26 smithi176 ceph-mon[34982]: pgmap v946: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 513 op/s 2024-09-10T23:12:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2181205476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:12:26.615 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:26 smithi120 ceph-mon[31247]: pgmap v946: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 513 op/s 2024-09-10T23:12:26.615 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2181205476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:12:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:26 smithi153 ceph-mon[34951]: pgmap v946: 97 pgs: 97 active+clean; 233 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 513 op/s 2024-09-10T23:12:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2181205476' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:12:26.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 180829 21.86 MB/sec execute 321 sec latency 58.206 ms 2024-09-10T23:12:27.008 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:12:27.309 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:12:27.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 -- 172.21.15.120:0/1306288132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1200109170 msgr2=0x7f120010f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 --2- 172.21.15.120:0/1306288132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1200109170 0x7f120010f970 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009ca0 tx=0x7f11f802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 -- 172.21.15.120:0/1306288132 shutdown_connections 2024-09-10T23:12:27.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 --2- 172.21.15.120:0/1306288132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1200109170 0x7f120010f970 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 --2- 172.21.15.120:0/1306288132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12001087f0 0x7f1200108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 --2- 172.21.15.120:0/1306288132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f12001031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 -- 172.21.15.120:0/1306288132 >> 172.21.15.120:0/1306288132 conn(0x7f12000fe830 msgr2=0x7f1200100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 -- 172.21.15.120:0/1306288132 shutdown_connections 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.867+0000 7f12051d8640 1 -- 172.21.15.120:0/1306288132 wait complete. 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 Processor -- start 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 -- start start 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f1200199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 0x7f120019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112df0 con 0x7f12001087f0 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112c70 con 0x7f1200102de0 2024-09-10T23:12:27.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f12051d8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112f70 con 0x7f1200109170 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58902/0 (socket says 172.21.15.120:58902) 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f11fe59c640 1 -- 172.21.15.120:0/3275617508 learned_addr learned my addr 172.21.15.120:0/3275617508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f11fed9d640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f1200199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.868+0000 7f11ff59e640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 0x7f120019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 -- 172.21.15.120:0/3275617508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 msgr2=0x7f120019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 0x7f120019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 -- 172.21.15.120:0/3275617508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 msgr2=0x7f1200199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f1200199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 -- 172.21.15.120:0/3275617508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019f310 con 0x7f12001087f0 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11ff59e640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 0x7f120019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fed9d640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f1200199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:27.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f11f400e9f0 tx=0x7f11f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11e7fff640 1 -- 172.21.15.120:0/3275617508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f400cd80 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120019f600 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11e7fff640 1 -- 172.21.15.120:0/3275617508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f40054d0 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.869+0000 7f11e7fff640 1 -- 172.21.15.120:0/3275617508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4010690 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.870+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001a5cc0 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 Processor -- start 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 -- start start 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11e7fff640 1 -- 172.21.15.120:0/3275617508 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f11f4004e90 con 0x7f12001087f0 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c88580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1bebea0 con 0x55d5e1c7a200 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1bdb2d0 con 0x55d5e1c86190 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f12075ad740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1bdf0e0 con 0x55d5e1c798e0 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11ff59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11ff59e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c79cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48348/0 (socket says 172.21.15.120:48348) 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11fed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58916/0 (socket says 172.21.15.120:58916) 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11ff59e640 1 -- 172.21.15.120:0/780689439 learned_addr learned my addr 172.21.15.120:0/780689439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.873+0000 7f11fe59c640 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 -- 172.21.15.120:0/780689439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 msgr2=0x55d5e1c79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 -- 172.21.15.120:0/780689439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 msgr2=0x55d5e1c88580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c88580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 -- 172.21.15.120:0/780689439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e1939650 con 0x55d5e1c7a200 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fe59c640 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c88580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11fed9d640 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fd0 tx=0x7f11f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f47b73bbdb8c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11e57fa640 1 -- 172.21.15.120:0/780689439 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004020 con 0x55d5e1c7a200 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11e57fa640 1 -- 172.21.15.120:0/780689439 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f00054d0 con 0x55d5e1c7a200 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.874+0000 7f11e57fa640 1 -- 172.21.15.120:0/780689439 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004c10 con 0x55d5e1c7a200 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f11e7fff640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 0x7f11e80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f11ff59e640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 0x7f11e80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- 172.21.15.120:0/780689439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 msgr2=0x55d5e1c85c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fd0 tx=0x7f11f00118d0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f11e7fff640 1 -- 172.21.15.120:0/3275617508 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11f40e5f80 con 0x7f12001087f0 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- 172.21.15.120:0/780689439 shutdown_connections 2024-09-10T23:12:27.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c88580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c85c50 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 --2- 172.21.15.120:0/780689439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- 172.21.15.120:0/780689439 >> 172.21.15.120:0/780689439 conn(0x55d5e1c77710 msgr2=0x55d5e1c788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- 172.21.15.120:0/780689439 shutdown_connections 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- 172.21.15.120:0/780689439 wait complete. 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f11ff59e640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 0x7f11e80b7820 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005fe0 tx=0x7f11f8005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.875+0000 7f12075ad740 1 -- start start 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 Processor -- start 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 0x55d5e1c85a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c86190 0x55d5e1c83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1a3c700 con 0x55d5e1c7a200 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1b76160 con 0x55d5e1c798e0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d5e1c88e10 con 0x55d5e1c86190 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f12075ad740 10 client.0 _ll_register_callbacks cb 0x55d5e1b8f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58918/0 (socket says 172.21.15.120:58918) 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fe59c640 1 -- 172.21.15.120:0/1767462307 learned_addr learned my addr 172.21.15.120:0/1767462307 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 0x55d5e1c85a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11ff59e640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c86190 0x55d5e1c83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 -- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c86190 msgr2=0x55d5e1c83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c86190 0x55d5e1c83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 -- 172.21.15.120:0/1767462307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 msgr2=0x55d5e1c82e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fed9d640 1 -- 172.21.15.120:0/1767462307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e1ba3c90 con 0x55d5e1c798e0 2024-09-10T23:12:27.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.876+0000 7f11fe59c640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11fed9d640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 0x55d5e1c85a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fa0 tx=0x7f11f000fc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3582dfe971c985 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0028020 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f00054d0 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11f0023470 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11f001d040 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 10 client.28523 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5e1c94730 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f00054d0 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f11f0023bd0 con 0x55d5e1c798e0 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 10 client.28523 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:27.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 10 client.28523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 20 client.28523 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1c97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 4 client.28523 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e1beddf0 con 0x55d5e1c95ae0 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.877+0000 7f11fe59c640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1c97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.878+0000 7f11e2ff5640 10 client.28523 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:27.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.878+0000 7f11fe59c640 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1c97ed0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da68407a7b24c36e server_cookie=a932abcd56924b00 in_seq=0 out_seq=0 2024-09-10T23:12:27.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f11e2ff5640 1 -- 172.21.15.120:0/1767462307 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5e1beddf0 con 0x55d5e1c95ae0 2024-09-10T23:12:27.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f11e2ff5640 10 client.28523 handle_command_reply: tid=0 2024-09-10T23:12:27.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 client.28523 shutdown 2024-09-10T23:12:27.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 msgr2=0x55d5e1c85a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 0x55d5e1c85a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fa0 tx=0x7f11f000fc80 comp rx=0 tx=0).stop 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 shutdown_connections 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1c97ed0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c86190 0x55d5e1c83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c7a200 0x55d5e1c82e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f11e2ff5640 0 client.28523 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 --2- 172.21.15.120:0/1767462307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c798e0 0x55d5e1c85a30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 >> 172.21.15.120:0/1767462307 conn(0x55d5e1c77710 msgr2=0x55d5e1c72420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 shutdown_connections 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.880+0000 7f12075ad740 1 -- 172.21.15.120:0/1767462307 wait complete. 2024-09-10T23:12:27.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.881+0000 7f12075ad740 20 client.28523 trim_cache size 0 max 16384 2024-09-10T23:12:27.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181201 21.84 MB/sec execute 322 sec latency 79.832 ms 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 Processor -- start 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 -- start start 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c92070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 0x55d5e1c87580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 0x55d5e1c87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12001aa0f0 con 0x55d5e1c86190 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ac970 con 0x55d5e1c7a200 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f12075ad740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12001aa760 con 0x55d5e1c798e0 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f11ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 0x55d5e1c87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f11ff59e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 0x55d5e1c87ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58934/0 (socket says 172.21.15.120:58934) 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f11ff59e640 1 -- 172.21.15.120:0/3610984673 learned_addr learned my addr 172.21.15.120:0/3610984673 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.896+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 0x55d5e1c87580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fed9d640 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c92070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 -- 172.21.15.120:0/3610984673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 msgr2=0x55d5e1c92070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c92070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 -- 172.21.15.120:0/3610984673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 msgr2=0x55d5e1c87ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 0x55d5e1c87ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 -- 172.21.15.120:0/3610984673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e1c85660 con 0x55d5e1c7a200 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 0x55d5e1c87580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11f40e3ab0 tx=0x7f11f4009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b906d4e27aec3bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11e4ff9640 1 -- 172.21.15.120:0/3610984673 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40a8610 con 0x55d5e1c7a200 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11e4ff9640 1 -- 172.21.15.120:0/3610984673 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f40ac280 con 0x55d5e1c7a200 2024-09-10T23:12:27.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.897+0000 7f11e4ff9640 1 -- 172.21.15.120:0/3610984673 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40abee0 con 0x55d5e1c7a200 2024-09-10T23:12:27.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- 172.21.15.120:0/3610984673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 msgr2=0x55d5e1c87580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 0x55d5e1c87580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11f40e3ab0 tx=0x7f11f4009720 comp rx=0 tx=0).stop 2024-09-10T23:12:27.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- 172.21.15.120:0/3610984673 shutdown_connections 2024-09-10T23:12:27.903 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c86190 0x55d5e1c87ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c7a200 0x55d5e1c87580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 --2- 172.21.15.120:0/3610984673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c798e0 0x55d5e1c92070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- 172.21.15.120:0/3610984673 >> 172.21.15.120:0/3610984673 conn(0x55d5e1c77710 msgr2=0x55d5e1c72420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- 172.21.15.120:0/3610984673 shutdown_connections 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- 172.21.15.120:0/3610984673 wait complete. 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 -- start start 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.898+0000 7f12075ad740 1 Processor -- start 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c798e0 0x55d5e1c8e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c89130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12001aae30 con 0x55d5e1c798e0 2024-09-10T23:12:27.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ac2a0 con 0x55d5e1c86190 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ab500 con 0x55d5e1c7a200 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f12075ad740 10 client.0 _ll_register_callbacks cb 0x7f11f40e9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48362/0 (socket says 172.21.15.120:48362) 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11ff59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c89130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 -- 172.21.15.120:0/3393854549 learned_addr learned my addr 172.21.15.120:0/3393854549 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 -- 172.21.15.120:0/3393854549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 msgr2=0x55d5e1c89130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c89130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 -- 172.21.15.120:0/3393854549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c798e0 msgr2=0x55d5e1c8e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c798e0 0x55d5e1c8e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 -- 172.21.15.120:0/3393854549 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e1c89960 con 0x55d5e1c7a200 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11fe59c640 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f11f40e3b00 tx=0x7f11f4005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=310f82e81e7944af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:27.905 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.899+0000 7f11e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40ac0c0 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f40ac280 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f11f40a89e0 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11f40ac540 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 10 client.28774 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5e1d034a0 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f40ac280 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f11f40c3020 con 0x55d5e1c7a200 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 10 client.28774 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 10 client.28774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 20 client.28774 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1d05ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:27.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 4 client.28774 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:12:27.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5e1be98a0 con 0x55d5e1c95ae0 2024-09-10T23:12:27.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.900+0000 7f11fed9d640 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1d05ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:27.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.901+0000 7f11e2ff5640 10 client.28774 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:27.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.901+0000 7f11fed9d640 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1d05ba0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53e7dee5e854023 server_cookie=29a3dbf1e2eb17f7 in_seq=0 out_seq=0 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.902+0000 7f11e2ff5640 1 -- 172.21.15.120:0/3393854549 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5e1be98a0 con 0x55d5e1c95ae0 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.902+0000 7f11e2ff5640 10 client.28774 handle_command_reply: tid=0 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.902+0000 7f12075ad740 1 client.28774 shutdown 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 msgr2=0x55d5e1c8e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f11f40e3b00 tx=0x7f11f4005c70 comp rx=0 tx=0).stop 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 shutdown_connections 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d5e1c95ae0 0x55d5e1d05ba0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d5e1c86190 0x55d5e1c89130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f11e2ff5640 0 client.28774 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5e1c7a200 0x55d5e1c8e960 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 --2- 172.21.15.120:0/3393854549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5e1c798e0 0x55d5e1c8e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 >> 172.21.15.120:0/3393854549 conn(0x55d5e1c77710 msgr2=0x55d5e1c844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 shutdown_connections 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 1 -- 172.21.15.120:0/3393854549 wait complete. 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.903+0000 7f12075ad740 20 client.28774 trim_cache size 0 max 16384 2024-09-10T23:12:27.909 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 msgr2=0x7f11e80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 0x7f11e80b7820 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005fe0 tx=0x7f11f8005ef0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 msgr2=0x7f12001997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f11f400e9f0 tx=0x7f11f400edf0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 shutdown_connections 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11e80b5360 0x7f11e80b7820 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1200109170 0x7f120019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12001087f0 0x7f12001997a0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 --2- 172.21.15.120:0/3275617508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1200102de0 0x7f1200199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 >> 172.21.15.120:0/3275617508 conn(0x7f12000fe830 msgr2=0x7f1200100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 shutdown_connections 2024-09-10T23:12:27.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:27.904+0000 7f12051d8640 1 -- 172.21.15.120:0/3275617508 wait complete. 2024-09-10T23:12:28.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:28 smithi176 ceph-mon[34982]: pgmap v947: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 643 op/s 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 -- 172.21.15.120:0/3093755705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6441029b0 msgr2=0x7fd644102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 --2- 172.21.15.120:0/3093755705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6441029b0 0x7fd644102e30 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd634009930 tx=0x7fd63402a420 comp rx=0 tx=0).stop 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 -- 172.21.15.120:0/3093755705 shutdown_connections 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 --2- 172.21.15.120:0/3093755705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd644103370 0x7fd644109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 --2- 172.21.15.120:0/3093755705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd6441029b0 0x7fd644102e30 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 --2- 172.21.15.120:0/3093755705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644101860 0x7fd644101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 -- 172.21.15.120:0/3093755705 >> 172.21.15.120:0/3093755705 conn(0x7fd6440fd140 msgr2=0x7fd6440ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 -- 172.21.15.120:0/3093755705 shutdown_connections 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.683+0000 7fd64c02a640 1 -- 172.21.15.120:0/3093755705 wait complete. 2024-09-10T23:12:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 Processor -- start 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 -- start start 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6441029b0 0x7fd644199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644103370 0x7fd64419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d4b0 con 0x7fd644101860 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d330 con 0x7fd6441029b0 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64c02a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d630 con 0x7fd644103370 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd649dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd6495c5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6441029b0 0x7fd644199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd649dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58962/0 (socket says 172.21.15.120:58962) 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd649dc6640 1 -- 172.21.15.120:0/1531572354 learned_addr learned my addr 172.21.15.120:0/1531572354 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644103370 0x7fd64419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd649dc6640 1 -- 172.21.15.120:0/1531572354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644103370 msgr2=0x7fd64419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.688 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.684+0000 7fd649dc6640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644103370 0x7fd64419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd649dc6640 1 -- 172.21.15.120:0/1531572354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6441029b0 msgr2=0x7fd644199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd649dc6640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6441029b0 0x7fd644199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd649dc6640 1 -- 172.21.15.120:0/1531572354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64419a8f0 con 0x7fd644101860 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd649dc6640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e9f0 tx=0x7fd64000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd63affd640 1 -- 172.21.15.120:0/1531572354 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64000cd80 con 0x7fd644101860 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd63affd640 1 -- 172.21.15.120:0/1531572354 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6400108e0 con 0x7fd644101860 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64419abe0 con 0x7fd644101860 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd63affd640 1 -- 172.21.15.120:0/1531572354 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640018980 con 0x7fd644101860 2024-09-10T23:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.685+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd644103ea0 con 0x7fd644101860 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 Processor -- start 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 -- start start 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55acb15466f0 con 0x55acb15d7190 2024-09-10T23:12:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55acb1535b20 con 0x55acb15d9bd0 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.688+0000 7fd64edf5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55acb1539930 con 0x55acb15d4860 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48394/0 (socket says 172.21.15.120:48394) 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 -- 172.21.15.120:0/811004422 learned_addr learned my addr 172.21.15.120:0/811004422 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd6495c5640 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd63affd640 1 -- 172.21.15.120:0/1531572354 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd640010430 con 0x7fd644101860 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 -- 172.21.15.120:0/811004422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 msgr2=0x55acb15dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 -- 172.21.15.120:0/811004422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 msgr2=0x55acb15d9580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15d9580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 -- 172.21.15.120:0/811004422 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb14fb530 con 0x55acb15d4860 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd6495c5640 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:12:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd63c003970 tx=0x7fd63c003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbb5ca8049370b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd62ffff640 1 -- 172.21.15.120:0/811004422 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011440 con 0x55acb15d4860 2024-09-10T23:12:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd62ffff640 1 -- 172.21.15.120:0/811004422 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c015030 con 0x55acb15d4860 2024-09-10T23:12:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.689+0000 7fd62ffff640 1 -- 172.21.15.120:0/811004422 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0109a0 con 0x55acb15d4860 2024-09-10T23:12:28.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd63affd640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 0x7fd6300b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd63affd640 1 -- 172.21.15.120:0/1531572354 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd6400e5e90 con 0x7fd644101860 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd649dc6640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 0x7fd6300b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 -- 172.21.15.120:0/811004422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 msgr2=0x55acb15d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd63c003970 tx=0x7fd63c003e30 comp rx=0 tx=0).stop 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 -- 172.21.15.120:0/811004422 shutdown_connections 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 --2- 172.21.15.120:0/811004422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15d6c50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 -- 172.21.15.120:0/811004422 >> 172.21.15.120:0/811004422 conn(0x55acb15cf9b0 msgr2=0x55acb15d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 -- 172.21.15.120:0/811004422 shutdown_connections 2024-09-10T23:12:28.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.690+0000 7fd64edf5740 1 -- 172.21.15.120:0/811004422 wait complete. 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd649dc6640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 0x7fd6300b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd6400e39c0 tx=0x7fd640018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd64edf5740 1 -- start start 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd64edf5740 1 Processor -- start 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.691+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64edf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55acb146dab0 con 0x55acb15d7190 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64edf5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55acb14db730 con 0x55acb15d9bd0 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64edf5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55acb15dcbb0 con 0x55acb15d4860 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58968/0 (socket says 172.21.15.120:58968) 2024-09-10T23:12:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 -- 172.21.15.120:0/687389617 learned_addr learned my addr 172.21.15.120:0/687389617 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64edf5740 10 client.0 _ll_register_callbacks cb 0x55acb152f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 -- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 msgr2=0x55acb15ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 -- 172.21.15.120:0/687389617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 msgr2=0x55acb15df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 -- 172.21.15.120:0/687389617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb1297790 con 0x55acb15d7190 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd6495c5640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd634009930 tx=0x7fd6340317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3efaa30278ad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.692+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634035070 con 0x55acb15d7190 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd634004450 con 0x55acb15d7190 2024-09-10T23:12:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63c015030 con 0x55acb15d7190 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd634005d50 con 0x55acb15d7190 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 10 client.19335 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55acb1310620 con 0x55acb15d7190 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c015030 con 0x55acb15d7190 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd6340038c0 con 0x55acb15d7190 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 10 client.19335 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 20 client.19335 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb15e97d0 0x55acb15ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 4 client.19335 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb1548640 con 0x55acb15e97d0 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.693+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb15e97d0 0x55acb15ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.694+0000 7fd62d7fa640 10 client.19335 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.694+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb15e97d0 0x55acb15ebc00 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ed5da4e58c4afd server_cookie=9dd85a6bd6d1d4ac in_seq=0 out_seq=0 2024-09-10T23:12:28.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:28 smithi120 ceph-mon[31247]: pgmap v947: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 643 op/s 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd62d7fa640 1 -- 172.21.15.120:0/687389617 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acb1548640 con 0x55acb15e97d0 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd62d7fa640 10 client.19335 handle_command_reply: tid=0 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 client.19335 shutdown 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 msgr2=0x55acb15de560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd634009930 tx=0x7fd6340317d0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 shutdown_connections 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb15e97d0 0x55acb15ebc00 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d7190 0x55acb15de560 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 --2- 172.21.15.120:0/687389617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d4860 0x55acb15ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd62d7fa640 0 client.19335 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 >> 172.21.15.120:0/687389617 conn(0x55acb15cf9b0 msgr2=0x55acb15cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 shutdown_connections 2024-09-10T23:12:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 1 -- 172.21.15.120:0/687389617 wait complete. 2024-09-10T23:12:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.699+0000 7fd64edf5740 20 client.19335 trim_cache size 0 max 16384 2024-09-10T23:12:28.703 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:28 smithi153 ceph-mon[34951]: pgmap v947: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 643 op/s 2024-09-10T23:12:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 Processor -- start 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 -- start start 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a5cd0 con 0x55acb15d4860 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a8550 con 0x55acb15d9bd0 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64edf5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a6340 con 0x55acb15d7190 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58984/0 (socket says 172.21.15.120:58984) 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd64a5c7640 1 -- 172.21.15.120:0/2756060137 learned_addr learned my addr 172.21.15.120:0/2756060137 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.716+0000 7fd6495c5640 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 -- 172.21.15.120:0/2756060137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 msgr2=0x55acb15de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 -- 172.21.15.120:0/2756060137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 msgr2=0x55acb15dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 -- 172.21.15.120:0/2756060137 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb15d98f0 con 0x55acb15d4860 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd63c004410 tx=0x7fd63c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302c3f5311327bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd62f7fe640 1 -- 172.21.15.120:0/2756060137 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c023040 con 0x55acb15d4860 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd62f7fe640 1 -- 172.21.15.120:0/2756060137 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c015040 con 0x55acb15d4860 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.717+0000 7fd62f7fe640 1 -- 172.21.15.120:0/2756060137 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011ca0 con 0x55acb15d4860 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- 172.21.15.120:0/2756060137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 msgr2=0x55acb15cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd63c004410 tx=0x7fd63c0096a0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- 172.21.15.120:0/2756060137 shutdown_connections 2024-09-10T23:12:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- 172.21.15.120:0/2756060137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15cad20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- 172.21.15.120:0/2756060137 >> 172.21.15.120:0/2756060137 conn(0x55acb15cf9b0 msgr2=0x55acb15ddec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- 172.21.15.120:0/2756060137 shutdown_connections 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- 172.21.15.120:0/2756060137 wait complete. 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- start start 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 Processor -- start 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a6a10 con 0x55acb15d4860 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a7e80 con 0x55acb15d9bd0 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.718+0000 7fd64edf5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441a70e0 con 0x55acb15d7190 2024-09-10T23:12:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64edf5740 10 client.0 _ll_register_callbacks cb 0x7fd634009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64a5c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58988/0 (socket says 172.21.15.120:58988) 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64a5c7640 1 -- 172.21.15.120:0/3307031306 learned_addr learned my addr 172.21.15.120:0/3307031306 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 -- 172.21.15.120:0/3307031306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 msgr2=0x55acb15e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 -- 172.21.15.120:0/3307031306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 msgr2=0x55acb15e56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15e56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 -- 172.21.15.120:0/3307031306 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb15e4e50 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd6495c5640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15e43d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd63402a940 tx=0x7fd634003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25eeeb4a39778b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd62d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634035040 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd634050910 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63c015040 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd634060070 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64edf5740 10 client.28792 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.719+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55acb15e3d50 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c015040 con 0x55acb15d7190 2024-09-10T23:12:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd63403a510 con 0x55acb15d7190 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 10 client.28792 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 10 client.28792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 20 client.28792 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb165f680 0x55acb1661aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 4 client.28792 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acb15440f0 con 0x55acb165f680 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb165f680 0x55acb1661aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd62d7fa640 10 client.28792 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:28.720+0000 7fd64a5c7640 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb165f680 0x55acb1661aa0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870343dfb2af5323 server_cookie=1620d6ceb36a5d30 in_seq=0 out_seq=0 2024-09-10T23:12:28.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181382 21.80 MB/sec execute 323 sec latency 279.626 ms 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd62d7fa640 1 -- 172.21.15.120:0/3307031306 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acb15440f0 con 0x55acb165f680 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd62d7fa640 10 client.28792 handle_command_reply: tid=0 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 client.28792 shutdown 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 msgr2=0x55acb15e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15e43d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd63402a940 tx=0x7fd634003820 comp rx=0 tx=0).stop 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 shutdown_connections 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55acb165f680 0x55acb1661aa0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55acb15d9bd0 0x55acb15e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd62d7fa640 0 client.28792 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55acb15d7190 0x55acb15e43d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 --2- 172.21.15.120:0/3307031306 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55acb15d4860 0x55acb15e56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 >> 172.21.15.120:0/3307031306 conn(0x55acb15cf9b0 msgr2=0x55acb15cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 shutdown_connections 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 1 -- 172.21.15.120:0/3307031306 wait complete. 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.603+0000 7fd64edf5740 20 client.28792 trim_cache size 0 max 16384 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:12:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 msgr2=0x7fd6300b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 0x7fd6300b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd6400e39c0 tx=0x7fd640018420 comp rx=0 tx=0).stop 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 msgr2=0x7fd644198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e9f0 tx=0x7fd64000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 shutdown_connections 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd6300b52e0 0x7fd6300b77a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd644103370 0x7fd64419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6441029b0 0x7fd644199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 --2- 172.21.15.120:0/1531572354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd644101860 0x7fd644198db0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 >> 172.21.15.120:0/1531572354 conn(0x7fd6440fd140 msgr2=0x7fd6440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 shutdown_connections 2024-09-10T23:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:29.604+0000 7fd64c02a640 1 -- 172.21.15.120:0/1531572354 wait complete. 2024-09-10T23:12:29.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:12:29.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:12:29.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181531 21.75 MB/sec execute 324 sec latency 455.606 ms 2024-09-10T23:12:30.079 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:12:30.568 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:30 smithi120 ceph-mon[31247]: pgmap v948: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 436 op/s 2024-09-10T23:12:30.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:30 smithi176 ceph-mon[34982]: pgmap v948: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 436 op/s 2024-09-10T23:12:30.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 -- 172.21.15.120:0/605894669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a4102da0 msgr2=0x7f23a4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/605894669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a4102da0 0x7f23a4103180 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f239402a410 comp rx=0 tx=0).stop 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 -- 172.21.15.120:0/605894669 shutdown_connections 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/605894669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 0x7f23a410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/605894669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a41087b0 0x7f23a4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/605894669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a4102da0 0x7f23a4103180 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 -- 172.21.15.120:0/605894669 >> 172.21.15.120:0/605894669 conn(0x7f23a40fe850 msgr2=0x7f23a4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 -- 172.21.15.120:0/605894669 shutdown_connections 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.668+0000 7f23aa3ef640 1 -- 172.21.15.120:0/605894669 wait complete. 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 Processor -- start 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 -- start start 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a4102da0 0x7f23a4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 0x7f23a419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4112df0 con 0x7f23a41087b0 2024-09-10T23:12:30.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f239bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f239bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59012/0 (socket says 172.21.15.120:59012) 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f239bfff640 1 -- 172.21.15.120:0/2514672322 learned_addr learned my addr 172.21.15.120:0/2514672322 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23a3fff640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a4102da0 0x7f23a4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23a898c640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 0x7f23a419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4112c70 con 0x7f23a4102da0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.669+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4112f70 con 0x7f23a4109130 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 -- 172.21.15.120:0/2514672322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 msgr2=0x7f23a419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 0x7f23a419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 -- 172.21.15.120:0/2514672322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a4102da0 msgr2=0x7f23a4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a4102da0 0x7f23a4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 -- 172.21.15.120:0/2514672322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a419f330 con 0x7f23a41087b0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f239bfff640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f238c00e9f0 tx=0x7f238c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f23a1ffb640 1 -- 172.21.15.120:0/2514672322 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c00cd80 con 0x7f23a41087b0 2024-09-10T23:12:30.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a419f620 con 0x7f23a41087b0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f23a1ffb640 1 -- 172.21.15.120:0/2514672322 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c0108e0 con 0x7f23a41087b0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f23a1ffb640 1 -- 172.21.15.120:0/2514672322 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c018980 con 0x7f23a41087b0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.670+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41a5eb0 con 0x7f23a41087b0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 Processor -- start 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 -- start start 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23a1ffb640 1 -- 172.21.15.120:0/2514672322 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f238c010430 con 0x7f23a41087b0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd34335e60 con 0x55bd343bf800 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd34325290 con 0x55bd343beee0 2024-09-10T23:12:30.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bd343290a0 con 0x55bd343cfba0 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59028/0 (socket says 172.21.15.120:59028) 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23a3fff640 1 -- 172.21.15.120:0/2756855997 learned_addr learned my addr 172.21.15.120:0/2756855997 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f239bfff640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.674+0000 7f23a898c640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 -- 172.21.15.120:0/2756855997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 msgr2=0x55bd343d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 -- 172.21.15.120:0/2756855997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 msgr2=0x55bd343bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 -- 172.21.15.120:0/2756855997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd34083930 con 0x55bd343bf800 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a898c640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a3fff640 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f2394032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec97ec4221da3f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a1ffb640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 0x7f23900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f239a7fc640 1 -- 172.21.15.120:0/2756855997 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394040070 con 0x55bd343bf800 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f239a7fc640 1 -- 172.21.15.120:0/2756855997 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394004030 con 0x55bd343bf800 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a898c640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 0x7f23900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f239a7fc640 1 -- 172.21.15.120:0/2756855997 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394035040 con 0x55bd343bf800 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.675+0000 7f23a1ffb640 1 -- 172.21.15.120:0/2514672322 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f238c0e5e90 con 0x7f23a41087b0 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 -- 172.21.15.120:0/2756855997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 msgr2=0x55bd343cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f2394032b20 comp rx=0 tx=0).stop 2024-09-10T23:12:30.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23a898c640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 0x7f23900b77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f239c009fd0 tx=0x7f239c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 -- 172.21.15.120:0/2756855997 shutdown_connections 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343bf800 0x55bd343cf660 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/2756855997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.676+0000 7f23ac7c4740 1 -- 172.21.15.120:0/2756855997 >> 172.21.15.120:0/2756855997 conn(0x55bd343bcd10 msgr2=0x55bd343bdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.677+0000 7f23ac7c4740 1 -- 172.21.15.120:0/2756855997 shutdown_connections 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.677+0000 7f23ac7c4740 1 -- 172.21.15.120:0/2756855997 wait complete. 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.677+0000 7f23ac7c4740 1 -- start start 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.677+0000 7f23ac7c4740 1 Processor -- start 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd34186890 con 0x55bd343cfba0 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55bd342bfd40 con 0x55bd343beee0 2024-09-10T23:12:30.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bd343d27f0 con 0x55bd343bf800 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23ac7c4740 10 client.0 _ll_register_callbacks cb 0x55bd34323c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23a898c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48450/0 (socket says 172.21.15.120:48450) 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23a898c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59044/0 (socket says 172.21.15.120:59044) 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 -- 172.21.15.120:0/1016517654 learned_addr learned my addr 172.21.15.120:0/1016517654 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23a3fff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 -- 172.21.15.120:0/1016517654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 msgr2=0x55bd343ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 -- 172.21.15.120:0/1016517654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 msgr2=0x55bd343cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f239bfff640 1 -- 172.21.15.120:0/1016517654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd343dbba0 con 0x55bd343bf800 2024-09-10T23:12:30.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.678+0000 7f23a898c640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f239bfff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f238c0a99a0 tx=0x7f238c0e6190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c43c9993bc6bed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0ac500 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2394004030 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f238c0abb90 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f238c0abea0 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 10 client.28804 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd343de5a0 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394004030 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f238c0e9020 con 0x55bd343bf800 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 10 client.28804 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 10 client.28804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:30.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 20 client.28804 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd343dfec0 0x55bd343e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 4 client.28804 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.679+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd34337db0 con 0x55bd343dfec0 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.680+0000 7f23a3fff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd343dfec0 0x55bd343e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.680+0000 7f23a3fff640 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd343dfec0 0x55bd343e22e0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755ccdc2ea2003ff server_cookie=413661d1adc7c48c in_seq=0 out_seq=0 2024-09-10T23:12:30.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.680+0000 7f23837fe640 10 client.28804 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.683+0000 7f23837fe640 1 -- 172.21.15.120:0/1016517654 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd34337db0 con 0x55bd343dfec0 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.683+0000 7f23837fe640 10 client.28804 handle_command_reply: tid=0 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.683+0000 7f23ac7c4740 1 client.28804 shutdown 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.683+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 msgr2=0x55bd343cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.683+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f238c0a99a0 tx=0x7f238c0e6190 comp rx=0 tx=0).stop 2024-09-10T23:12:30.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 shutdown_connections 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd343dfec0 0x55bd343e22e0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343bf800 0x55bd343cd040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/1016517654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343beee0 0x55bd343ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 >> 172.21.15.120:0/1016517654 conn(0x55bd343bcd10 msgr2=0x55bd343bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23837fe640 0 client.28804 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 shutdown_connections 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 1 -- 172.21.15.120:0/1016517654 wait complete. 2024-09-10T23:12:30.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.684+0000 7f23ac7c4740 20 client.28804 trim_cache size 0 max 16384 2024-09-10T23:12:30.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:30 smithi153 ceph-mon[34951]: pgmap v948: 97 pgs: 97 active+clean; 242 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 436 op/s 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 Processor -- start 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 -- start start 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41aa130 con 0x55bd343beee0 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41ac9b0 con 0x55bd343bf800 2024-09-10T23:12:30.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41aa7a0 con 0x55bd343cfba0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59046/0 (socket says 172.21.15.120:59046) 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.706+0000 7f23a3fff640 1 -- 172.21.15.120:0/3580201870 learned_addr learned my addr 172.21.15.120:0/3580201870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a898c640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f239bfff640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 -- 172.21.15.120:0/3580201870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 msgr2=0x55bd343cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 -- 172.21.15.120:0/3580201870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 msgr2=0x55bd343cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 -- 172.21.15.120:0/3580201870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd343cb000 con 0x55bd343beee0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f239bfff640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f23a3fff640 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f2394000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0e8740df6a1ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f2399ffb640 1 -- 172.21.15.120:0/3580201870 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394040050 con 0x55bd343beee0 2024-09-10T23:12:30.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f2399ffb640 1 -- 172.21.15.120:0/3580201870 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23940314a0 con 0x55bd343beee0 2024-09-10T23:12:30.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.707+0000 7f2399ffb640 1 -- 172.21.15.120:0/3580201870 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394035040 con 0x55bd343beee0 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- 172.21.15.120:0/3580201870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 msgr2=0x55bd343cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f2394000bc0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- 172.21.15.120:0/3580201870 shutdown_connections 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343cfba0 0x55bd343cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/3580201870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343beee0 0x55bd343cd000 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- 172.21.15.120:0/3580201870 >> 172.21.15.120:0/3580201870 conn(0x55bd343bcd10 msgr2=0x55bd343d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.713 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- 172.21.15.120:0/3580201870 shutdown_connections 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- 172.21.15.120:0/3580201870 wait complete. 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.708+0000 7f23ac7c4740 1 -- start start 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 Processor -- start 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343beee0 0x55bd343d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41aae70 con 0x55bd343cfba0 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41ac2e0 con 0x55bd343bf800 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41ab540 con 0x55bd343beee0 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23a898c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23a898c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59054/0 (socket says 172.21.15.120:59054) 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23a3fff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343beee0 0x55bd343d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.714 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23ac7c4740 10 client.0 _ll_register_callbacks cb 0x7f238c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23a898c640 1 -- 172.21.15.120:0/939194678 learned_addr learned my addr 172.21.15.120:0/939194678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 -- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343beee0 msgr2=0x55bd343d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343beee0 0x55bd343d8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 -- 172.21.15.120:0/939194678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 msgr2=0x55bd343e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 -- 172.21.15.120:0/939194678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3444bdf0 con 0x55bd343bf800 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23a898c640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f239bfff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343e79a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f238c0a8d00 tx=0x7f238c018dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf2bd119eeeb3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.709+0000 7f23837fe640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0ac080 con 0x55bd343bf800 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23940314a0 con 0x55bd343bf800 2024-09-10T23:12:30.715 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f238c0a8280 con 0x55bd343bf800 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f238c0ac500 con 0x55bd343bf800 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 10 client.28559 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd3444d040 con 0x55bd343bf800 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23940314a0 con 0x55bd343bf800 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f238c0b22f0 con 0x55bd343bf800 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 10 client.28559 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 10 client.28559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 20 client.28559 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd3444d5d0 0x55bd3444f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 4 client.28559 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd34333860 con 0x55bd3444d5d0 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.710+0000 7f23a3fff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd3444d5d0 0x55bd3444f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:30.716 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.711+0000 7f23837fe640 10 client.28559 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:30.717 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.711+0000 7f23a3fff640 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd3444d5d0 0x55bd3444f9f0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbaa5fe00e52b826 server_cookie=c7c59628daf77e9c in_seq=0 out_seq=0 2024-09-10T23:12:30.723 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23837fe640 1 -- 172.21.15.120:0/939194678 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd34333860 con 0x55bd3444d5d0 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23837fe640 10 client.28559 handle_command_reply: tid=0 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 client.28559 shutdown 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 msgr2=0x55bd343e79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343e79a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f238c0a8d00 tx=0x7f238c018dd0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 shutdown_connections 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55bd3444d5d0 0x55bd3444f9f0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd343cfba0 0x55bd343e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bd343bf800 0x55bd343e79a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23837fe640 0 client.28559 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 --2- 172.21.15.120:0/939194678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd343beee0 0x55bd343d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 >> 172.21.15.120:0/939194678 conn(0x55bd343bcd10 msgr2=0x55bd343cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 shutdown_connections 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.722+0000 7f23ac7c4740 1 -- 172.21.15.120:0/939194678 wait complete. 2024-09-10T23:12:30.724 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.723+0000 7f23ac7c4740 20 client.28559 trim_cache size 0 max 16384 2024-09-10T23:12:30.725 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:12:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:12:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "15d1876e-977a-45d9-8082-947af36d1a52", 2024-09-10T23:12:30.725 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:12:30.725 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.723+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 msgr2=0x7f23900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.723+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 0x7f23900b77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f239c009fd0 tx=0x7f239c009500 comp rx=0 tx=0).stop 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.723+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 msgr2=0x7f23a41997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.723+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f238c00e9f0 tx=0x7f238c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 shutdown_connections 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23900b52e0 0x7f23900b77a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23a4109130 0x7f23a419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23a41087b0 0x7f23a41997c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.726 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 --2- 172.21.15.120:0/2514672322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23a4102da0 0x7f23a4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:30.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 >> 172.21.15.120:0/2514672322 conn(0x7f23a40fe850 msgr2=0x7f23a4100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:30.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 shutdown_connections 2024-09-10T23:12:30.727 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:30.724+0000 7f23aa3ef640 1 -- 172.21.15.120:0/2514672322 wait complete. 2024-09-10T23:12:30.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 181982 21.74 MB/sec execute 325 sec latency 68.087 ms 2024-09-10T23:12:31.461 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:12:31.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:31 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:12:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:31.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:31 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:12:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:31 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:12:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:31.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:31 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:12:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:31 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:12:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:31 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:12:31.769 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:12:31.882 INFO:tasks.workunit.client.0.smithi120.stdout: 10 182139 21.70 MB/sec execute 326 sec latency 305.640 ms 2024-09-10T23:12:32.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 -- 172.21.15.120:0/3717779954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0102400 msgr2=0x7faad01027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 --2- 172.21.15.120:0/3717779954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0102400 0x7faad01027e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7faac0009900 tx=0x7faac002a410 comp rx=0 tx=0).stop 2024-09-10T23:12:32.318 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 -- 172.21.15.120:0/3717779954 shutdown_connections 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 --2- 172.21.15.120:0/3717779954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 --2- 172.21.15.120:0/3717779954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad01044a0 0x7faad0104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 --2- 172.21.15.120:0/3717779954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad0102400 0x7faad01027e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 -- 172.21.15.120:0/3717779954 >> 172.21.15.120:0/3717779954 conn(0x7faad0078140 msgr2=0x7faad00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 -- 172.21.15.120:0/3717779954 shutdown_connections 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.315+0000 7faad4e19640 1 -- 172.21.15.120:0/3717779954 wait complete. 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 Processor -- start 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 -- start start 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad0102400 0x7faad01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad0111c80 con 0x7faad01044a0 2024-09-10T23:12:32.319 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad0111b00 con 0x7faad0104e40 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faad4e19640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faad0111e00 con 0x7faad0102400 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faacdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faacdd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59076/0 (socket says 172.21.15.120:59076) 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faaced9d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faacdd9b640 1 -- 172.21.15.120:0/3609377275 learned_addr learned my addr 172.21.15.120:0/3609377275 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.316+0000 7faace59c640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad0102400 0x7faad01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 -- 172.21.15.120:0/3609377275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad0102400 msgr2=0x7faad01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.320 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad0102400 0x7faad01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 -- 172.21.15.120:0/3609377275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 msgr2=0x7faad019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 -- 172.21.15.120:0/3609377275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad019f350 con 0x7faad01044a0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faaced9d640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faabf7fe640 1 -- 172.21.15.120:0/3609377275 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400cd80 con 0x7faad01044a0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faabf7fe640 1 -- 172.21.15.120:0/3609377275 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac40108e0 con 0x7faad01044a0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faabf7fe640 1 -- 172.21.15.120:0/3609377275 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4018980 con 0x7faad01044a0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad019f640 con 0x7faad01044a0 2024-09-10T23:12:32.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.317+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a5d00 con 0x7faad01044a0 2024-09-10T23:12:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 Processor -- start 2024-09-10T23:12:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 -- start start 2024-09-10T23:12:32.329 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b827e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b827ed60 0x5610b8281150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610b81e4e50 con 0x5610b826e9c0 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5610b81d4280 con 0x5610b827ed60 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.320+0000 7faad71ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5610b81d8090 con 0x5610b826e0a0 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:48480/0 (socket says 172.21.15.120:48480) 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 -- 172.21.15.120:0/486356588 learned_addr learned my addr 172.21.15.120:0/486356588 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faace59c640 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b827e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faabf7fe640 1 -- 172.21.15.120:0/3609377275 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7faac4010430 con 0x7faad01044a0 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 -- 172.21.15.120:0/486356588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b827ed60 msgr2=0x5610b8281150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b827ed60 0x5610b8281150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.330 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 -- 172.21.15.120:0/486356588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 msgr2=0x5610b827e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b827e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 -- 172.21.15.120:0/486356588 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b7f32650 con 0x5610b826e0a0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.321+0000 7faaced9d640 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faac8011820 tx=0x7faac8011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d65418ad7cf356fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faabcff9640 1 -- 172.21.15.120:0/486356588 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80112f0 con 0x5610b826e0a0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faabcff9640 1 -- 172.21.15.120:0/486356588 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8003e20 con 0x5610b826e0a0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faabcff9640 1 -- 172.21.15.120:0/486356588 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac80044c0 con 0x5610b826e0a0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faabf7fe640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 0x7faab80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faabf7fe640 1 -- 172.21.15.120:0/3609377275 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7faac40e5e90 con 0x7faad01044a0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.322+0000 7faace59c640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 0x7faab80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 -- 172.21.15.120:0/486356588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 msgr2=0x5610b826e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faac8011820 tx=0x7faac8011ce0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 -- 172.21.15.120:0/486356588 shutdown_connections 2024-09-10T23:12:32.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b827ed60 0x5610b8281150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b827e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 --2- 172.21.15.120:0/486356588 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b826e0a0 0x5610b826e480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 -- 172.21.15.120:0/486356588 >> 172.21.15.120:0/486356588 conn(0x5610b826bed0 msgr2=0x5610b826d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faace59c640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 0x7faab80b77a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faac0031440 tx=0x7faac0005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 -- 172.21.15.120:0/486356588 shutdown_connections 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.323+0000 7faad71ee740 1 -- 172.21.15.120:0/486356588 wait complete. 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 -- start start 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 Processor -- start 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b827e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b827bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b827ed60 0x5610b827c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b827e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610b8035700 con 0x5610b826e0a0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5610b816f110 con 0x5610b826e9c0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b827e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59088/0 (socket says 172.21.15.120:59088) 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5610b8281a60 con 0x5610b827ed60 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faace59c640 1 -- 172.21.15.120:0/3338366080 learned_addr learned my addr 172.21.15.120:0/3338366080 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b827bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faad71ee740 10 client.0 _ll_register_callbacks cb 0x5610b8188990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 -- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b827ed60 msgr2=0x5610b827c210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b827ed60 0x5610b827c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 -- 172.21.15.120:0/3338366080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 msgr2=0x5610b827e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b827e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.324+0000 7faacdd9b640 1 -- 172.21.15.120:0/3338366080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b819cc40 con 0x5610b826e9c0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faacdd9b640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b827bb40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5610b826e050 tx=0x7faac40ac080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8d85cd2a614855 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40abb90 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac8003e20 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7faac4018b20 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faac40ac730 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 10 client.28568 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610b8275700 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac8003e20 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7faac40ab400 con 0x5610b826e9c0 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 10 client.28568 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 10 client.28568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.325+0000 7faad71ee740 20 client.28568 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b8276b10 0x5610b8276ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faad71ee740 4 client.28568 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b81e6da0 con 0x5610b8276b10 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faace59c640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b8276b10 0x5610b8276ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faab5ffb640 10 client.28568 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.326+0000 7faace59c640 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b8276b10 0x5610b8276ef0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c63a02c2fc57e53 server_cookie=58dc3972e10fd5fd in_seq=0 out_seq=0 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.327+0000 7faab5ffb640 1 -- 172.21.15.120:0/3338366080 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610b81e6da0 con 0x5610b8276b10 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.327+0000 7faab5ffb640 10 client.28568 handle_command_reply: tid=0 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.327+0000 7faad71ee740 1 client.28568 shutdown 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.327+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 msgr2=0x5610b827bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.327+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b827bb40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5610b826e050 tx=0x7faac40ac080 comp rx=0 tx=0).stop 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 shutdown_connections 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b8276b10 0x5610b8276ef0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b827ed60 0x5610b827c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faab5ffb640 0 client.28568 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b827bb40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 --2- 172.21.15.120:0/3338366080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b827e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 >> 172.21.15.120:0/3338366080 conn(0x5610b826bed0 msgr2=0x5610b8271a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 shutdown_connections 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 1 -- 172.21.15.120:0/3338366080 wait complete. 2024-09-10T23:12:32.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.328+0000 7faad71ee740 20 client.28568 trim_cache size 0 max 16384 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 Processor -- start 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 -- start start 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b828ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b82711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad01aa0e0 con 0x5610b826e0a0 2024-09-10T23:12:32.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad01ac960 con 0x5610b826e9c0 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faad71ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faad01aa750 con 0x5610b8276b10 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faace59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:59090/0 (socket says 172.21.15.120:59090) 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faace59c640 1 -- 172.21.15.120:0/1255033106 learned_addr learned my addr 172.21.15.120:0/1255033106 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.354+0000 7faacdd9b640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b828ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faaced9d640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b82711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 -- 172.21.15.120:0/1255033106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 msgr2=0x5610b82711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b82711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 -- 172.21.15.120:0/1255033106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 msgr2=0x5610b828ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b828ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 -- 172.21.15.120:0/1255033106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b8277260 con 0x5610b826e0a0 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faacdd9b640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b828ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:32.357 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faace59c640 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7faac0006010 tx=0x7faac003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79c307f5ff9bad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faab7fff640 1 -- 172.21.15.120:0/1255033106 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0056070 con 0x5610b826e0a0 2024-09-10T23:12:32.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faab7fff640 1 -- 172.21.15.120:0/1255033106 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac0005040 con 0x5610b826e0a0 2024-09-10T23:12:32.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.355+0000 7faab7fff640 1 -- 172.21.15.120:0/1255033106 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0035040 con 0x5610b826e0a0 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 -- 172.21.15.120:0/1255033106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 msgr2=0x5610b826d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7faac0006010 tx=0x7faac003f000 comp rx=0 tx=0).stop 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 -- 172.21.15.120:0/1255033106 shutdown_connections 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b82711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e9c0 0x5610b828ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 --2- 172.21.15.120:0/1255033106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e0a0 0x5610b826d710 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 -- 172.21.15.120:0/1255033106 >> 172.21.15.120:0/1255033106 conn(0x5610b826bed0 msgr2=0x5610b826d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.356+0000 7faad71ee740 1 -- 172.21.15.120:0/1255033106 shutdown_connections 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 -- 172.21.15.120:0/1255033106 wait complete. 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 -- start start 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 Processor -- start 2024-09-10T23:12:32.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b82907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b8290d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faad01aae20 con 0x5610b826e9c0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faad01ac290 con 0x5610b826e0a0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faad01ab4f0 con 0x5610b8276b10 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.358+0000 7faad71ee740 10 client.0 _ll_register_callbacks cb 0x7faac400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faace59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:45282/0 (socket says 172.21.15.120:45282) 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faace59c640 1 -- 172.21.15.120:0/4108893519 learned_addr learned my addr 172.21.15.120:0/4108893519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b82907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faaced9d640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b8290d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 -- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 msgr2=0x5610b8290d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b8290d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 -- 172.21.15.120:0/4108893519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 msgr2=0x5610b8290280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 -- 172.21.15.120:0/4108893519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b82f6570 con 0x5610b826e9c0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faace59c640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faacdd9b640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b82907c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7faac40ac6c0 tx=0x7faac4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36a5486660059ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40a8d80 con 0x5610b826e9c0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7faac4018b20 con 0x5610b826e9c0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.359+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac0005040 con 0x5610b826e9c0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faac40b1160 con 0x5610b826e9c0 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 10 client.19377 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:32.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610b82f7880 con 0x5610b826e9c0 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac0005040 con 0x5610b826e9c0 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7faac40c5050 con 0x5610b826e9c0 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 10 client.19377 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 10 client.19377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 20 client.19377 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b827ed60 0x5610b82f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 4 client.19377 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610b81e2850 con 0x5610b827ed60 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.360+0000 7faace59c640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b827ed60 0x5610b82f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.361+0000 7faab5ffb640 10 client.19377 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:32.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.361+0000 7faace59c640 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b827ed60 0x5610b82f9e10 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1a98bc79dc00ea server_cookie=ff0efe16665ecdc4 in_seq=0 out_seq=0 2024-09-10T23:12:32.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faab5ffb640 1 -- 172.21.15.120:0/4108893519 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9948 (crc 0 0 0) 0x5610b81e2850 con 0x5610b827ed60 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faab5ffb640 10 client.19377 handle_command_reply: tid=0 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 client.19377 shutdown 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 msgr2=0x5610b82907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b82907c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7faac40ac6c0 tx=0x7faac4000bc0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 shutdown_connections 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5610b827ed60 0x5610b82f9e10 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faab5ffb640 0 client.19377 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5610b8276b10 0x5610b8290d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610b826e9c0 0x5610b82907c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 --2- 172.21.15.120:0/4108893519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5610b826e0a0 0x5610b8290280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.363+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 >> 172.21.15.120:0/4108893519 conn(0x5610b826bed0 msgr2=0x5610b82711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 shutdown_connections 2024-09-10T23:12:32.367 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad71ee740 1 -- 172.21.15.120:0/4108893519 wait complete. 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad71ee740 20 client.19377 trim_cache size 0 max 16384 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "15d1876e-977a-45d9-8082-947af36d1a52", 2024-09-10T23:12:32.368 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:12:32.369 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:12:32.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:12:32.375 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:12:32.378 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.381 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:12:32.382 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.384 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:12:32.385 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.386 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:12:32.388 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 msgr2=0x7faab80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 0x7faab80b77a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faac0031440 tx=0x7faac0005b90 comp rx=0 tx=0).stop 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 msgr2=0x7faad01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7faac400e9f0 tx=0x7faac400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 shutdown_connections 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7faab80b52e0 0x7faab80b77a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.389 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faad0104e40 0x7faad019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faad01044a0 0x7faad01997e0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 --2- 172.21.15.120:0/3609377275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faad0102400 0x7faad01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 >> 172.21.15.120:0/3609377275 conn(0x7faad0078140 msgr2=0x7faad00fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 shutdown_connections 2024-09-10T23:12:32.390 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:12:32.364+0000 7faad4e19640 1 -- 172.21.15.120:0/3609377275 wait complete. 2024-09-10T23:12:32.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:32 smithi176 ceph-mon[34982]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-09-10T23:12:32.599 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:32 smithi153 ceph-mon[34951]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-09-10T23:12:32.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:32 smithi120 ceph-mon[31247]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-09-10T23:12:32.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 182365 21.66 MB/sec execute 327 sec latency 448.515 ms 2024-09-10T23:12:33.352 INFO:tasks.cephfs.filesystem:scrub status for tag:15d1876e-977a-45d9-8082-947af36d1a52 - {'path': '/', 'tag': '15d1876e-977a-45d9-8082-947af36d1a52', 'options': 'recursive,force'} 2024-09-10T23:12:33.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:33 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:33 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:33 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:12:33.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 182821 21.67 MB/sec execute 328 sec latency 217.103 ms 2024-09-10T23:12:34.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:34 smithi176 ceph-mon[34982]: pgmap v950: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-09-10T23:12:34.596 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:34 smithi120 ceph-mon[31247]: pgmap v950: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-09-10T23:12:34.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:34 smithi153 ceph-mon[34951]: pgmap v950: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-09-10T23:12:34.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 183306 21.69 MB/sec execute 329 sec latency 85.494 ms 2024-09-10T23:12:35.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:12:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:12:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:12:35.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 183818 21.73 MB/sec execute 330 sec latency 86.757 ms 2024-09-10T23:12:36.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:36 smithi120 ceph-mon[31247]: pgmap v951: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-09-10T23:12:36.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:36 smithi153 ceph-mon[34951]: pgmap v951: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-09-10T23:12:36.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:36 smithi176 ceph-mon[34982]: pgmap v951: 97 pgs: 97 active+clean; 229 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-09-10T23:12:36.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 184230 21.76 MB/sec execute 331 sec latency 73.225 ms 2024-09-10T23:12:37.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 184562 21.73 MB/sec execute 332 sec latency 121.419 ms 2024-09-10T23:12:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:38 smithi120 ceph-mon[31247]: pgmap v952: 97 pgs: 97 active+clean; 233 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 677 op/s 2024-09-10T23:12:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:38 smithi153 ceph-mon[34951]: pgmap v952: 97 pgs: 97 active+clean; 233 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 677 op/s 2024-09-10T23:12:38.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:38 smithi176 ceph-mon[34982]: pgmap v952: 97 pgs: 97 active+clean; 233 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 677 op/s 2024-09-10T23:12:38.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 184792 21.70 MB/sec execute 333 sec latency 355.031 ms 2024-09-10T23:12:39.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 185107 21.69 MB/sec execute 334 sec latency 85.359 ms 2024-09-10T23:12:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:40 smithi120 ceph-mon[31247]: pgmap v953: 97 pgs: 97 active+clean; 235 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-09-10T23:12:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:40 smithi153 ceph-mon[34951]: pgmap v953: 97 pgs: 97 active+clean; 235 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-09-10T23:12:40.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:40 smithi176 ceph-mon[34982]: pgmap v953: 97 pgs: 97 active+clean; 235 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 633 op/s 2024-09-10T23:12:40.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 185569 21.67 MB/sec execute 335 sec latency 103.905 ms 2024-09-10T23:12:41.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 186093 21.70 MB/sec execute 336 sec latency 58.749 ms 2024-09-10T23:12:42.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:42 smithi120 ceph-mon[31247]: pgmap v954: 97 pgs: 97 active+clean; 239 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T23:12:42.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:42 smithi153 ceph-mon[34951]: pgmap v954: 97 pgs: 97 active+clean; 239 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T23:12:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.703+0000 7fa5d35f8640 1 -- 172.21.15.120:0/2076241228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 msgr2=0x7fa5cc10ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.703+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/2076241228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc10ab50 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ca0 tx=0x7fa5c402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- 172.21.15.120:0/2076241228 shutdown_connections 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/2076241228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc10ab50 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/2076241228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc105b20 0x7fa5cc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/2076241228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- 172.21.15.120:0/2076241228 >> 172.21.15.120:0/2076241228 conn(0x7fa5cc0fd150 msgr2=0x7fa5cc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- 172.21.15.120:0/2076241228 shutdown_connections 2024-09-10T23:12:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- 172.21.15.120:0/2076241228 wait complete. 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 Processor -- start 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- start start 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc105b20 0x7fa5cc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10e600 con 0x7fa5cc108690 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10e480 con 0x7fa5cc105b20 2024-09-10T23:12:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.704+0000 7fa5d35f8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10e780 con 0x7fa5cc06bf10 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc105b20 0x7fa5cc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44678/0 (socket says 172.21.15.120:44678) 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1394640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 -- 172.21.15.120:0/1558843145 learned_addr learned my addr 172.21.15.120:0/1558843145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 -- 172.21.15.120:0/1558843145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 msgr2=0x7fa5cc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 -- 172.21.15.120:0/1558843145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc105b20 msgr2=0x7fa5cc199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc105b20 0x7fa5cc199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 -- 172.21.15.120:0/1558843145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19a870 con 0x7fa5cc108690 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1394640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d1b95640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40097b0 tx=0x7fa5c4031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5c27fc640 1 -- 172.21.15.120:0/1558843145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003b70 con 0x7fa5cc108690 2024-09-10T23:12:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5c27fc640 1 -- 172.21.15.120:0/1558843145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5c4003d10 con 0x7fa5cc108690 2024-09-10T23:12:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19ab00 con 0x7fa5cc108690 2024-09-10T23:12:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5c27fc640 1 -- 172.21.15.120:0/1558843145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c402fc10 con 0x7fa5cc108690 2024-09-10T23:12:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.705+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a1230 con 0x7fa5cc108690 2024-09-10T23:12:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.708+0000 7fa5c27fc640 1 -- 172.21.15.120:0/1558843145 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa5c40305c0 con 0x7fa5cc108690 2024-09-10T23:12:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.708+0000 7fa5d63cd740 1 Processor -- start 2024-09-10T23:12:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 -- start start 2024-09-10T23:12:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612af20f6f0 con 0x5612af29d860 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612af1feb20 con 0x5612af2a2bd0 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5612af202930 con 0x5612af2a0190 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44690/0 (socket says 172.21.15.120:44690) 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 -- 172.21.15.120:0/957053310 learned_addr learned my addr 172.21.15.120:0/957053310 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d1394640 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5c27fc640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 0x7fa5b80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 -- 172.21.15.120:0/957053310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 msgr2=0x5612af2a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 -- 172.21.15.120:0/957053310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 msgr2=0x5612af2a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 -- 172.21.15.120:0/957053310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612af1c4530 con 0x5612af29d860 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d1b95640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 0x7fa5b80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009710 tx=0x7fa5bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e72f74ea1e4bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.709+0000 7fa5c27fc640 1 -- 172.21.15.120:0/1558843145 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa5c4106ec0 con 0x7fa5cc108690 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5b77fe640 1 -- 172.21.15.120:0/957053310 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc013e50 con 0x5612af29d860 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5b77fe640 1 -- 172.21.15.120:0/957053310 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc013120 con 0x5612af29d860 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5b77fe640 1 -- 172.21.15.120:0/957053310 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0035f0 con 0x5612af29d860 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 -- 172.21.15.120:0/957053310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 msgr2=0x5612af29fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d1b95640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 0x7fa5b80b7820 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40c9c80 tx=0x7fa5c404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009710 tx=0x7fa5bc009780 comp rx=0 tx=0).stop 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 -- 172.21.15.120:0/957053310 shutdown_connections 2024-09-10T23:12:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/957053310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29fc50 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.710+0000 7fa5d63cd740 1 -- 172.21.15.120:0/957053310 >> 172.21.15.120:0/957053310 conn(0x5612af2989b0 msgr2=0x5612af299740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- 172.21.15.120:0/957053310 shutdown_connections 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- 172.21.15.120:0/957053310 wait complete. 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- start start 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 Processor -- start 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af2a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612af136ab0 con 0x5612af2a2bd0 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5612af1a4730 con 0x5612af29d860 2024-09-10T23:12:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5612af2a5b50 con 0x5612af2a0190 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d63cd740 10 client.0 _ll_register_callbacks cb 0x5612af1f86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af2a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d1394640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51194/0 (socket says 172.21.15.120:51194) 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.711+0000 7fa5d0b93640 1 -- 172.21.15.120:0/1894824481 learned_addr learned my addr 172.21.15.120:0/1894824481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 -- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 msgr2=0x5612af2a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 -- 172.21.15.120:0/1894824481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 msgr2=0x5612af2a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af2a8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 -- 172.21.15.120:0/1894824481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612aef60790 con 0x5612af29d860 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0096e0 tx=0x7fa5bc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649764a68c45fb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc024070 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa5bc0050f0 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5bc013120 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5bc0052f0 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d63cd740 10 client.28586 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612aefd9620 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.712+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc013120 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa5bc010760 con 0x5612af29d860 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 10 client.28586 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 10 client.28586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 20 client.28586 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af2b24a0 0x5612af2b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 4 client.28586 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612af211640 con 0x5612af2b24a0 2024-09-10T23:12:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d1394640 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af2b24a0 0x5612af2b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5b4ff9640 10 client.28586 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.713+0000 7fa5d1394640 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af2b24a0 0x5612af2b4890 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e7b3ff52161c6d server_cookie=6d2d742ad1974daa in_seq=0 out_seq=0 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/1894824481 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612af211640 con 0x5612af2b24a0 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5b4ff9640 10 client.28586 handle_command_reply: tid=0 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 client.28586 shutdown 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 msgr2=0x5612af2a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0096e0 tx=0x7fa5bc005850 comp rx=0 tx=0).stop 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 shutdown_connections 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af2b24a0 0x5612af2b4890 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af2a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5b4ff9640 0 client.28586 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/1894824481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af29d860 0x5612af2a6ed0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 >> 172.21.15.120:0/1894824481 conn(0x5612af2989b0 msgr2=0x5612af297bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 shutdown_connections 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 1 -- 172.21.15.120:0/1894824481 wait complete. 2024-09-10T23:12:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.718+0000 7fa5d63cd740 20 client.28586 trim_cache size 0 max 16384 2024-09-10T23:12:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 Processor -- start 2024-09-10T23:12:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 -- start start 2024-09-10T23:12:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1a7d80 con 0x5612af29d860 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1aa600 con 0x5612af2a2bd0 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1a83f0 con 0x5612af2a0190 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:44706/0 (socket says 172.21.15.120:44706) 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 -- 172.21.15.120:0/2008777006 learned_addr learned my addr 172.21.15.120:0/2008777006 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d1394640 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 -- 172.21.15.120:0/2008777006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 msgr2=0x5612af2b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 -- 172.21.15.120:0/2008777006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 msgr2=0x5612af2b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.735+0000 7fa5d0b93640 1 -- 172.21.15.120:0/2008777006 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612af2a7240 con 0x5612af29d860 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00f790 tx=0x7fa5bc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d2d144e828eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5b6ffd640 1 -- 172.21.15.120:0/2008777006 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc024070 con 0x5612af29d860 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5b6ffd640 1 -- 172.21.15.120:0/2008777006 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc009780 con 0x5612af29d860 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5b6ffd640 1 -- 172.21.15.120:0/2008777006 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc003850 con 0x5612af29d860 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5d63cd740 1 -- 172.21.15.120:0/2008777006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 msgr2=0x5612af29bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.736+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00f790 tx=0x7fa5bc01c040 comp rx=0 tx=0).stop 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- 172.21.15.120:0/2008777006 shutdown_connections 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a2bd0 0x5612af2b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af2a0190 0x5612af2b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/2008777006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af29d860 0x5612af29bf00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- 172.21.15.120:0/2008777006 >> 172.21.15.120:0/2008777006 conn(0x5612af2989b0 msgr2=0x5612af29a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- 172.21.15.120:0/2008777006 shutdown_connections 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- 172.21.15.120:0/2008777006 wait complete. 2024-09-10T23:12:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- start start 2024-09-10T23:12:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 Processor -- start 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af29d860 0x5612af2ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af31d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1a8ac0 con 0x5612af2a2bd0 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1a9f30 con 0x5612af2a0190 2024-09-10T23:12:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1a9190 con 0x5612af29d860 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d63cd740 10 client.0 _ll_register_callbacks cb 0x7fa5bc009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af31d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.737+0000 7fa5d1394640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d0b93640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af29d860 0x5612af2ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:51216/0 (socket says 172.21.15.120:51216) 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 -- 172.21.15.120:0/3728109152 learned_addr learned my addr 172.21.15.120:0/3728109152 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 -- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af29d860 msgr2=0x5612af2ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af29d860 0x5612af2ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 -- 172.21.15.120:0/3728109152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 msgr2=0x5612af31d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af31d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 -- 172.21.15.120:0/3728109152 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612af2b4c00 con 0x5612af2a0190 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d1394640 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5612af2b5090 tx=0x7fa5c8012c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e8ca466ccefd80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8010c90 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa5c8025440 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5bc009780 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5c8010e30 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d63cd740 10 client.28592 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.738+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612af2a84b0 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc009780 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa5c802e540 con 0x5612af2a0190 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 10 client.28592 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 10 client.28592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 20 client.28592 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af31fa60 0x5612af321e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 4 client.28592 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:12:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612af20d0f0 con 0x5612af31fa60 2024-09-10T23:12:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af31fa60 0x5612af321e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5b4ff9640 10 client.28592 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:42.739+0000 7fa5d0b93640 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af31fa60 0x5612af321e80 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=470cc38391be1dfd server_cookie=449b4e1c4dde3510 in_seq=0 out_seq=0 2024-09-10T23:12:42.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:42 smithi176 ceph-mon[34982]: pgmap v954: 97 pgs: 97 active+clean; 239 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-10T23:12:42.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 186669 21.71 MB/sec execute 337 sec latency 112.840 ms 2024-09-10T23:12:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.613+0000 7fa5b4ff9640 1 -- 172.21.15.120:0/3728109152 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612af20d0f0 con 0x5612af31fa60 2024-09-10T23:12:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.613+0000 7fa5b4ff9640 10 client.28592 handle_command_reply: tid=0 2024-09-10T23:12:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.613+0000 7fa5d63cd740 1 client.28592 shutdown 2024-09-10T23:12:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.613+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 msgr2=0x5612af31b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.613+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5612af2b5090 tx=0x7fa5c8012c40 comp rx=0 tx=0).stop 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 shutdown_connections 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5612af31fa60 0x5612af321e80 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5b4ff9640 0 client.28592 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612af2a2bd0 0x5612af31d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5612af2a0190 0x5612af31b330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 --2- 172.21.15.120:0/3728109152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612af29d860 0x5612af2ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 >> 172.21.15.120:0/3728109152 conn(0x5612af2989b0 msgr2=0x5612af29b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 shutdown_connections 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 1 -- 172.21.15.120:0/3728109152 wait complete. 2024-09-10T23:12:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.614+0000 7fa5d63cd740 20 client.28592 trim_cache size 0 max 16384 2024-09-10T23:12:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:12:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:12:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:12:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 msgr2=0x7fa5b80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 0x7fa5b80b7820 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40c9c80 tx=0x7fa5c404a000 comp rx=0 tx=0).stop 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 msgr2=0x7fa5cc199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40097b0 tx=0x7fa5c4031770 comp rx=0 tx=0).stop 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 shutdown_connections 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa5b80b5360 0x7fa5b80b7820 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa5cc108690 0x7fa5cc199fb0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5cc105b20 0x7fa5cc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 --2- 172.21.15.120:0/1558843145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5cc06bf10 0x7fa5cc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 >> 172.21.15.120:0/1558843145 conn(0x7fa5cc0fd150 msgr2=0x7fa5cc103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 shutdown_connections 2024-09-10T23:12:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:43.615+0000 7fa5d35f8640 1 -- 172.21.15.120:0/1558843145 wait complete. 2024-09-10T23:12:43.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 186801 21.67 MB/sec execute 338 sec latency 389.634 ms 2024-09-10T23:12:44.646 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:44 smithi120 ceph-mon[31247]: pgmap v955: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T23:12:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:44 smithi153 ceph-mon[34951]: pgmap v955: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T23:12:44.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:44 smithi176 ceph-mon[34982]: pgmap v955: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T23:12:44.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 187282 21.71 MB/sec execute 339 sec latency 72.370 ms 2024-09-10T23:12:45.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 187778 21.74 MB/sec execute 340 sec latency 43.078 ms 2024-09-10T23:12:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:46 smithi120 ceph-mon[31247]: pgmap v956: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-09-10T23:12:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:46 smithi153 ceph-mon[34951]: pgmap v956: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-09-10T23:12:46.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:46 smithi176 ceph-mon[34982]: pgmap v956: 97 pgs: 97 active+clean; 241 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 402 op/s 2024-09-10T23:12:46.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 188133 21.72 MB/sec execute 341 sec latency 100.307 ms 2024-09-10T23:12:47.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 188442 21.69 MB/sec execute 342 sec latency 93.999 ms 2024-09-10T23:12:48.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:48 smithi120 ceph-mon[31247]: pgmap v957: 97 pgs: 97 active+clean; 236 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-10T23:12:48.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:48 smithi153 ceph-mon[34951]: pgmap v957: 97 pgs: 97 active+clean; 236 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-10T23:12:48.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:48 smithi176 ceph-mon[34982]: pgmap v957: 97 pgs: 97 active+clean; 236 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-10T23:12:48.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 188712 21.68 MB/sec execute 343 sec latency 245.466 ms 2024-09-10T23:12:49.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 189107 21.66 MB/sec execute 344 sec latency 185.257 ms 2024-09-10T23:12:50.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:50 smithi120 ceph-mon[31247]: pgmap v958: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-10T23:12:50.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:50 smithi153 ceph-mon[34951]: pgmap v958: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-10T23:12:50.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:50 smithi176 ceph-mon[34982]: pgmap v958: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-10T23:12:50.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 189603 21.68 MB/sec execute 345 sec latency 68.600 ms 2024-09-10T23:12:51.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 190227 21.70 MB/sec execute 346 sec latency 106.573 ms 2024-09-10T23:12:52.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:52 smithi120 ceph-mon[31247]: pgmap v959: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-10T23:12:52.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:52 smithi153 ceph-mon[34951]: pgmap v959: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-10T23:12:52.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:52 smithi176 ceph-mon[34982]: pgmap v959: 97 pgs: 97 active+clean; 232 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-09-10T23:12:52.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 190740 21.75 MB/sec execute 347 sec latency 51.355 ms 2024-09-10T23:12:53.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:53 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:53.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:53 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:53.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:53 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:12:53.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191103 21.73 MB/sec execute 348 sec latency 176.337 ms 2024-09-10T23:12:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:54 smithi120 ceph-mon[31247]: pgmap v960: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T23:12:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:54 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:54 smithi153 ceph-mon[34951]: pgmap v960: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T23:12:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:54 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:54 smithi176 ceph-mon[34982]: pgmap v960: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-09-10T23:12:54.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:54 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:54.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191382 21.73 MB/sec execute 349 sec latency 199.732 ms 2024-09-10T23:12:55.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 191716 21.72 MB/sec execute 350 sec latency 135.274 ms 2024-09-10T23:12:56.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: pgmap v961: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T23:12:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: pgmap v961: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: pgmap v961: 97 pgs: 97 active+clean; 240 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 516 op/s 2024-09-10T23:12:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:12:56.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:12:56.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:12:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 -- 172.21.15.120:0/2002094861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1044a0 msgr2=0x7fd1ac10ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2002094861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac10ae20 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009cd0 tx=0x7fd1a402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 -- 172.21.15.120:0/2002094861 shutdown_connections 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2002094861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac10ae20 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2002094861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2002094861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 0x7fd1ac102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 -- 172.21.15.120:0/2002094861 >> 172.21.15.120:0/2002094861 conn(0x7fd1ac0fe250 msgr2=0x7fd1ac100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 -- 172.21.15.120:0/2002094861 shutdown_connections 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.708+0000 7fd1b2033640 1 -- 172.21.15.120:0/2002094861 wait complete. 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 Processor -- start 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 -- start start 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 0x7fd1ac198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac10e600 con 0x7fd1ac103ae0 2024-09-10T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac10e480 con 0x7fd1ac102990 2024-09-10T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1b2033640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac10e780 con 0x7fd1ac1044a0 2024-09-10T23:12:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38262/0 (socket says 172.21.15.120:38262) 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 -- 172.21.15.120:0/2671704374 learned_addr learned my addr 172.21.15.120:0/2671704374 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1abfff640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 0x7fd1ac198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 -- 172.21.15.120:0/2671704374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac1044a0 msgr2=0x7fd1ac199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 -- 172.21.15.120:0/2671704374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 msgr2=0x7fd1ac198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 0x7fd1ac198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.709+0000 7fd1aaffd640 1 -- 172.21.15.120:0/2671704374 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac19a890 con 0x7fd1ac103ae0 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd19400e9f0 tx=0x7fd19400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1a8ff9640 1 -- 172.21.15.120:0/2671704374 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19400cd80 con 0x7fd1ac103ae0 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac19ab80 con 0x7fd1ac103ae0 2024-09-10T23:12:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1a8ff9640 1 -- 172.21.15.120:0/2671704374 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1940054d0 con 0x7fd1ac103ae0 2024-09-10T23:12:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1a8ff9640 1 -- 172.21.15.120:0/2671704374 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194010690 con 0x7fd1ac103ae0 2024-09-10T23:12:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.710+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac1a1250 con 0x7fd1ac103ae0 2024-09-10T23:12:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1a8ff9640 1 -- 172.21.15.120:0/2671704374 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd194004e90 con 0x7fd1ac103ae0 2024-09-10T23:12:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 Processor -- start 2024-09-10T23:12:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 -- start start 2024-09-10T23:12:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643eb7336f0 con 0x5643eb7c1860 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643eb722b20 con 0x5643eb7c4190 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.712+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5643eb726930 con 0x5643eb7c6bd0 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38274/0 (socket says 172.21.15.120:38274) 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 -- 172.21.15.120:0/650517686 learned_addr learned my addr 172.21.15.120:0/650517686 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 -- 172.21.15.120:0/650517686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 msgr2=0x5643eb7c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 -- 172.21.15.120:0/650517686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 msgr2=0x5643eb7c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 -- 172.21.15.120:0/650517686 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643eb6e8530 con 0x5643eb7c1860 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1a8ff9640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 0x7fd1980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1abfff640 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd1a402ae00 tx=0x7fd1a4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3316d797752ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 0x7fd1980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1a8ff9640 1 -- 172.21.15.120:0/2671704374 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd1940e5fb0 con 0x7fd1ac103ae0 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.713+0000 7fd1a1ffb640 1 -- 172.21.15.120:0/650517686 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4031430 con 0x5643eb7c1860 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1a1ffb640 1 -- 172.21.15.120:0/650517686 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a4036040 con 0x5643eb7c1860 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1a1ffb640 1 -- 172.21.15.120:0/650517686 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4004330 con 0x5643eb7c1860 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 0x7fd1980b77a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009da0 tx=0x7fd19c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 -- 172.21.15.120:0/650517686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 msgr2=0x5643eb7c3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd1a402ae00 tx=0x7fd1a4009a20 comp rx=0 tx=0).stop 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 -- 172.21.15.120:0/650517686 shutdown_connections 2024-09-10T23:12:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/650517686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7c3c50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 -- 172.21.15.120:0/650517686 >> 172.21.15.120:0/650517686 conn(0x5643eb7bc9b0 msgr2=0x5643eb7bd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 -- 172.21.15.120:0/650517686 shutdown_connections 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.714+0000 7fd1b4e04740 1 -- 172.21.15.120:0/650517686 wait complete. 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 -- start start 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 Processor -- start 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 0x5643eb7cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c6bd0 0x5643eb7cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5643eb65aab0 con 0x5643eb7c1860 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5643eb6c8730 con 0x5643eb7c6bd0 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5643eb7c9af0 con 0x5643eb7c4190 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1b4e04740 10 client.0 _ll_register_callbacks cb 0x5643eb71c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38280/0 (socket says 172.21.15.120:38280) 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1ab7fe640 1 -- 172.21.15.120:0/1934299231 learned_addr learned my addr 172.21.15.120:0/1934299231 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 0x5643eb7cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.715+0000 7fd1abfff640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c6bd0 0x5643eb7cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 -- 172.21.15.120:0/1934299231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c6bd0 msgr2=0x5643eb7cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c6bd0 0x5643eb7cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 -- 172.21.15.120:0/1934299231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 msgr2=0x5643eb7cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 -- 172.21.15.120:0/1934299231 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643eb484790 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 0x5643eb7cb500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1940e3b30 tx=0x7fd1940e9680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2644d9bacb5209f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940ac570 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a4036040 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd1940e9c40 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1940abf10 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 10 client.28855 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643eb4fd620 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a4036040 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd1940b58a0 con 0x5643eb7c4190 2024-09-10T23:12:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 10 client.28855 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 10 client.28855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.716+0000 7fd1b4e04740 20 client.28855 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb7d68f0 0x5643eb7d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd1b4e04740 4 client.28855 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643eb735640 con 0x5643eb7d68f0 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb7d68f0 0x5643eb7d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb7d68f0 0x5643eb7d8ce0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5948920cf1b840f0 server_cookie=ec0044845b276004 in_seq=0 out_seq=0 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.717+0000 7fd18affd640 10 client.28855 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd18affd640 1 -- 172.21.15.120:0/1934299231 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643eb735640 con 0x5643eb7d68f0 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd18affd640 10 client.28855 handle_command_reply: tid=0 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 client.28855 shutdown 2024-09-10T23:12:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 msgr2=0x5643eb7cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 0x5643eb7cb500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1940e3b30 tx=0x7fd1940e9680 comp rx=0 tx=0).stop 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 shutdown_connections 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb7d68f0 0x5643eb7d8ce0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c6bd0 0x5643eb7cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c4190 0x5643eb7cb500 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/1934299231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd18affd640 0 client.28855 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 >> 172.21.15.120:0/1934299231 conn(0x5643eb7bc9b0 msgr2=0x5643eb7bbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 shutdown_connections 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 1 -- 172.21.15.120:0/1934299231 wait complete. 2024-09-10T23:12:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.719+0000 7fd1b4e04740 20 client.28855 trim_cache size 0 max 16384 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 Processor -- start 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 -- start start 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a7d50 con 0x5643eb7c1860 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1aa5d0 con 0x5643eb7c4190 2024-09-10T23:12:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a83c0 con 0x5643eb7c6bd0 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38288/0 (socket says 172.21.15.120:38288) 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 -- 172.21.15.120:0/2946611179 learned_addr learned my addr 172.21.15.120:0/2946611179 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1abfff640 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 -- 172.21.15.120:0/2946611179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 msgr2=0x5643eb7bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7bd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 -- 172.21.15.120:0/2946611179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 msgr2=0x5643eb7bd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7bd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.734+0000 7fd1ab7fe640 1 -- 172.21.15.120:0/2946611179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643eb7c68f0 con 0x5643eb7c1860 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009b80 tx=0x7fd19c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f989a9d9958545 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1a17fa640 1 -- 172.21.15.120:0/2946611179 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c003d10 con 0x5643eb7c1860 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1a17fa640 1 -- 172.21.15.120:0/2946611179 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19c0054d0 con 0x5643eb7c1860 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1a17fa640 1 -- 172.21.15.120:0/2946611179 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0047a0 con 0x5643eb7c1860 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 -- 172.21.15.120:0/2946611179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 msgr2=0x5643eb7ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009b80 tx=0x7fd19c01b000 comp rx=0 tx=0).stop 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 -- 172.21.15.120:0/2946611179 shutdown_connections 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7bd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c4190 0x5643eb7bd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/2946611179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c1860 0x5643eb7ca0a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 -- 172.21.15.120:0/2946611179 >> 172.21.15.120:0/2946611179 conn(0x5643eb7bc9b0 msgr2=0x5643eb7c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 -- 172.21.15.120:0/2946611179 shutdown_connections 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.735+0000 7fd1b4e04740 1 -- 172.21.15.120:0/2946611179 wait complete. 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 -- start start 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 Processor -- start 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c1860 0x5643eb7d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a8a90 con 0x5643eb7c4190 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a9f00 con 0x5643eb7c1860 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1a9160 con 0x5643eb7c6bd0 2024-09-10T23:12:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1b4e04740 10 client.0 _ll_register_callbacks cb 0x7fd1940e9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38292/0 (socket says 172.21.15.120:38292) 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1aaffd640 1 -- 172.21.15.120:0/433999093 learned_addr learned my addr 172.21.15.120:0/433999093 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c1860 0x5643eb7d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.736+0000 7fd1abfff640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 -- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 msgr2=0x5643eb7d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 -- 172.21.15.120:0/433999093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c1860 msgr2=0x5643eb7d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c1860 0x5643eb7d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 -- 172.21.15.120:0/433999093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643eb7d0390 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1aaffd640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd1940e3e50 tx=0x7fd1940e92f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8272b81e6962e405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194097060 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd1940aba00 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd19c0054d0 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1940ac0f0 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 10 client.19413 fetch_fsmap learned FSMap version 26 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643eb843950 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19c0054d0 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd1940a80f0 con 0x5643eb7c4190 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 10 client.19413 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:12:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 10 client.19413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.737+0000 7fd1b4e04740 20 client.19413 populate_metadata read hostname 'smithi120' 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb8442d0 0x5643eb8466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd1b4e04740 4 client.19413 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643eb7310f0 con 0x5643eb8442d0 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb8442d0 0x5643eb8466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd18affd640 10 client.19413 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:56.738+0000 7fd1ab7fe640 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb8442d0 0x5643eb8466f0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f816a6af518afd server_cookie=2b3fab7415249b31 in_seq=0 out_seq=0 2024-09-10T23:12:56.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 192103 21.71 MB/sec execute 351 sec latency 91.497 ms 2024-09-10T23:12:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd18affd640 1 -- 172.21.15.120:0/433999093 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643eb7310f0 con 0x5643eb8442d0 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd18affd640 10 client.19413 handle_command_reply: tid=0 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 client.19413 shutdown 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 msgr2=0x5643eb7d18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd1940e3e50 tx=0x7fd1940e92f0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 shutdown_connections 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5643eb8442d0 0x5643eb8466f0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643eb7c6bd0 0x5643eb7d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5643eb7c4190 0x5643eb7d18d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 --2- 172.21.15.120:0/433999093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643eb7c1860 0x5643eb7d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 >> 172.21.15.120:0/433999093 conn(0x5643eb7bc9b0 msgr2=0x5643eb7c0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd18affd640 0 client.19413 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:12:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 shutdown_connections 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 1 -- 172.21.15.120:0/433999093 wait complete. 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.592+0000 7fd1b4e04740 20 client.19413 trim_cache size 0 max 16384 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:12:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 msgr2=0x7fd1980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 0x7fd1980b77a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009da0 tx=0x7fd19c005f90 comp rx=0 tx=0).stop 2024-09-10T23:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 msgr2=0x7fd1ac1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd19400e9f0 tx=0x7fd19400edf0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 shutdown_connections 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd1980b52e0 0x7fd1980b77a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1ac1044a0 0x7fd1ac199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac103ae0 0x7fd1ac1993a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 --2- 172.21.15.120:0/2671704374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1ac102990 0x7fd1ac198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 >> 172.21.15.120:0/2671704374 conn(0x7fd1ac0fe250 msgr2=0x7fd1ac106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 shutdown_connections 2024-09-10T23:12:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:12:57.593+0000 7fd1b2033640 1 -- 172.21.15.120:0/2671704374 wait complete. 2024-09-10T23:12:57.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 192293 21.67 MB/sec execute 352 sec latency 508.797 ms 2024-09-10T23:12:58.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:57 smithi176 ceph-mon[34982]: pgmap v962: 97 pgs: 97 active+clean; 243 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-10T23:12:58.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:57 smithi120 ceph-mon[31247]: pgmap v962: 97 pgs: 97 active+clean; 243 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-10T23:12:58.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:57 smithi153 ceph-mon[34951]: pgmap v962: 97 pgs: 97 active+clean; 243 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-10T23:12:58.883 INFO:tasks.workunit.client.0.smithi120.stdout: 10 192783 21.66 MB/sec execute 353 sec latency 115.814 ms 2024-09-10T23:12:59.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 193132 21.68 MB/sec execute 354 sec latency 136.517 ms 2024-09-10T23:13:00.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:12:59 smithi176 ceph-mon[34982]: pgmap v963: 97 pgs: 97 active+clean; 245 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 397 op/s 2024-09-10T23:13:00.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:12:59 smithi120 ceph-mon[31247]: pgmap v963: 97 pgs: 97 active+clean; 245 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 397 op/s 2024-09-10T23:13:00.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:12:59 smithi153 ceph-mon[34951]: pgmap v963: 97 pgs: 97 active+clean; 245 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.3 MiB/s wr, 397 op/s 2024-09-10T23:13:00.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 193818 21.70 MB/sec execute 355 sec latency 53.157 ms 2024-09-10T23:13:01.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 194332 21.74 MB/sec execute 356 sec latency 69.666 ms 2024-09-10T23:13:02.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:01 smithi176 ceph-mon[34982]: pgmap v964: 97 pgs: 97 active+clean; 248 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-10T23:13:02.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:01 smithi120 ceph-mon[31247]: pgmap v964: 97 pgs: 97 active+clean; 248 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-10T23:13:02.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:01 smithi153 ceph-mon[34951]: pgmap v964: 97 pgs: 97 active+clean; 248 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 425 op/s 2024-09-10T23:13:02.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 194827 21.75 MB/sec execute 357 sec latency 73.286 ms 2024-09-10T23:13:03.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:13:03.689 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:03.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 195158 21.75 MB/sec execute 358 sec latency 96.520 ms 2024-09-10T23:13:04.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:03 smithi176 ceph-mon[34982]: pgmap v965: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:03 smithi120 ceph-mon[31247]: pgmap v965: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:13:04.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:03 smithi153 ceph-mon[34951]: pgmap v965: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 -- 172.21.15.120:0/3969216146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 msgr2=0x7fb4d410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 --2- 172.21.15.120:0/3969216146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d410e820 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009ca0 tx=0x7fb4cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 -- 172.21.15.120:0/3969216146 shutdown_connections 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 --2- 172.21.15.120:0/3969216146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d410e820 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 --2- 172.21.15.120:0/3969216146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d41076a0 0x7fb4d4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 --2- 172.21.15.120:0/3969216146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d4101c90 0x7fb4d4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.552+0000 7fb4db66f640 1 -- 172.21.15.120:0/3969216146 >> 172.21.15.120:0/3969216146 conn(0x7fb4d40fd700 msgr2=0x7fb4d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.553+0000 7fb4db66f640 1 -- 172.21.15.120:0/3969216146 shutdown_connections 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.553+0000 7fb4db66f640 1 -- 172.21.15.120:0/3969216146 wait complete. 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.553+0000 7fb4db66f640 1 Processor -- start 2024-09-10T23:13:04.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.553+0000 7fb4db66f640 1 -- start start 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.553+0000 7fb4db66f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4db66f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d41076a0 0x7fb4d4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4db66f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4db66f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d406b1d0 con 0x7fb4d4108020 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4db66f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d406b050 con 0x7fb4d4101c90 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4db66f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d406b350 con 0x7fb4d41076a0 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44820/0 (socket says 172.21.15.120:44820) 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 -- 172.21.15.120:0/1146883895 learned_addr learned my addr 172.21.15.120:0/1146883895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d41076a0 0x7fb4d4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 -- 172.21.15.120:0/1146883895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d41076a0 msgr2=0x7fb4d4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.643 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d41076a0 0x7fb4d4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 -- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 msgr2=0x7fb4d419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d940b640 1 -- 172.21.15.120:0/1146883895 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d419f2f0 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.554+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.556+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d770 tx=0x7fb4d000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.557+0000 7fb4ca7fc640 1 -- 172.21.15.120:0/1146883895 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0017070 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.557+0000 7fb4ca7fc640 1 -- 172.21.15.120:0/1146883895 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d00054d0 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.558+0000 7fb4ca7fc640 1 -- 172.21.15.120:0/1146883895 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004c30 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.558+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d419f5e0 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.558+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41115c0 con 0x7fb4d4101c90 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.561+0000 7fb4dda44740 1 Processor -- start 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.561+0000 7fb4dda44740 1 -- start start 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b50120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa80ab7e50 con 0x55aa80b4dd30 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa80aa7280 con 0x55aa80b41da0 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4dda44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55aa80aab090 con 0x55aa80b41480 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4d8c0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b50120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4d8c0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b50120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42390/0 (socket says 172.21.15.120:42390) 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/2232389734 learned_addr learned my addr 172.21.15.120:0/2232389734 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.562+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 -- 172.21.15.120:0/2232389734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 msgr2=0x55aa80b4d7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 -- 172.21.15.120:0/2232389734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 msgr2=0x55aa80b50120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b50120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 -- 172.21.15.120:0/2232389734 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa80805650 con 0x55aa80b41480 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.563+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b41860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3592f7afb166fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.564+0000 7fb4ca7fc640 1 -- 172.21.15.120:0/1146883895 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb4d0009c30 con 0x7fb4d4101c90 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.564+0000 7fb4bf7fe640 1 -- 172.21.15.120:0/2232389734 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc031cf0 con 0x55aa80b41480 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.564+0000 7fb4bf7fe640 1 -- 172.21.15.120:0/2232389734 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc036040 con 0x55aa80b41480 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.564+0000 7fb4bf7fe640 1 -- 172.21.15.120:0/2232389734 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004040 con 0x55aa80b41480 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.565+0000 7fb4dda44740 1 -- 172.21.15.120:0/2232389734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 msgr2=0x55aa80b41860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.565+0000 7fb4dda44740 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b41860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc006010 tx=0x7fb4cc005fa0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.565+0000 7fb4dda44740 1 -- 172.21.15.120:0/2232389734 shutdown_connections 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.565+0000 7fb4dda44740 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.565+0000 7fb4dda44740 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.566+0000 7fb4dda44740 1 --2- 172.21.15.120:0/2232389734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b41860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.566+0000 7fb4dda44740 1 -- 172.21.15.120:0/2232389734 >> 172.21.15.120:0/2232389734 conn(0x55aa80b3f2b0 msgr2=0x55aa80b40450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.566+0000 7fb4ca7fc640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 0x7fb4c00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.566+0000 7fb4ca7fc640 1 -- 172.21.15.120:0/1146883895 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb4d00e5430 con 0x7fb4d4101c90 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.567+0000 7fb4dda44740 1 -- 172.21.15.120:0/2232389734 shutdown_connections 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.567+0000 7fb4dda44740 1 -- 172.21.15.120:0/2232389734 wait complete. 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.568+0000 7fb4dda44740 1 -- start start 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.568+0000 7fb4dda44740 1 Processor -- start 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 0x7fb4c00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b4d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b4d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.569+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b4d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40620/0 (socket says 172.21.15.120:40620) 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4d940b640 1 -- 172.21.15.120:0/1959140701 learned_addr learned my addr 172.21.15.120:0/1959140701 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 0x7fb4c00b78f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005fd0 tx=0x7fb4c4011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa80908700 con 0x55aa80b4dd30 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aa80a42110 con 0x55aa80b41da0 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.570+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55aa80b508f0 con 0x55aa80b41480 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4dda44740 10 client.0 _ll_register_callbacks cb 0x55aa80a5b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 msgr2=0x55aa80b4d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b4d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1959140701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 msgr2=0x55aa80b4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa80a6fc40 con 0x55aa80b41da0 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4aa30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40042c0 tx=0x7fb4c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a39529389d50e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.571+0000 7fb4bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4026630 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb4c40267d0 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc036040 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4c4026a10 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 10 client.28619 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa80b5c2e0 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc036040 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb4c4019070 con 0x55aa80b41da0 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 10 client.28619 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 10 client.28619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 20 client.28619 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:04.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.572+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80b5d860 0x55aa80b5fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.573+0000 7fb4dda44740 4 client.28619 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.573+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa80ab9da0 con 0x55aa80b5d860 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.574+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80b5d860 0x55aa80b5fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.578+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80b5d860 0x55aa80b5fc80 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da89d7a735367d93 server_cookie=417bc230c2263d49 in_seq=0 out_seq=0 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.578+0000 7fb4bcff9640 10 client.28619 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.580+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1959140701 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa80ab9da0 con 0x55aa80b5d860 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.580+0000 7fb4bcff9640 10 client.28619 handle_command_reply: tid=0 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.580+0000 7fb4dda44740 1 client.28619 shutdown 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 msgr2=0x55aa80b4aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4aa30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40042c0 tx=0x7fb4c4004730 comp rx=0 tx=0).stop 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 shutdown_connections 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80b5d860 0x55aa80b5fc80 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4bcff9640 0 client.28619 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:04.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b4dd30 0x55aa80b4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b41da0 0x55aa80b4aa30 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40042c0 tx=0x7fb4c4004730 comp rx=0 tx=0).stop 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1959140701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b41480 0x55aa80b4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.581+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 >> 172.21.15.120:0/1959140701 conn(0x55aa80b3f2b0 msgr2=0x55aa80b44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.582+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 shutdown_connections 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.582+0000 7fb4dda44740 1 -- 172.21.15.120:0/1959140701 wait complete. 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.582+0000 7fb4dda44740 20 client.28619 trim_cache size 0 max 16384 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.608+0000 7fb4dda44740 1 Processor -- start 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.608+0000 7fb4dda44740 1 -- start start 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b40370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b5d860 0x55aa80b499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800c3d0 con 0x55aa80b41480 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800ec50 con 0x55aa80b4dd30 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4dda44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800ca40 con 0x55aa80b5d860 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4d940b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42398/0 (socket says 172.21.15.120:42398) 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.609+0000 7fb4d940b640 1 -- 172.21.15.120:0/677039327 learned_addr learned my addr 172.21.15.120:0/677039327 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 -- 172.21.15.120:0/677039327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b5d860 msgr2=0x55aa80b499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b5d860 0x55aa80b499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 -- 172.21.15.120:0/677039327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 msgr2=0x55aa80b40370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b40370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 -- 172.21.15.120:0/677039327 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa80b4ada0 con 0x55aa80b41480 2024-09-10T23:13:04.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4d940b640 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000df50 tx=0x7fb4d00e2dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793e3eece6333c1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4beffd640 1 -- 172.21.15.120:0/677039327 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0020040 con 0x55aa80b41480 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.610+0000 7fb4beffd640 1 -- 172.21.15.120:0/677039327 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d00ac2c0 con 0x55aa80b41480 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4beffd640 1 -- 172.21.15.120:0/677039327 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00b4cc0 con 0x55aa80b41480 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 -- 172.21.15.120:0/677039327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 msgr2=0x55aa80b3ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000df50 tx=0x7fb4d00e2dd0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 -- 172.21.15.120:0/677039327 shutdown_connections 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80b5d860 0x55aa80b499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b40370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 --2- 172.21.15.120:0/677039327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b41480 0x55aa80b3ebc0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.611+0000 7fb4dda44740 1 -- 172.21.15.120:0/677039327 >> 172.21.15.120:0/677039327 conn(0x55aa80b3f2b0 msgr2=0x55aa80b492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.612+0000 7fb4dda44740 1 -- 172.21.15.120:0/677039327 shutdown_connections 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.612+0000 7fb4dda44740 1 -- 172.21.15.120:0/677039327 wait complete. 2024-09-10T23:13:04.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.612+0000 7fb4dda44740 1 -- start start 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.612+0000 7fb4dda44740 1 Processor -- start 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.612+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80bc3870 0x55aa80bc3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800d110 con 0x55aa80b5d860 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800e580 con 0x55aa80b4dd30 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b800d7e0 con 0x55aa80bc3870 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 10 client.0 _ll_register_callbacks cb 0x7fb4c4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42414/0 (socket says 172.21.15.120:42414) 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1154298010 learned_addr learned my addr 172.21.15.120:0/1154298010 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d9c0c640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80bc3870 0x55aa80bc3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80bc3870 msgr2=0x55aa80bc3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80bc3870 0x55aa80bc3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1154298010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 msgr2=0x55aa80b526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 -- 172.21.15.120:0/1154298010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa80bc4390 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4d8c0a640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb4c401e040 tx=0x7fb4c401a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511cd267321c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4026430 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d00ac2c0 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.613+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb4c40265d0 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4c4041050 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 10 client.19431 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa80bcb810 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d00ac2c0 con 0x55aa80b5d860 2024-09-10T23:13:04.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fb4c4019070 con 0x55aa80b5d860 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 10 client.19431 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 10 client.19431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 20 client.19431 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80bcbf30 0x55aa80bce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 4 client.19431 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.614+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa80ab5850 con 0x55aa80bcbf30 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.617+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80bcbf30 0x55aa80bce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.618+0000 7fb4bcff9640 10 client.19431 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.618+0000 7fb4d940b640 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80bcbf30 0x55aa80bce350 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8522379d01afe5 server_cookie=f138658a9647031a in_seq=0 out_seq=0 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.626+0000 7fb4bcff9640 1 -- 172.21.15.120:0/1154298010 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9746 (crc 0 0 0) 0x55aa80ab5850 con 0x55aa80bcbf30 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.626+0000 7fb4bcff9640 10 client.19431 handle_command_reply: tid=0 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.626+0000 7fb4dda44740 1 client.19431 shutdown 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.629+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 msgr2=0x55aa80b52c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.629+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb4c401e040 tx=0x7fb4c401a6c0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 shutdown_connections 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aa80bcbf30 0x55aa80bce350 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aa80bc3870 0x55aa80bc3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa80b5d860 0x55aa80b52c60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 --2- 172.21.15.120:0/1154298010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aa80b4dd30 0x55aa80b526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4bcff9640 0 client.19431 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 >> 172.21.15.120:0/1154298010 conn(0x55aa80b3f2b0 msgr2=0x55aa80b49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 shutdown_connections 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 1 -- 172.21.15.120:0/1154298010 wait complete. 2024-09-10T23:13:04.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.630+0000 7fb4dda44740 20 client.19431 trim_cache size 0 max 16384 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.658 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.659 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:13:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.661 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.662 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.663 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.664 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.666 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.667 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.668 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.669 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.670 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.671 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:13:04.672 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.673 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:13:04.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:04.678 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 msgr2=0x7fb4c00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 0x7fb4c00b78f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005fd0 tx=0x7fb4c4011040 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 msgr2=0x7fb4d4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d770 tx=0x7fb4d000dc30 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 shutdown_connections 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb4c00b5430 0x7fb4c00b78f0 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005fd0 tx=0x7fb4c4011040 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4d4108020 0x7fb4d419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4d41076a0 0x7fb4d4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 --2- 172.21.15.120:0/1146883895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4d4101c90 0x7fb4d4199240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 >> 172.21.15.120:0/1146883895 conn(0x7fb4d40fd700 msgr2=0x7fb4d4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 shutdown_connections 2024-09-10T23:13:04.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:04.632+0000 7fb4db66f640 1 -- 172.21.15.120:0/1146883895 wait complete. 2024-09-10T23:13:04.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 195434 21.72 MB/sec execute 359 sec latency 225.059 ms 2024-09-10T23:13:05.599 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:13:05.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:13:05.859 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:05.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 195762 21.72 MB/sec execute 360 sec latency 65.413 ms 2024-09-10T23:13:05.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:05 smithi120 ceph-mon[31247]: pgmap v966: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2024-09-10T23:13:06.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:05 smithi176 ceph-mon[34982]: pgmap v966: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2024-09-10T23:13:06.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:05 smithi153 ceph-mon[34951]: pgmap v966: 97 pgs: 97 active+clean; 231 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2024-09-10T23:13:06.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 -- 172.21.15.120:0/1671779173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 msgr2=0x7f23cc104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:06.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 --2- 172.21.15.120:0/1671779173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc104080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009900 tx=0x7f23c802a410 comp rx=0 tx=0).stop 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 -- 172.21.15.120:0/1671779173 shutdown_connections 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 --2- 172.21.15.120:0/1671779173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc10d100 0x7f23cc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 --2- 172.21.15.120:0/1671779173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc1045c0 0x7f23cc10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 --2- 172.21.15.120:0/1671779173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc104080 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 -- 172.21.15.120:0/1671779173 >> 172.21.15.120:0/1671779173 conn(0x7f23cc0fd700 msgr2=0x7f23cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 -- 172.21.15.120:0/1671779173 shutdown_connections 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.395+0000 7f23d4778640 1 -- 172.21.15.120:0/1671779173 wait complete. 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 Processor -- start 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 -- start start 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 0x7f23cc101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:06.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc10d100 0x7f23cc1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112d70 con 0x7f23cc103ca0 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112bf0 con 0x7f23cc10d100 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d4778640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112ef0 con 0x7f23cc1045c0 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42434/0 (socket says 172.21.15.120:42434) 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 -- 172.21.15.120:0/209504150 learned_addr learned my addr 172.21.15.120:0/209504150 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d1d13640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 0x7f23cc101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2d15640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc10d100 0x7f23cc1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 -- 172.21.15.120:0/209504150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 msgr2=0x7f23cc101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 0x7f23cc101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 -- 172.21.15.120:0/209504150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc10d100 msgr2=0x7f23cc1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc10d100 0x7f23cc1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d2514640 1 -- 172.21.15.120:0/209504150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc1a7680 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.396+0000 7f23d1d13640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 0x7f23cc101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23d2514640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005940 tx=0x7f23c80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c803f030 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c80044c0 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8035070 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc1a7910 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.397+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc1a7e40 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.398+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23cc104080 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.400+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f23c803b9d0 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.404+0000 7f23c37fe640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 0x7f23b80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.404+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f23c81064a0 con 0x7f23cc103ca0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.407+0000 7f23d1d13640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 0x7f23b80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:06.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.407+0000 7f23d1d13640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 0x7f23b80b78a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f23cc103440 tx=0x7f23bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:06.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.409+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f23c80cd330 con 0x7f23cc103ca0 2024-09-10T23:13:06.655 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.653+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23cc1a8230 con 0x7f23cc103ca0 2024-09-10T23:13:06.656 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.654+0000 7f23c37fe640 1 -- 172.21.15.120:0/209504150 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f23c80ccc30 con 0x7f23cc103ca0 2024-09-10T23:13:06.656 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:13:06.656 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:13:06.657 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 msgr2=0x7f23b80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 0x7f23b80b78a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f23cc103440 tx=0x7f23bc005f90 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 msgr2=0x7f23cc101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005940 tx=0x7f23c80040c0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 shutdown_connections 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f23b80b53e0 0x7f23b80b78a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23cc10d100 0x7f23cc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23cc1045c0 0x7f23cc101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 --2- 172.21.15.120:0/209504150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23cc103ca0 0x7f23cc101950 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:06.659 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 >> 172.21.15.120:0/209504150 conn(0x7f23cc0fd700 msgr2=0x7f23cc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 shutdown_connections 2024-09-10T23:13:06.660 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:06.657+0000 7f23d4778640 1 -- 172.21.15.120:0/209504150 wait complete. 2024-09-10T23:13:06.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 196203 21.69 MB/sec execute 361 sec latency 63.164 ms 2024-09-10T23:13:06.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/209504150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:07.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/209504150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:07.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/209504150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:07.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:13:07.651 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:07.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 196619 21.71 MB/sec execute 362 sec latency 66.957 ms 2024-09-10T23:13:08.045 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:07 smithi120 ceph-mon[31247]: pgmap v967: 97 pgs: 97 active+clean; 238 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-10T23:13:08.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:07 smithi176 ceph-mon[34982]: pgmap v967: 97 pgs: 97 active+clean; 238 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 -- 172.21.15.120:0/2308660846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf01076a0 msgr2=0x7fbaf0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 --2- 172.21.15.120:0/2308660846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf01076a0 0x7fbaf0107ae0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009900 tx=0x7fbad802a410 comp rx=0 tx=0).stop 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 -- 172.21.15.120:0/2308660846 shutdown_connections 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 --2- 172.21.15.120:0/2308660846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf0108020 0x7fbaf010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 --2- 172.21.15.120:0/2308660846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf01076a0 0x7fbaf0107ae0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 --2- 172.21.15.120:0/2308660846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 -- 172.21.15.120:0/2308660846 >> 172.21.15.120:0/2308660846 conn(0x7fbaf00fd700 msgr2=0x7fbaf00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 -- 172.21.15.120:0/2308660846 shutdown_connections 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.132+0000 7fbaf480b640 1 -- 172.21.15.120:0/2308660846 wait complete. 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.133+0000 7fbaf480b640 1 Processor -- start 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.133+0000 7fbaf480b640 1 -- start start 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.133+0000 7fbaf480b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf01076a0 0x7fbaf019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0111ce0 con 0x7fbaf0108020 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0111b60 con 0x7fbaf01076a0 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf0111e60 con 0x7fbaf0101c90 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40654/0 (socket says 172.21.15.120:40654) 2024-09-10T23:13:08.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42448/0 (socket says 172.21.15.120:42448) 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaee59c640 1 -- 172.21.15.120:0/100519748 learned_addr learned my addr 172.21.15.120:0/100519748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf01076a0 0x7fbaf019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 -- 172.21.15.120:0/100519748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 msgr2=0x7fbaf019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 -- 172.21.15.120:0/100519748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf01076a0 msgr2=0x7fbaf019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf01076a0 0x7fbaf019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 -- 172.21.15.120:0/100519748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf01a5530 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaee59c640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaeed9d640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbae800be20 tx=0x7fbae800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbae7fff640 1 -- 172.21.15.120:0/100519748 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8019070 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf01a5820 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbae7fff640 1 -- 172.21.15.120:0/100519748 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae80148f0 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.134+0000 7fbae7fff640 1 -- 172.21.15.120:0/100519748 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8012840 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.135+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf01a5d30 con 0x7fbaf0108020 2024-09-10T23:13:08.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 Processor -- start 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbae7fff640 1 -- 172.21.15.120:0/100519748 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbae8012a00 con 0x7fbaf0108020 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 -- start start 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 0x5611722d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722e1db0 0x5611722e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561172247ea0 con 0x5611722d10f0 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611722372d0 con 0x5611722e1db0 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56117223b0e0 con 0x5611722d1a10 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 0x5611722d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 0x5611722d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42454/0 (socket says 172.21.15.120:42454) 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbae5d9b640 1 -- 172.21.15.120:0/641755034 learned_addr learned my addr 172.21.15.120:0/641755034 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.138+0000 7fbaee59c640 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722e1db0 0x5611722e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbae7fff640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 0x7fbadc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 -- 172.21.15.120:0/641755034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722e1db0 msgr2=0x5611722e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722e1db0 0x5611722e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 -- 172.21.15.120:0/641755034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 msgr2=0x5611722d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 0x5611722d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 -- 172.21.15.120:0/641755034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561171f95650 con 0x5611722d1a10 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaeed9d640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 0x7fbadc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaee59c640 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722e1870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbae0010550 tx=0x7fbae0010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a5fcb5ba4f4c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbae7fff640 1 -- 172.21.15.120:0/100519748 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbae80e55b0 con 0x7fbaf0108020 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbaeed9d640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 0x7fbadc0b77a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbae80a8420 tx=0x7fbae8070a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbae4d99640 1 -- 172.21.15.120:0/641755034 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011760 con 0x5611722d1a10 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.139+0000 7fbae4d99640 1 -- 172.21.15.120:0/641755034 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae0011db0 con 0x5611722d1a10 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbae4d99640 1 -- 172.21.15.120:0/641755034 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae00193d0 con 0x5611722d1a10 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 -- 172.21.15.120:0/641755034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 msgr2=0x5611722e1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722e1870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbae0010550 tx=0x7fbae0010a10 comp rx=0 tx=0).stop 2024-09-10T23:13:08.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 -- 172.21.15.120:0/641755034 shutdown_connections 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722e1db0 0x5611722e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722e1870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/641755034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d10f0 0x5611722d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 -- 172.21.15.120:0/641755034 >> 172.21.15.120:0/641755034 conn(0x5611722cef20 msgr2=0x5611722d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 -- 172.21.15.120:0/641755034 shutdown_connections 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.140+0000 7fbaf6be0740 1 -- 172.21.15.120:0/641755034 wait complete. 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 -- start start 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 Processor -- start 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722e1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722e1db0 0x5611722deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561172098700 con 0x5611722d1a10 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5611721d2160 con 0x5611722d10f0 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5611722e4a60 con 0x5611722e1db0 2024-09-10T23:13:08.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaf6be0740 10 client.0 _ll_register_callbacks cb 0x5611721eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42470/0 (socket says 172.21.15.120:42470) 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 -- 172.21.15.120:0/935028571 learned_addr learned my addr 172.21.15.120:0/935028571 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722e1db0 0x5611722deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 -- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722e1db0 msgr2=0x5611722deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722e1db0 0x5611722deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 -- 172.21.15.120:0/935028571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 msgr2=0x5611722e1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722e1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.141+0000 7fbaee59c640 1 -- 172.21.15.120:0/935028571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611721ffc90 con 0x5611722d1a10 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaee59c640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbae0000c00 tx=0x7fbae000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4c2809fc826732 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:08.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae000ee20 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbae0004050 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae0011db0 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbae0004270 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 10 client.19449 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611722f0310 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae0011db0 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbae0005460 con 0x5611722d1a10 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 10 client.19449 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 20 client.19449 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5611722f1710 0x5611722f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 4 client.19449 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.142+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561172249df0 con 0x5611722f1710 2024-09-10T23:13:08.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.143+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5611722f1710 0x5611722f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.143+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5611722f1710 0x5611722f3b00 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17d6f439e350af9 server_cookie=ad3f4795f5ef3242 in_seq=0 out_seq=0 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.143+0000 7fbad5ffb640 10 client.19449 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbad5ffb640 1 -- 172.21.15.120:0/935028571 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561172249df0 con 0x5611722f1710 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbad5ffb640 10 client.19449 handle_command_reply: tid=0 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 client.19449 shutdown 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 msgr2=0x5611722dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbae0000c00 tx=0x7fbae000ec30 comp rx=0 tx=0).stop 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 shutdown_connections 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5611722f1710 0x5611722f3b00 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722e1db0 0x5611722deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722d1a10 0x5611722dea70 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbad5ffb640 0 client.19449 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/935028571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722e1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 >> 172.21.15.120:0/935028571 conn(0x5611722cef20 msgr2=0x5611722d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 shutdown_connections 2024-09-10T23:13:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 1 -- 172.21.15.120:0/935028571 wait complete. 2024-09-10T23:13:08.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.144+0000 7fbaf6be0740 20 client.19449 trim_cache size 0 max 16384 2024-09-10T23:13:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.157+0000 7fbaf6be0740 1 Processor -- start 2024-09-10T23:13:08.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.157+0000 7fbaf6be0740 1 -- start start 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d10f0 0x5611722edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d1a10 0x5611722d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01aa0e0 con 0x5611722e1db0 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01ac960 con 0x5611722d1a10 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01aa750 con 0x5611722d10f0 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42484/0 (socket says 172.21.15.120:42484) 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 -- 172.21.15.120:0/2353563224 learned_addr learned my addr 172.21.15.120:0/2353563224 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbaee59c640 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d1a10 0x5611722d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d10f0 0x5611722edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 -- 172.21.15.120:0/2353563224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d10f0 msgr2=0x5611722edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d10f0 0x5611722edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 -- 172.21.15.120:0/2353563224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d1a10 msgr2=0x5611722d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d1a10 0x5611722d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 -- 172.21.15.120:0/2353563224 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611722ceb90 con 0x5611722e1db0 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbad802a930 tx=0x7fbad8032bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58b05f61e3f6284 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbad7fff640 1 -- 172.21.15.120:0/2353563224 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8035040 con 0x5611722e1db0 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbad7fff640 1 -- 172.21.15.120:0/2353563224 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad8005500 con 0x5611722e1db0 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.158+0000 7fbad7fff640 1 -- 172.21.15.120:0/2353563224 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8003850 con 0x5611722e1db0 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- 172.21.15.120:0/2353563224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 msgr2=0x5611722ddf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbad802a930 tx=0x7fbad8032bf0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- 172.21.15.120:0/2353563224 shutdown_connections 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722ddf80 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d1a10 0x5611722d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/2353563224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d10f0 0x5611722edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- 172.21.15.120:0/2353563224 >> 172.21.15.120:0/2353563224 conn(0x5611722cef20 msgr2=0x5611722cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- 172.21.15.120:0/2353563224 shutdown_connections 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- 172.21.15.120:0/2353563224 wait complete. 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 -- start start 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.159+0000 7fbaf6be0740 1 Processor -- start 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01aae20 con 0x5611722e1db0 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01ac290 con 0x5611722d10f0 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf01ab4f0 con 0x5611722d1a10 2024-09-10T23:13:08.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaf6be0740 10 client.0 _ll_register_callbacks cb 0x7fbad8032b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40674/0 (socket says 172.21.15.120:40674) 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbae5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42490/0 (socket says 172.21.15.120:42490) 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 -- 172.21.15.120:0/4101876731 learned_addr learned my addr 172.21.15.120:0/4101876731 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 -- 172.21.15.120:0/4101876731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 msgr2=0x5611722ea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722ea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 -- 172.21.15.120:0/4101876731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 msgr2=0x5611722e6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 -- 172.21.15.120:0/4101876731 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611722e7630 con 0x5611722d1a10 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbaee59c640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b630 tx=0x7fbae0012f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0094bd9e483e0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:08.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.160+0000 7fbad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011820 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fbae00119c0 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad8005500 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbae0011be0 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 10 client.28900 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56117235f090 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad8005500 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fbae0005430 con 0x5611722d1a10 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 10 client.28900 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 10 client.28900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 20 client.28900 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56117235f800 0x561172361c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 4 client.28900 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:13:08.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611722458a0 con 0x56117235f800 2024-09-10T23:13:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.161+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56117235f800 0x561172361c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.162+0000 7fbae5d9b640 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56117235f800 0x561172361c20 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551ffaf9f8add82e server_cookie=10bd39a92b3478ac in_seq=0 out_seq=0 2024-09-10T23:13:08.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.162+0000 7fbad5ffb640 10 client.28900 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbad5ffb640 1 -- 172.21.15.120:0/4101876731 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611722458a0 con 0x56117235f800 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbad5ffb640 10 client.28900 handle_command_reply: tid=0 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 client.28900 shutdown 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 msgr2=0x5611722ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b630 tx=0x7fbae0012f00 comp rx=0 tx=0).stop 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 shutdown_connections 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56117235f800 0x561172361c20 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5611722e1db0 0x5611722e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5611722d1a10 0x5611722ea620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 --2- 172.21.15.120:0/4101876731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5611722d10f0 0x5611722ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 >> 172.21.15.120:0/4101876731 conn(0x5611722cef20 msgr2=0x5611722d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbad5ffb640 0 client.28900 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 shutdown_connections 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.166+0000 7fbaf6be0740 1 -- 172.21.15.120:0/4101876731 wait complete. 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf6be0740 20 client.28900 trim_cache size 0 max 16384 2024-09-10T23:13:08.174 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 msgr2=0x7fbadc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 0x7fbadc0b77a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbae80a8420 tx=0x7fbae8070a60 comp rx=0 tx=0).stop 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 msgr2=0x7fbaf019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbae800be20 tx=0x7fbae800bf10 comp rx=0 tx=0).stop 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 shutdown_connections 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fbadc0b52e0 0x7fbadc0b77a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbaf0108020 0x7fbaf019abf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaf01076a0 0x7fbaf019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 --2- 172.21.15.120:0/100519748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaf0101c90 0x7fbaf019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:08.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 >> 172.21.15.120:0/100519748 conn(0x7fbaf00fd700 msgr2=0x7fbaf00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:08.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 shutdown_connections 2024-09-10T23:13:08.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:08.167+0000 7fbaf480b640 1 -- 172.21.15.120:0/100519748 wait complete. 2024-09-10T23:13:08.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:07 smithi153 ceph-mon[34951]: pgmap v967: 97 pgs: 97 active+clean; 238 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-09-10T23:13:08.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197189 21.72 MB/sec execute 363 sec latency 87.847 ms 2024-09-10T23:13:09.884 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197539 21.72 MB/sec execute 364 sec latency 159.974 ms 2024-09-10T23:13:09.940 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:13:09.940 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:13:10.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:09 smithi176 ceph-mon[34982]: pgmap v968: 97 pgs: 97 active+clean; 238 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-09-10T23:13:10.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:09 smithi120 ceph-mon[31247]: pgmap v968: 97 pgs: 97 active+clean; 238 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-09-10T23:13:10.197 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:10.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:09 smithi153 ceph-mon[34951]: pgmap v968: 97 pgs: 97 active+clean; 238 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 -- 172.21.15.120:0/3042843576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc101890 msgr2=0x7fe7cc101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/3042843576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc101890 0x7fe7cc101c90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009930 tx=0x7fe7bc02a420 comp rx=0 tx=0).stop 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 -- 172.21.15.120:0/3042843576 shutdown_connections 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/3042843576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1033a0 0x7fe7cc109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/3042843576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1029e0 0x7fe7cc102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/3042843576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc101890 0x7fe7cc101c90 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 -- 172.21.15.120:0/3042843576 >> 172.21.15.120:0/3042843576 conn(0x7fe7cc0fd150 msgr2=0x7fe7cc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 -- 172.21.15.120:0/3042843576 shutdown_connections 2024-09-10T23:13:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.688+0000 7fe7d21ad640 1 -- 172.21.15.120:0/3042843576 wait complete. 2024-09-10T23:13:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 Processor -- start 2024-09-10T23:13:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 -- start start 2024-09-10T23:13:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc1029e0 0x7fe7cc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1033a0 0x7fe7cc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10c3f0 con 0x7fe7cc1029e0 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10c270 con 0x7fe7cc101890 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7d21ad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10c570 con 0x7fe7cc1033a0 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44946/0 (socket says 172.21.15.120:44946) 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7cb7fe640 1 -- 172.21.15.120:0/640996471 learned_addr learned my addr 172.21.15.120:0/640996471 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc1029e0 0x7fe7cc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.689+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1033a0 0x7fe7cc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 -- 172.21.15.120:0/640996471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1033a0 msgr2=0x7fe7cc19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1033a0 0x7fe7cc19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 -- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc1029e0 msgr2=0x7fe7cc19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc1029e0 0x7fe7cc19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 -- 172.21.15.120:0/640996471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc19cb40 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009a50 tx=0x7fe7bc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7c97fa640 1 -- 172.21.15.120:0/640996471 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc03f030 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc19cdd0 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7c97fa640 1 -- 172.21.15.120:0/640996471 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7bc003cb0 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7c97fa640 1 -- 172.21.15.120:0/640996471 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc035070 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.690+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc1a3500 con 0x7fe7cc101890 2024-09-10T23:13:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.693+0000 7fe7c97fa640 1 -- 172.21.15.120:0/640996471 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe7bc03bb90 con 0x7fe7cc101890 2024-09-10T23:13:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.693+0000 7fe7c97fa640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 0x7fe7b80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 Processor -- start 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 0x7fe7b80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7c97fa640 1 -- 172.21.15.120:0/640996471 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe7bc106810 con 0x7fe7cc101890 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 -- start start 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f88c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f89190 0x55b475f8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b475ef86f0 con 0x55b475f8bbd0 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b475ee7b20 con 0x55b475f89190 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7d5059740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b475eeb930 con 0x55b475f86860 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42498/0 (socket says 172.21.15.120:42498) 2024-09-10T23:13:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 -- 172.21.15.120:0/3694819600 learned_addr learned my addr 172.21.15.120:0/3694819600 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 0x7fe7b80b77a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009c20 tx=0x7fe7b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f89190 0x55b475f8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 -- 172.21.15.120:0/3694819600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 msgr2=0x55b475f88c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f88c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f88c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 -- 172.21.15.120:0/3694819600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f89190 msgr2=0x55b475f8b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f89190 0x55b475f8b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cbfff640 1 -- 172.21.15.120:0/3694819600 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b475ead530 con 0x55b475f8bbd0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.694+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f88c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.695+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400ae60 tx=0x7fe7c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8641b8eef0a41758 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.695+0000 7fe7c1ffb640 1 -- 172.21.15.120:0/3694819600 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4003b70 con 0x55b475f8bbd0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.695+0000 7fe7c1ffb640 1 -- 172.21.15.120:0/3694819600 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c4003d10 con 0x55b475f8bbd0 2024-09-10T23:13:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.695+0000 7fe7c1ffb640 1 -- 172.21.15.120:0/3694819600 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4010ba0 con 0x55b475f8bbd0 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- 172.21.15.120:0/3694819600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 msgr2=0x55b475f8e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400ae60 tx=0x7fe7c400f790 comp rx=0 tx=0).stop 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- 172.21.15.120:0/3694819600 shutdown_connections 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f8e090 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f89190 0x55b475f8b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3694819600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f88c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- 172.21.15.120:0/3694819600 >> 172.21.15.120:0/3694819600 conn(0x55b475f819b0 msgr2=0x55b475f82740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- 172.21.15.120:0/3694819600 shutdown_connections 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- 172.21.15.120:0/3694819600 wait complete. 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.696+0000 7fe7d5059740 1 -- start start 2024-09-10T23:13:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 Processor -- start 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f8fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f89190 0x55b475f90540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b475e1fab0 con 0x55b475f8bbd0 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55b475e8d730 con 0x55b475f86860 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b475f8ebb0 con 0x55b475f89190 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7d5059740 10 client.0 _ll_register_callbacks cb 0x55b475ee16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42502/0 (socket says 172.21.15.120:42502) 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 -- 172.21.15.120:0/522396715 learned_addr learned my addr 172.21.15.120:0/522396715 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f8fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f89190 0x55b475f90540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 -- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f89190 msgr2=0x55b475f90540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f89190 0x55b475f90540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 -- 172.21.15.120:0/522396715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 msgr2=0x55b475f8fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f8fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.697+0000 7fe7cbfff640 1 -- 172.21.15.120:0/522396715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b475c49790 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b010 tx=0x7fe7c4004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eb8f76cc81b37c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4026070 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe7c4010d30 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c4003d10 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7c4032050 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 10 client.19467 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b475cc2620 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c4003d10 con 0x55b475f8bbd0 2024-09-10T23:13:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe7c4018710 con 0x55b475f8bbd0 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 10 client.19467 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 10 client.19467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 20 client.19467 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b475f9b3d0 0x55b475f9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 4 client.19467 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.698+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b475efa640 con 0x55b475f9b3d0 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.699+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b475f9b3d0 0x55b475f9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.699+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b475f9b3d0 0x55b475f9d7c0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf1dd472229004b server_cookie=e7f4330882b7da05 in_seq=0 out_seq=0 2024-09-10T23:13:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.699+0000 7fe7aaffd640 10 client.19467 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.701+0000 7fe7aaffd640 1 -- 172.21.15.120:0/522396715 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b475efa640 con 0x55b475f9b3d0 2024-09-10T23:13:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.701+0000 7fe7aaffd640 10 client.19467 handle_command_reply: tid=0 2024-09-10T23:13:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.701+0000 7fe7d5059740 1 client.19467 shutdown 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.701+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 msgr2=0x55b475f91110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.701+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b010 tx=0x7fe7c4004cd0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 shutdown_connections 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b475f9b3d0 0x55b475f9d7c0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f8bbd0 0x55b475f91110 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7aaffd640 0 client.19467 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f89190 0x55b475f90540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 --2- 172.21.15.120:0/522396715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f8fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 >> 172.21.15.120:0/522396715 conn(0x55b475f819b0 msgr2=0x55b475f80bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 shutdown_connections 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 1 -- 172.21.15.120:0/522396715 wait complete. 2024-09-10T23:13:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.702+0000 7fe7d5059740 20 client.19467 trim_cache size 0 max 16384 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 Processor -- start 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 -- start start 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f8bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f8bbd0 0x55b475f85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1aa0a0 con 0x55b475f89190 2024-09-10T23:13:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1ac920 con 0x55b475f8bbd0 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.734+0000 7fe7d5059740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1aa710 con 0x55b475f86860 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42508/0 (socket says 172.21.15.120:42508) 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 -- 172.21.15.120:0/3473705160 learned_addr learned my addr 172.21.15.120:0/3473705160 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 -- 172.21.15.120:0/3473705160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 msgr2=0x55b475f8bfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f8bfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 -- 172.21.15.120:0/3473705160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f8bbd0 msgr2=0x55b475f85cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f8bbd0 0x55b475f85cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 -- 172.21.15.120:0/3473705160 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b475f9db30 con 0x55b475f89190 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009f00 tx=0x7fe7b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57848fb121da7c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c17fa640 1 -- 172.21.15.120:0/3473705160 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4003cf0 con 0x55b475f89190 2024-09-10T23:13:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c17fa640 1 -- 172.21.15.120:0/3473705160 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b4003e90 con 0x55b475f89190 2024-09-10T23:13:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.735+0000 7fe7c17fa640 1 -- 172.21.15.120:0/3473705160 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4026770 con 0x55b475f89190 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 -- 172.21.15.120:0/3473705160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 msgr2=0x55b475f8c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009f00 tx=0x7fe7b4014000 comp rx=0 tx=0).stop 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 -- 172.21.15.120:0/3473705160 shutdown_connections 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f8bbd0 0x55b475f85cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f8c4f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 --2- 172.21.15.120:0/3473705160 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f86860 0x55b475f8bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 -- 172.21.15.120:0/3473705160 >> 172.21.15.120:0/3473705160 conn(0x55b475f819b0 msgr2=0x55b475f82450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 -- 172.21.15.120:0/3473705160 shutdown_connections 2024-09-10T23:13:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.736+0000 7fe7d5059740 1 -- 172.21.15.120:0/3473705160 wait complete. 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 -- start start 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 Processor -- start 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f8bbd0 0x55b475f968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1aade0 con 0x55b475f89190 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1ac250 con 0x55b475f86860 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1ab4b0 con 0x55b475f8bbd0 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.737+0000 7fe7d5059740 10 client.0 _ll_register_callbacks cb 0x7fe7c400ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7cb7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44990/0 (socket says 172.21.15.120:44990) 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7cb7fe640 1 -- 172.21.15.120:0/4183784785 learned_addr learned my addr 172.21.15.120:0/4183784785 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7cbfff640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f8bbd0 0x55b475f968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 -- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f8bbd0 msgr2=0x55b475f968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f8bbd0 0x55b475f968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 -- 172.21.15.120:0/4183784785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 msgr2=0x55b475f976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 -- 172.21.15.120:0/4183784785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b475f96e20 con 0x55b475f89190 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7c3fff640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f963a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4013fd0 tx=0x7fe7b401b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74a610e2b2d94f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4032020 con 0x55b475f89190 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe7b401bc00 con 0x55b475f89190 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.738+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b4003e90 con 0x55b475f89190 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7b4005050 con 0x55b475f89190 2024-09-10T23:13:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 10 client.19473 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b475f95d80 con 0x55b475f89190 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b40046c0 con 0x55b475f89190 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe7b4025070 con 0x55b475f89190 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 10 client.19473 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 10 client.19473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 20 client.19473 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b4760088c0 0x55b47600ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 4 client.19473 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.739+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe7b4003e90 con 0x55b4760088c0 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.740+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b4760088c0 0x55b47600ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.740+0000 7fe7aaffd640 10 client.19473 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:10.740+0000 7fe7cb7fe640 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b4760088c0 0x55b47600ace0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3abb0e14b3e5f9 server_cookie=9e92ad7f6530d38e in_seq=0 out_seq=0 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 -- 172.21.15.120:0/2088438567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102de0 msgr2=0x7f78dc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 --2- 172.21.15.120:0/2088438567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102de0 0x7f78dc1031c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009900 tx=0x7f78d802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 -- 172.21.15.120:0/2088438567 shutdown_connections 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 --2- 172.21.15.120:0/2088438567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc109170 0x7f78dc10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 --2- 172.21.15.120:0/2088438567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc1087f0 0x7f78dc108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 --2- 172.21.15.120:0/2088438567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc102de0 0x7f78dc1031c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 -- 172.21.15.120:0/2088438567 >> 172.21.15.120:0/2088438567 conn(0x7f78dc0fe830 msgr2=0x7f78dc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.763+0000 7f78e3a77640 1 -- 172.21.15.120:0/2088438567 shutdown_connections 2024-09-10T23:13:10.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 -- 172.21.15.120:0/2088438567 wait complete. 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 Processor -- start 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 -- start start 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc102de0 0x7f78dc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 0x7f78dc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112e80 con 0x7f78dc1087f0 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112d00 con 0x7f78dc102de0 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e3a77640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc113000 con 0x7f78dc109170 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42532/0 (socket says 172.21.15.120:42532) 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e1012640 1 -- 172.21.15.120:0/3750612988 learned_addr learned my addr 172.21.15.120:0/3750612988 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e1813640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc102de0 0x7f78dc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.764+0000 7f78e2014640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 0x7f78dc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 -- 172.21.15.120:0/3750612988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 msgr2=0x7f78dc075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 0x7f78dc075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 -- 172.21.15.120:0/3750612988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc102de0 msgr2=0x7f78dc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc102de0 0x7f78dc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 -- 172.21.15.120:0/3750612988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc1a7710 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e2014640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 0x7f78dc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e1012640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00e9f0 tx=0x7f78cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78d2ffd640 1 -- 172.21.15.120:0/3750612988 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00cd80 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78d2ffd640 1 -- 172.21.15.120:0/3750612988 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc0054d0 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc1a7a00 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78d2ffd640 1 -- 172.21.15.120:0/3750612988 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc010690 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.765+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a7f70 con 0x7f78dc1087f0 2024-09-10T23:13:10.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78d2ffd640 1 -- 172.21.15.120:0/3750612988 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f78cc004e90 con 0x7f78dc1087f0 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 Processor -- start 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 -- start start 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594e4345e60 con 0x5594e43db790 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594e4335290 con 0x5594e43ceee0 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5594e43390a0 con 0x5594e43cf800 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42548/0 (socket says 172.21.15.120:42548) 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.768+0000 7f78e1012640 1 -- 172.21.15.120:0/721442871 learned_addr learned my addr 172.21.15.120:0/721442871 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78d2ffd640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 0x7f78c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e2014640 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1813640 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 -- 172.21.15.120:0/721442871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 msgr2=0x5594e43db250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43db250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 -- 172.21.15.120:0/721442871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 msgr2=0x5594e43cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 -- 172.21.15.120:0/721442871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e4093930 con 0x5594e43db790 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e2014640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 0x7f78c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78d2ffd640 1 -- 172.21.15.120:0/3750612988 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f78cc0e5f90 con 0x7f78dc1087f0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e1012640 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0903e0 tx=0x7f78cc070fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea66607224fc62c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78c7fff640 1 -- 172.21.15.120:0/721442871 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0ac170 con 0x5594e43db790 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78e2014640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 0x7f78c80b77a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f78dc077560 tx=0x7f78d4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78c7fff640 1 -- 172.21.15.120:0/721442871 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc0ac330 con 0x5594e43db790 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.769+0000 7f78c7fff640 1 -- 172.21.15.120:0/721442871 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc092420 con 0x5594e43db790 2024-09-10T23:13:10.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 -- 172.21.15.120:0/721442871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 msgr2=0x5594e43ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0903e0 tx=0x7f78cc070fa0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 -- 172.21.15.120:0/721442871 shutdown_connections 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43db790 0x5594e43ddb80 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/721442871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 -- 172.21.15.120:0/721442871 >> 172.21.15.120:0/721442871 conn(0x5594e43ccd10 msgr2=0x5594e43cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 -- 172.21.15.120:0/721442871 shutdown_connections 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.770+0000 7f78e5e4c740 1 -- 172.21.15.120:0/721442871 wait complete. 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 -- start start 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 Processor -- start 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43ceee0 0x5594e43db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5594e4196890 con 0x5594e43cf800 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5594e42cfd40 con 0x5594e43db790 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.771+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5594e43de3e0 con 0x5594e43ceee0 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e5e4c740 10 client.0 _ll_register_callbacks cb 0x5594e4333c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42560/0 (socket says 172.21.15.120:42560) 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 -- 172.21.15.120:0/2644968576 learned_addr learned my addr 172.21.15.120:0/2644968576 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 -- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43ceee0 msgr2=0x5594e43db250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43ceee0 0x5594e43db250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 -- 172.21.15.120:0/2644968576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 msgr2=0x5594e43d8e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43d8e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 -- 172.21.15.120:0/2644968576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e43f8720 con 0x5594e43cf800 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e1012640 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0a8df0 tx=0x7f78cc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3496c15658b03cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc071740 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f78cc0e80c0 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0ac330 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78cc0abf90 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e5e4c740 10 client.19485 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.772+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594e44038e0 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc0ac330 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f78cc0b5920 con 0x5594e43cf800 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 10 client.19485 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 20 client.19485 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4404c30 0x5594e4407020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 4 client.19485 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594e4347db0 con 0x5594e4404c30 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e1813640 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4404c30 0x5594e4407020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78c57fa640 10 client.19485 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.773+0000 7f78e1813640 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4404c30 0x5594e4407020 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530b6a17d8984eb8 server_cookie=575fa601339a2128 in_seq=0 out_seq=0 2024-09-10T23:13:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.827+0000 7f78c57fa640 1 -- 172.21.15.120:0/2644968576 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594e4347db0 con 0x5594e4404c30 2024-09-10T23:13:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.827+0000 7f78c57fa640 10 client.19485 handle_command_reply: tid=0 2024-09-10T23:13:10.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.827+0000 7f78e5e4c740 1 client.19485 shutdown 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.827+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 msgr2=0x5594e43d86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.827+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0a8df0 tx=0x7f78cc0048f0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 shutdown_connections 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4404c30 0x5594e4407020 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d86e0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/2644968576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43ceee0 0x5594e43db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.829+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 >> 172.21.15.120:0/2644968576 conn(0x5594e43ccd10 msgr2=0x5594e43cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.831+0000 7f78c57fa640 0 client.19485 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.831+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 shutdown_connections 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.834+0000 7f78e5e4c740 1 -- 172.21.15.120:0/2644968576 wait complete. 2024-09-10T23:13:10.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.834+0000 7f78e5e4c740 20 client.19485 trim_cache size 0 max 16384 2024-09-10T23:13:10.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.873+0000 7f78e5e4c740 1 Processor -- start 2024-09-10T23:13:10.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.873+0000 7f78e5e4c740 1 -- start start 2024-09-10T23:13:10.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.873+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.873+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43db790 0x5594e43cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e5e4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc06eb30 con 0x5594e43cf800 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e1012640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42566/0 (socket says 172.21.15.120:42566) 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e1012640 1 -- 172.21.15.120:0/1712408333 learned_addr learned my addr 172.21.15.120:0/1712408333 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0713b0 con 0x5594e43ceee0 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc06f1a0 con 0x5594e43db790 2024-09-10T23:13:10.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.874+0000 7f78e2014640 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43db790 0x5594e43cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1012640 1 -- 172.21.15.120:0/1712408333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43db790 msgr2=0x5594e43cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1012640 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43db790 0x5594e43cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.879 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1012640 1 -- 172.21.15.120:0/1712408333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 msgr2=0x5594e43d58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1813640 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1012640 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43d58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.878+0000 7f78e1012640 1 -- 172.21.15.120:0/1712408333 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e43dae80 con 0x5594e43cf800 2024-09-10T23:13:10.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.879+0000 7f78e1012640 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c6a0 tx=0x7f78cc009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552fe842a3b488b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 197927 21.73 MB/sec execute 365 sec latency 118.879 ms 2024-09-10T23:13:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.881+0000 7f78c77fe640 1 -- 172.21.15.120:0/1712408333 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0ac330 con 0x5594e43cf800 2024-09-10T23:13:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.881+0000 7f78c77fe640 1 -- 172.21.15.120:0/1712408333 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc0acf20 con 0x5594e43cf800 2024-09-10T23:13:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.881+0000 7f78c77fe640 1 -- 172.21.15.120:0/1712408333 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0aba80 con 0x5594e43cf800 2024-09-10T23:13:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.883+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 msgr2=0x5594e43d5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.883+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c6a0 tx=0x7f78cc009720 comp rx=0 tx=0).stop 2024-09-10T23:13:10.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.887+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 shutdown_connections 2024-09-10T23:13:10.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.887+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43db790 0x5594e43cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.887+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43cf800 0x5594e43d5e30 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.887+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1712408333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43ceee0 0x5594e43d58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.887+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 >> 172.21.15.120:0/1712408333 conn(0x5594e43ccd10 msgr2=0x5594e43d6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.892+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 shutdown_connections 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.894+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1712408333 wait complete. 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.894+0000 7f78e5e4c740 1 -- start start 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.895+0000 7f78e5e4c740 1 Processor -- start 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.895+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43ceee0 0x5594e43f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.895+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.898 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e5e4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e1012640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e1012640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40796/0 (socket says 172.21.15.120:40796) 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e1012640 1 -- 172.21.15.120:0/1304489464 learned_addr learned my addr 172.21.15.120:0/1304489464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e2014640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc06f870 con 0x5594e43ceee0 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc070ce0 con 0x5594e43db790 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.896+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc06ff40 con 0x5594e43cf800 2024-09-10T23:13:10.899 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.897+0000 7f78e5e4c740 10 client.0 _ll_register_callbacks cb 0x7f78cc0a8f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:10.904 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.902+0000 7f78e1813640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43ceee0 0x5594e43f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e2014640 1 -- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 msgr2=0x5594e43f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e2014640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e2014640 1 -- 172.21.15.120:0/1304489464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43ceee0 msgr2=0x5594e43f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e2014640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43ceee0 0x5594e43f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e2014640 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e43f0000 con 0x5594e43db790 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.912+0000 7f78e1012640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:10.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.913+0000 7f78e2014640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43ef800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5594e43d98b0 tx=0x7f78d4004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8fcde3abed2e6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:10.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.918+0000 7f78c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:13:10.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.918+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4029060 con 0x5594e43db790 2024-09-10T23:13:10.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.918+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f78d401bb40 con 0x5594e43db790 2024-09-10T23:13:10.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.918+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc0acf20 con 0x5594e43db790 2024-09-10T23:13:10.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.922+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78d4015070 con 0x5594e43db790 2024-09-10T23:13:10.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.925+0000 7f78e5e4c740 10 client.28664 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:10.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.925+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594e4472560 con 0x5594e43db790 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d401bd40 con 0x5594e43db790 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f78d401a520 con 0x5594e43db790 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 10 client.28664 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 10 client.28664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 20 client.28664 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4472c70 0x5594e4475090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 4 client.28664 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:13:10.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.930+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f78cc0acf20 con 0x5594e4472c70 2024-09-10T23:13:10.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.932+0000 7f78e1813640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4472c70 0x5594e4475090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:10.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.944+0000 7f78e1813640 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4472c70 0x5594e4475090 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b073fa585c28eb server_cookie=a1770e5518c25662 in_seq=0 out_seq=0 2024-09-10T23:13:10.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.944+0000 7f78c57fa640 10 client.28664 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78c57fa640 1 -- 172.21.15.120:0/1304489464 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f78cc0acf20 con 0x5594e4472c70 2024-09-10T23:13:10.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78c57fa640 10 client.28664 handle_command_reply: tid=0 2024-09-10T23:13:10.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78e5e4c740 1 client.28664 shutdown 2024-09-10T23:13:10.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 msgr2=0x5594e43ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43ef800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5594e43d98b0 tx=0x7f78d4004b50 comp rx=0 tx=0).stop 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.958+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 shutdown_connections 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5594e4472c70 0x5594e4475090 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594e43db790 0x5594e43ef800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594e43cf800 0x5594e43f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 --2- 172.21.15.120:0/1304489464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5594e43ceee0 0x5594e43f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.961 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 >> 172.21.15.120:0/1304489464 conn(0x5594e43ccd10 msgr2=0x5594e43cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78c57fa640 0 client.28664 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 shutdown_connections 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 1 -- 172.21.15.120:0/1304489464 wait complete. 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.959+0000 7f78e5e4c740 20 client.28664 trim_cache size 0 max 16384 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:13:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "1e09b8ea-07fb-48f0-a34b-c815ff45daaa", 2024-09-10T23:13:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:13:10.963 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:10.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.960+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 msgr2=0x7f78c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.960+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 0x7f78c80b77a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f78dc077560 tx=0x7f78d4004a40 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.960+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 msgr2=0x7f78dc0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.960+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00e9f0 tx=0x7f78cc00edf0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.960+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 shutdown_connections 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f78c80b52e0 0x7f78c80b77a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78dc109170 0x7f78dc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78dc1087f0 0x7f78dc0759e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 --2- 172.21.15.120:0/3750612988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78dc102de0 0x7f78dc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 >> 172.21.15.120:0/3750612988 conn(0x7f78dc0fe830 msgr2=0x7f78dc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 shutdown_connections 2024-09-10T23:13:10.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:10.961+0000 7f78e3a77640 1 -- 172.21.15.120:0/3750612988 wait complete. 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.487+0000 7fe7aaffd640 1 -- 172.21.15.120:0/4183784785 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe7b4003e90 con 0x55b4760088c0 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7aaffd640 10 client.19473 handle_command_reply: tid=0 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 client.19473 shutdown 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 msgr2=0x55b475f963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f963a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4013fd0 tx=0x7fe7b401b740 comp rx=0 tx=0).stop 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 shutdown_connections 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55b4760088c0 0x55b47600ace0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b475f8bbd0 0x55b475f968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b475f89190 0x55b475f963a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7aaffd640 0 client.19473 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 --2- 172.21.15.120:0/4183784785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b475f86860 0x55b475f976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 >> 172.21.15.120:0/4183784785 conn(0x55b475f819b0 msgr2=0x55b475f8bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 shutdown_connections 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 1 -- 172.21.15.120:0/4183784785 wait complete. 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d5059740 20 client.19473 trim_cache size 0 max 16384 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:13:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:13:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:13:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 msgr2=0x7fe7b80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 0x7fe7b80b77a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009c20 tx=0x7fe7b4005f90 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 msgr2=0x7fe7cc19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.488+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009a50 tx=0x7fe7bc0042f0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 shutdown_connections 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe7b80b52e0 0x7fe7b80b77a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7cc1033a0 0x7fe7cc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe7cc1029e0 0x7fe7cc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 --2- 172.21.15.120:0/640996471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc101890 0x7fe7cc19b000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 >> 172.21.15.120:0/640996471 conn(0x7fe7cc0fd150 msgr2=0x7fe7cc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 shutdown_connections 2024-09-10T23:13:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:11.489+0000 7fe7d21ad640 1 -- 172.21.15.120:0/640996471 wait complete. 2024-09-10T23:13:11.684 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:13:11.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198036 21.69 MB/sec execute 366 sec latency 450.978 ms 2024-09-10T23:13:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:11 smithi120 ceph-mon[31247]: pgmap v969: 97 pgs: 97 active+clean; 244 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-09-10T23:13:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:11 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:13:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:11 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:11 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:13:11.957 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:11 smithi176 ceph-mon[34982]: pgmap v969: 97 pgs: 97 active+clean; 244 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-09-10T23:13:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:11 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:13:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:11 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:12.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:11 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:13:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:11 smithi153 ceph-mon[34951]: pgmap v969: 97 pgs: 97 active+clean; 244 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-09-10T23:13:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:11 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:13:12.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:11 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:12.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:11 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:13:12.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 -- 172.21.15.120:0/1977321642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c981076d0 msgr2=0x7f1c98107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/1977321642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c981076d0 0x7f1c98107b10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c8802a410 comp rx=0 tx=0).stop 2024-09-10T23:13:12.583 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 -- 172.21.15.120:0/1977321642 shutdown_connections 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/1977321642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c98108050 0x7f1c9810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/1977321642 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c981076d0 0x7f1c98107b10 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/1977321642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c98101cc0 0x7f1c981020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 -- 172.21.15.120:0/1977321642 >> 172.21.15.120:0/1977321642 conn(0x7f1c980fd770 msgr2=0x7f1c980ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 -- 172.21.15.120:0/1977321642 shutdown_connections 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.568+0000 7f1c9fb27640 1 -- 172.21.15.120:0/1977321642 wait complete. 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.569+0000 7f1c9fb27640 1 Processor -- start 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.569+0000 7f1c9fb27640 1 -- start start 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.569+0000 7f1c9fb27640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.584 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981076d0 0x7f1c981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c98108050 0x7f1c9819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98111d10 con 0x7f1c98101cc0 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98111b90 con 0x7f1c981076d0 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98111e90 con 0x7f1c98108050 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42606/0 (socket says 172.21.15.120:42606) 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/4027415539 learned_addr learned my addr 172.21.15.120:0/4027415539 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981076d0 0x7f1c981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/4027415539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c98108050 msgr2=0x7f1c9819ec50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c98108050 0x7f1c9819ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/4027415539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981076d0 msgr2=0x7f1c981998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981076d0 0x7f1c981998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/4027415539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9819f420 con 0x7f1c98101cc0 2024-09-10T23:13:12.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9f0 tx=0x7f1c9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c8effd640 1 -- 172.21.15.120:0/4027415539 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400cd80 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9819f710 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c8effd640 1 -- 172.21.15.120:0/4027415539 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c940108e0 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c8effd640 1 -- 172.21.15.120:0/4027415539 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94018980 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.570+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c981a5dd0 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 Processor -- start 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1c8effd640 1 -- 172.21.15.120:0/4027415539 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1c94010430 con 0x7f1c98101cc0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 -- start start 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 0x55c311a28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c31199ee50 con 0x55c311a28da0 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c31198e280 con 0x55c311a28480 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c311992090 con 0x55c311a39140 2024-09-10T23:13:12.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1c9e0c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 0x55c311a28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42620/0 (socket says 172.21.15.120:42620) 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.573+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/4166610882 learned_addr learned my addr 172.21.15.120:0/4166610882 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/4166610882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 msgr2=0x55c311a3b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/4166610882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 msgr2=0x55c311a38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c8effd640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 0x7f1c840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/4166610882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3116ec650 con 0x55c311a28480 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:12.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 0x7f1c840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c8effd640 1 -- 172.21.15.120:0/4027415539 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c940e5fa0 con 0x7f1c98101cc0 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 0x55c311a28860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009ce0 tx=0x7f1c9000b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931bccb9bfd0068 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c83fff640 1 -- 172.21.15.120:0/4166610882 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90014610 con 0x55c311a28480 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c83fff640 1 -- 172.21.15.120:0/4166610882 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90014d60 con 0x55c311a28480 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.574+0000 7f1c83fff640 1 -- 172.21.15.120:0/4166610882 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001f9c0 con 0x55c311a28480 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- 172.21.15.120:0/4166610882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 msgr2=0x55c311a28860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 0x55c311a28860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009ce0 tx=0x7f1c9000b7e0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 0x7f1c840b77a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009fd0 tx=0x7f1c9001c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- 172.21.15.120:0/4166610882 shutdown_connections 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28da0 0x55c311a38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/4166610882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28480 0x55c311a28860 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- 172.21.15.120:0/4166610882 >> 172.21.15.120:0/4166610882 conn(0x55c311a262b0 msgr2=0x55c311a27450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- 172.21.15.120:0/4166610882 shutdown_connections 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- 172.21.15.120:0/4166610882 wait complete. 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.575+0000 7f1ca1efc740 1 -- start start 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 Processor -- start 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c3117ef700 con 0x55c311a28480 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c311929110 con 0x55c311a28da0 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c311a3bdf0 con 0x55c311a39140 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1ca1efc740 10 client.0 _ll_register_callbacks cb 0x55c311942990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.589 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42632/0 (socket says 172.21.15.120:42632) 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.576+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2375283742 learned_addr learned my addr 172.21.15.120:0/2375283742 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 msgr2=0x55c311a363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2375283742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 msgr2=0x55c311a35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2375283742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c311956c40 con 0x55c311a28480 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1c940a8d60 tx=0x7f1c94018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ae70a9f428274 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c817fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940cf070 con 0x55c311a28480 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c940ab9f0 con 0x55c311a28480 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c90014d60 con 0x55c311a28480 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c940a8630 con 0x55c311a28480 2024-09-10T23:13:12.590 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.577+0000 7f1ca1efc740 10 client.19503 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c311a47740 con 0x55c311a28480 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90014d60 con 0x55c311a28480 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1c940adac0 con 0x55c311a28480 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 10 client.19503 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 20 client.19503 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311a48c00 0x55c311a4b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 4 client.19503 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.578+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3119a0da0 con 0x55c311a48c00 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.579+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311a48c00 0x55c311a4b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.579+0000 7f1c817fa640 10 client.19503 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:12.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.579+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311a48c00 0x55c311a4b070 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4773c16e6372c555 server_cookie=b72ef26f79a3ebd2 in_seq=0 out_seq=0 2024-09-10T23:13:12.592 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.582+0000 7f1c817fa640 1 -- 172.21.15.120:0/2375283742 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3119a0da0 con 0x55c311a48c00 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.582+0000 7f1c817fa640 10 client.19503 handle_command_reply: tid=0 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.582+0000 7f1ca1efc740 1 client.19503 shutdown 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.582+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 msgr2=0x55c311a389e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.582+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1c940a8d60 tx=0x7f1c94018420 comp rx=0 tx=0).stop 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 shutdown_connections 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311a48c00 0x55c311a4b070 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1c817fa640 0 client.19503 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2375283742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a389e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 >> 172.21.15.120:0/2375283742 conn(0x55c311a262b0 msgr2=0x55c311a2bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.593 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 shutdown_connections 2024-09-10T23:13:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2375283742 wait complete. 2024-09-10T23:13:12.594 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.583+0000 7f1ca1efc740 20 client.19503 trim_cache size 0 max 16384 2024-09-10T23:13:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.600+0000 7f1ca1efc740 1 Processor -- start 2024-09-10T23:13:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.600+0000 7f1ca1efc740 1 -- start start 2024-09-10T23:13:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 0x55c311a45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.603 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981aa1c0 con 0x55c311a39140 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981aca20 con 0x55c311a28da0 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981aa810 con 0x55c311a28480 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42638/0 (socket says 172.21.15.120:42638) 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/3436101981 learned_addr learned my addr 172.21.15.120:0/3436101981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 0x55c311a45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/3436101981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 msgr2=0x55c311a45080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 0x55c311a45080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.604 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/3436101981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 msgr2=0x55c311a39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 -- 172.21.15.120:0/3436101981 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c311a34d90 con 0x55c311a39140 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 0x55c311a45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c9e0c4640 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009c10 tx=0x7f1c90009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486c96b60084037f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.601+0000 7f1c837fe640 1 -- 172.21.15.120:0/3436101981 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c900098f0 con 0x55c311a39140 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1c837fe640 1 -- 172.21.15.120:0/3436101981 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90027070 con 0x55c311a39140 2024-09-10T23:13:12.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1c837fe640 1 -- 172.21.15.120:0/3436101981 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000fcf0 con 0x55c311a39140 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 -- 172.21.15.120:0/3436101981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 msgr2=0x55c311a37b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009c10 tx=0x7f1c90009c40 comp rx=0 tx=0).stop 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 -- 172.21.15.120:0/3436101981 shutdown_connections 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a39140 0x55c311a37b70 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/3436101981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a28480 0x55c311a45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 -- 172.21.15.120:0/3436101981 >> 172.21.15.120:0/3436101981 conn(0x55c311a262b0 msgr2=0x55c311a37460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 -- 172.21.15.120:0/3436101981 shutdown_connections 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.602+0000 7f1ca1efc740 1 -- 172.21.15.120:0/3436101981 wait complete. 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.603+0000 7f1ca1efc740 1 -- start start 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.603+0000 7f1ca1efc740 1 Processor -- start 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.603+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.603+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a3e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.603+0000 7f1ca1efc740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981aaee0 con 0x55c311a28480 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981ac350 con 0x55c311a28da0 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1ca1efc740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981ab5b0 con 0x55c311a39140 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1ca1efc740 10 client.0 _ll_register_callbacks cb 0x7f1c94009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42654/0 (socket says 172.21.15.120:42654) 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2966175454 learned_addr learned my addr 172.21.15.120:0/2966175454 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a3e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 msgr2=0x55c311a3e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2966175454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 msgr2=0x55c311a3e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a3e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 -- 172.21.15.120:0/2966175454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c311a3eee0 con 0x55c311a28480 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a3e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c9d8c3640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ade90 tx=0x7f1c940abc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d962e52c73cb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c817fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940cf070 con 0x55c311a28480 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1c940a82f0 con 0x55c311a28480 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.604+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c90027070 con 0x55c311a28480 2024-09-10T23:13:12.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c940e82d0 con 0x55c311a28480 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 10 client.19509 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c311ab6cd0 con 0x55c311a28480 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90027070 con 0x55c311a28480 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1c940b2400 con 0x55c311a28480 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 10 client.19509 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 20 client.19509 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311ab7400 0x55c311ab9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 4 client.19509 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.605+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c31199c850 con 0x55c311ab7400 2024-09-10T23:13:12.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.606+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311ab7400 0x55c311ab9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:12.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.608+0000 7f1c9d0c2640 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311ab7400 0x55c311ab9820 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2793ebba22f423 server_cookie=7cabb037e4cf6050 in_seq=0 out_seq=0 2024-09-10T23:13:12.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.608+0000 7f1c817fa640 10 client.19509 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.610+0000 7f1c817fa640 1 -- 172.21.15.120:0/2966175454 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10226 (crc 0 0 0) 0x55c31199c850 con 0x55c311ab7400 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.610+0000 7f1c817fa640 10 client.19509 handle_command_reply: tid=0 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 client.19509 shutdown 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 msgr2=0x55c311a41950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ade90 tx=0x7f1c940abc20 comp rx=0 tx=0).stop 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 shutdown_connections 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c311ab7400 0x55c311ab9820 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c311a39140 0x55c311a3e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c311a28da0 0x55c311a3e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1c817fa640 0 client.19509 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:12.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 --2- 172.21.15.120:0/2966175454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c311a28480 0x55c311a41950 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 >> 172.21.15.120:0/2966175454 conn(0x55c311a262b0 msgr2=0x55c311a37c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 shutdown_connections 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 1 -- 172.21.15.120:0/2966175454 wait complete. 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.611+0000 7f1ca1efc740 20 client.19509 trim_cache size 0 max 16384 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:13:12.614 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "1e09b8ea-07fb-48f0-a34b-c815ff45daaa", 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.615 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:13:12.616 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.617 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.618 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.619 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:13:12.620 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.621 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:13:12.622 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.623 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.624 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:13:12.625 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.626 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.627 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.628 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.629 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.630 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:13:12.631 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.632 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.633 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.634 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.635 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:12.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 msgr2=0x7f1c840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 0x7f1c840b77a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009fd0 tx=0x7f1c9001c000 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 msgr2=0x7f1c98199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9f0 tx=0x7f1c9400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 shutdown_connections 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1c840b52e0 0x7f1c840b77a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c98108050 0x7f1c9819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981076d0 0x7f1c981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 --2- 172.21.15.120:0/4027415539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c98101cc0 0x7f1c98199370 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 >> 172.21.15.120:0/4027415539 conn(0x7f1c980fd770 msgr2=0x7f1c98104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 shutdown_connections 2024-09-10T23:13:12.638 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:12.612+0000 7f1c9fb27640 1 -- 172.21.15.120:0/4027415539 wait complete. 2024-09-10T23:13:12.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198164 21.64 MB/sec execute 367 sec latency 300.326 ms 2024-09-10T23:13:13.625 INFO:tasks.cephfs.filesystem:scrub status for tag:1e09b8ea-07fb-48f0-a34b-c815ff45daaa - {'path': '/', 'tag': '1e09b8ea-07fb-48f0-a34b-c815ff45daaa', 'options': 'recursive,force'} 2024-09-10T23:13:13.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198593 21.66 MB/sec execute 368 sec latency 65.166 ms 2024-09-10T23:13:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:13 smithi120 ceph-mon[31247]: pgmap v970: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-09-10T23:13:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:13 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:14.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:13 smithi176 ceph-mon[34982]: pgmap v970: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-09-10T23:13:14.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:13 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:14.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:13 smithi153 ceph-mon[34951]: pgmap v970: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-09-10T23:13:14.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:13 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:14.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 198877 21.64 MB/sec execute 369 sec latency 215.780 ms 2024-09-10T23:13:15.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199193 21.63 MB/sec execute 370 sec latency 73.767 ms 2024-09-10T23:13:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:15 smithi120 ceph-mon[31247]: pgmap v971: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-09-10T23:13:15.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:13:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:15 smithi176 ceph-mon[34982]: pgmap v971: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-09-10T23:13:16.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:13:16.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:15 smithi153 ceph-mon[34951]: pgmap v971: 97 pgs: 97 active+clean; 246 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-09-10T23:13:16.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:13:16.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 199651 21.62 MB/sec execute 371 sec latency 48.119 ms 2024-09-10T23:13:17.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 200060 21.61 MB/sec execute 372 sec latency 79.064 ms 2024-09-10T23:13:18.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:17 smithi176 ceph-mon[34982]: pgmap v972: 97 pgs: 97 active+clean; 230 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-10T23:13:18.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:17 smithi120 ceph-mon[31247]: pgmap v972: 97 pgs: 97 active+clean; 230 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-10T23:13:18.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:17 smithi153 ceph-mon[34951]: pgmap v972: 97 pgs: 97 active+clean; 230 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-10T23:13:18.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 200673 21.65 MB/sec execute 373 sec latency 37.160 ms 2024-09-10T23:13:19.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 201116 21.64 MB/sec execute 374 sec latency 218.497 ms 2024-09-10T23:13:20.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:19 smithi176 ceph-mon[34982]: pgmap v973: 97 pgs: 97 active+clean; 237 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-09-10T23:13:20.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:19 smithi120 ceph-mon[31247]: pgmap v973: 97 pgs: 97 active+clean; 237 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-09-10T23:13:20.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:19 smithi153 ceph-mon[34951]: pgmap v973: 97 pgs: 97 active+clean; 237 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-09-10T23:13:20.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 201559 21.67 MB/sec execute 375 sec latency 64.801 ms 2024-09-10T23:13:21.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 202104 21.70 MB/sec execute 376 sec latency 57.770 ms 2024-09-10T23:13:22.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:21 smithi176 ceph-mon[34982]: pgmap v974: 97 pgs: 97 active+clean; 245 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-09-10T23:13:22.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:21 smithi120 ceph-mon[31247]: pgmap v974: 97 pgs: 97 active+clean; 245 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-09-10T23:13:22.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:21 smithi153 ceph-mon[34951]: pgmap v974: 97 pgs: 97 active+clean; 245 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-09-10T23:13:22.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 202452 21.69 MB/sec execute 377 sec latency 79.618 ms 2024-09-10T23:13:23.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 202854 21.69 MB/sec execute 378 sec latency 42.538 ms 2024-09-10T23:13:24.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:23 smithi176 ceph-mon[34982]: pgmap v975: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-10T23:13:24.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:23 smithi120 ceph-mon[31247]: pgmap v975: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-10T23:13:24.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:23 smithi153 ceph-mon[34951]: pgmap v975: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.622+0000 7ffa046dd640 1 -- 172.21.15.120:0/272868866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc104ba0 msgr2=0x7ff9fc104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.622+0000 7ffa046dd640 1 --2- 172.21.15.120:0/272868866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc104fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009900 tx=0x7ff9f802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 -- 172.21.15.120:0/272868866 shutdown_connections 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 --2- 172.21.15.120:0/272868866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc1066b0 0x7ff9fc10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 --2- 172.21.15.120:0/272868866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 --2- 172.21.15.120:0/272868866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc104fa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 -- 172.21.15.120:0/272868866 >> 172.21.15.120:0/272868866 conn(0x7ff9fc100440 msgr2=0x7ff9fc102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 -- 172.21.15.120:0/272868866 shutdown_connections 2024-09-10T23:13:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.623+0000 7ffa046dd640 1 -- 172.21.15.120:0/272868866 wait complete. 2024-09-10T23:13:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 Processor -- start 2024-09-10T23:13:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 -- start start 2024-09-10T23:13:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc1066b0 0x7ff9fc19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc110810 con 0x7ff9fc105cf0 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc110690 con 0x7ff9fc1066b0 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa046dd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc110990 con 0x7ff9fc104ba0 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa02479640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa02479640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:59766/0 (socket says 172.21.15.120:59766) 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.624+0000 7ffa02479640 1 -- 172.21.15.120:0/4031452621 learned_addr learned my addr 172.21.15.120:0/4031452621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa01c78640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 -- 172.21.15.120:0/4031452621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc1066b0 msgr2=0x7ff9fc19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc1066b0 0x7ff9fc19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 -- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 msgr2=0x7ff9fc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 -- 172.21.15.120:0/4031452621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc19c8b0 con 0x7ff9fc104ba0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa01c78640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa02479640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009850 tx=0x7ff9f80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ff9f37fe640 1 -- 172.21.15.120:0/4031452621 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f803f030 con 0x7ff9fc104ba0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc19cb40 con 0x7ff9fc104ba0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ff9f37fe640 1 -- 172.21.15.120:0/4031452621 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f8003cb0 con 0x7ff9fc104ba0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.625+0000 7ff9f37fe640 1 -- 172.21.15.120:0/4031452621 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8035070 con 0x7ff9fc104ba0 2024-09-10T23:13:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.626+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1a3090 con 0x7ff9fc104ba0 2024-09-10T23:13:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.629+0000 7ffa074a9740 1 Processor -- start 2024-09-10T23:13:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.630+0000 7ffa074a9740 1 -- start start 2024-09-10T23:13:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.630+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.630+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f7fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.630+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.630+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:59028/0 (socket says 172.21.15.120:59028) 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199f828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa074a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560199ee46f0 con 0x560199f7d9a0 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa074a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560199ed3b20 con 0x560199f7b070 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa074a9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560199ed7930 con 0x560199f803e0 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 -- 172.21.15.120:0/3567893294 learned_addr learned my addr 172.21.15.120:0/3567893294 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa01c78640 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199f828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 -- 172.21.15.120:0/3567893294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 msgr2=0x560199f828a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199f828a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 -- 172.21.15.120:0/3567893294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 msgr2=0x560199f7fd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f7fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.631+0000 7ffa02c7a640 1 -- 172.21.15.120:0/3567893294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560199e99530 con 0x560199f7b070 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.632+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009e70 tx=0x7ff9f400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a82ef564a52dfc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.632+0000 7ff9f37fe640 1 -- 172.21.15.120:0/4031452621 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff9f803bc00 con 0x7ff9fc104ba0 2024-09-10T23:13:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.633+0000 7ff9f37fe640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 0x7ff9e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.633+0000 7ffa02479640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 0x7ff9e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.634+0000 7ff9f37fe640 1 -- 172.21.15.120:0/4031452621 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff9f8106250 con 0x7ff9fc104ba0 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.634+0000 7ffa02479640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 0x7ff9e80b77a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80c9cb0 tx=0x7ff9f800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.634+0000 7ff9f0ff9640 1 -- 172.21.15.120:0/3567893294 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004d20 con 0x560199f7b070 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.634+0000 7ff9f0ff9640 1 -- 172.21.15.120:0/3567893294 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f400ee70 con 0x560199f7b070 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.634+0000 7ff9f0ff9640 1 -- 172.21.15.120:0/3567893294 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4005020 con 0x560199f7b070 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.635+0000 7ffa074a9740 1 -- 172.21.15.120:0/3567893294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 msgr2=0x560199f7d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.635+0000 7ffa074a9740 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009e70 tx=0x7ff9f400e720 comp rx=0 tx=0).stop 2024-09-10T23:13:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.637+0000 7ffa074a9740 1 -- 172.21.15.120:0/3567893294 shutdown_connections 2024-09-10T23:13:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.637+0000 7ffa074a9740 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199f828a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.637+0000 7ffa074a9740 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f7fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.637+0000 7ffa074a9740 1 --2- 172.21.15.120:0/3567893294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7b070 0x560199f7d460 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.637+0000 7ffa074a9740 1 -- 172.21.15.120:0/3567893294 >> 172.21.15.120:0/3567893294 conn(0x560199f761c0 msgr2=0x560199f76f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.639+0000 7ffa074a9740 1 -- 172.21.15.120:0/3567893294 shutdown_connections 2024-09-10T23:13:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.639+0000 7ffa074a9740 1 -- 172.21.15.120:0/3567893294 wait complete. 2024-09-10T23:13:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.640+0000 7ffa074a9740 1 -- start start 2024-09-10T23:13:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.640+0000 7ffa074a9740 1 Processor -- start 2024-09-10T23:13:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.640+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7b070 0x560199f84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f85a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560199e0bab0 con 0x560199f7d9a0 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560199e79730 con 0x560199f803e0 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa01c78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa01c78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37346/0 (socket says 172.21.15.120:37346) 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa01c78640 1 -- 172.21.15.120:0/1414568678 learned_addr learned my addr 172.21.15.120:0/1414568678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560199f833c0 con 0x560199f7b070 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.641+0000 7ffa074a9740 10 client.0 _ll_register_callbacks cb 0x560199ecd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.642+0000 7ffa01c78640 1 -- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7b070 msgr2=0x560199f84790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.642+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7b070 0x560199f84790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.642+0000 7ffa01c78640 1 -- 172.21.15.120:0/1414568678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 msgr2=0x560199f85a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.642+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f85a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.642+0000 7ffa01c78640 1 -- 172.21.15.120:0/1414568678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560199c35790 con 0x560199f7d9a0 2024-09-10T23:13:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.643+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec013740 tx=0x7ff9ec013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a404555c20f17606 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.643+0000 7ff9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.643+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec011dd0 con 0x560199f7d9a0 2024-09-10T23:13:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.643+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff9ec005020 con 0x560199f7d9a0 2024-09-10T23:13:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.646+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f400ee70 con 0x560199f7d9a0 2024-09-10T23:13:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.647+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9ec005220 con 0x560199f7d9a0 2024-09-10T23:13:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.647+0000 7ffa074a9740 10 client.19518 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.647+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560199cae620 con 0x560199f7d9a0 2024-09-10T23:13:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f400ee70 con 0x560199f7d9a0 2024-09-10T23:13:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff9ec01a030 con 0x560199f7d9a0 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 10 client.19518 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 10 client.19518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 20 client.19518 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199f90520 0x560199f929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 4 client.19518 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.648+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560199ee6640 con 0x560199f90520 2024-09-10T23:13:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.654+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199f90520 0x560199f929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.659+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199f90520 0x560199f929a0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e1cd60f760c4b1 server_cookie=4ff4d990681d2de2 in_seq=0 out_seq=0 2024-09-10T23:13:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.660+0000 7ff9e5ffb640 10 client.19518 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.663+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1414568678 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560199ee6640 con 0x560199f90520 2024-09-10T23:13:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.663+0000 7ff9e5ffb640 10 client.19518 handle_command_reply: tid=0 2024-09-10T23:13:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.663+0000 7ffa074a9740 1 client.19518 shutdown 2024-09-10T23:13:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.665+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 msgr2=0x560199f84e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.665+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec013740 tx=0x7ff9ec013c00 comp rx=0 tx=0).stop 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 shutdown_connections 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199f90520 0x560199f929a0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f85a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7d9a0 0x560199f84e30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1414568678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7b070 0x560199f84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.667+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 >> 172.21.15.120:0/1414568678 conn(0x560199f761c0 msgr2=0x560199f7dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.668+0000 7ff9e5ffb640 0 client.19518 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.672+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 shutdown_connections 2024-09-10T23:13:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.672+0000 7ffa074a9740 1 -- 172.21.15.120:0/1414568678 wait complete. 2024-09-10T23:13:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.673+0000 7ffa074a9740 20 client.19518 trim_cache size 0 max 16384 2024-09-10T23:13:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 Processor -- start 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 -- start start 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7d9a0 0x560199f79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a5c40 con 0x560199f7b070 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a84c0 con 0x560199f803e0 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa074a9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a62b0 con 0x560199f7d9a0 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37352/0 (socket says 172.21.15.120:37352) 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa01c78640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7d9a0 0x560199f79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 -- 172.21.15.120:0/324568519 learned_addr learned my addr 172.21.15.120:0/324568519 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 -- 172.21.15.120:0/324568519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7d9a0 msgr2=0x560199f79750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7d9a0 0x560199f79750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 -- 172.21.15.120:0/324568519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 msgr2=0x560199f84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.693+0000 7ffa02c7a640 1 -- 172.21.15.120:0/324568519 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560199f837c0 con 0x560199f7b070 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400ec20 tx=0x7ff9f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618379a0776a87d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ff9e7fff640 1 -- 172.21.15.120:0/324568519 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4009ac0 con 0x560199f7b070 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ff9e7fff640 1 -- 172.21.15.120:0/324568519 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f4003e20 con 0x560199f7b070 2024-09-10T23:13:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ff9e7fff640 1 -- 172.21.15.120:0/324568519 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4025460 con 0x560199f7b070 2024-09-10T23:13:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ffa074a9740 1 -- 172.21.15.120:0/324568519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 msgr2=0x560199f79210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.694+0000 7ffa074a9740 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400ec20 tx=0x7ff9f4004a40 comp rx=0 tx=0).stop 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- 172.21.15.120:0/324568519 shutdown_connections 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f803e0 0x560199f84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f7d9a0 0x560199f79750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- 172.21.15.120:0/324568519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f79210 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- 172.21.15.120:0/324568519 >> 172.21.15.120:0/324568519 conn(0x560199f761c0 msgr2=0x560199f68af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- 172.21.15.120:0/324568519 shutdown_connections 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- 172.21.15.120:0/324568519 wait complete. 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- start start 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 Processor -- start 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 0x560199f8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199ffb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a6980 con 0x560199f7b070 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a7df0 con 0x560199f7d9a0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1a7050 con 0x560199f803e0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa02c7a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37368/0 (socket says 172.21.15.120:37368) 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa02c7a640 1 -- 172.21.15.120:0/1182169518 learned_addr learned my addr 172.21.15.120:0/1182169518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa074a9740 10 client.0 _ll_register_callbacks cb 0x7ff9ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.695+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199ffb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 0x560199f8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 -- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 msgr2=0x560199ffb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199ffb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 -- 172.21.15.120:0/1182169518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 msgr2=0x560199f8c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 0x560199f8c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 -- 172.21.15.120:0/1182169518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560199ffbb10 con 0x560199f7b070 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 0x560199f8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa02c7a640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4003ce0 tx=0x7ff9f4009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7538caeb7242eb0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ff9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40041e0 con 0x560199f7b070 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff9f4028030 con 0x560199f7b070 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f4003e20 con 0x560199f7b070 2024-09-10T23:13:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4004380 con 0x560199f7b070 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa074a9740 10 client.19524 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560199ffce00 con 0x560199f7b070 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.696+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f4025460 con 0x560199f7b070 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ff9f4018470 con 0x560199f7b070 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 10 client.19524 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 10 client.19524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 20 client.19524 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199ffd590 0x560199fff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 4 client.19524 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff9f4003e20 con 0x560199ffd590 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199ffd590 0x560199fff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ffa01c78640 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199ffd590 0x560199fff9b0 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78402e359a347f8e server_cookie=7cdeba6091c34b5f in_seq=0 out_seq=0 2024-09-10T23:13:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:24.697+0000 7ff9e5ffb640 10 client.19524 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:24.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 203078 21.66 MB/sec execute 379 sec latency 400.967 ms 2024-09-10T23:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ff9e5ffb640 1 -- 172.21.15.120:0/1182169518 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff9f4003e20 con 0x560199ffd590 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ff9e5ffb640 10 client.19524 handle_command_reply: tid=0 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 client.19524 shutdown 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 msgr2=0x560199f8baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4003ce0 tx=0x7ff9f4009610 comp rx=0 tx=0).stop 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 shutdown_connections 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560199ffd590 0x560199fff9b0 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560199f803e0 0x560199ffb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ff9e5ffb640 0 client.19524 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560199f7d9a0 0x560199f8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 --2- 172.21.15.120:0/1182169518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560199f7b070 0x560199f8baf0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 >> 172.21.15.120:0/1182169518 conn(0x560199f761c0 msgr2=0x560199f79210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 shutdown_connections 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 1 -- 172.21.15.120:0/1182169518 wait complete. 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.728+0000 7ffa074a9740 20 client.19524 trim_cache size 0 max 16384 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 msgr2=0x7ff9e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 0x7ff9e80b77a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80c9cb0 tx=0x7ff9f800b040 comp rx=0 tx=0).stop 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 msgr2=0x7ff9fc19ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009850 tx=0x7ff9f80042f0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 shutdown_connections 2024-09-10T23:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff9e80b52e0 0x7ff9e80b77a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9fc1066b0 0x7ff9fc19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9fc105cf0 0x7ff9fc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 --2- 172.21.15.120:0/4031452621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff9fc104ba0 0x7ff9fc19ad70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 >> 172.21.15.120:0/4031452621 conn(0x7ff9fc100440 msgr2=0x7ff9fc109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 shutdown_connections 2024-09-10T23:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:25.729+0000 7ffa046dd640 1 -- 172.21.15.120:0/4031452621 wait complete. 2024-09-10T23:13:25.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 203177 21.61 MB/sec execute 380 sec latency 523.172 ms 2024-09-10T23:13:26.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:25 smithi176 ceph-mon[34982]: pgmap v976: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-09-10T23:13:26.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:25 smithi120 ceph-mon[31247]: pgmap v976: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-09-10T23:13:26.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:25 smithi153 ceph-mon[34951]: pgmap v976: 97 pgs: 97 active+clean; 247 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-09-10T23:13:26.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 203665 21.61 MB/sec execute 381 sec latency 161.046 ms 2024-09-10T23:13:27.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 204292 21.64 MB/sec execute 382 sec latency 59.259 ms 2024-09-10T23:13:28.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:27 smithi176 ceph-mon[34982]: pgmap v977: 97 pgs: 97 active+clean; 240 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-09-10T23:13:28.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:27 smithi120 ceph-mon[31247]: pgmap v977: 97 pgs: 97 active+clean; 240 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-09-10T23:13:28.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:27 smithi153 ceph-mon[34951]: pgmap v977: 97 pgs: 97 active+clean; 240 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-09-10T23:13:28.885 INFO:tasks.workunit.client.0.smithi120.stdout: 10 204836 21.67 MB/sec execute 383 sec latency 66.957 ms 2024-09-10T23:13:29.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 205279 21.69 MB/sec execute 384 sec latency 198.639 ms 2024-09-10T23:13:30.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:29 smithi176 ceph-mon[34982]: pgmap v978: 97 pgs: 97 active+clean; 238 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-09-10T23:13:30.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:29 smithi120 ceph-mon[31247]: pgmap v978: 97 pgs: 97 active+clean; 238 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-09-10T23:13:30.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:29 smithi153 ceph-mon[34951]: pgmap v978: 97 pgs: 97 active+clean; 238 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-09-10T23:13:30.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 205675 21.70 MB/sec execute 385 sec latency 88.542 ms 2024-09-10T23:13:31.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 205999 21.68 MB/sec execute 386 sec latency 67.314 ms 2024-09-10T23:13:32.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:31 smithi176 ceph-mon[34982]: pgmap v979: 97 pgs: 97 active+clean; 238 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T23:13:32.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:31 smithi120 ceph-mon[31247]: pgmap v979: 97 pgs: 97 active+clean; 238 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T23:13:32.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:31 smithi153 ceph-mon[34951]: pgmap v979: 97 pgs: 97 active+clean; 238 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T23:13:32.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 206392 21.68 MB/sec execute 387 sec latency 53.675 ms 2024-09-10T23:13:33.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 206807 21.66 MB/sec execute 388 sec latency 61.299 ms 2024-09-10T23:13:34.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:33 smithi176 ceph-mon[34982]: pgmap v980: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 560 op/s 2024-09-10T23:13:34.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:33 smithi120 ceph-mon[31247]: pgmap v980: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 560 op/s 2024-09-10T23:13:34.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:33 smithi153 ceph-mon[34951]: pgmap v980: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 560 op/s 2024-09-10T23:13:34.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 207197 21.67 MB/sec execute 389 sec latency 195.928 ms 2024-09-10T23:13:35.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 207610 21.67 MB/sec execute 390 sec latency 228.054 ms 2024-09-10T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:35 smithi176 ceph-mon[34982]: pgmap v981: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-09-10T23:13:36.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:35 smithi120 ceph-mon[31247]: pgmap v981: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-09-10T23:13:36.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:35 smithi153 ceph-mon[34951]: pgmap v981: 97 pgs: 97 active+clean; 244 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 471 op/s 2024-09-10T23:13:36.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 208112 21.67 MB/sec execute 391 sec latency 106.506 ms 2024-09-10T23:13:37.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 208622 21.70 MB/sec execute 392 sec latency 55.405 ms 2024-09-10T23:13:38.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:37 smithi176 ceph-mon[34982]: pgmap v982: 97 pgs: 97 active+clean; 254 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-09-10T23:13:38.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:37 smithi120 ceph-mon[31247]: pgmap v982: 97 pgs: 97 active+clean; 254 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-09-10T23:13:38.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:37 smithi153 ceph-mon[34951]: pgmap v982: 97 pgs: 97 active+clean; 254 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- 172.21.15.120:0/1297696018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc1018b0 msgr2=0x7f89bc101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1297696018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc1018b0 0x7f89bc101cb0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009930 tx=0x7f89b802a420 comp rx=0 tx=0).stop 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- 172.21.15.120:0/1297696018 shutdown_connections 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1297696018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1033c0 0x7f89bc109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1297696018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc102a00 0x7f89bc102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1297696018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc1018b0 0x7f89bc101cb0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- 172.21.15.120:0/1297696018 >> 172.21.15.120:0/1297696018 conn(0x7f89bc0fd150 msgr2=0x7f89bc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- 172.21.15.120:0/1297696018 shutdown_connections 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- 172.21.15.120:0/1297696018 wait complete. 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 Processor -- start 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.829+0000 7f89c33c2640 1 -- start start 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1018b0 0x7f89bc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 0x7f89bc19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc10e600 con 0x7f89bc102a00 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc10e480 con 0x7f89bc1033c0 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc10e780 con 0x7f89bc1018b0 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51880/0 (socket says 172.21.15.120:51880) 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 -- 172.21.15.120:0/1128042405 learned_addr learned my addr 172.21.15.120:0/1128042405 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c115e640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1018b0 0x7f89bc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c195f640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 0x7f89bc19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 -- 172.21.15.120:0/1128042405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1018b0 msgr2=0x7f89bc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1018b0 0x7f89bc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 -- 172.21.15.120:0/1128042405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 msgr2=0x7f89bc19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 0x7f89bc19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 -- 172.21.15.120:0/1128042405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc19a8f0 con 0x7f89bc102a00 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c195f640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 0x7f89bc19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c095d640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c920 tx=0x7f89ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89b27fc640 1 -- 172.21.15.120:0/1128042405 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac007c70 con 0x7f89bc102a00 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89b27fc640 1 -- 172.21.15.120:0/1128042405 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ac007e10 con 0x7f89bc102a00 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc19abe0 con 0x7f89bc102a00 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89b27fc640 1 -- 172.21.15.120:0/1128042405 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac018820 con 0x7f89bc102a00 2024-09-10T23:13:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.830+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a12b0 con 0x7f89bc102a00 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 Processor -- start 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89b27fc640 1 -- 172.21.15.120:0/1128042405 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f89ac00f450 con 0x7f89bc102a00 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 -- start start 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55f190 0x55ad2f561580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f564090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f4ce6f0 con 0x55ad2f561bd0 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f4bdb20 con 0x55ad2f55f190 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c6274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f4c1930 con 0x55ad2f55c860 2024-09-10T23:13:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.833+0000 7f89c195f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:42682/0 (socket says 172.21.15.120:42682) 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 -- 172.21.15.120:0/3217035509 learned_addr learned my addr 172.21.15.120:0/3217035509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c095d640 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f564090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c115e640 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55f190 0x55ad2f561580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 -- 172.21.15.120:0/3217035509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55f190 msgr2=0x55ad2f561580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55f190 0x55ad2f561580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 -- 172.21.15.120:0/3217035509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 msgr2=0x55ad2f564090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f564090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 -- 172.21.15.120:0/3217035509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2f483530 con 0x55ad2f55c860 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89b27fc640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 0x7f89a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c195f640 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ee0 tx=0x7f89b400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27b2a049e1a8df69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89c115e640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 0x7f89a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89b27fc640 1 -- 172.21.15.120:0/1128042405 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f89ac0e5d20 con 0x7f89bc102a00 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89a77fe640 1 -- 172.21.15.120:0/3217035509 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4016070 con 0x55ad2f55c860 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89a77fe640 1 -- 172.21.15.120:0/3217035509 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4004d20 con 0x55ad2f55c860 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.834+0000 7f89a77fe640 1 -- 172.21.15.120:0/3217035509 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40117a0 con 0x55ad2f55c860 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- 172.21.15.120:0/3217035509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 msgr2=0x55ad2f55ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c115e640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 0x7f89a80b77a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006010 tx=0x7f89b80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ee0 tx=0x7f89b400e6c0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- 172.21.15.120:0/3217035509 shutdown_connections 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f564090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55f190 0x55ad2f561580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 --2- 172.21.15.120:0/3217035509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55c860 0x55ad2f55ec50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- 172.21.15.120:0/3217035509 >> 172.21.15.120:0/3217035509 conn(0x55ad2f5579b0 msgr2=0x55ad2f558740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- 172.21.15.120:0/3217035509 shutdown_connections 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- 172.21.15.120:0/3217035509 wait complete. 2024-09-10T23:13:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 -- start start 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.835+0000 7f89c6274740 1 Processor -- start 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f566530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f567100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f3f5ab0 con 0x55ad2f55c860 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f463730 con 0x55ad2f561bd0 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ad2f564bb0 con 0x55ad2f55f190 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c115e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c115e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51904/0 (socket says 172.21.15.120:51904) 2024-09-10T23:13:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c095d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f566530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c115e640 1 -- 172.21.15.120:0/2150856199 learned_addr learned my addr 172.21.15.120:0/2150856199 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c6274740 10 client.0 _ll_register_callbacks cb 0x55ad2f4b76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.836+0000 7f89c195f640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f567100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 -- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 msgr2=0x55ad2f566530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f566530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 -- 172.21.15.120:0/2150856199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 msgr2=0x55ad2f567100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f567100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 -- 172.21.15.120:0/2150856199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2f21f790 con 0x55ad2f55c860 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c115e640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009930 tx=0x7f89b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21419c74304549e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b804c030 con 0x55ad2f55c860 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f89b8003800 con 0x55ad2f55c860 2024-09-10T23:13:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b4004d20 con 0x55ad2f55c860 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b8035040 con 0x55ad2f55c860 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c6274740 10 client.19536 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad2f298620 con 0x55ad2f55c860 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4004d20 con 0x55ad2f55c860 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.837+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f89b8048a80 con 0x55ad2f55c860 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 10 client.19536 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 10 client.19536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 20 client.19536 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f571770 0x55ad2f573bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 4 client.19536 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad2f4d0640 con 0x55ad2f571770 2024-09-10T23:13:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c095d640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f571770 0x55ad2f573bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89a4ff9640 10 client.19536 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.838+0000 7f89c095d640 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f571770 0x55ad2f573bc0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2095289dbf18fc4 server_cookie=a981932b4dc6467d in_seq=0 out_seq=0 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89a4ff9640 1 -- 172.21.15.120:0/2150856199 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad2f4d0640 con 0x55ad2f571770 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89a4ff9640 10 client.19536 handle_command_reply: tid=0 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 client.19536 shutdown 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 msgr2=0x55ad2f565e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009930 tx=0x7f89b8009260 comp rx=0 tx=0).stop 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 shutdown_connections 2024-09-10T23:13:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f571770 0x55ad2f573bc0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f567100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f566530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 --2- 172.21.15.120:0/2150856199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f565e90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.840+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 >> 172.21.15.120:0/2150856199 conn(0x55ad2f5579b0 msgr2=0x55ad2f556bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.841+0000 7f89a4ff9640 0 client.19536 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.841+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 shutdown_connections 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.841+0000 7f89c6274740 1 -- 172.21.15.120:0/2150856199 wait complete. 2024-09-10T23:13:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.841+0000 7f89c6274740 20 client.19536 trim_cache size 0 max 16384 2024-09-10T23:13:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 Processor -- start 2024-09-10T23:13:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 -- start start 2024-09-10T23:13:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55c860 0x55ad2f573780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f55bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1a7f00 con 0x55ad2f561bd0 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1aa780 con 0x55ad2f55c860 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.859+0000 7f89c6274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1a8570 con 0x55ad2f55f190 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51914/0 (socket says 172.21.15.120:51914) 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 -- 172.21.15.120:0/4094176882 learned_addr learned my addr 172.21.15.120:0/4094176882 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c115e640 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55c860 0x55ad2f573780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c095d640 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f55bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 -- 172.21.15.120:0/4094176882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 msgr2=0x55ad2f55bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f55bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 -- 172.21.15.120:0/4094176882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55c860 msgr2=0x55ad2f573780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55c860 0x55ad2f573780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 -- 172.21.15.120:0/4094176882 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2f55efc0 con 0x55ad2f561bd0 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89c195f640 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ee0 tx=0x7f89b40094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e804be177eab49f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89a6ffd640 1 -- 172.21.15.120:0/4094176882 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4016040 con 0x55ad2f561bd0 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89a6ffd640 1 -- 172.21.15.120:0/4094176882 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4003b70 con 0x55ad2f561bd0 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.860+0000 7f89a6ffd640 1 -- 172.21.15.120:0/4094176882 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011740 con 0x55ad2f561bd0 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- 172.21.15.120:0/4094176882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 msgr2=0x55ad2f55c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009ee0 tx=0x7f89b40094d0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- 172.21.15.120:0/4094176882 shutdown_connections 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f561bd0 0x55ad2f55c200 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f55bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 --2- 172.21.15.120:0/4094176882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f55c860 0x55ad2f573780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- 172.21.15.120:0/4094176882 >> 172.21.15.120:0/4094176882 conn(0x55ad2f5579b0 msgr2=0x55ad2f55aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- 172.21.15.120:0/4094176882 shutdown_connections 2024-09-10T23:13:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- 172.21.15.120:0/4094176882 wait complete. 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 -- start start 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.861+0000 7f89c6274740 1 Processor -- start 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f56c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f56c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1a8c40 con 0x55ad2f55c860 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1aa0b0 con 0x55ad2f561bd0 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1a9310 con 0x55ad2f55f190 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c6274740 10 client.0 _ll_register_callbacks cb 0x7f89b8032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51922/0 (socket says 172.21.15.120:51922) 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 -- 172.21.15.120:0/4203444518 learned_addr learned my addr 172.21.15.120:0/4203444518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c095d640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f56c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c195f640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f56c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 -- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 msgr2=0x55ad2f56c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f56c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 -- 172.21.15.120:0/4203444518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 msgr2=0x55ad2f56c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f56c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 -- 172.21.15.120:0/4203444518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2f56ce70 con 0x55ad2f55c860 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c195f640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f56c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89c115e640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005ab0 tx=0x7f89b8031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb577a168ae0159 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.862+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b804c040 con 0x55ad2f55c860 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f89b8003800 con 0x55ad2f55c860 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b4003b70 con 0x55ad2f55c860 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b803ce60 con 0x55ad2f55c860 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 10 client.19542 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad2f5de6d0 con 0x55ad2f55c860 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4003b70 con 0x55ad2f55c860 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f89b8048a50 con 0x55ad2f55c860 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 10 client.19542 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 10 client.19542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 20 client.19542 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f5deea0 0x55ad2f5e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 4 client.19542 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad2f573f50 con 0x55ad2f5deea0 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.863+0000 7f89c095d640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f5deea0 0x55ad2f5e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.864+0000 7f89c095d640 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f5deea0 0x55ad2f5e12c0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eab2d9835d201c2 server_cookie=b65103a7a84ccec4 in_seq=0 out_seq=0 2024-09-10T23:13:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:38.864+0000 7f89a4ff9640 10 client.19542 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:38.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 209177 21.73 MB/sec execute 393 sec latency 70.006 ms 2024-09-10T23:13:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.704+0000 7f89a4ff9640 1 -- 172.21.15.120:0/4203444518 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad2f573f50 con 0x55ad2f5deea0 2024-09-10T23:13:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.704+0000 7f89a4ff9640 10 client.19542 handle_command_reply: tid=0 2024-09-10T23:13:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.704+0000 7f89c6274740 1 client.19542 shutdown 2024-09-10T23:13:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.704+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 msgr2=0x55ad2f56d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.704+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005ab0 tx=0x7f89b8031c40 comp rx=0 tx=0).stop 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 shutdown_connections 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55ad2f5deea0 0x55ad2f5e12c0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ad2f561bd0 0x55ad2f56c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89a4ff9640 0 client.19542 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ad2f55f190 0x55ad2f56c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 --2- 172.21.15.120:0/4203444518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ad2f55c860 0x55ad2f56d6c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 >> 172.21.15.120:0/4203444518 conn(0x55ad2f5579b0 msgr2=0x55ad2f55aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 shutdown_connections 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 1 -- 172.21.15.120:0/4203444518 wait complete. 2024-09-10T23:13:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.705+0000 7f89c6274740 20 client.19542 trim_cache size 0 max 16384 2024-09-10T23:13:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:13:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:13:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:13:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:13:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 msgr2=0x7f89a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 0x7f89a80b77a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006010 tx=0x7f89b80314a0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 msgr2=0x7f89bc199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c920 tx=0x7f89ac00cde0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 shutdown_connections 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f89a80b52e0 0x7f89a80b77a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1033c0 0x7f89bc19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89bc102a00 0x7f89bc199400 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 --2- 172.21.15.120:0/1128042405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89bc1018b0 0x7f89bc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 >> 172.21.15.120:0/1128042405 conn(0x7f89bc0fd150 msgr2=0x7f89bc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 shutdown_connections 2024-09-10T23:13:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:39.706+0000 7f89c33c2640 1 -- 172.21.15.120:0/1128042405 wait complete. 2024-09-10T23:13:39.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 209277 21.69 MB/sec execute 394 sec latency 454.865 ms 2024-09-10T23:13:40.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:39 smithi176 ceph-mon[34982]: pgmap v983: 97 pgs: 97 active+clean; 258 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.2 MiB/s wr, 456 op/s 2024-09-10T23:13:40.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:39 smithi120 ceph-mon[31247]: pgmap v983: 97 pgs: 97 active+clean; 258 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.2 MiB/s wr, 456 op/s 2024-09-10T23:13:40.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:39 smithi153 ceph-mon[34951]: pgmap v983: 97 pgs: 97 active+clean; 258 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.2 MiB/s wr, 456 op/s 2024-09-10T23:13:40.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 209600 21.69 MB/sec execute 395 sec latency 186.773 ms 2024-09-10T23:13:41.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 209959 21.68 MB/sec execute 396 sec latency 74.924 ms 2024-09-10T23:13:42.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:41 smithi176 ceph-mon[34982]: pgmap v984: 97 pgs: 97 active+clean; 249 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 410 op/s 2024-09-10T23:13:42.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:41 smithi120 ceph-mon[31247]: pgmap v984: 97 pgs: 97 active+clean; 249 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 410 op/s 2024-09-10T23:13:42.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:41 smithi153 ceph-mon[34951]: pgmap v984: 97 pgs: 97 active+clean; 249 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 410 op/s 2024-09-10T23:13:42.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 210366 21.66 MB/sec execute 397 sec latency 80.405 ms 2024-09-10T23:13:43.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:13:43.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 210840 21.67 MB/sec execute 398 sec latency 60.532 ms 2024-09-10T23:13:43.891 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:43 smithi120 ceph-mon[31247]: pgmap v985: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T23:13:44.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:43 smithi176 ceph-mon[34982]: pgmap v985: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T23:13:44.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:43 smithi153 ceph-mon[34951]: pgmap v985: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 -- 172.21.15.120:0/2206379555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc104590 msgr2=0x7f7bcc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/2206379555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc104590 0x7f7bcc10cb70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009900 tx=0x7f7bbc02a410 comp rx=0 tx=0).stop 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 -- 172.21.15.120:0/2206379555 shutdown_connections 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/2206379555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 0x7f7bcc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/2206379555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc104590 0x7f7bcc10cb70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/2206379555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc103b90 0x7f7bcc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 -- 172.21.15.120:0/2206379555 >> 172.21.15.120:0/2206379555 conn(0x7f7bcc0fd700 msgr2=0x7f7bcc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 -- 172.21.15.120:0/2206379555 shutdown_connections 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.393+0000 7f7bd3d43640 1 -- 172.21.15.120:0/2206379555 wait complete. 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 Processor -- start 2024-09-10T23:13:44.409 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 -- start start 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 0x7f7bcc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 0x7f7bcc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc112d60 con 0x7f7bcc103b90 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc112be0 con 0x7f7bcc104590 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd3d43640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc112ee0 con 0x7f7bcc10d140 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35832/0 (socket says 172.21.15.120:35832) 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd12de640 1 -- 172.21.15.120:0/1873397962 learned_addr learned my addr 172.21.15.120:0/1873397962 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd1adf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 0x7f7bcc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.394+0000 7f7bd22e0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 0x7f7bcc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.410 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 -- 172.21.15.120:0/1873397962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 msgr2=0x7f7bcc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 0x7f7bcc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 -- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 msgr2=0x7f7bcc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 0x7f7bcc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 -- 172.21.15.120:0/1873397962 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc19f3c0 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 0x7f7bcc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005940 tx=0x7f7bbc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bc2ffd640 1 -- 172.21.15.120:0/1873397962 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc048070 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bc2ffd640 1 -- 172.21.15.120:0/1873397962 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bbc03ab20 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc19f650 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bc2ffd640 1 -- 172.21.15.120:0/1873397962 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc035050 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.395+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc1018f0 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bc2ffd640 1 -- 172.21.15.120:0/1873397962 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7bbc03a670 con 0x7f7bcc104590 2024-09-10T23:13:44.411 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 Processor -- start 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 -- start start 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bc2ffd640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 0x7f7bb80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f2317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f234120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.398+0000 7f7bd6118740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56456f19be50 con 0x56456f225480 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd6118740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56456f18b280 con 0x56456f231d30 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd6118740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56456f18f090 con 0x56456f225da0 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55390/0 (socket says 172.21.15.120:55390) 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 -- 172.21.15.120:0/1919995472 learned_addr learned my addr 172.21.15.120:0/1919995472 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 0x7f7bb80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.412 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bc2ffd640 1 -- 172.21.15.120:0/1873397962 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc109e00 con 0x7f7bcc104590 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f234120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f2317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 -- 172.21.15.120:0/1919995472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 msgr2=0x56456f2317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f2317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 -- 172.21.15.120:0/1919995472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 msgr2=0x56456f234120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f234120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 -- 172.21.15.120:0/1919995472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456eee9650 con 0x56456f225480 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 0x7f7bb80b77a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800afb0 tx=0x7f7bc800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4014850 tx=0x7f7bc4009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570d95349d485e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.399+0000 7f7bb7fff640 1 -- 172.21.15.120:0/1919995472 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc401a030 con 0x56456f225480 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bb7fff640 1 -- 172.21.15.120:0/1919995472 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc4012890 con 0x56456f225480 2024-09-10T23:13:44.413 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bb7fff640 1 -- 172.21.15.120:0/1919995472 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40052e0 con 0x56456f225480 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 -- 172.21.15.120:0/1919995472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 msgr2=0x56456f225860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4014850 tx=0x7f7bc4009ab0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 -- 172.21.15.120:0/1919995472 shutdown_connections 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f234120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f2317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1919995472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f225860 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 -- 172.21.15.120:0/1919995472 >> 172.21.15.120:0/1919995472 conn(0x56456f2232b0 msgr2=0x56456f224450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 -- 172.21.15.120:0/1919995472 shutdown_connections 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.400+0000 7f7bd6118740 1 -- 172.21.15.120:0/1919995472 wait complete. 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 -- start start 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 Processor -- start 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f225480 0x56456f2315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.414 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f231d30 0x56456f22ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56456efec700 con 0x56456f225da0 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56456f126110 con 0x56456f225480 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56456f2349b0 con 0x56456f231d30 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd6118740 10 client.0 _ll_register_callbacks cb 0x56456f13f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55392/0 (socket says 172.21.15.120:55392) 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd12de640 1 -- 172.21.15.120:0/1551243955 learned_addr learned my addr 172.21.15.120:0/1551243955 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f225480 0x56456f2315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.401+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f231d30 0x56456f22ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 -- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f231d30 msgr2=0x56456f22ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f231d30 0x56456f22ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 -- 172.21.15.120:0/1551243955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f225480 msgr2=0x56456f2315a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f225480 0x56456f2315a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 -- 172.21.15.120:0/1551243955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456f153c40 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd12de640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006010 tx=0x7f7bbc03af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65601b497be937c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0cfd00 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc0d9690 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc4012890 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bbc003e40 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 10 client.19554 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56456f240260 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc4012890 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7bbc0cbee0 con 0x56456f225da0 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 10 client.19554 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 10 client.19554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:44.416 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 20 client.19554 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2416a0 0x56456f243b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 4 client.19554 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.402+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56456f19dda0 con 0x56456f2416a0 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.403+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2416a0 0x56456f243b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.403+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2416a0 0x56456f243b00 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da5662eb51afd3e server_cookie=e2da0d8c27fa494c in_seq=0 out_seq=0 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.403+0000 7f7bb57fa640 10 client.19554 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bb57fa640 1 -- 172.21.15.120:0/1551243955 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56456f19dda0 con 0x56456f2416a0 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bb57fa640 10 client.19554 handle_command_reply: tid=0 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 client.19554 shutdown 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 msgr2=0x56456f22e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006010 tx=0x7f7bbc03af00 comp rx=0 tx=0).stop 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 shutdown_connections 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2416a0 0x56456f243b00 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f231d30 0x56456f22ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bb57fa640 0 client.19554 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f22e9f0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 --2- 172.21.15.120:0/1551243955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f225480 0x56456f2315a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 >> 172.21.15.120:0/1551243955 conn(0x56456f2232b0 msgr2=0x56456f228e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 shutdown_connections 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 1 -- 172.21.15.120:0/1551243955 wait complete. 2024-09-10T23:13:44.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.405+0000 7f7bd6118740 20 client.19554 trim_cache size 0 max 16384 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 Processor -- start 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 -- start start 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f222bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f233120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1aa070 con 0x56456f225480 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1ac8d0 con 0x56456f231d30 2024-09-10T23:13:44.426 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd6118740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1aa6c0 con 0x56456f225da0 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd1adf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f222bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd1adf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55408/0 (socket says 172.21.15.120:55408) 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd22e0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f233120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.424+0000 7f7bd1adf640 1 -- 172.21.15.120:0/4004669176 learned_addr learned my addr 172.21.15.120:0/4004669176 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 -- 172.21.15.120:0/4004669176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 msgr2=0x56456f222bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f222bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 -- 172.21.15.120:0/4004669176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 msgr2=0x56456f233120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f233120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.427 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 -- 172.21.15.120:0/4004669176 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456f225bd0 con 0x56456f225480 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x56456f244340 tx=0x7f7bc8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e4c270f7a1686b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bb77fe640 1 -- 172.21.15.120:0/4004669176 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8033070 con 0x56456f225480 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bb77fe640 1 -- 172.21.15.120:0/4004669176 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc801ca50 con 0x56456f225480 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.425+0000 7f7bb77fe640 1 -- 172.21.15.120:0/4004669176 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8016070 con 0x56456f225480 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 -- 172.21.15.120:0/4004669176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 msgr2=0x56456f224a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x56456f244340 tx=0x7f7bc8012000 comp rx=0 tx=0).stop 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 -- 172.21.15.120:0/4004669176 shutdown_connections 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f233120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225da0 0x56456f222bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 --2- 172.21.15.120:0/4004669176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225480 0x56456f224a80 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.426+0000 7f7bd6118740 1 -- 172.21.15.120:0/4004669176 >> 172.21.15.120:0/4004669176 conn(0x56456f2232b0 msgr2=0x56456f224370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.429 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.427+0000 7f7bd6118740 1 -- 172.21.15.120:0/4004669176 shutdown_connections 2024-09-10T23:13:44.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.427+0000 7f7bd6118740 1 -- 172.21.15.120:0/4004669176 wait complete. 2024-09-10T23:13:44.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.427+0000 7f7bd6118740 1 -- start start 2024-09-10T23:13:44.430 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.427+0000 7f7bd6118740 1 Processor -- start 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 0x56456f23a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f22adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1aad90 con 0x56456f225da0 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1ac200 con 0x56456f231d30 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1ab460 con 0x56456f225480 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd6118740 10 client.0 _ll_register_callbacks cb 0x7f7bbc0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:44.432 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55420/0 (socket says 172.21.15.120:55420) 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 -- 172.21.15.120:0/2133690230 learned_addr learned my addr 172.21.15.120:0/2133690230 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd1adf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 0x56456f23a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd22e0640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f22adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 -- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 msgr2=0x56456f23a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 0x56456f23a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 -- 172.21.15.120:0/2133690230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 msgr2=0x56456f22adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f22adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 -- 172.21.15.120:0/2133690230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456f22b5f0 con 0x56456f225da0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 0x56456f23a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bd12de640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0cc250 tx=0x7f7bbc0ccbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f2a0a32e567d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.428+0000 7f7bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0cfd00 con 0x56456f225da0 2024-09-10T23:13:44.433 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7bbc0cf9f0 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc801ca50 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bbc0cc430 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 10 client.19560 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56456f22c900 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc801ca50 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f7bbc0cf260 con 0x56456f225da0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 10 client.19560 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 20 client.19560 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2af830 0x56456f2b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 4 client.19560 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.429+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56456f199850 con 0x56456f2af830 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.430+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2af830 0x56456f2b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:44.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.430+0000 7f7bb57fa640 10 client.19560 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:44.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.430+0000 7f7bd1adf640 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2af830 0x56456f2b1c20 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3e974fe5886d0c server_cookie=3a5ed297652d2577 in_seq=0 out_seq=0 2024-09-10T23:13:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.434+0000 7f7bb57fa640 1 -- 172.21.15.120:0/2133690230 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10030 (crc 0 0 0) 0x56456f199850 con 0x56456f2af830 2024-09-10T23:13:44.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.434+0000 7f7bb57fa640 10 client.19560 handle_command_reply: tid=0 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.434+0000 7f7bd6118740 1 client.19560 shutdown 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.434+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 msgr2=0x56456f23a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.434+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0cc250 tx=0x7f7bbc0ccbe0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 shutdown_connections 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56456f2af830 0x56456f2b1c20 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56456f231d30 0x56456f22adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bb57fa640 0 client.19560 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56456f225da0 0x56456f23a5e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 --2- 172.21.15.120:0/2133690230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56456f225480 0x56456f23a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 >> 172.21.15.120:0/2133690230 conn(0x56456f2232b0 msgr2=0x56456f230020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 shutdown_connections 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 1 -- 172.21.15.120:0/2133690230 wait complete. 2024-09-10T23:13:44.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd6118740 20 client.19560 trim_cache size 0 max 16384 2024-09-10T23:13:44.438 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:44.438 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:13:44.439 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.441 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.442 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.443 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.444 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:13:44.445 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.446 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.447 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:13:44.448 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.449 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.450 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.451 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:13:44.452 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.454 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.455 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.456 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.457 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.458 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:44.459 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 msgr2=0x7f7bb80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 0x7f7bb80b77a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800afb0 tx=0x7f7bc800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 msgr2=0x7f7bcc199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005940 tx=0x7f7bbc004270 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 shutdown_connections 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7bb80b52e0 0x7f7bb80b77a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bcc10d140 0x7f7bcc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7bcc104590 0x7f7bcc199850 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 --2- 172.21.15.120:0/1873397962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7bcc103b90 0x7f7bcc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.435+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 >> 172.21.15.120:0/1873397962 conn(0x7f7bcc0fd700 msgr2=0x7f7bcc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.436+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 shutdown_connections 2024-09-10T23:13:44.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:44.436+0000 7f7bd3d43640 1 -- 172.21.15.120:0/1873397962 wait complete. 2024-09-10T23:13:44.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 211403 21.69 MB/sec execute 399 sec latency 65.496 ms 2024-09-10T23:13:45.404 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:13:45.404 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:13:45.706 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:45.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 211832 21.68 MB/sec execute 400 sec latency 198.782 ms 2024-09-10T23:13:46.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:45 smithi176 ceph-mon[34982]: pgmap v986: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-09-10T23:13:46.144 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:45 smithi120 ceph-mon[31247]: pgmap v986: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-09-10T23:13:46.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:45 smithi153 ceph-mon[34951]: pgmap v986: 97 pgs: 97 active+clean; 239 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 -- 172.21.15.120:0/3457162292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff200101c90 msgr2=0x7ff200102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 --2- 172.21.15.120:0/3457162292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff200101c90 0x7ff200102070 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009ca0 tx=0x7ff1f002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 -- 172.21.15.120:0/3457162292 shutdown_connections 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 --2- 172.21.15.120:0/3457162292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff20010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 --2- 172.21.15.120:0/3457162292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2001076a0 0x7ff200107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 --2- 172.21.15.120:0/3457162292 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff200101c90 0x7ff200102070 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 -- 172.21.15.120:0/3457162292 >> 172.21.15.120:0/3457162292 conn(0x7ff2000fd700 msgr2=0x7ff2000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:46.253 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 -- 172.21.15.120:0/3457162292 shutdown_connections 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.230+0000 7ff204dd2640 1 -- 172.21.15.120:0/3457162292 wait complete. 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 Processor -- start 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 -- start start 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 0x7ff20019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff2001a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff200111ce0 con 0x7ff2001076a0 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff200111b60 con 0x7ff200101c90 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff204dd2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff200111e60 con 0x7ff200108020 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1fe59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 0x7ff20019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1f5d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55428/0 (socket says 172.21.15.120:55428) 2024-09-10T23:13:46.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1f5d9b640 1 -- 172.21.15.120:0/2037328549 learned_addr learned my addr 172.21.15.120:0/2037328549 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1fe59c640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 0x7ff20019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35900/0 (socket says 172.21.15.120:35900) 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.231+0000 7ff1fed9d640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff2001a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 -- 172.21.15.120:0/2037328549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 msgr2=0x7ff2001a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff2001a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 -- 172.21.15.120:0/2037328549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 msgr2=0x7ff20019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 0x7ff20019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 -- 172.21.15.120:0/2037328549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2001a14b0 con 0x7ff2001076a0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1fed9d640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff2001a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f5d9b640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c920 tx=0x7ff1e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8007c70 con 0x7ff2001076a0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2001a17a0 con 0x7ff2001076a0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8007e10 con 0x7ff2001076a0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8018820 con 0x7ff2001076a0 2024-09-10T23:13:46.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.232+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2001a7e60 con 0x7ff2001076a0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.234+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff200102070 con 0x7ff2001076a0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.235+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff1e800f450 con 0x7ff2001076a0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.235+0000 7ff1f7fff640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 0x7ff1ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.236+0000 7ff1fe59c640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 0x7ff1ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.236+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ff1e80e5cb0 con 0x7ff2001076a0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.236+0000 7ff1fe59c640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 0x7ff1ec0b77a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009430 tx=0x7ff1f0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:46.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.242+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1e8014030 con 0x7ff2001076a0 2024-09-10T23:13:46.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.484+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2001a8150 con 0x7ff2001076a0 2024-09-10T23:13:46.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.484+0000 7ff1f7fff640 1 -- 172.21.15.120:0/2037328549 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7ff1e80cde20 con 0x7ff2001076a0 2024-09-10T23:13:46.486 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:13:46.486 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:13:46.487 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:13:46.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 msgr2=0x7ff1ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:46.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 0x7ff1ec0b77a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009430 tx=0x7ff1f0009360 comp rx=0 tx=0).stop 2024-09-10T23:13:46.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 msgr2=0x7ff20019b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:46.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c920 tx=0x7ff1e800cde0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 shutdown_connections 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ff1ec0b52e0 0x7ff1ec0b77a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff200108020 0x7ff2001a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff2001076a0 0x7ff20019b940 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 --2- 172.21.15.120:0/2037328549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff200101c90 0x7ff20019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.487+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 >> 172.21.15.120:0/2037328549 conn(0x7ff2000fd700 msgr2=0x7ff20006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.488+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 shutdown_connections 2024-09-10T23:13:46.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:46.488+0000 7ff204dd2640 1 -- 172.21.15.120:0/2037328549 wait complete. 2024-09-10T23:13:46.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 212380 21.72 MB/sec execute 401 sec latency 56.092 ms 2024-09-10T23:13:47.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2037328549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:47.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:46 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2037328549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:47.270 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:13:47.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:46 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2037328549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:13:47.547 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:47.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 212888 21.74 MB/sec execute 402 sec latency 60.283 ms 2024-09-10T23:13:47.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:47 smithi120 ceph-mon[31247]: pgmap v987: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 671 op/s 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 -- 172.21.15.120:0/4071036171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa090101cc0 msgr2=0x7fa0901020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 --2- 172.21.15.120:0/4071036171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa090101cc0 0x7fa0901020a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa084009900 tx=0x7fa08402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 -- 172.21.15.120:0/4071036171 shutdown_connections 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 --2- 172.21.15.120:0/4071036171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 0x7fa09010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 --2- 172.21.15.120:0/4071036171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0901076d0 0x7fa090107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 --2- 172.21.15.120:0/4071036171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa090101cc0 0x7fa0901020a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 -- 172.21.15.120:0/4071036171 >> 172.21.15.120:0/4071036171 conn(0x7fa0900fd770 msgr2=0x7fa0900ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 -- 172.21.15.120:0/4071036171 shutdown_connections 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.090+0000 7fa09724b640 1 -- 172.21.15.120:0/4071036171 wait complete. 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 Processor -- start 2024-09-10T23:13:48.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 -- start start 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0901076d0 0x7fa090197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 0x7fa09019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa090110c10 con 0x7fa0901076d0 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa090110a90 con 0x7fa090101cc0 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.091+0000 7fa09724b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa090110d90 con 0x7fa090108050 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35922/0 (socket says 172.21.15.120:35922) 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 -- 172.21.15.120:0/691510260 learned_addr learned my addr 172.21.15.120:0/691510260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa08ffff640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0901076d0 0x7fa090197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa0957e8640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 0x7fa09019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 -- 172.21.15.120:0/691510260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 msgr2=0x7fa09019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 0x7fa09019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 -- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0901076d0 msgr2=0x7fa090197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0901076d0 0x7fa090197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 -- 172.21.15.120:0/691510260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09019d170 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa094fe7640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa084005940 tx=0x7fa0840040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.092+0000 7fa08dffb640 1 -- 172.21.15.120:0/691510260 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08403f030 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.093+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09019d400 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.093+0000 7fa08dffb640 1 -- 172.21.15.120:0/691510260 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0840044c0 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.093+0000 7fa08dffb640 1 -- 172.21.15.120:0/691510260 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084035070 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.093+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0901a3b20 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.096+0000 7fa099620740 1 Processor -- start 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.096+0000 7fa099620740 1 -- start start 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.096+0000 7fa08dffb640 1 -- 172.21.15.120:0/691510260 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa08403b9f0 con 0x7fa090101cc0 2024-09-10T23:13:48.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aca9034ea0 con 0x55aca90bedf0 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aca90242d0 con 0x55aca90cad80 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa099620740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55aca90280e0 con 0x55aca90be4d0 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52484/0 (socket says 172.21.15.120:52484) 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 -- 172.21.15.120:0/376293877 learned_addr learned my addr 172.21.15.120:0/376293877 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa08ffff640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa094fe7640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 -- 172.21.15.120:0/376293877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 msgr2=0x55aca90cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 -- 172.21.15.120:0/376293877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 msgr2=0x55aca90ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.115 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 -- 172.21.15.120:0/376293877 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca8d82650 con 0x55aca90be4d0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa08ffff640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.097+0000 7fa0957e8640 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa088009fc0 tx=0x7fa088009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da555406c6ad4f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa07affd640 1 -- 172.21.15.120:0/376293877 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088003b70 con 0x55aca90be4d0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa07affd640 1 -- 172.21.15.120:0/376293877 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa088003d10 con 0x55aca90be4d0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa07affd640 1 -- 172.21.15.120:0/376293877 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088007a60 con 0x55aca90be4d0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa08dffb640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 0x7fa0800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa094fe7640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 0x7fa0800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.098+0000 7fa08dffb640 1 -- 172.21.15.120:0/691510260 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa084106260 con 0x7fa090101cc0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- 172.21.15.120:0/376293877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 msgr2=0x55aca90be8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa094fe7640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 0x7fa0800b77a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa084104100 tx=0x7fa08410b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa088009fc0 tx=0x7fa088009500 comp rx=0 tx=0).stop 2024-09-10T23:13:48.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- 172.21.15.120:0/376293877 shutdown_connections 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 --2- 172.21.15.120:0/376293877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90be8b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- 172.21.15.120:0/376293877 >> 172.21.15.120:0/376293877 conn(0x55aca90bc300 msgr2=0x55aca90bd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- 172.21.15.120:0/376293877 shutdown_connections 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- 172.21.15.120:0/376293877 wait complete. 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 -- start start 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.099+0000 7fa099620740 1 Processor -- start 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.117 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aca8e85700 con 0x55aca90be4d0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55aca8fbf160 con 0x55aca90cad80 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55aca90cda00 con 0x55aca90bedf0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa099620740 10 client.0 _ll_register_callbacks cb 0x55aca8fd89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55454/0 (socket says 172.21.15.120:55454) 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 -- 172.21.15.120:0/3515369047 learned_addr learned my addr 172.21.15.120:0/3515369047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa08ffff640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 -- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 msgr2=0x55aca90c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.118 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 -- 172.21.15.120:0/3515369047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 msgr2=0x55aca90c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.100+0000 7fa0957e8640 1 -- 172.21.15.120:0/3515369047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca8fecc90 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa0957e8640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa088009370 tx=0x7fa088009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd03da6e9be7c6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa06ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088004c40 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa088004de0 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa088003d10 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa088005050 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa099620740 10 client.19578 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aca90d93e0 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.101+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa088003d10 con 0x55aca90be4d0 2024-09-10T23:13:48.119 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa088032050 con 0x55aca90be4d0 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 10 client.19578 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 20 client.19578 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca90dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 4 client.19578 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.102+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca9036df0 con 0x55aca90da840 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.103+0000 7fa08ffff640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca90dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.103+0000 7fa08ffff640 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca90dcc30 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2645680f9450ad89 server_cookie=16fd5999ca160dc0 in_seq=0 out_seq=0 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.103+0000 7fa06ffff640 10 client.19578 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa06ffff640 1 -- 172.21.15.120:0/3515369047 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aca9036df0 con 0x55aca90da840 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa06ffff640 10 client.19578 handle_command_reply: tid=0 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 client.19578 shutdown 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 msgr2=0x55aca90ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.120 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa088009370 tx=0x7fa088009280 comp rx=0 tx=0).stop 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 shutdown_connections 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca90dcc30 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa06ffff640 0 client.19578 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 --2- 172.21.15.120:0/3515369047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90ca6f0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 >> 172.21.15.120:0/3515369047 conn(0x55aca90bc300 msgr2=0x55aca90c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 shutdown_connections 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 1 -- 172.21.15.120:0/3515369047 wait complete. 2024-09-10T23:13:48.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.109+0000 7fa099620740 20 client.19578 trim_cache size 0 max 16384 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 Processor -- start 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 -- start start 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901aa050 con 0x55aca90be4d0 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901ac8d0 con 0x55aca90cad80 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa099620740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901aa6c0 con 0x55aca90bedf0 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa0957e8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55470/0 (socket says 172.21.15.120:55470) 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.125+0000 7fa0957e8640 1 -- 172.21.15.120:0/1748926561 learned_addr learned my addr 172.21.15.120:0/1748926561 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:48.128 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa08ffff640 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 -- 172.21.15.120:0/1748926561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 msgr2=0x55aca90c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 -- 172.21.15.120:0/1748926561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 msgr2=0x55aca90c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 -- 172.21.15.120:0/1748926561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca90c78f0 con 0x55aca90be4d0 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa0957e8640 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa088009950 tx=0x7fa088009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b4471b6882650 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa07a7fc640 1 -- 172.21.15.120:0/1748926561 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088038070 con 0x55aca90be4d0 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa07a7fc640 1 -- 172.21.15.120:0/1748926561 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0880044b0 con 0x55aca90be4d0 2024-09-10T23:13:48.129 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.126+0000 7fa07a7fc640 1 -- 172.21.15.120:0/1748926561 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088004e40 con 0x55aca90be4d0 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- 172.21.15.120:0/1748926561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 msgr2=0x55aca90bbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa088009950 tx=0x7fa088009fc0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- 172.21.15.120:0/1748926561 shutdown_connections 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90bedf0 0x55aca90c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 --2- 172.21.15.120:0/1748926561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90be4d0 0x55aca90bbc10 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- 172.21.15.120:0/1748926561 >> 172.21.15.120:0/1748926561 conn(0x55aca90bc300 msgr2=0x55aca90c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- 172.21.15.120:0/1748926561 shutdown_connections 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- 172.21.15.120:0/1748926561 wait complete. 2024-09-10T23:13:48.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.127+0000 7fa099620740 1 -- start start 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 Processor -- start 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901aad90 con 0x55aca90bedf0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901ac200 con 0x55aca90cad80 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901ab460 con 0x55aca90be4d0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa099620740 10 client.0 _ll_register_callbacks cb 0x7fa07c00b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55480/0 (socket says 172.21.15.120:55480) 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 -- 172.21.15.120:0/2380798704 learned_addr learned my addr 172.21.15.120:0/2380798704 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa0957e8640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa0957e8640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 -- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 msgr2=0x55aca90d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 -- 172.21.15.120:0/2380798704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 msgr2=0x55aca90cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.128+0000 7fa08ffff640 1 -- 172.21.15.120:0/2380798704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca90ce1c0 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa08ffff640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00b440 tx=0x7fa07c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeee1b9c68f7d15d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa06ffff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c017800 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fa07c0179a0 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0880044b0 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa07c017ba0 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa099620740 10 client.19584 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aca9147e40 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.129+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0880044b0 con 0x55aca90bedf0 2024-09-10T23:13:48.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fa07c034c90 con 0x55aca90bedf0 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 10 client.19584 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 10 client.19584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 20 client.19584 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca914a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 4 client.19584 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aca90328a0 con 0x55aca90da840 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa0957e8640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca914a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa06ffff640 10 client.19584 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:48.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.130+0000 7fa0957e8640 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca914a660 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e027d80532cd72 server_cookie=83c5426788027b20 in_seq=0 out_seq=0 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa06ffff640 1 -- 172.21.15.120:0/2380798704 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aca90328a0 con 0x55aca90da840 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa06ffff640 10 client.19584 handle_command_reply: tid=0 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 client.19584 shutdown 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 msgr2=0x55aca90d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00b440 tx=0x7fa07c005f90 comp rx=0 tx=0).stop 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 shutdown_connections 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55aca90da840 0x55aca914a660 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa06ffff640 0 client.19584 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aca90cad80 0x55aca90cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aca90bedf0 0x55aca90d3210 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 --2- 172.21.15.120:0/2380798704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aca90be4d0 0x55aca90d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.132+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 >> 172.21.15.120:0/2380798704 conn(0x55aca90bc300 msgr2=0x55aca90bd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.133+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 shutdown_connections 2024-09-10T23:13:48.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.133+0000 7fa099620740 1 -- 172.21.15.120:0/2380798704 wait complete. 2024-09-10T23:13:48.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.133+0000 7fa099620740 20 client.19584 trim_cache size 0 max 16384 2024-09-10T23:13:48.138 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 msgr2=0x7fa0800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 0x7fa0800b77a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa084104100 tx=0x7fa08410b040 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 msgr2=0x7fa0901970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa084005940 tx=0x7fa0840040c0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 shutdown_connections 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fa0800b52e0 0x7fa0800b77a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa090108050 0x7fa09019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0901076d0 0x7fa090197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 --2- 172.21.15.120:0/691510260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa090101cc0 0x7fa0901970c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 >> 172.21.15.120:0/691510260 conn(0x7fa0900fd770 msgr2=0x7fa0900ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 shutdown_connections 2024-09-10T23:13:48.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:48.134+0000 7fa09724b640 1 -- 172.21.15.120:0/691510260 wait complete. 2024-09-10T23:13:48.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:47 smithi153 ceph-mon[34951]: pgmap v987: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 671 op/s 2024-09-10T23:13:48.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:47 smithi176 ceph-mon[34982]: pgmap v987: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 671 op/s 2024-09-10T23:13:48.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 213236 21.74 MB/sec execute 403 sec latency 64.477 ms 2024-09-10T23:13:49.884 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:13:49.885 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:13:49.888 INFO:tasks.workunit.client.0.smithi120.stdout: 10 213680 21.73 MB/sec execute 404 sec latency 75.491 ms 2024-09-10T23:13:50.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:49 smithi120 ceph-mon[31247]: pgmap v988: 97 pgs: 97 active+clean; 248 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-09-10T23:13:50.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:49 smithi153 ceph-mon[34951]: pgmap v988: 97 pgs: 97 active+clean; 248 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-09-10T23:13:50.241 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:50.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:49 smithi176 ceph-mon[34982]: pgmap v988: 97 pgs: 97 active+clean; 248 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-09-10T23:13:50.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.785+0000 7f111ca02640 1 -- 172.21.15.120:0/2270614471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118102220 msgr2=0x7f11181066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.785+0000 7f111ca02640 1 --2- 172.21.15.120:0/2270614471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118102220 0x7f11181066f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f110c009860 tx=0x7f110c02a380 comp rx=0 tx=0).stop 2024-09-10T23:13:50.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.786+0000 7f111ca02640 1 -- 172.21.15.120:0/2270614471 shutdown_connections 2024-09-10T23:13:50.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.786+0000 7f111ca02640 1 --2- 172.21.15.120:0/2270614471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f1118107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.786+0000 7f111ca02640 1 --2- 172.21.15.120:0/2270614471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118102220 0x7f11181066f0 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f110c009860 tx=0x7f110c02a380 comp rx=0 tx=0).stop 2024-09-10T23:13:50.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.786+0000 7f111ca02640 1 --2- 172.21.15.120:0/2270614471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1118101900 0x7f1118101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.786+0000 7f111ca02640 1 -- 172.21.15.120:0/2270614471 >> 172.21.15.120:0/2270614471 conn(0x7f11180fd5b0 msgr2=0x7f11180ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.788+0000 7f111ca02640 1 -- 172.21.15.120:0/2270614471 shutdown_connections 2024-09-10T23:13:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.789+0000 7f111ca02640 1 -- 172.21.15.120:0/2270614471 wait complete. 2024-09-10T23:13:50.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.789+0000 7f111ca02640 1 Processor -- start 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.790+0000 7f111ca02640 1 -- start start 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.790+0000 7f111ca02640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118101900 0x7f111806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.790+0000 7f111ca02640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.790+0000 7f111ca02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11180744b0 0x7f1118074960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.790+0000 7f111ca02640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f111806d040 con 0x7f11180744b0 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f111ca02640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f111806cec0 con 0x7f1118101900 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f111ca02640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f111806d1c0 con 0x7f1118106d40 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52524/0 (socket says 172.21.15.120:52524) 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 -- 172.21.15.120:0/1398919556 learned_addr learned my addr 172.21.15.120:0/1398919556 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 -- 172.21.15.120:0/1398919556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118101900 msgr2=0x7f111806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118101900 0x7f111806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 -- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11180744b0 msgr2=0x7f1118074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11180744b0 0x7f1118074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.791+0000 7f1116ffd640 1 -- 172.21.15.120:0/1398919556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180750b0 con 0x7f1118106d40 2024-09-10T23:13:50.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.792+0000 7f1116ffd640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f110c02a8f0 tx=0x7f110c0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.796+0000 7f1114ff9640 1 -- 172.21.15.120:0/1398919556 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c040070 con 0x7f1118106d40 2024-09-10T23:13:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.796+0000 7f1114ff9640 1 -- 172.21.15.120:0/1398919556 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c0046c0 con 0x7f1118106d40 2024-09-10T23:13:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.796+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118107cc0 con 0x7f1118106d40 2024-09-10T23:13:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.796+0000 7f1114ff9640 1 -- 172.21.15.120:0/1398919556 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c037040 con 0x7f1118106d40 2024-09-10T23:13:50.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.796+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1118108120 con 0x7f1118106d40 2024-09-10T23:13:50.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.800+0000 7f1114ff9640 1 -- 172.21.15.120:0/1398919556 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f110c03a030 con 0x7f1118106d40 2024-09-10T23:13:50.802 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.801+0000 7f1114ff9640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 0x7f11000b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.801+0000 7f1114ff9640 1 -- 172.21.15.120:0/1398919556 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f110c104e60 con 0x7f1118106d40 2024-09-10T23:13:50.803 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.801+0000 7f11177fe640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 0x7f11000b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.803+0000 7f1120039740 1 Processor -- start 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 -- start start 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 0x55e8420774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e842083460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 0x55e842085d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e841fede60 con 0x55e842077a10 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e841fdd290 con 0x55e8420839a0 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1120039740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e841fe10a0 con 0x55e8420770f0 2024-09-10T23:13:50.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f11177fe640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 0x7f11000b79e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1110005e30 tx=0x7f1110005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f11177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 0x55e8420774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f11177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 0x55e8420774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52528/0 (socket says 172.21.15.120:52528) 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f11177fe640 1 -- 172.21.15.120:0/1448647912 learned_addr learned my addr 172.21.15.120:0/1448647912 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.804+0000 7f1117fff640 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 0x55e842085d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.805+0000 7f1117fff640 1 -- 172.21.15.120:0/1448647912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 msgr2=0x55e8420774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.805+0000 7f1117fff640 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 0x55e8420774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.805+0000 7f1117fff640 1 -- 172.21.15.120:0/1448647912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 msgr2=0x55e842083460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.805+0000 7f1117fff640 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e842083460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.805+0000 7f1117fff640 1 -- 172.21.15.120:0/1448647912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e841d3b930 con 0x55e8420839a0 2024-09-10T23:13:50.808 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.806+0000 7f1117fff640 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 0x55e842085d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f11080096a0 tx=0x7f1108009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8246f163f631805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.808+0000 7f10fdffb640 1 -- 172.21.15.120:0/1448647912 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108015020 con 0x55e8420839a0 2024-09-10T23:13:50.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.809+0000 7f10fdffb640 1 -- 172.21.15.120:0/1448647912 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110800bd70 con 0x55e8420839a0 2024-09-10T23:13:50.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.809+0000 7f10fdffb640 1 -- 172.21.15.120:0/1448647912 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108010380 con 0x55e8420839a0 2024-09-10T23:13:50.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- 172.21.15.120:0/1448647912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 msgr2=0x55e842085d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.811 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 0x55e842085d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f11080096a0 tx=0x7f1108009b60 comp rx=0 tx=0).stop 2024-09-10T23:13:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- 172.21.15.120:0/1448647912 shutdown_connections 2024-09-10T23:13:50.813 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420839a0 0x55e842085d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e842083460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 --2- 172.21.15.120:0/1448647912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420770f0 0x55e8420774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- 172.21.15.120:0/1448647912 >> 172.21.15.120:0/1448647912 conn(0x55e842074f20 msgr2=0x55e8420760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- 172.21.15.120:0/1448647912 shutdown_connections 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- 172.21.15.120:0/1448647912 wait complete. 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.810+0000 7f1120039740 1 -- start start 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 Processor -- start 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e8420808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842081040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e841e3e890 con 0x55e842077a10 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e841f77d40 con 0x55e8420770f0 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e8420866f0 con 0x55e8420839a0 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.811+0000 7f1120039740 10 client.0 _ll_register_callbacks cb 0x55e841fdbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35982/0 (socket says 172.21.15.120:35982) 2024-09-10T23:13:50.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 -- 172.21.15.120:0/2205693729 learned_addr learned my addr 172.21.15.120:0/2205693729 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 -- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 msgr2=0x55e842081040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842081040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 -- 172.21.15.120:0/2205693729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 msgr2=0x55e8420808b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e8420808b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.812+0000 7f11177fe640 1 -- 172.21.15.120:0/2205693729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84208fa70 con 0x55e8420770f0 2024-09-10T23:13:50.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.813+0000 7f11177fe640 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f111000fe00 tx=0x7f11100092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539b8d914469bceb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.813+0000 7f10f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.813+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110026430 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.813+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110800bd70 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.814+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f111001d430 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.814+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f111001d630 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.814+0000 7f1120039740 10 client.28739 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.814+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e842092450 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.815+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110800bd70 con 0x55e8420770f0 2024-09-10T23:13:50.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.815+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f11100319a0 con 0x55e8420770f0 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 10 client.28739 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 10 client.28739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 20 client.28739 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842093910 0x55e842095d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 4 client.28739 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.816+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e841fefdb0 con 0x55e842093910 2024-09-10T23:13:50.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.817+0000 7f1116ffd640 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842093910 0x55e842095d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.818+0000 7f1116ffd640 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842093910 0x55e842095d70 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3c18567dd8ce3 server_cookie=38e33877b7947857 in_seq=0 out_seq=0 2024-09-10T23:13:50.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.818+0000 7f10f6ffd640 10 client.28739 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.819+0000 7f10f6ffd640 1 -- 172.21.15.120:0/2205693729 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e841fefdb0 con 0x55e842093910 2024-09-10T23:13:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.819+0000 7f10f6ffd640 10 client.28739 handle_command_reply: tid=0 2024-09-10T23:13:50.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.819+0000 7f1120039740 1 client.28739 shutdown 2024-09-10T23:13:50.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 msgr2=0x55e842083440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f111000fe00 tx=0x7f11100092c0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 shutdown_connections 2024-09-10T23:13:50.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842093910 0x55e842095d70 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842081040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842077a10 0x55e8420808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 --2- 172.21.15.120:0/2205693729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e842083440 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 >> 172.21.15.120:0/2205693729 conn(0x55e842074f20 msgr2=0x55e84207aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f10f6ffd640 0 client.28739 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.820+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 shutdown_connections 2024-09-10T23:13:50.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.821+0000 7f1120039740 1 -- 172.21.15.120:0/2205693729 wait complete. 2024-09-10T23:13:50.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.824+0000 7f1120039740 20 client.28739 trim_cache size 0 max 16384 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.871+0000 7f1120039740 1 Processor -- start 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.871+0000 7f1120039740 1 -- start start 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e84207bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 0x55e84207c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 0x55e84207f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f11177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e84207bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f11177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e84207bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35990/0 (socket says 172.21.15.120:35990) 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 0x55e84207c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 0x55e84207c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52552/0 (socket says 172.21.15.120:52552) 2024-09-10T23:13:50.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f111806fd00 con 0x55e8420839a0 2024-09-10T23:13:50.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11181c65d0 con 0x55e8420770f0 2024-09-10T23:13:50.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f1120039740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f111810fe80 con 0x55e842077a10 2024-09-10T23:13:50.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.872+0000 7f11177fe640 1 -- 172.21.15.120:0/3072038657 learned_addr learned my addr 172.21.15.120:0/3072038657 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.874+0000 7f1117fff640 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 0x55e84207f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 -- 172.21.15.120:0/3072038657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 msgr2=0x55e84207c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 0x55e84207c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 -- 172.21.15.120:0/3072038657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 msgr2=0x55e84207bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e84207bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 -- 172.21.15.120:0/3072038657 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8420759a0 con 0x55e8420839a0 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f1117fff640 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 0x55e84207f110 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f11080096a0 tx=0x7f1108006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac757ebbf9d3946f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.875+0000 7f10fd7fa640 1 -- 172.21.15.120:0/3072038657 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108015050 con 0x55e8420839a0 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.876+0000 7f10fd7fa640 1 -- 172.21.15.120:0/3072038657 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110800bd70 con 0x55e8420839a0 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.876+0000 7f10fd7fa640 1 -- 172.21.15.120:0/3072038657 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110801add0 con 0x55e8420839a0 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 -- 172.21.15.120:0/3072038657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 msgr2=0x55e84207f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 0x55e84207f110 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f11080096a0 tx=0x7f1108006da0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 -- 172.21.15.120:0/3072038657 shutdown_connections 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e8420839a0 0x55e84207f110 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e842077a10 0x55e84207c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 --2- 172.21.15.120:0/3072038657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e8420770f0 0x55e84207bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.877+0000 7f1120039740 1 -- 172.21.15.120:0/3072038657 >> 172.21.15.120:0/3072038657 conn(0x55e842074f20 msgr2=0x55e84207fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.879+0000 7f1120039740 1 -- 172.21.15.120:0/3072038657 shutdown_connections 2024-09-10T23:13:50.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.879+0000 7f1120039740 1 -- 172.21.15.120:0/3072038657 wait complete. 2024-09-10T23:13:50.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.879+0000 7f1120039740 1 -- start start 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.880+0000 7f1120039740 1 Processor -- start 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.880+0000 7f1120039740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842087fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1120039740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1120039740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842088990 0x55e8420ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f11177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842087fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.884 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f11177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842087fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52566/0 (socket says 172.21.15.120:52566) 2024-09-10T23:13:50.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1116ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35994/0 (socket says 172.21.15.120:35994) 2024-09-10T23:13:50.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f11177fe640 1 -- 172.21.15.120:0/4094013563 learned_addr learned my addr 172.21.15.120:0/4094013563 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:50.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110550 con 0x55e842088990 2024-09-10T23:13:50.885 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.881+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11181c5f00 con 0x55e842088500 2024-09-10T23:13:50.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.884+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110c20 con 0x55e8420839a0 2024-09-10T23:13:50.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.884+0000 7f1120039740 10 client.0 _ll_register_callbacks cb 0x7f111000fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:50.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 213901 21.70 MB/sec execute 405 sec latency 290.846 ms 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 -- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 msgr2=0x55e842087fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842087fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 -- 172.21.15.120:0/4094013563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842088990 msgr2=0x55e8420ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842088990 0x55e8420ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e842100590 con 0x55e842088500 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.885+0000 7f1116ffd640 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e84207a270 tx=0x7f110c0c7d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795c713462a85b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:50.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.886+0000 7f10f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:13:50.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.886+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0d0130 con 0x55e842088500 2024-09-10T23:13:50.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.886+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f110c004870 con 0x55e842088500 2024-09-10T23:13:50.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.886+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110800bd70 con 0x55e842088500 2024-09-10T23:13:50.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.886+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f110c0d4be0 con 0x55e842088500 2024-09-10T23:13:50.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.887+0000 7f1120039740 10 client.28745 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:50.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.887+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e842101910 con 0x55e842088500 2024-09-10T23:13:50.890 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.888+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c102e50 con 0x55e842088500 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f110c0ca8b0 con 0x55e842088500 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 10 client.28745 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 10 client.28745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 20 client.28745 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842102010 0x55e842104430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 4 client.28745 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:13:50.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.889+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f110800bd70 con 0x55e842102010 2024-09-10T23:13:50.895 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.894+0000 7f11177fe640 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842102010 0x55e842104430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:50.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.904+0000 7f11177fe640 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842102010 0x55e842104430 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5981fd64e43f509b server_cookie=4c0736962b05ddf1 in_seq=0 out_seq=0 2024-09-10T23:13:50.906 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.905+0000 7f10f6ffd640 10 client.28745 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:50.909 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.908+0000 7f10f6ffd640 1 -- 172.21.15.120:0/4094013563 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f110800bd70 con 0x55e842102010 2024-09-10T23:13:50.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.908+0000 7f10f6ffd640 10 client.28745 handle_command_reply: tid=0 2024-09-10T23:13:50.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.908+0000 7f1120039740 1 client.28745 shutdown 2024-09-10T23:13:50.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.909+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 msgr2=0x55e8420fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.909+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55e84207a270 tx=0x7f110c0c7d50 comp rx=0 tx=0).stop 2024-09-10T23:13:50.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 shutdown_connections 2024-09-10T23:13:50.913 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e842102010 0x55e842104430 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e842088990 0x55e8420ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f10f6ffd640 0 client.28745 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e842088500 0x55e8420fd830 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 --2- 172.21.15.120:0/4094013563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e8420839a0 0x55e842087fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 >> 172.21.15.120:0/4094013563 conn(0x55e842074f20 msgr2=0x55e84207fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.911+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 shutdown_connections 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.912+0000 7f1120039740 1 -- 172.21.15.120:0/4094013563 wait complete. 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.912+0000 7f1120039740 20 client.28745 trim_cache size 0 max 16384 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:50.914 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "fafad612-5cd2-4024-9a30-621cf6373623", 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 msgr2=0x7f11000b79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 0x7f11000b79e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1110005e30 tx=0x7f1110005d80 comp rx=0 tx=0).stop 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 msgr2=0x7f111806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:50.915 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f110c02a8f0 tx=0x7f110c0043b0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 shutdown_connections 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f11000b5520 0x7f11000b79e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11180744b0 0x7f1118074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.913+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1118106d40 0x7f111806f110 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.914+0000 7f111ca02640 1 --2- 172.21.15.120:0/1398919556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1118101900 0x7f111806ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:50.916 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.914+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 >> 172.21.15.120:0/1398919556 conn(0x7f11180fd5b0 msgr2=0x7f11180fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:50.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.916+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 shutdown_connections 2024-09-10T23:13:50.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:50.916+0000 7f111ca02640 1 -- 172.21.15.120:0/1398919556 wait complete. 2024-09-10T23:13:51.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214029 21.66 MB/sec execute 406 sec latency 285.949 ms 2024-09-10T23:13:51.907 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:13:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:51 smithi120 ceph-mon[31247]: pgmap v989: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T23:13:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:51 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:13:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:51 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:52.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:51 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:13:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:51 smithi153 ceph-mon[34951]: pgmap v989: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T23:13:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:51 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:13:52.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:51 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:52.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:51 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:13:52.238 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:13:52.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:51 smithi176 ceph-mon[34982]: pgmap v989: 97 pgs: 97 active+clean; 246 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-09-10T23:13:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:51 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:13:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:51 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:52.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:51 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:13:52.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 -- 172.21.15.120:0/1596747922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80101620 msgr2=0x7efe80101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 --2- 172.21.15.120:0/1596747922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80101620 0x7efe80101aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe7400c890 tx=0x7efe7402a470 comp rx=0 tx=0).stop 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 -- 172.21.15.120:0/1596747922 shutdown_connections 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 --2- 172.21.15.120:0/1596747922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 0x7efe8010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 --2- 172.21.15.120:0/1596747922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80101620 0x7efe80101aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 --2- 172.21.15.120:0/1596747922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80103f60 0x7efe80104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 -- 172.21.15.120:0/1596747922 >> 172.21.15.120:0/1596747922 conn(0x7efe80076e00 msgr2=0x7efe800796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 -- 172.21.15.120:0/1596747922 shutdown_connections 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.773+0000 7efe8611d640 1 -- 172.21.15.120:0/1596747922 wait complete. 2024-09-10T23:13:52.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.774+0000 7efe8611d640 1 Processor -- start 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.774+0000 7efe8611d640 1 -- start start 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 0x7efe801974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80103f60 0x7efe801979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe80110c20 con 0x7efe80101fe0 2024-09-10T23:13:52.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe80110aa0 con 0x7efe80103f60 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe8611d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe80110da0 con 0x7efe80101620 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52578/0 (socket says 172.21.15.120:52578) 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 -- 172.21.15.120:0/3865994163 learned_addr learned my addr 172.21.15.120:0/3865994163 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7ffff640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80103f60 0x7efe801979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7effd640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 0x7efe801974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 -- 172.21.15.120:0/3865994163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80103f60 msgr2=0x7efe801979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80103f60 0x7efe801979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 -- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 msgr2=0x7efe801974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 0x7efe801974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.775+0000 7efe7f7fe640 1 -- 172.21.15.120:0/3865994163 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe801abb30 con 0x7efe80101620 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.776+0000 7efe7f7fe640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe700058c0 tx=0x7efe70009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.776+0000 7efe7cff9640 1 -- 172.21.15.120:0/3865994163 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70009e30 con 0x7efe80101620 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.776+0000 7efe7cff9640 1 -- 172.21.15.120:0/3865994163 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe70005020 con 0x7efe80101620 2024-09-10T23:13:52.779 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.776+0000 7efe7cff9640 1 -- 172.21.15.120:0/3865994163 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004bc0 con 0x7efe80101620 2024-09-10T23:13:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.776+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe801abe20 con 0x7efe80101620 2024-09-10T23:13:52.780 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.777+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801ac320 con 0x7efe80101620 2024-09-10T23:13:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 Processor -- start 2024-09-10T23:13:52.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 -- start start 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a929c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a929c950 0x5647a929ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647a9206e50 con 0x5647a929c950 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a91f6280 con 0x5647a92909c0 2024-09-10T23:13:52.782 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.780+0000 7efe884f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5647a91fa090 con 0x5647a92900a0 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52594/0 (socket says 172.21.15.120:52594) 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 -- 172.21.15.120:0/806792934 learned_addr learned my addr 172.21.15.120:0/806792934 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 -- 172.21.15.120:0/806792934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 msgr2=0x5647a929c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a929c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 -- 172.21.15.120:0/806792934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a929c950 msgr2=0x5647a929ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.783 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a929c950 0x5647a929ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.781+0000 7efe7ffff640 1 -- 172.21.15.120:0/806792934 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a8f54650 con 0x5647a92900a0 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.782+0000 7efe7cff9640 1 -- 172.21.15.120:0/3865994163 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efe700054d0 con 0x7efe80101620 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.782+0000 7efe7ffff640 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe78009fd0 tx=0x7efe780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f237ebb1a1c9c066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.782+0000 7efe65ffb640 1 -- 172.21.15.120:0/806792934 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78004d20 con 0x5647a92900a0 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.782+0000 7efe65ffb640 1 -- 172.21.15.120:0/806792934 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe78004050 con 0x5647a92900a0 2024-09-10T23:13:52.784 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.782+0000 7efe65ffb640 1 -- 172.21.15.120:0/806792934 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78005280 con 0x5647a92900a0 2024-09-10T23:13:52.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.783+0000 7efe7cff9640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 0x7efe680b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.783+0000 7efe884f2740 1 -- 172.21.15.120:0/806792934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 msgr2=0x5647a9290480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.783+0000 7efe884f2740 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe78009fd0 tx=0x7efe780118d0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.785 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.783+0000 7efe7cff9640 1 -- 172.21.15.120:0/3865994163 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe700e50e0 con 0x7efe80101620 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 -- 172.21.15.120:0/806792934 shutdown_connections 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a929c950 0x5647a929ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a929c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 --2- 172.21.15.120:0/806792934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a92900a0 0x5647a9290480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 -- 172.21.15.120:0/806792934 >> 172.21.15.120:0/806792934 conn(0x5647a928ded0 msgr2=0x5647a928f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe7f7fe640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 0x7efe680b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 -- 172.21.15.120:0/806792934 shutdown_connections 2024-09-10T23:13:52.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 -- 172.21.15.120:0/806792934 wait complete. 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.784+0000 7efe884f2740 1 -- start start 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe7f7fe640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 0x7efe680b78e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efe7000cfd0 tx=0x7efe700046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 Processor -- start 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a929c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9299790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a9299f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647a9057700 con 0x5647a92900a0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5647a9191110 con 0x5647a92909c0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5647a929f5d0 con 0x5647a929c950 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.785+0000 7efe884f2740 10 client.0 _ll_register_callbacks cb 0x5647a91aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9299790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9299790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36034/0 (socket says 172.21.15.120:36034) 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7effd640 1 -- 172.21.15.120:0/36243072 learned_addr learned my addr 172.21.15.120:0/36243072 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a9299f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a929c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 -- 172.21.15.120:0/36243072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 msgr2=0x5647a9299790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9299790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 -- 172.21.15.120:0/36243072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 msgr2=0x5647a929c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.789 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a929c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 -- 172.21.15.120:0/36243072 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a91bec40 con 0x5647a929c950 2024-09-10T23:13:52.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe7ffff640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a9299f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe78009fa0 tx=0x7efe78004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e74be3fcb2e828 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.786+0000 7efe5effd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7802d060 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe780044c0 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe78004050 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe78004a00 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe884f2740 10 client.29035 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647a92ab000 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.787+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe78004050 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efe780056f0 con 0x5647a929c950 2024-09-10T23:13:52.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 10 client.29035 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 10 client.29035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 20 client.29035 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a92ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 4 client.29035 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a9208da0 con 0x5647a92ac400 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.788+0000 7efe7effd640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a92ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.789+0000 7efe5effd640 10 client.29035 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.789+0000 7efe7effd640 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a92ae7f0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00c22a4593f96f2 server_cookie=8ae5e8c0cc922329 in_seq=0 out_seq=0 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe5effd640 1 -- 172.21.15.120:0/36243072 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5647a9208da0 con 0x5647a92ac400 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe5effd640 10 client.29035 handle_command_reply: tid=0 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 client.29035 shutdown 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 msgr2=0x5647a9299f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a9299f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe78009fa0 tx=0x7efe78004950 comp rx=0 tx=0).stop 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 shutdown_connections 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a92ae7f0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe5effd640 0 client.29035 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a9299f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9299790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 --2- 172.21.15.120:0/36243072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a929c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 >> 172.21.15.120:0/36243072 conn(0x5647a928ded0 msgr2=0x5647a9293a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 shutdown_connections 2024-09-10T23:13:52.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.791+0000 7efe884f2740 1 -- 172.21.15.120:0/36243072 wait complete. 2024-09-10T23:13:52.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.792+0000 7efe884f2740 20 client.29035 trim_cache size 0 max 16384 2024-09-10T23:13:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 Processor -- start 2024-09-10T23:13:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 -- start start 2024-09-10T23:13:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a9296a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a929dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b2ae0 con 0x5647a92900a0 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b5360 con 0x5647a92909c0 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.813+0000 7efe884f2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b3150 con 0x5647a929c950 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:36040/0 (socket says 172.21.15.120:36040) 2024-09-10T23:13:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 -- 172.21.15.120:0/1111158038 learned_addr learned my addr 172.21.15.120:0/1111158038 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a9296a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 -- 172.21.15.120:0/1111158038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 msgr2=0x5647a929dd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a929dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 -- 172.21.15.120:0/1111158038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 msgr2=0x5647a9296a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a9296a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7effd640 1 -- 172.21.15.120:0/1111158038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a928e950 con 0x5647a92909c0 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.814+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a9296a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.815+0000 7efe7effd640 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fa0 tx=0x7efe74030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6863725d107dfef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.815+0000 7efe657fa640 1 -- 172.21.15.120:0/1111158038 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74030b80 con 0x5647a92909c0 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.815+0000 7efe657fa640 1 -- 172.21.15.120:0/1111158038 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe740054f0 con 0x5647a92909c0 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.815+0000 7efe657fa640 1 -- 172.21.15.120:0/1111158038 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74004c60 con 0x5647a92909c0 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- 172.21.15.120:0/1111158038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 msgr2=0x5647a9296f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fa0 tx=0x7efe74030890 comp rx=0 tx=0).stop 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- 172.21.15.120:0/1111158038 shutdown_connections 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a929dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a9296f40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 --2- 172.21.15.120:0/1111158038 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a9296a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- 172.21.15.120:0/1111158038 >> 172.21.15.120:0/1111158038 conn(0x5647a928ded0 msgr2=0x5647a928ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- 172.21.15.120:0/1111158038 shutdown_connections 2024-09-10T23:13:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- 172.21.15.120:0/1111158038 wait complete. 2024-09-10T23:13:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.816+0000 7efe884f2740 1 -- start start 2024-09-10T23:13:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 Processor -- start 2024-09-10T23:13:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a92a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a92a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a92959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a92a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a92a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55594/0 (socket says 172.21.15.120:55594) 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe7ffff640 1 -- 172.21.15.120:0/1051669536 learned_addr learned my addr 172.21.15.120:0/1051669536 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b3820 con 0x5647a92900a0 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b4c90 con 0x5647a92909c0 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b3ef0 con 0x5647a929c950 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.817+0000 7efe884f2740 10 client.0 _ll_register_callbacks cb 0x7efe78009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a92959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 -- 172.21.15.120:0/1051669536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 msgr2=0x5647a92a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a92a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 -- 172.21.15.120:0/1051669536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 msgr2=0x5647a92a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a92a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a92960e0 con 0x5647a929c950 2024-09-10T23:13:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe7ffff640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a92959f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe7800a010 tx=0x7efe78004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4197632fed85441b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.818+0000 7efe5effd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7802d070 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efe78009b10 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe740054f0 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe780042c0 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe884f2740 10 client.29038 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.819+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647a9297340 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe78004460 con 0x5647a929c950 2024-09-10T23:13:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7efe780288c0 con 0x5647a929c950 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 10 client.29038 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 10 client.29038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 20 client.29038 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a931c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 4 client.29038 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efe740054f0 con 0x5647a92ac400 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.820+0000 7efe7effd640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a931c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.821+0000 7efe5effd640 10 client.29038 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.821+0000 7efe7effd640 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a931c110 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c21bffc071737d server_cookie=9a86654f70b5f21 in_seq=0 out_seq=0 2024-09-10T23:13:52.825 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe5effd640 1 -- 172.21.15.120:0/1051669536 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10510 (crc 0 0 0) 0x7efe740054f0 con 0x5647a92ac400 2024-09-10T23:13:52.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe5effd640 10 client.29038 handle_command_reply: tid=0 2024-09-10T23:13:52.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 client.29038 shutdown 2024-09-10T23:13:52.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 msgr2=0x5647a92959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a92959f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe7800a010 tx=0x7efe78004950 comp rx=0 tx=0).stop 2024-09-10T23:13:52.826 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 shutdown_connections 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5647a92ac400 0x5647a931c110 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5647a929c950 0x5647a92959f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5647a92909c0 0x5647a92a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 --2- 172.21.15.120:0/1051669536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647a92900a0 0x5647a92a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 >> 172.21.15.120:0/1051669536 conn(0x5647a928ded0 msgr2=0x5647a929adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.824+0000 7efe5effd640 0 client.29038 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.825+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 shutdown_connections 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.825+0000 7efe884f2740 1 -- 172.21.15.120:0/1051669536 wait complete. 2024-09-10T23:13:52.827 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.825+0000 7efe884f2740 20 client.29038 trim_cache size 0 max 16384 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "fafad612-5cd2-4024-9a30-621cf6373623", 2024-09-10T23:13:52.828 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:13:52.830 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.831 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.832 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.833 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:13:52.834 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.835 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.836 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:13:52.837 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.838 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.839 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.840 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.841 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:13:52.842 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.843 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.844 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:13:52.845 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.846 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.847 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.848 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:13:52.849 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:13:52.850 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:52.851 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:13:52.851 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 msgr2=0x7efe680b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 0x7efe680b78e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efe7000cfd0 tx=0x7efe700046c0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 msgr2=0x7efe80196f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe700058c0 tx=0x7efe70009be0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 shutdown_connections 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efe680b5420 0x7efe680b78e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.852 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe80103f60 0x7efe801979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe80101fe0 0x7efe801974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 --2- 172.21.15.120:0/3865994163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe80101620 0x7efe80196f60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 >> 172.21.15.120:0/3865994163 conn(0x7efe80076e00 msgr2=0x7efe80105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 shutdown_connections 2024-09-10T23:13:52.853 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:13:52.827+0000 7efe8611d640 1 -- 172.21.15.120:0/3865994163 wait complete. 2024-09-10T23:13:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/2949374446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4103380 msgr2=0x7f6dc4109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/2949374446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4103380 0x7f6dc4109d00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009cd0 tx=0x7f6dbc02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/2949374446 shutdown_connections 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/2949374446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4103380 0x7f6dc4109d00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/2949374446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 0x7f6dc4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/2949374446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4101870 0x7f6dc4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/2949374446 >> 172.21.15.120:0/2949374446 conn(0x7f6dc40fd130 msgr2=0x7f6dc40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/2949374446 shutdown_connections 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.837+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/2949374446 wait complete. 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 Processor -- start 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 -- start start 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4101870 0x7f6dc4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 0x7f6dc4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410c440 con 0x7f6dc4103380 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410c2c0 con 0x7f6dc41029c0 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc9dbb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410c5c0 con 0x7f6dc4101870 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55606/0 (socket says 172.21.15.120:55606) 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc3fff640 1 -- 172.21.15.120:0/594448530 learned_addr learned my addr 172.21.15.120:0/594448530 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.838+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4101870 0x7f6dc4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 0x7f6dc4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 -- 172.21.15.120:0/594448530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4101870 msgr2=0x7f6dc4196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4101870 0x7f6dc4196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 -- 172.21.15.120:0/594448530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 msgr2=0x7f6dc4197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 0x7f6dc4197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 -- 172.21.15.120:0/594448530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc4198640 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc3fff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02ae00 tx=0x7f6dbc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc17fa640 1 -- 172.21.15.120:0/594448530 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc004040 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41988d0 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc17fa640 1 -- 172.21.15.120:0/594448530 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc004690 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.839+0000 7f6dc17fa640 1 -- 172.21.15.120:0/594448530 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc030de0 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.840+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc419f000 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.844+0000 7f6dc17fa640 1 -- 172.21.15.120:0/594448530 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6dbc030070 con 0x7f6dc4103380 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.844+0000 7f6dccc65740 1 Processor -- start 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 -- start start 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bebd0 0x560abe6c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560abe62b6f0 con 0x560abe6b9860 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560abe61ab20 con 0x560abe6bebd0 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dccc65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560abe61e930 con 0x560abe6bc190 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.845+0000 7f6dc17fa640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 0x7f6db00b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55612/0 (socket says 172.21.15.120:55612) 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 -- 172.21.15.120:0/1343756653 learned_addr learned my addr 172.21.15.120:0/1343756653 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dc17fa640 1 -- 172.21.15.120:0/594448530 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dbc02e030 con 0x7f6dc4103380 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52632/0 (socket says 172.21.15.120:52632) 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bebd0 0x560abe6c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 -- 172.21.15.120:0/1343756653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 msgr2=0x560abe6be580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dc3fff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 0x7f6db00b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 -- 172.21.15.120:0/1343756653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bebd0 msgr2=0x560abe6c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bebd0 0x560abe6c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dbbfff640 1 -- 172.21.15.120:0/1343756653 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abe5e0530 con 0x560abe6b9860 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.846+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.847+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6db4018990 tx=0x7f6db400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54143bb2a8a0259 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.847+0000 7f6db9ffb640 1 -- 172.21.15.120:0/1343756653 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400b9d0 con 0x560abe6b9860 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.847+0000 7f6db9ffb640 1 -- 172.21.15.120:0/1343756653 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4018110 con 0x560abe6b9860 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.847+0000 7f6db9ffb640 1 -- 172.21.15.120:0/1343756653 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4016ab0 con 0x560abe6b9860 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.848+0000 7f6dccc65740 1 -- 172.21.15.120:0/1343756653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 msgr2=0x560abe6bbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.848+0000 7f6dccc65740 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6db4018990 tx=0x7f6db400d430 comp rx=0 tx=0).stop 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dc3fff640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 0x7f6db00b78a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc031040 tx=0x7f6dbc009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 -- 172.21.15.120:0/1343756653 shutdown_connections 2024-09-10T23:13:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bebd0 0x560abe6c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bc190 0x560abe6be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 --2- 172.21.15.120:0/1343756653 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6b9860 0x560abe6bbc50 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 -- 172.21.15.120:0/1343756653 >> 172.21.15.120:0/1343756653 conn(0x560abe6b49b0 msgr2=0x560abe6b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 -- 172.21.15.120:0/1343756653 shutdown_connections 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 -- 172.21.15.120:0/1343756653 wait complete. 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 -- start start 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.849+0000 7f6dccc65740 1 Processor -- start 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6b9860 0x560abe6c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bebd0 0x560abe6c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560abe552ab0 con 0x560abe6bc190 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560abe5c0730 con 0x560abe6b9860 2024-09-10T23:13:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560abe6c1bb0 con 0x560abe6bebd0 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dccc65740 10 client.0 _ll_register_callbacks cb 0x560abe6146b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55628/0 (socket says 172.21.15.120:55628) 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 -- 172.21.15.120:0/2642637162 learned_addr learned my addr 172.21.15.120:0/2642637162 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 -- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bebd0 msgr2=0x560abe6c41f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bebd0 0x560abe6c41f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 -- 172.21.15.120:0/2642637162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6b9860 msgr2=0x560abe6c2f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6b9860 0x560abe6c2f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.850+0000 7f6dc37fe640 1 -- 172.21.15.120:0/2642637162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abe37c790 con 0x560abe6bc190 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00a4d0 tx=0x7f6dac00a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38f122e3dac2461 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01b020 con 0x560abe6bc190 2024-09-10T23:13:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db4018110 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dac009980 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dac009c20 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6dccc65740 10 client.19614 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560abe3f5620 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4018110 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.851+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dac028020 con 0x560abe6bc190 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 10 client.19614 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 10 client.19614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 20 client.19614 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe6ce9f0 0x560abe6d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 4 client.19614 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abe62d640 con 0x560abe6ce9f0 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.852+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe6ce9f0 0x560abe6d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.853+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe6ce9f0 0x560abe6d0e60 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ad2fdbcb90353e server_cookie=92ead90224f122d2 in_seq=0 out_seq=0 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.853+0000 7f6da2ffd640 10 client.19614 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6da2ffd640 1 -- 172.21.15.120:0/2642637162 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560abe62d640 con 0x560abe6ce9f0 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6da2ffd640 10 client.19614 handle_command_reply: tid=0 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 client.19614 shutdown 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 msgr2=0x560abe6c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00a4d0 tx=0x7f6dac00a990 comp rx=0 tx=0).stop 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 shutdown_connections 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe6ce9f0 0x560abe6d0e60 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6bebd0 0x560abe6c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bc190 0x560abe6c3620 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 --2- 172.21.15.120:0/2642637162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6b9860 0x560abe6c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 >> 172.21.15.120:0/2642637162 conn(0x560abe6b49b0 msgr2=0x560abe6b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6da2ffd640 0 client.19614 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 shutdown_connections 2024-09-10T23:13:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.854+0000 7f6dccc65740 1 -- 172.21.15.120:0/2642637162 wait complete. 2024-09-10T23:13:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.855+0000 7f6dccc65740 20 client.19614 trim_cache size 0 max 16384 2024-09-10T23:13:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 Processor -- start 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 -- start start 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.872+0000 7f6dccc65740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a5bc0 con 0x560abe6bebd0 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dccc65740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a8440 con 0x560abe6bc190 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dccc65740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a6230 con 0x560abe6b9860 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52664/0 (socket says 172.21.15.120:52664) 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 -- 172.21.15.120:0/4135881012 learned_addr learned my addr 172.21.15.120:0/4135881012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 -- 172.21.15.120:0/4135881012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 msgr2=0x560abe6bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 -- 172.21.15.120:0/4135881012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 msgr2=0x560abe6c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 -- 172.21.15.120:0/4135881012 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abe6be8f0 con 0x560abe6b9860 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6db4018960 tx=0x7f6db4016980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93dec71e3e4c8d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.873+0000 7f6db97fa640 1 -- 172.21.15.120:0/4135881012 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db402e040 con 0x560abe6b9860 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.874+0000 7f6db97fa640 1 -- 172.21.15.120:0/4135881012 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4018070 con 0x560abe6b9860 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.874+0000 7f6db97fa640 1 -- 172.21.15.120:0/4135881012 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400b9d0 con 0x560abe6b9860 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.874+0000 7f6dccc65740 1 -- 172.21.15.120:0/4135881012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 msgr2=0x560abe6befb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.874+0000 7f6dccc65740 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6db4018960 tx=0x7f6db4016980 comp rx=0 tx=0).stop 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 -- 172.21.15.120:0/4135881012 shutdown_connections 2024-09-10T23:13:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 --2- 172.21.15.120:0/4135881012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6befb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 -- 172.21.15.120:0/4135881012 >> 172.21.15.120:0/4135881012 conn(0x560abe6b49b0 msgr2=0x560abe6afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 -- 172.21.15.120:0/4135881012 shutdown_connections 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 -- 172.21.15.120:0/4135881012 wait complete. 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 -- start start 2024-09-10T23:13:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 Processor -- start 2024-09-10T23:13:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.875+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52680/0 (socket says 172.21.15.120:52680) 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dbbfff640 1 -- 172.21.15.120:0/737323121 learned_addr learned my addr 172.21.15.120:0/737323121 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a6900 con 0x560abe6bebd0 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a7d70 con 0x560abe6bc190 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a6fd0 con 0x560abe6b9860 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dccc65740 10 client.0 _ll_register_callbacks cb 0x7f6dac00b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 -- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 msgr2=0x560abe6c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 -- 172.21.15.120:0/737323121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 msgr2=0x560abe6c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dc37fe640 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abe6c9720 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.876+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dc37fe640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6c8320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00d010 tx=0x7f6dac009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae2260f3ed8fa257 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01b040 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db4018070 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6dac0052c0 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dac03a070 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 10 client.28772 fetch_fsmap learned FSMap version 26 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560abe6caae0 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4018070 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6dac028070 con 0x560abe6bc190 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 10 client.28772 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 10 client.28772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:13:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.877+0000 7f6dccc65740 20 client.28772 populate_metadata read hostname 'smithi120' 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe73bb40 0x560abe73df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6dccc65740 4 client.28772 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560abe6290f0 con 0x560abe73bb40 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe73bb40 0x560abe73df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6da2ffd640 10 client.28772 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:52.878+0000 7f6dbbfff640 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe73bb40 0x560abe73df60 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa23cef88145b90b server_cookie=b4d31c8fcca6b84c in_seq=0 out_seq=0 2024-09-10T23:13:52.886 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214084 21.61 MB/sec execute 407 sec latency 654.932 ms 2024-09-10T23:13:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6da2ffd640 1 -- 172.21.15.120:0/737323121 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560abe6290f0 con 0x560abe73bb40 2024-09-10T23:13:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6da2ffd640 10 client.28772 handle_command_reply: tid=0 2024-09-10T23:13:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6dccc65740 1 client.28772 shutdown 2024-09-10T23:13:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 msgr2=0x560abe6c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6c8320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00d010 tx=0x7f6dac009990 comp rx=0 tx=0).stop 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 shutdown_connections 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560abe73bb40 0x560abe73df60 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.534+0000 7f6da2ffd640 0 client.28772 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560abe6bebd0 0x560abe6c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560abe6bc190 0x560abe6c8320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 --2- 172.21.15.120:0/737323121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560abe6b9860 0x560abe6c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 >> 172.21.15.120:0/737323121 conn(0x560abe6b49b0 msgr2=0x560abe6b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 shutdown_connections 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 1 -- 172.21.15.120:0/737323121 wait complete. 2024-09-10T23:13:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.535+0000 7f6dccc65740 20 client.28772 trim_cache size 0 max 16384 2024-09-10T23:13:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:13:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:13:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:13:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 msgr2=0x7f6db00b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 0x7f6db00b78a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc031040 tx=0x7f6dbc009870 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 msgr2=0x7f6dc4197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc02ae00 tx=0x7f6dbc003ba0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 shutdown_connections 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6db00b53e0 0x7f6db00b78a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dc4103380 0x7f6dc4197d80 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6dc41029c0 0x7f6dc4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 --2- 172.21.15.120:0/594448530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dc4101870 0x7f6dc4196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:13:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.536+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 >> 172.21.15.120:0/594448530 conn(0x7f6dc40fd130 msgr2=0x7f6dc4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:13:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.537+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 shutdown_connections 2024-09-10T23:13:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:13:53.537+0000 7f6dc9dbb640 1 -- 172.21.15.120:0/594448530 wait complete. 2024-09-10T23:13:53.719 INFO:tasks.cephfs.filesystem:scrub status for tag:fafad612-5cd2-4024-9a30-621cf6373623 - {'path': '/', 'tag': 'fafad612-5cd2-4024-9a30-621cf6373623', 'options': 'recursive,force'} 2024-09-10T23:13:53.887 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214337 21.60 MB/sec execute 408 sec latency 400.363 ms 2024-09-10T23:13:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:53 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:53 smithi120 ceph-mon[31247]: pgmap v990: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 611 op/s 2024-09-10T23:13:54.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:53 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:54.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:53 smithi153 ceph-mon[34951]: pgmap v990: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 611 op/s 2024-09-10T23:13:54.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:53 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:13:54.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:53 smithi176 ceph-mon[34982]: pgmap v990: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 611 op/s 2024-09-10T23:13:54.887 INFO:tasks.workunit.client.0.smithi120.stdout: 10 214821 21.60 MB/sec execute 409 sec latency 80.983 ms 2024-09-10T23:13:55.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 215151 21.58 MB/sec execute 410 sec latency 238.791 ms 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: pgmap v991: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 309 op/s 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.196 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:55 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: pgmap v991: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 309 op/s 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:55 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: pgmap v991: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 309 op/s 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:55 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:56.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 215599 21.61 MB/sec execute 411 sec latency 182.519 ms 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:57.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:57.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:57.697 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:13:57.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:13:57.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 216129 21.63 MB/sec execute 412 sec latency 71.019 ms 2024-09-10T23:13:58.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:13:58 smithi176 ceph-mon[34982]: pgmap v992: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 625 op/s 2024-09-10T23:13:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:13:58 smithi120 ceph-mon[31247]: pgmap v992: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 625 op/s 2024-09-10T23:13:58.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:13:58 smithi153 ceph-mon[34951]: pgmap v992: 97 pgs: 97 active+clean; 254 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.8 MiB/s wr, 625 op/s 2024-09-10T23:13:58.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 216581 21.65 MB/sec execute 413 sec latency 80.131 ms 2024-09-10T23:13:59.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 216965 21.66 MB/sec execute 414 sec latency 58.405 ms 2024-09-10T23:14:00.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:00 smithi176 ceph-mon[34982]: pgmap v993: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-10T23:14:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:00 smithi120 ceph-mon[31247]: pgmap v993: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-10T23:14:00.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:00 smithi153 ceph-mon[34951]: pgmap v993: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-10T23:14:00.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 217184 21.61 MB/sec execute 415 sec latency 342.425 ms 2024-09-10T23:14:01.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 217484 21.59 MB/sec execute 416 sec latency 115.547 ms 2024-09-10T23:14:02.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:02 smithi176 ceph-mon[34982]: pgmap v994: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 590 op/s 2024-09-10T23:14:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:02 smithi120 ceph-mon[31247]: pgmap v994: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 590 op/s 2024-09-10T23:14:02.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:02 smithi153 ceph-mon[34951]: pgmap v994: 97 pgs: 97 active+clean; 242 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 590 op/s 2024-09-10T23:14:02.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 218016 21.60 MB/sec execute 417 sec latency 57.854 ms 2024-09-10T23:14:03.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 218590 21.62 MB/sec execute 418 sec latency 84.351 ms 2024-09-10T23:14:04.500 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:04 smithi153 ceph-mon[34951]: pgmap v995: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-09-10T23:14:04.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:04 smithi176 ceph-mon[34982]: pgmap v995: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-09-10T23:14:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:04 smithi120 ceph-mon[31247]: pgmap v995: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-09-10T23:14:04.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 219154 21.65 MB/sec execute 419 sec latency 93.066 ms 2024-09-10T23:14:05.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 219554 21.65 MB/sec execute 420 sec latency 221.364 ms 2024-09-10T23:14:06.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:06 smithi176 ceph-mon[34982]: pgmap v996: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-09-10T23:14:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 -- 172.21.15.120:0/1757188982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b041029b0 msgr2=0x7f4b04102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 --2- 172.21.15.120:0/1757188982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b041029b0 0x7f4b04102e30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009930 tx=0x7f4af402a420 comp rx=0 tx=0).stop 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 -- 172.21.15.120:0/1757188982 shutdown_connections 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 --2- 172.21.15.120:0/1757188982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b04103370 0x7f4b04109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 --2- 172.21.15.120:0/1757188982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b041029b0 0x7f4b04102e30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.632+0000 7f4b0c175640 1 --2- 172.21.15.120:0/1757188982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04101860 0x7f4b04101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- 172.21.15.120:0/1757188982 >> 172.21.15.120:0/1757188982 conn(0x7f4b040fd140 msgr2=0x7f4b040ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- 172.21.15.120:0/1757188982 shutdown_connections 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- 172.21.15.120:0/1757188982 wait complete. 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 Processor -- start 2024-09-10T23:14:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- start start 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b041029b0 0x7f4b04103e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04103370 0x7f4b041043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0410d4b0 con 0x7f4b04101860 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0410d330 con 0x7f4b04103370 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0c175640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0410d630 con 0x7f4b041029b0 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37736/0 (socket says 172.21.15.120:37736) 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b09f11640 1 -- 172.21.15.120:0/2889695284 learned_addr learned my addr 172.21.15.120:0/2889695284 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.633+0000 7f4b0a712640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04103370 0x7f4b041043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09710640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b041029b0 0x7f4b04103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 -- 172.21.15.120:0/2889695284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b041029b0 msgr2=0x7f4b04103e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b041029b0 0x7f4b04103e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 -- 172.21.15.120:0/2889695284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04103370 msgr2=0x7f4b041043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04103370 0x7f4b041043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 -- 172.21.15.120:0/2889695284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b04104c30 con 0x7f4b04101860 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b09f11640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000d950 tx=0x7f4b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4afaffd640 1 -- 172.21.15.120:0/2889695284 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00017070 con 0x7f4b04101860 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4afaffd640 1 -- 172.21.15.120:0/2889695284 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b00009c30 con 0x7f4b04101860 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4afaffd640 1 -- 172.21.15.120:0/2889695284 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000f620 con 0x7f4b04101860 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b041a2e90 con 0x7f4b04101860 2024-09-10T23:14:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.634+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b041a3300 con 0x7f4b04101860 2024-09-10T23:14:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.636+0000 7f4b0f022740 1 Processor -- start 2024-09-10T23:14:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.636+0000 7f4afaffd640 1 -- 172.21.15.120:0/2889695284 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4b00010460 con 0x7f4b04101860 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 -- start start 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea7bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea7e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c0e9eb6f0 con 0x559c0ea79860 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c0e9dab20 con 0x559c0ea7c190 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0f022740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559c0e9de930 con 0x559c0ea7ebd0 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0a712640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea7bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea7e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09710640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09710640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41032/0 (socket says 172.21.15.120:41032) 2024-09-10T23:14:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09710640 1 -- 172.21.15.120:0/1715245794 learned_addr learned my addr 172.21.15.120:0/1715245794 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b0a712640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea7bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37750/0 (socket says 172.21.15.120:37750) 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 -- 172.21.15.120:0/1715245794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 msgr2=0x559c0ea81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea81090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 -- 172.21.15.120:0/1715245794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 msgr2=0x559c0ea7bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4afaffd640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 0x7f4af00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea7bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.637+0000 7f4b09f11640 1 -- 172.21.15.120:0/1715245794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0e9a0530 con 0x559c0ea7c190 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4b09710640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 0x7f4af00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4afaffd640 1 -- 172.21.15.120:0/2889695284 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4b000e56f0 con 0x7f4b04101860 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea7e580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e31f0 tx=0x7f4b000a86a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a18138e7cf882e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4aeffff640 1 -- 172.21.15.120:0/1715245794 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00020070 con 0x559c0ea7c190 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4aeffff640 1 -- 172.21.15.120:0/1715245794 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b000abaa0 con 0x559c0ea7c190 2024-09-10T23:14:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.638+0000 7f4aeffff640 1 -- 172.21.15.120:0/1715245794 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000ab700 con 0x559c0ea7c190 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- 172.21.15.120:0/1715245794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 msgr2=0x559c0ea7e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b09710640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 0x7f4af00b7820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005c40 tx=0x7f4af40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea7e580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e31f0 tx=0x7f4b000a86a0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- 172.21.15.120:0/1715245794 shutdown_connections 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea7e580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1715245794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea7bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- 172.21.15.120:0/1715245794 >> 172.21.15.120:0/1715245794 conn(0x559c0ea749b0 msgr2=0x559c0ea75740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- 172.21.15.120:0/1715245794 shutdown_connections 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- 172.21.15.120:0/1715245794 wait complete. 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 -- start start 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.639+0000 7f4b0f022740 1 Processor -- start 2024-09-10T23:14:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 0x559c0ea82f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559c0e912ab0 con 0x559c0ea7c190 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x559c0e980730 con 0x559c0ea79860 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559c0ea81bb0 con 0x559c0ea7ebd0 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0f022740 10 client.0 _ll_register_callbacks cb 0x559c0e9d46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37766/0 (socket says 172.21.15.120:37766) 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 0x559c0ea82f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 -- 172.21.15.120:0/612921173 learned_addr learned my addr 172.21.15.120:0/612921173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b0a712640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 -- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 msgr2=0x559c0ea841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 -- 172.21.15.120:0/612921173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 msgr2=0x559c0ea82f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 0x559c0ea82f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 -- 172.21.15.120:0/612921173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0e73c790 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09f11640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 0x559c0ea82f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4b09710640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005ab0 tx=0x7f4af4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30cf27561d401e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.640+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4035040 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4af40038b0 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b000abaa0 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4af4003ab0 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 10 client.19632 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c0e7b5620 con 0x559c0ea7c190 2024-09-10T23:14:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b000abaa0 con 0x559c0ea7c190 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4af4044070 con 0x559c0ea7c190 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 10 client.19632 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 10 client.19632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 20 client.19632 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0ea8e920 0x559c0ea90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 4 client.19632 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c0e9ed640 con 0x559c0ea8e920 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.641+0000 7f4b09f11640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0ea8e920 0x559c0ea90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.642+0000 7f4aed7fa640 10 client.19632 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.642+0000 7f4b09f11640 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0ea8e920 0x559c0ea90d30 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da20f6d6ebdacf62 server_cookie=149478e46cc83650 in_seq=0 out_seq=0 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4aed7fa640 1 -- 172.21.15.120:0/612921173 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c0e9ed640 con 0x559c0ea8e920 2024-09-10T23:14:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4aed7fa640 10 client.19632 handle_command_reply: tid=0 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 client.19632 shutdown 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 msgr2=0x559c0ea83620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005ab0 tx=0x7f4af4009260 comp rx=0 tx=0).stop 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 shutdown_connections 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0ea8e920 0x559c0ea90d30 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea7c190 0x559c0ea83620 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 --2- 172.21.15.120:0/612921173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea79860 0x559c0ea82f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4aed7fa640 0 client.19632 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.643+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 >> 172.21.15.120:0/612921173 conn(0x559c0ea749b0 msgr2=0x559c0ea73bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.644+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 shutdown_connections 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.644+0000 7f4b0f022740 1 -- 172.21.15.120:0/612921173 wait complete. 2024-09-10T23:14:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.644+0000 7f4b0f022740 20 client.19632 trim_cache size 0 max 16384 2024-09-10T23:14:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.662+0000 7f4b0f022740 1 Processor -- start 2024-09-10T23:14:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.662+0000 7f4b0f022740 1 -- start start 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.662+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea78d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0f022740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041aa000 con 0x559c0ea79860 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0f022740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041ac880 con 0x559c0ea7c190 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0f022740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041aa670 con 0x559c0ea7ebd0 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09710640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea78d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37768/0 (socket says 172.21.15.120:37768) 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 -- 172.21.15.120:0/200068512 learned_addr learned my addr 172.21.15.120:0/200068512 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b0a712640 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 -- 172.21.15.120:0/200068512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 msgr2=0x559c0ea792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 -- 172.21.15.120:0/200068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 msgr2=0x559c0ea78d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea78d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 -- 172.21.15.120:0/200068512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0ea81fb0 con 0x559c0ea79860 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09710640 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea78d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4b09f11640 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e3590 tx=0x7f4b000a8710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64432022d4daa08f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4aef7fe640 1 -- 172.21.15.120:0/200068512 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00020040 con 0x559c0ea79860 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4aef7fe640 1 -- 172.21.15.120:0/200068512 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b000abaa0 con 0x559c0ea79860 2024-09-10T23:14:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.663+0000 7f4aef7fe640 1 -- 172.21.15.120:0/200068512 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000b09c0 con 0x559c0ea79860 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 -- 172.21.15.120:0/200068512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 msgr2=0x559c0ea841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e3590 tx=0x7f4b000a8710 comp rx=0 tx=0).stop 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 -- 172.21.15.120:0/200068512 shutdown_connections 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7ebd0 0x559c0ea792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7c190 0x559c0ea78d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 --2- 172.21.15.120:0/200068512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea841d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.664+0000 7f4b0f022740 1 -- 172.21.15.120:0/200068512 >> 172.21.15.120:0/200068512 conn(0x559c0ea749b0 msgr2=0x559c0ea6faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 -- 172.21.15.120:0/200068512 shutdown_connections 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 -- 172.21.15.120:0/200068512 wait complete. 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 -- start start 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 Processor -- start 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.665+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0f022740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 0x559c0eaf9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0f022740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041aad40 con 0x559c0ea79860 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0f022740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041ac1b0 con 0x559c0ea7ebd0 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0f022740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b041ab410 con 0x559c0ea7c190 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0f022740 10 client.0 _ll_register_callbacks cb 0x7f4af4032e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09710640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37774/0 (socket says 172.21.15.120:37774) 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09710640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:41056/0 (socket says 172.21.15.120:41056) 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 -- 172.21.15.120:0/1964491533 learned_addr learned my addr 172.21.15.120:0/1964491533 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0a712640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 0x559c0eaf9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 -- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 msgr2=0x559c0ea8ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 -- 172.21.15.120:0/1964491533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 msgr2=0x559c0eaf9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 0x559c0eaf9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 -- 172.21.15.120:0/1964491533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0eafa060 con 0x559c0ea79860 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09710640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b0a712640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 0x559c0eaf9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4b09f11640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e32f0 tx=0x7f4b00010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a2ac71ad4598ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.666+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00020070 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4b000a8200 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b000abaa0 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b00009de0 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 10 client.19638 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c0eafb370 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b000abaa0 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4b000e8ce0 con 0x559c0ea79860 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 10 client.19638 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 10 client.19638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 20 client.19638 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0eafb820 0x559c0eafdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 4 client.19638 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c0e9e90f0 con 0x559c0eafb820 2024-09-10T23:14:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.667+0000 7f4b09710640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0eafb820 0x559c0eafdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.668+0000 7f4aed7fa640 10 client.19638 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:06.668+0000 7f4b09710640 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0eafb820 0x559c0eafdc10 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c7b4ae3f31f875 server_cookie=8e5438d3bd6f052c in_seq=0 out_seq=0 2024-09-10T23:14:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:06 smithi120 ceph-mon[31247]: pgmap v996: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-09-10T23:14:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:06 smithi153 ceph-mon[34951]: pgmap v996: 97 pgs: 97 active+clean; 247 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-09-10T23:14:06.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 219924 21.65 MB/sec execute 421 sec latency 168.104 ms 2024-09-10T23:14:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.509+0000 7f4aed7fa640 1 -- 172.21.15.120:0/1964491533 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c0e9e90f0 con 0x559c0eafb820 2024-09-10T23:14:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4aed7fa640 10 client.19638 handle_command_reply: tid=0 2024-09-10T23:14:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 client.19638 shutdown 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 msgr2=0x559c0ea8a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4b000e32f0 tx=0x7f4b00010cb0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 shutdown_connections 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x559c0eafb820 0x559c0eafdc10 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4aed7fa640 0 client.19638 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559c0ea7ebd0 0x559c0eaf9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559c0ea7c190 0x559c0ea8ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 --2- 172.21.15.120:0/1964491533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559c0ea79860 0x559c0ea8a4a0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 >> 172.21.15.120:0/1964491533 conn(0x559c0ea749b0 msgr2=0x559c0ea78d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.510+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 shutdown_connections 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.511+0000 7f4b0f022740 1 -- 172.21.15.120:0/1964491533 wait complete. 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.511+0000 7f4b0f022740 20 client.19638 trim_cache size 0 max 16384 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:14:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 msgr2=0x7f4af00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 0x7f4af00b7820 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005c40 tx=0x7f4af40314a0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 msgr2=0x7f4b04105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000d950 tx=0x7f4b0000de10 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 shutdown_connections 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4af00b5360 0x7f4af00b7820 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b04103370 0x7f4b041043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b041029b0 0x7f4b04103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 --2- 172.21.15.120:0/2889695284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4b04101860 0x7f4b04105620 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 >> 172.21.15.120:0/2889695284 conn(0x7f4b040fd140 msgr2=0x7f4b040febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 shutdown_connections 2024-09-10T23:14:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:07.512+0000 7f4b0c175640 1 -- 172.21.15.120:0/2889695284 wait complete. 2024-09-10T23:14:07.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 220114 21.64 MB/sec execute 422 sec latency 426.655 ms 2024-09-10T23:14:08.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:08 smithi176 ceph-mon[34982]: pgmap v997: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-09-10T23:14:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:08 smithi120 ceph-mon[31247]: pgmap v997: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-09-10T23:14:08.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:08 smithi153 ceph-mon[34951]: pgmap v997: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-09-10T23:14:08.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 220476 21.63 MB/sec execute 423 sec latency 66.149 ms 2024-09-10T23:14:09.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 220839 21.62 MB/sec execute 424 sec latency 73.878 ms 2024-09-10T23:14:10.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:10 smithi176 ceph-mon[34982]: pgmap v998: 97 pgs: 97 active+clean; 254 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:14:10.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:10 smithi120 ceph-mon[31247]: pgmap v998: 97 pgs: 97 active+clean; 254 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:14:10.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:10 smithi153 ceph-mon[34951]: pgmap v998: 97 pgs: 97 active+clean; 254 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 404 op/s 2024-09-10T23:14:10.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 221128 21.60 MB/sec execute 425 sec latency 154.788 ms 2024-09-10T23:14:11.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 221617 21.61 MB/sec execute 426 sec latency 101.046 ms 2024-09-10T23:14:12.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:12 smithi176 ceph-mon[34982]: pgmap v999: 97 pgs: 97 active+clean; 253 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 400 op/s 2024-09-10T23:14:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:12 smithi120 ceph-mon[31247]: pgmap v999: 97 pgs: 97 active+clean; 253 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 400 op/s 2024-09-10T23:14:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:12 smithi153 ceph-mon[34951]: pgmap v999: 97 pgs: 97 active+clean; 253 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 400 op/s 2024-09-10T23:14:12.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 222097 21.61 MB/sec execute 427 sec latency 171.764 ms 2024-09-10T23:14:13.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 222644 21.63 MB/sec execute 428 sec latency 170.889 ms 2024-09-10T23:14:14.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:14 smithi176 ceph-mon[34982]: pgmap v1000: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-10T23:14:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:14 smithi120 ceph-mon[31247]: pgmap v1000: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-10T23:14:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:14 smithi153 ceph-mon[34951]: pgmap v1000: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 642 op/s 2024-09-10T23:14:14.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 223157 21.65 MB/sec execute 429 sec latency 68.172 ms 2024-09-10T23:14:15.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 223431 21.63 MB/sec execute 430 sec latency 244.152 ms 2024-09-10T23:14:16.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:16 smithi176 ceph-mon[34982]: pgmap v1001: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:14:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:16 smithi120 ceph-mon[31247]: pgmap v1001: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:14:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:16 smithi153 ceph-mon[34951]: pgmap v1001: 97 pgs: 97 active+clean; 242 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-09-10T23:14:16.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 223819 21.65 MB/sec execute 431 sec latency 304.421 ms 2024-09-10T23:14:17.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 224147 21.64 MB/sec execute 432 sec latency 111.190 ms 2024-09-10T23:14:18.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:18 smithi176 ceph-mon[34982]: pgmap v1002: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 604 op/s 2024-09-10T23:14:18.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:18 smithi120 ceph-mon[31247]: pgmap v1002: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 604 op/s 2024-09-10T23:14:18.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:18 smithi153 ceph-mon[34951]: pgmap v1002: 97 pgs: 97 active+clean; 246 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 604 op/s 2024-09-10T23:14:18.889 INFO:tasks.workunit.client.0.smithi120.stdout: 10 224529 21.62 MB/sec execute 433 sec latency 77.620 ms 2024-09-10T23:14:19.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 224943 21.62 MB/sec execute 434 sec latency 80.693 ms 2024-09-10T23:14:20.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:20 smithi176 ceph-mon[34982]: pgmap v1003: 97 pgs: 97 active+clean; 247 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 580 op/s 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 -- 172.21.15.120:0/2875149798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac103ae0 msgr2=0x7fd0ac103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/2875149798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac103f60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009930 tx=0x7fd0a002a420 comp rx=0 tx=0).stop 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 -- 172.21.15.120:0/2875149798 shutdown_connections 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/2875149798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/2875149798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac103f60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/2875149798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102990 0x7fd0ac102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 -- 172.21.15.120:0/2875149798 >> 172.21.15.120:0/2875149798 conn(0x7fd0ac0fe250 msgr2=0x7fd0ac100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 -- 172.21.15.120:0/2875149798 shutdown_connections 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 -- 172.21.15.120:0/2875149798 wait complete. 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.605+0000 7fd0b1a57640 1 Processor -- start 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 -- start start 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac10e5e0 con 0x7fd0ac102990 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac10e460 con 0x7fd0ac1044a0 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b1a57640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac10e760 con 0x7fd0ac103ae0 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41844/0 (socket says 172.21.15.120:41844) 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 -- 172.21.15.120:0/4124717979 learned_addr learned my addr 172.21.15.120:0/4124717979 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 -- 172.21.15.120:0/4124717979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac103ae0 msgr2=0x7fd0ac1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 -- 172.21.15.120:0/4124717979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 msgr2=0x7fd0ac19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 -- 172.21.15.120:0/4124717979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac19a8e0 con 0x7fd0ac102990 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea60 tx=0x7fd09c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.606+0000 7fd0b0a55640 1 -- 172.21.15.120:0/4124717979 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00cde0 con 0x7fd0ac102990 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.607+0000 7fd0b0a55640 1 -- 172.21.15.120:0/4124717979 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09c0035f0 con 0x7fd0ac102990 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.607+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ac19abd0 con 0x7fd0ac102990 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.607+0000 7fd0b0a55640 1 -- 172.21.15.120:0/4124717979 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c010690 con 0x7fd0ac102990 2024-09-10T23:14:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.607+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ac1a12a0 con 0x7fd0ac102990 2024-09-10T23:14:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 Processor -- start 2024-09-10T23:14:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 -- start start 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b0a55640 1 -- 172.21.15.120:0/4124717979 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd09c004000 con 0x7fd0ac102990 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 0x55d063b77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d063ae76f0 con 0x55d063b7abd0 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d063ad6b20 con 0x55d063b75860 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.609+0000 7fd0b4826740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d063ada930 con 0x55d063b78190 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41856/0 (socket says 172.21.15.120:41856) 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/408193811 learned_addr learned my addr 172.21.15.120:0/408193811 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 0x55d063b77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/408193811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 msgr2=0x55d063b7a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/408193811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 msgr2=0x55d063b77c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 0x55d063b77c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/408193811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d063a9c530 con 0x55d063b7abd0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 0x55d063b77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002a940 tx=0x7fd0a0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beef17878a42d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0b0a55640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 0x7fd0940b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd091ffb640 1 -- 172.21.15.120:0/408193811 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035030 con 0x55d063b7abd0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 0x7fd0940b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd091ffb640 1 -- 172.21.15.120:0/408193811 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a0040070 con 0x55d063b7abd0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd091ffb640 1 -- 172.21.15.120:0/408193811 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00038e0 con 0x55d063b7abd0 2024-09-10T23:14:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.610+0000 7fd0b0a55640 1 -- 172.21.15.120:0/4124717979 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd09c0e5de0 con 0x7fd0ac102990 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- 172.21.15.120:0/408193811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 msgr2=0x55d063b7d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 0x7fd0940b7820 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac199b20 tx=0x7fd0a4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002a940 tx=0x7fd0a0005de0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- 172.21.15.120:0/408193811 shutdown_connections 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b7abd0 0x55d063b7d090 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 --2- 172.21.15.120:0/408193811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b75860 0x55d063b77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- 172.21.15.120:0/408193811 >> 172.21.15.120:0/408193811 conn(0x55d063b709b0 msgr2=0x55d063b71740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- 172.21.15.120:0/408193811 shutdown_connections 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- 172.21.15.120:0/408193811 wait complete. 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.611+0000 7fd0b4826740 1 -- start start 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 Processor -- start 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d063a0eab0 con 0x55d063b75860 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55d063a7c730 con 0x55d063b7abd0 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d063b7db50 con 0x55d063b78190 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0b4826740 10 client.0 _ll_register_callbacks cb 0x55d063ad06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41866/0 (socket says 172.21.15.120:41866) 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3269802158 learned_addr learned my addr 172.21.15.120:0/3269802158 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 msgr2=0x55d063b7f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3269802158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 msgr2=0x55d063b800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.612+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3269802158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d063838790 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0a8ba0 tx=0x7fd09c003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5007a3f10b1b0f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ac3a0 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd09c0ab830 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a0040070 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd09c0abd40 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 10 client.19650 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0638b1620 con 0x55d063b75860 2024-09-10T23:14:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a0040070 con 0x55d063b75860 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd09c0a7f20 con 0x55d063b75860 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 10 client.19650 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 10 client.19650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 20 client.19650 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063b8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 4 client.19650 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d063ae9640 con 0x55d063b8a780 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.613+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063b8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.614+0000 7fd08affd640 10 client.19650 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.614+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063b8cb70 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f753003e7721dd65 server_cookie=68d10979c4479d61 in_seq=0 out_seq=0 2024-09-10T23:14:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.618+0000 7fd08affd640 1 -- 172.21.15.120:0/3269802158 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d063ae9640 con 0x55d063b8a780 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.618+0000 7fd08affd640 10 client.19650 handle_command_reply: tid=0 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 client.19650 shutdown 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 msgr2=0x55d063b7ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0a8ba0 tx=0x7fd09c003cf0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 shutdown_connections 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063b8cb70 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd08affd640 0 client.19650 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3269802158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7ee70 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 >> 172.21.15.120:0/3269802158 conn(0x55d063b709b0 msgr2=0x55d063b6fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 shutdown_connections 2024-09-10T23:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 1 -- 172.21.15.120:0/3269802158 wait complete. 2024-09-10T23:14:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.619+0000 7fd0b4826740 20 client.19650 trim_cache size 0 max 16384 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.633+0000 7fd0b4826740 1 Processor -- start 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.633+0000 7fd0b4826740 1 -- start start 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.633+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.633+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b7b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0b4826740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1a7e60 con 0x55d063b75860 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0b4826740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1aa6e0 con 0x55d063b7abd0 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0b4826740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1a84d0 con 0x55d063b78190 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41870/0 (socket says 172.21.15.120:41870) 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3288077580 learned_addr learned my addr 172.21.15.120:0/3288077580 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b7b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3288077580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 msgr2=0x55d063b7afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3288077580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 msgr2=0x55d063b7b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b7b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 -- 172.21.15.120:0/3288077580 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d063b8cee0 con 0x55d063b75860 2024-09-10T23:14:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ac2d0 tx=0x7fd09c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ccf8d4c6980548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0917fa640 1 -- 172.21.15.120:0/3288077580 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0abf20 con 0x55d063b75860 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0917fa640 1 -- 172.21.15.120:0/3288077580 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09c0b10b0 con 0x55d063b75860 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.634+0000 7fd0917fa640 1 -- 172.21.15.120:0/3288077580 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ac5d0 con 0x55d063b75860 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- 172.21.15.120:0/3288077580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 msgr2=0x55d063b7f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ac2d0 tx=0x7fd09c00ef90 comp rx=0 tx=0).stop 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- 172.21.15.120:0/3288077580 shutdown_connections 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063b7b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b78190 0x55d063b7afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 --2- 172.21.15.120:0/3288077580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b75860 0x55d063b7f510 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- 172.21.15.120:0/3288077580 >> 172.21.15.120:0/3288077580 conn(0x55d063b709b0 msgr2=0x55d063b7ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- 172.21.15.120:0/3288077580 shutdown_connections 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- 172.21.15.120:0/3288077580 wait complete. 2024-09-10T23:14:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 -- start start 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.635+0000 7fd0b4826740 1 Processor -- start 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b75860 0x55d063bf5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063bfa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1a8ba0 con 0x55d063b78190 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1aa010 con 0x55d063b7abd0 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac1a9270 con 0x55d063b75860 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 10 client.0 _ll_register_callbacks cb 0x7fd09c0e95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41874/0 (socket says 172.21.15.120:41874) 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/1507495461 learned_addr learned my addr 172.21.15.120:0/1507495461 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b75860 0x55d063bf5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0ab7fe640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063bfa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b75860 msgr2=0x55d063bf5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b75860 0x55d063bf5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/1507495461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 msgr2=0x55d063bfa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063bfa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 -- 172.21.15.120:0/1507495461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d063b80450 con 0x55d063b78190 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0aa7fc640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009850 tx=0x7fd0a0031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c16f96fc04a83f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd08affd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035040 con 0x55d063b78190 2024-09-10T23:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd0a0004030 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.636+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c0b10b0 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0a005f070 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 10 client.19656 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d063bfb950 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09c0b10b0 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd0a0058480 con 0x55d063b78190 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 10 client.19656 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 10 client.19656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 20 client.19656 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063bfdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 4 client.19656 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:14:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d063ae50f0 con 0x55d063b8a780 2024-09-10T23:14:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063bfdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd08affd640 10 client.19656 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:20.637+0000 7fd0aaffd640 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063bfdf20 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163c0157a2088240 server_cookie=2267899eed5a89e7 in_seq=0 out_seq=0 2024-09-10T23:14:20.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:20 smithi153 ceph-mon[34951]: pgmap v1003: 97 pgs: 97 active+clean; 247 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 580 op/s 2024-09-10T23:14:20.701 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:20 smithi120 ceph-mon[31247]: pgmap v1003: 97 pgs: 97 active+clean; 247 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 580 op/s 2024-09-10T23:14:20.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 225356 21.61 MB/sec execute 435 sec latency 222.146 ms 2024-09-10T23:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd08affd640 1 -- 172.21.15.120:0/1507495461 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d063ae50f0 con 0x55d063b8a780 2024-09-10T23:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd08affd640 10 client.19656 handle_command_reply: tid=0 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 client.19656 shutdown 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 msgr2=0x55d063bf7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009850 tx=0x7fd0a0031ba0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 shutdown_connections 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55d063b8a780 0x55d063bfdf20 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d063b7abd0 0x55d063bfa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d063b78190 0x55d063bf7b70 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 --2- 172.21.15.120:0/1507495461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d063b75860 0x55d063bf5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd08affd640 0 client.19656 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 >> 172.21.15.120:0/1507495461 conn(0x55d063b709b0 msgr2=0x55d063b7afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 shutdown_connections 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 1 -- 172.21.15.120:0/1507495461 wait complete. 2024-09-10T23:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.554+0000 7fd0b4826740 20 client.19656 trim_cache size 0 max 16384 2024-09-10T23:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 msgr2=0x7fd0940b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 0x7fd0940b7820 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac199b20 tx=0x7fd0a4009500 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 msgr2=0x7fd0ac198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea60 tx=0x7fd09c00ef20 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 shutdown_connections 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0940b5360 0x7fd0940b7820 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac1044a0 0x7fd0ac19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0ac103ae0 0x7fd0ac1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 --2- 172.21.15.120:0/4124717979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0ac102990 0x7fd0ac198da0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 >> 172.21.15.120:0/4124717979 conn(0x7fd0ac0fe250 msgr2=0x7fd0ac106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 shutdown_connections 2024-09-10T23:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:21.555+0000 7fd0b1a57640 1 -- 172.21.15.120:0/4124717979 wait complete. 2024-09-10T23:14:21.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 225565 21.60 MB/sec execute 436 sec latency 712.683 ms 2024-09-10T23:14:22.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:22 smithi176 ceph-mon[34982]: pgmap v1004: 97 pgs: 97 active+clean; 253 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 562 op/s 2024-09-10T23:14:22.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:22 smithi120 ceph-mon[31247]: pgmap v1004: 97 pgs: 97 active+clean; 253 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 562 op/s 2024-09-10T23:14:22.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:22 smithi153 ceph-mon[34951]: pgmap v1004: 97 pgs: 97 active+clean; 253 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 562 op/s 2024-09-10T23:14:22.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 226063 21.59 MB/sec execute 437 sec latency 70.431 ms 2024-09-10T23:14:23.721 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:14:23.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 226650 21.63 MB/sec execute 438 sec latency 38.110 ms 2024-09-10T23:14:24.021 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:24.393 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:24 smithi120 ceph-mon[31247]: pgmap v1005: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-09-10T23:14:24.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:24 smithi176 ceph-mon[34982]: pgmap v1005: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-09-10T23:14:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- 172.21.15.120:0/796867308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c10d130 msgr2=0x7f3d7c10f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 --2- 172.21.15.120:0/796867308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c10d130 0x7f3d7c10f590 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009900 tx=0x7f3d7402a410 comp rx=0 tx=0).stop 2024-09-10T23:14:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- 172.21.15.120:0/796867308 shutdown_connections 2024-09-10T23:14:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 --2- 172.21.15.120:0/796867308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c10d130 0x7f3d7c10f590 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 --2- 172.21.15.120:0/796867308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 --2- 172.21.15.120:0/796867308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c103cd0 0x7f3d7c1040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- 172.21.15.120:0/796867308 >> 172.21.15.120:0/796867308 conn(0x7f3d7c0fd770 msgr2=0x7f3d7c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- 172.21.15.120:0/796867308 shutdown_connections 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- 172.21.15.120:0/796867308 wait complete. 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 Processor -- start 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.631+0000 7f3d837f5640 1 -- start start 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c103cd0 0x7f3d7c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c10d130 0x7f3d7c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c112da0 con 0x7f3d7c1045f0 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c112c20 con 0x7f3d7c10d130 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d837f5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c112f20 con 0x7f3d7c103cd0 2024-09-10T23:14:24.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d81591640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c103cd0 0x7f3d7c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57576/0 (socket says 172.21.15.120:57576) 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 -- 172.21.15.120:0/3214392473 learned_addr learned my addr 172.21.15.120:0/3214392473 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d81d92640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c10d130 0x7f3d7c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 -- 172.21.15.120:0/3214392473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c103cd0 msgr2=0x7f3d7c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c103cd0 0x7f3d7c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 -- 172.21.15.120:0/3214392473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c10d130 msgr2=0x7f3d7c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c10d130 0x7f3d7c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 -- 172.21.15.120:0/3214392473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c19f360 con 0x7f3d7c1045f0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d80d90640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e9c0 tx=0x7f3d6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.632+0000 7f3d727fc640 1 -- 172.21.15.120:0/3214392473 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00ccc0 con 0x7f3d7c1045f0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.633+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c19f650 con 0x7f3d7c1045f0 2024-09-10T23:14:24.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.633+0000 7f3d727fc640 1 -- 172.21.15.120:0/3214392473 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c0035f0 con 0x7f3d7c1045f0 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.633+0000 7f3d727fc640 1 -- 172.21.15.120:0/3214392473 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00ce60 con 0x7f3d7c1045f0 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.633+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c1a5d10 con 0x7f3d7c1045f0 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.635+0000 7f3d727fc640 1 -- 172.21.15.120:0/3214392473 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3d6c004000 con 0x7f3d7c1045f0 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.635+0000 7f3d85bca740 1 Processor -- start 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.635+0000 7f3d85bca740 1 -- start start 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c4c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556759bb3e50 con 0x556759c3dda0 2024-09-10T23:14:24.647 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556759ba3280 con 0x556759c3d480 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d85bca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556759ba7090 con 0x556759c49d30 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81591640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81591640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57594/0 (socket says 172.21.15.120:57594) 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81591640 1 -- 172.21.15.120:0/4235338318 learned_addr learned my addr 172.21.15.120:0/4235338318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d80d90640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c4c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 -- 172.21.15.120:0/4235338318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 msgr2=0x556759c4c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c4c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 -- 172.21.15.120:0/4235338318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 msgr2=0x556759c497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.648 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 -- 172.21.15.120:0/4235338318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556759901650 con 0x556759c3d480 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d80d90640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81591640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d727fc640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 0x7f3d680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81591640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 0x7f3d680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.636+0000 7f3d81d92640 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3d860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d74005940 tx=0x7f3d74032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de45c5c7f476f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d727fc640 1 -- 172.21.15.120:0/3214392473 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d6c0e5e70 con 0x7f3d7c1045f0 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d677fe640 1 -- 172.21.15.120:0/4235338318 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74040070 con 0x556759c3d480 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d677fe640 1 -- 172.21.15.120:0/4235338318 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74004030 con 0x556759c3d480 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d677fe640 1 -- 172.21.15.120:0/4235338318 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74035040 con 0x556759c3d480 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d85bca740 1 -- 172.21.15.120:0/4235338318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 msgr2=0x556759c3d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d81591640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 0x7f3d680b77a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x556759c3c190 tx=0x7f3d78009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.637+0000 7f3d85bca740 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3d860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d74005940 tx=0x7f3d74032b20 comp rx=0 tx=0).stop 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- 172.21.15.120:0/4235338318 shutdown_connections 2024-09-10T23:14:24.649 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c4c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- 172.21.15.120:0/4235338318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3d860 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- 172.21.15.120:0/4235338318 >> 172.21.15.120:0/4235338318 conn(0x556759c3b2b0 msgr2=0x556759c3c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- 172.21.15.120:0/4235338318 shutdown_connections 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- 172.21.15.120:0/4235338318 wait complete. 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- start start 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 Processor -- start 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 0x556759c46a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556759a04700 con 0x556759c49d30 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556759b3e110 con 0x556759c3d480 2024-09-10T23:14:24.650 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556759c4c950 con 0x556759c3dda0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.638+0000 7f3d85bca740 10 client.0 _ll_register_callbacks cb 0x556759b57990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 0x556759c46a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57596/0 (socket says 172.21.15.120:57596) 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 -- 172.21.15.120:0/2384875365 learned_addr learned my addr 172.21.15.120:0/2384875365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 0x556759c46a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37248/0 (socket says 172.21.15.120:37248) 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81591640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 -- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 msgr2=0x556759c46a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 0x556759c46a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 -- 172.21.15.120:0/2384875365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 msgr2=0x556759c49620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c49620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 -- 172.21.15.120:0/2384875365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556759b6bc40 con 0x556759c49d30 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d81d92640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009a20 tx=0x7f3d740041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d69e33c300ccaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.651 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7404c070 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d74004030 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d740055f0 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d74058050 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d85bca740 10 client.19668 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.639+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556759c58280 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74004030 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3d74004c20 con 0x556759c49d30 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 10 client.19668 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:24.652 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 10 client.19668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 20 client.19668 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759c595d0 0x556759c5b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 4 client.19668 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556759bb5da0 con 0x556759c595d0 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d81591640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759c595d0 0x556759c5b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d81591640 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759c595d0 0x556759c5b9c0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ea5d3e037bcb3 server_cookie=853e57e36b00e4d5 in_seq=0 out_seq=0 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.640+0000 7f3d64ff9640 10 client.19668 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d64ff9640 1 -- 172.21.15.120:0/2384875365 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556759bb5da0 con 0x556759c595d0 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d64ff9640 10 client.19668 handle_command_reply: tid=0 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 client.19668 shutdown 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 msgr2=0x556759c470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009a20 tx=0x7f3d740041e0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 shutdown_connections 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759c595d0 0x556759c5b9c0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759c470a0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3dda0 0x556759c46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2384875365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c49620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 >> 172.21.15.120:0/2384875365 conn(0x556759c3b2b0 msgr2=0x556759c40e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d64ff9640 0 client.19668 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 shutdown_connections 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 1 -- 172.21.15.120:0/2384875365 wait complete. 2024-09-10T23:14:24.654 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.642+0000 7f3d85bca740 20 client.19668 trim_cache size 0 max 16384 2024-09-10T23:14:24.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 Processor -- start 2024-09-10T23:14:24.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 -- start start 2024-09-10T23:14:24.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.664 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ac300 con 0x556759c3dda0 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1aeb80 con 0x556759c3d480 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.662+0000 7f3d85bca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ac970 con 0x556759c49d30 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57606/0 (socket says 172.21.15.120:57606) 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 -- 172.21.15.120:0/2741445873 learned_addr learned my addr 172.21.15.120:0/2741445873 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d81d92640 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d81591640 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 -- 172.21.15.120:0/2741445873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 msgr2=0x556759c441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 -- 172.21.15.120:0/2741445873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 msgr2=0x556759c3abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 -- 172.21.15.120:0/2741445873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556759c49990 con 0x556759c3dda0 2024-09-10T23:14:24.665 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d80d90640 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00c6a0 tx=0x7f3d6c0a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9809a7866386a7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d66ffd640 1 -- 172.21.15.120:0/2741445873 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0e3e60 con 0x556759c3dda0 2024-09-10T23:14:24.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d66ffd640 1 -- 172.21.15.120:0/2741445873 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c0ac170 con 0x556759c3dda0 2024-09-10T23:14:24.666 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.663+0000 7f3d66ffd640 1 -- 172.21.15.120:0/2741445873 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b1140 con 0x556759c3dda0 2024-09-10T23:14:24.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 -- 172.21.15.120:0/2741445873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 msgr2=0x556759c43c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.668 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00c6a0 tx=0x7f3d6c0a8d70 comp rx=0 tx=0).stop 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 -- 172.21.15.120:0/2741445873 shutdown_connections 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c49d30 0x556759c441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c3dda0 0x556759c43c80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 --2- 172.21.15.120:0/2741445873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3d480 0x556759c3abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 -- 172.21.15.120:0/2741445873 >> 172.21.15.120:0/2741445873 conn(0x556759c3b2b0 msgr2=0x556759c480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 -- 172.21.15.120:0/2741445873 shutdown_connections 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.664+0000 7f3d85bca740 1 -- 172.21.15.120:0/2741445873 wait complete. 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 -- start start 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 Processor -- start 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3d480 0x556759c5cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 0x556759c5d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ad040 con 0x556759c49d30 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ae4b0 con 0x556759c3dda0 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ad710 con 0x556759c3d480 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d85bca740 10 client.0 _ll_register_callbacks cb 0x7f3d78009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d81d92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d81d92640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57614/0 (socket says 172.21.15.120:57614) 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d81d92640 1 -- 172.21.15.120:0/1893206929 learned_addr learned my addr 172.21.15.120:0/1893206929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d80d90640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 0x556759c5d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.665+0000 7f3d81591640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3d480 0x556759c5cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 -- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3d480 msgr2=0x556759c5cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3d480 0x556759c5cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 -- 172.21.15.120:0/1893206929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 msgr2=0x556759c5d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 0x556759c5d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 -- 172.21.15.120:0/1893206929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556759cc57b0 con 0x556759c49d30 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d80d90640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 0x556759c5d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:24.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d81d92640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099f0 tx=0x7f3d74003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87763e532a14d37c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7404c040 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d6c0ac170 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3d74003e00 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d74040050 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d85bca740 10 client.19674 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556759cc6aa0 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.666+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c0ac170 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3d740046c0 con 0x556759c49d30 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 10 client.19674 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 10 client.19674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 20 client.19674 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759cc7210 0x556759cc9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:24.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 4 client.19674 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:14:24.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556759bb1850 con 0x556759cc7210 2024-09-10T23:14:24.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d81591640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759cc7210 0x556759cc9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:24.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d64ff9640 10 client.19674 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:24.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.667+0000 7f3d81591640 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759cc7210 0x556759cc9630 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5db4e066a6f235 server_cookie=3f5ad55dcb718b75 in_seq=0 out_seq=0 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d64ff9640 1 -- 172.21.15.120:0/1893206929 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10314 (crc 0 0 0) 0x556759bb1850 con 0x556759cc7210 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d64ff9640 10 client.19674 handle_command_reply: tid=0 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 client.19674 shutdown 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 msgr2=0x556759cc5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099f0 tx=0x7f3d74003970 comp rx=0 tx=0).stop 2024-09-10T23:14:24.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 shutdown_connections 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556759cc7210 0x556759cc9630 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556759c49d30 0x556759cc5140 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556759c3dda0 0x556759c5d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 --2- 172.21.15.120:0/1893206929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556759c3d480 0x556759c5cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 >> 172.21.15.120:0/1893206929 conn(0x556759c3b2b0 msgr2=0x556759c452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d64ff9640 0 client.19674 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 shutdown_connections 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 1 -- 172.21.15.120:0/1893206929 wait complete. 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.671+0000 7f3d85bca740 20 client.19674 trim_cache size 0 max 16384 2024-09-10T23:14:24.674 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:14:24.675 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.676 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.677 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.678 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:14:24.679 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.680 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.681 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:14:24.683 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.685 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:14:24.686 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:14:24.687 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.688 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.689 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.690 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.691 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:14:24.693 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.695 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:14:24.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 msgr2=0x7f3d680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 0x7f3d680b77a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x556759c3c190 tx=0x7f3d78009500 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 msgr2=0x7f3d7c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e9c0 tx=0x7f3d6c00ee80 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 shutdown_connections 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3d680b52e0 0x7f3d680b77a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d7c10d130 0x7f3d7c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d7c1045f0 0x7f3d7c1997f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 --2- 172.21.15.120:0/3214392473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d7c103cd0 0x7f3d7c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 >> 172.21.15.120:0/3214392473 conn(0x7f3d7c0fd770 msgr2=0x7f3d7c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 shutdown_connections 2024-09-10T23:14:24.697 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:24.672+0000 7f3d837f5640 1 -- 172.21.15.120:0/3214392473 wait complete. 2024-09-10T23:14:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:24 smithi153 ceph-mon[34951]: pgmap v1005: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-09-10T23:14:24.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 227090 21.65 MB/sec execute 439 sec latency 52.499 ms 2024-09-10T23:14:25.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:14:25.591 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:14:25.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 227457 21.66 MB/sec execute 440 sec latency 90.912 ms 2024-09-10T23:14:25.933 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:26.323 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:26 smithi120 ceph-mon[31247]: pgmap v1006: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-09-10T23:14:26.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:26 smithi176 ceph-mon[34982]: pgmap v1006: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-09-10T23:14:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- 172.21.15.120:0/348679563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd81087f0 msgr2=0x7fcdd8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 --2- 172.21.15.120:0/348679563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd81087f0 0x7fcdd8108c30 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009900 tx=0x7fcdc402a410 comp rx=0 tx=0).stop 2024-09-10T23:14:26.598 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- 172.21.15.120:0/348679563 shutdown_connections 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 --2- 172.21.15.120:0/348679563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd8109170 0x7fcdd810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 --2- 172.21.15.120:0/348679563 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd81087f0 0x7fcdd8108c30 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 --2- 172.21.15.120:0/348679563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8102de0 0x7fcdd81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- 172.21.15.120:0/348679563 >> 172.21.15.120:0/348679563 conn(0x7fcdd80fe830 msgr2=0x7fcdd8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- 172.21.15.120:0/348679563 shutdown_connections 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- 172.21.15.120:0/348679563 wait complete. 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 Processor -- start 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.406+0000 7fcddf05d640 1 -- start start 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd81087f0 0x7fcdd8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8109170 0x7fcdd819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112e30 con 0x7fcdd8102de0 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112cb0 con 0x7fcdd81087f0 2024-09-10T23:14:26.599 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112fb0 con 0x7fcdd8109170 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57634/0 (socket says 172.21.15.120:57634) 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 -- 172.21.15.120:0/2168542860 learned_addr learned my addr 172.21.15.120:0/2168542860 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcdd7fff640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd81087f0 0x7fcdd8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddd5fa640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8109170 0x7fcdd819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 -- 172.21.15.120:0/2168542860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8109170 msgr2=0x7fcdd819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8109170 0x7fcdd819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 -- 172.21.15.120:0/2168542860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd81087f0 msgr2=0x7fcdd8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd81087f0 0x7fcdd8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 -- 172.21.15.120:0/2168542860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd819f390 con 0x7fcdd8102de0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddcdf9640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00cd80 con 0x7fcdd8102de0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc0054d0 con 0x7fcdd8102de0 2024-09-10T23:14:26.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd819f680 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.407+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc010690 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.408+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd81a5d40 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.409+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd81031c0 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.410+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcdcc004e90 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.410+0000 7fcdd5ffb640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 0x7fcdc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.411+0000 7fcdd7fff640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 0x7fcdc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.411+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcdcc0e5fb0 con 0x7fcdd8102de0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.411+0000 7fcdd7fff640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 0x7fcdc80b77a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031440 tx=0x7fcdc4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:26.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.415+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcdcc0acea0 con 0x7fcdd8102de0 2024-09-10T23:14:26.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.640+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdd802e5c0 con 0x7fcdd8102de0 2024-09-10T23:14:26.642 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.641+0000 7fcdd5ffb640 1 -- 172.21.15.120:0/2168542860 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fcdcc0ac7a0 con 0x7fcdd8102de0 2024-09-10T23:14:26.643 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:14:26.643 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:14:26.643 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:14:26.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 msgr2=0x7fcdc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:26.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 0x7fcdc80b77a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031440 tx=0x7fcdc4005b50 comp rx=0 tx=0).stop 2024-09-10T23:14:26.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 msgr2=0x7fcdd81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:26.645 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00edf0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 shutdown_connections 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcdc80b52e0 0x7fcdc80b77a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcdd8109170 0x7fcdd819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdd81087f0 0x7fcdd8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 --2- 172.21.15.120:0/2168542860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdd8102de0 0x7fcdd81992e0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 >> 172.21.15.120:0/2168542860 conn(0x7fcdd80fe830 msgr2=0x7fcdd8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 shutdown_connections 2024-09-10T23:14:26.646 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:26.644+0000 7fcddf05d640 1 -- 172.21.15.120:0/2168542860 wait complete. 2024-09-10T23:14:26.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:26 smithi153 ceph-mon[34951]: pgmap v1006: 97 pgs: 97 active+clean; 258 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 375 op/s 2024-09-10T23:14:26.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 227753 21.64 MB/sec execute 441 sec latency 202.536 ms 2024-09-10T23:14:27.334 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:14:27.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:27 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2168542860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:27.606 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:27.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2168542860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:27.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:27 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2168542860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:14:27.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 228050 21.61 MB/sec execute 442 sec latency 151.858 ms 2024-09-10T23:14:28.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 -- 172.21.15.120:0/3030439250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a001076a0 msgr2=0x7f9a00107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 --2- 172.21.15.120:0/3030439250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a001076a0 0x7f9a00107ae0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009900 tx=0x7f99f002a410 comp rx=0 tx=0).stop 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 -- 172.21.15.120:0/3030439250 shutdown_connections 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 --2- 172.21.15.120:0/3030439250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00108020 0x7f9a0010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 --2- 172.21.15.120:0/3030439250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a001076a0 0x7f9a00107ae0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 --2- 172.21.15.120:0/3030439250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a00101c90 0x7f9a00102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 -- 172.21.15.120:0/3030439250 >> 172.21.15.120:0/3030439250 conn(0x7f9a000fd700 msgr2=0x7f9a000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 -- 172.21.15.120:0/3030439250 shutdown_connections 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.124+0000 7f9a07371640 1 -- 172.21.15.120:0/3030439250 wait complete. 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 Processor -- start 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 -- start start 2024-09-10T23:14:28.153 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101c90 0x7f9a00199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a001076a0 0x7f9a00199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00111ce0 con 0x7f9a00108020 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00111b60 con 0x7f9a00101c90 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a07371640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00111e60 con 0x7f9a001076a0 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57652/0 (socket says 172.21.15.120:57652) 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 -- 172.21.15.120:0/714406876 learned_addr learned my addr 172.21.15.120:0/714406876 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0490c640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a001076a0 0x7f9a00199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0510d640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101c90 0x7f9a00199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 -- 172.21.15.120:0/714406876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a001076a0 msgr2=0x7f9a00199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a001076a0 0x7f9a00199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 -- 172.21.15.120:0/714406876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101c90 msgr2=0x7f9a00199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101c90 0x7f9a00199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.125+0000 7f9a0590e640 1 -- 172.21.15.120:0/714406876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0019f3d0 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f9a0590e640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009fd0 tx=0x7f99f800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f99f67fc640 1 -- 172.21.15.120:0/714406876 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8017070 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0019f6c0 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f99f67fc640 1 -- 172.21.15.120:0/714406876 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f8003e20 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f99f67fc640 1 -- 172.21.15.120:0/714406876 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010460 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.126+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a5d80 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.128+0000 7f9a09746740 1 Processor -- start 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f99f67fc640 1 -- 172.21.15.120:0/714406876 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f99f8003970 con 0x7f9a00108020 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 -- start start 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cd48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3c4aea0 con 0x55c7f3cd44d0 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3c3a2d0 con 0x55c7f3cd4df0 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a09746740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3c3e0e0 con 0x55c7f3ce5190 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0490c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cd48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0490c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cd48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57670/0 (socket says 172.21.15.120:57670) 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0490c640 1 -- 172.21.15.120:0/1454613957 learned_addr learned my addr 172.21.15.120:0/1454613957 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0490c640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 -- 172.21.15.120:0/1454613957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 msgr2=0x55c7f3ce7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 -- 172.21.15.120:0/1454613957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 msgr2=0x55c7f3cd48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cd48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.156 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0510d640 1 -- 172.21.15.120:0/1454613957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f3998650 con 0x55c7f3cd4df0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.129+0000 7f9a0490c640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f99f67fc640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 0x7f99ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f9a0510d640 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce4c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005fd0 tx=0x7f99fc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a9ec26e0f84c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f9a0490c640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 0x7f99ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f99eb7fe640 1 -- 172.21.15.120:0/1454613957 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01c480 con 0x55c7f3cd4df0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f99f67fc640 1 -- 172.21.15.120:0/714406876 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f99f80e5880 con 0x7f9a00108020 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f99eb7fe640 1 -- 172.21.15.120:0/1454613957 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc01cad0 con 0x55c7f3cd4df0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.130+0000 7f99eb7fe640 1 -- 172.21.15.120:0/1454613957 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc011a70 con 0x55c7f3cd4df0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- 172.21.15.120:0/1454613957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 msgr2=0x55c7f3ce4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a0490c640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 0x7f99ec0b77a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005c40 tx=0x7f99f0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce4c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005fd0 tx=0x7f99fc0107a0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- 172.21.15.120:0/1454613957 shutdown_connections 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.157 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce4c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 --2- 172.21.15.120:0/1454613957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cd48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- 172.21.15.120:0/1454613957 >> 172.21.15.120:0/1454613957 conn(0x55c7f3cd2300 msgr2=0x55c7f3cd34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- 172.21.15.120:0/1454613957 shutdown_connections 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- 172.21.15.120:0/1454613957 wait complete. 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 -- start start 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.131+0000 7f9a09746740 1 Processor -- start 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ce4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3a9b700 con 0x55c7f3cd4df0 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3bd5160 con 0x55c7f3ce5190 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c7f3ce7e40 con 0x55c7f3cd44d0 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a09746740 10 client.0 _ll_register_callbacks cb 0x55c7f3bee9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:28.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0510d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ce4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49836/0 (socket says 172.21.15.120:49836) 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0510d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ce4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37320/0 (socket says 172.21.15.120:37320) 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 -- 172.21.15.120:0/198264260 learned_addr learned my addr 172.21.15.120:0/198264260 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0490c640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 -- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 msgr2=0x55c7f3ce4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ce4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 -- 172.21.15.120:0/198264260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd4df0 msgr2=0x55c7f3ce1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.132+0000 7f9a0590e640 1 -- 172.21.15.120:0/198264260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f3c02c90 con 0x55c7f3ce5190 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a0590e640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99f80abdc0 tx=0x7f99f80abb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903444c5ee4f92b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8020070 con 0x55c7f3ce5190 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f99f80ab520 con 0x55c7f3ce5190 2024-09-10T23:14:28.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc01cad0 con 0x55c7f3ce5190 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f99f80ac0c0 con 0x55c7f3ce5190 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 10 client.28844 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7f3cf3740 con 0x55c7f3ce5190 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc01cad0 con 0x55c7f3ce5190 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f99f80aad90 con 0x55c7f3ce5190 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 10 client.28844 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 10 client.28844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 20 client.28844 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3cf4c00 0x55c7f3cf7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 4 client.28844 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.133+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f3c4cdf0 con 0x55c7f3cf4c00 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.134+0000 7f9a0510d640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3cf4c00 0x55c7f3cf7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.134+0000 7f99e8ff9640 10 client.28844 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.134+0000 7f9a0510d640 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3cf4c00 0x55c7f3cf7060 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7793386b666f1479 server_cookie=5222baf683b7899a in_seq=0 out_seq=0 2024-09-10T23:14:28.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f99e8ff9640 1 -- 172.21.15.120:0/198264260 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7f3c4cdf0 con 0x55c7f3cf4c00 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f99e8ff9640 10 client.28844 handle_command_reply: tid=0 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 client.28844 shutdown 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 msgr2=0x55c7f3ce2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f99f80abdc0 tx=0x7f99f80abb00 comp rx=0 tx=0).stop 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 shutdown_connections 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3cf4c00 0x55c7f3cf7060 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3ce5190 0x55c7f3ce2480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ce1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 --2- 172.21.15.120:0/198264260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ce4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f99e8ff9640 0 client.28844 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 >> 172.21.15.120:0/198264260 conn(0x55c7f3cd2300 msgr2=0x55c7f3cd7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 shutdown_connections 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 1 -- 172.21.15.120:0/198264260 wait complete. 2024-09-10T23:14:28.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.135+0000 7f9a09746740 20 client.28844 trim_cache size 0 max 16384 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 Processor -- start 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 -- start start 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3cd36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3cfc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001aa100 con 0x55c7f3cd44d0 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac960 con 0x55c7f3cd4df0 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a09746740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001aa750 con 0x55c7f3ce5190 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57684/0 (socket says 172.21.15.120:57684) 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 -- 172.21.15.120:0/2698553591 learned_addr learned my addr 172.21.15.120:0/2698553591 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0590e640 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3cfc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0490c640 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3cd36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 -- 172.21.15.120:0/2698553591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 msgr2=0x55c7f3cfc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3cfc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 -- 172.21.15.120:0/2698553591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 msgr2=0x55c7f3cd36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3cd36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 -- 172.21.15.120:0/2698553591 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f3ce1c50 con 0x55c7f3cd44d0 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.149+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0115e0 tx=0x7f99fc011570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0bbc68e8d2d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f99eaffd640 1 -- 172.21.15.120:0/2698553591 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004220 con 0x55c7f3cd44d0 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f99eaffd640 1 -- 172.21.15.120:0/2698553591 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc00be60 con 0x55c7f3cd44d0 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f99eaffd640 1 -- 172.21.15.120:0/2698553591 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc011ae0 con 0x55c7f3cd44d0 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 -- 172.21.15.120:0/2698553591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 msgr2=0x55c7f3cf1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0115e0 tx=0x7f99fc011570 comp rx=0 tx=0).stop 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 -- 172.21.15.120:0/2698553591 shutdown_connections 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3ce5190 0x55c7f3cfc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd4df0 0x55c7f3cd36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 --2- 172.21.15.120:0/2698553591 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3cd44d0 0x55c7f3cf1070 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.150+0000 7f9a09746740 1 -- 172.21.15.120:0/2698553591 >> 172.21.15.120:0/2698553591 conn(0x55c7f3cd2300 msgr2=0x55c7f3cd2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- 172.21.15.120:0/2698553591 shutdown_connections 2024-09-10T23:14:28.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- 172.21.15.120:0/2698553591 wait complete. 2024-09-10T23:14:28.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- start start 2024-09-10T23:14:28.164 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 Processor -- start 2024-09-10T23:14:28.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ced480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ceda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001aae20 con 0x55c7f3ce5190 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac290 con 0x55c7f3cd44d0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ab4f0 con 0x55c7f3cd4df0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.151+0000 7f9a09746740 10 client.0 _ll_register_callbacks cb 0x7f99f80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0490c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ceda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57686/0 (socket says 172.21.15.120:57686) 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0490c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ceda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37346/0 (socket says 172.21.15.120:37346) 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 -- 172.21.15.120:0/2312642916 learned_addr learned my addr 172.21.15.120:0/2312642916 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ced480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 -- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 msgr2=0x55c7f3ceda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ceda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 -- 172.21.15.120:0/2312642916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd44d0 msgr2=0x55c7f3ced480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ced480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 -- 172.21.15.120:0/2312642916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f3ceaa10 con 0x55c7f3ce5190 2024-09-10T23:14:28.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a0590e640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f99f80b5070 tx=0x7f99f80b1f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348895ec9434150c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f99e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8014030 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f99f80ab320 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc00be60 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f99f8020070 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a09746740 10 client.19698 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.152+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7f3d627a0 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc00be60 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f99f80c1b90 con 0x55c7f3ce5190 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 10 client.19698 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 20 client.19698 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3d62ef0 0x55c7f3d65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:28.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 4 client.19698 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7f3c488a0 con 0x55c7f3d62ef0 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3d62ef0 0x55c7f3d65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f99e8ff9640 10 client.19698 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.153+0000 7f9a0510d640 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3d62ef0 0x55c7f3d65310 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4610d77edf2512f server_cookie=4ac2db6e529e0b14 in_seq=0 out_seq=0 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f99e8ff9640 1 -- 172.21.15.120:0/2312642916 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7f3c488a0 con 0x55c7f3d62ef0 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f99e8ff9640 10 client.19698 handle_command_reply: tid=0 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 client.19698 shutdown 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 msgr2=0x55c7f3cea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f99f80b5070 tx=0x7f99f80b1f80 comp rx=0 tx=0).stop 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 shutdown_connections 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55c7f3d62ef0 0x55c7f3d65310 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f99e8ff9640 0 client.19698 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:28.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7f3ce5190 0x55c7f3cea1e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c7f3cd4df0 0x55c7f3ceda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 --2- 172.21.15.120:0/2312642916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c7f3cd44d0 0x55c7f3ced480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.155+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 >> 172.21.15.120:0/2312642916 conn(0x55c7f3cd2300 msgr2=0x55c7f3ce3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 shutdown_connections 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a09746740 1 -- 172.21.15.120:0/2312642916 wait complete. 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a09746740 20 client.19698 trim_cache size 0 max 16384 2024-09-10T23:14:28.170 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 msgr2=0x7f99ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 0x7f99ec0b77a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005c40 tx=0x7f99f0005b50 comp rx=0 tx=0).stop 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 msgr2=0x7f9a0019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009fd0 tx=0x7f99f800b260 comp rx=0 tx=0).stop 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 shutdown_connections 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f99ec0b52e0 0x7f99ec0b77a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a00108020 0x7f9a0019ec00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a001076a0 0x7f9a00199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 --2- 172.21.15.120:0/714406876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00101c90 0x7f9a00199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:28.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.156+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 >> 172.21.15.120:0/714406876 conn(0x7f9a000fd700 msgr2=0x7f9a0006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:28.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.157+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 shutdown_connections 2024-09-10T23:14:28.172 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:28.157+0000 7f9a07371640 1 -- 172.21.15.120:0/714406876 wait complete. 2024-09-10T23:14:28.446 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:28 smithi120 ceph-mon[31247]: pgmap v1007: 97 pgs: 97 active+clean; 245 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 613 op/s 2024-09-10T23:14:28.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:28 smithi176 ceph-mon[34982]: pgmap v1007: 97 pgs: 97 active+clean; 245 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 613 op/s 2024-09-10T23:14:28.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:28 smithi153 ceph-mon[34951]: pgmap v1007: 97 pgs: 97 active+clean; 245 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 613 op/s 2024-09-10T23:14:28.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 228495 21.61 MB/sec execute 443 sec latency 63.621 ms 2024-09-10T23:14:29.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229026 21.62 MB/sec execute 444 sec latency 82.794 ms 2024-09-10T23:14:29.950 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:14:29.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:14:30.241 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:30.517 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:30 smithi120 ceph-mon[31247]: pgmap v1008: 97 pgs: 97 active+clean; 238 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:14:30.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:30 smithi176 ceph-mon[34982]: pgmap v1008: 97 pgs: 97 active+clean; 238 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:14:30.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:30 smithi153 ceph-mon[34951]: pgmap v1008: 97 pgs: 97 active+clean; 238 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 532 op/s 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 -- 172.21.15.120:0/1383564545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 msgr2=0x7f5914107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 --2- 172.21.15.120:0/1383564545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f5914107ae0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5904009900 tx=0x7f590402a410 comp rx=0 tx=0).stop 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 -- 172.21.15.120:0/1383564545 shutdown_connections 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 --2- 172.21.15.120:0/1383564545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 0x7f591410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 --2- 172.21.15.120:0/1383564545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f5914107ae0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 --2- 172.21.15.120:0/1383564545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 -- 172.21.15.120:0/1383564545 >> 172.21.15.120:0/1383564545 conn(0x7f59140fd700 msgr2=0x7f59140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 -- 172.21.15.120:0/1383564545 shutdown_connections 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.753+0000 7f591b865640 1 -- 172.21.15.120:0/1383564545 wait complete. 2024-09-10T23:14:30.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 Processor -- start 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 -- start start 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 0x7f5914198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5914111ca0 con 0x7f59141076a0 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5914111b20 con 0x7f5914101c90 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f591b865640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5914111e20 con 0x7f5914108020 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5918e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5918e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57712/0 (socket says 172.21.15.120:57712) 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5918e00640 1 -- 172.21.15.120:0/577066690 learned_addr learned my addr 172.21.15.120:0/577066690 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919e02640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 0x7f5914198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 -- 172.21.15.120:0/577066690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 msgr2=0x7f5914198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 0x7f5914198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 -- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 msgr2=0x7f59141983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5919601640 1 -- 172.21.15.120:0/577066690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59141a7680 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.754+0000 7f5918e00640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f5919601640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914197ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f591000d6d0 tx=0x7f591000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f590a7fc640 1 -- 172.21.15.120:0/577066690 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017070 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f590a7fc640 1 -- 172.21.15.120:0/577066690 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5910009820 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f590a7fc640 1 -- 172.21.15.120:0/577066690 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004c40 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59141a7970 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.755+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59141a7e30 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.757+0000 7f590a7fc640 1 -- 172.21.15.120:0/577066690 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5910005220 con 0x7f5914101c90 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 Processor -- start 2024-09-10T23:14:30.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 -- start start 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x558494932a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494932fa0 0x558494935390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558494894e60 con 0x5584949266f0 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558494884290 con 0x558494927010 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f591dc3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5584948880a0 con 0x558494932fa0 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57720/0 (socket says 172.21.15.120:57720) 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919601640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x558494932a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 -- 172.21.15.120:0/3830192548 learned_addr learned my addr 172.21.15.120:0/3830192548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5918e00640 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494932fa0 0x558494935390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 -- 172.21.15.120:0/3830192548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494932fa0 msgr2=0x558494935390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494932fa0 0x558494935390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.766 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 -- 172.21.15.120:0/3830192548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 msgr2=0x558494932a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x558494932a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919e02640 1 -- 172.21.15.120:0/3830192548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584945e2930 con 0x5584949266f0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.758+0000 7f5919601640 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x558494932a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f590a7fc640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 0x7f59000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f5919e02640 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f590c010370 tx=0x7f590c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216b78df27ebb30a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f5919601640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 0x7f59000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f58ff7fe640 1 -- 172.21.15.120:0/3830192548 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c016070 con 0x5584949266f0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f58ff7fe640 1 -- 172.21.15.120:0/3830192548 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c004d20 con 0x5584949266f0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f58ff7fe640 1 -- 172.21.15.120:0/3830192548 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c005420 con 0x5584949266f0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.759+0000 7f590a7fc640 1 -- 172.21.15.120:0/577066690 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f59100e55f0 con 0x7f5914101c90 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- 172.21.15.120:0/3830192548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 msgr2=0x558494926ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f5919601640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 0x7f59000b77a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f59100a8460 tx=0x7f5910093040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f590c010370 tx=0x7f590c010830 comp rx=0 tx=0).stop 2024-09-10T23:14:30.767 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- 172.21.15.120:0/3830192548 shutdown_connections 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494932fa0 0x558494935390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x558494932a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3830192548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x558494926ad0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- 172.21.15.120:0/3830192548 >> 172.21.15.120:0/3830192548 conn(0x558494924520 msgr2=0x5584949256c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- 172.21.15.120:0/3830192548 shutdown_connections 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- 172.21.15.120:0/3830192548 wait complete. 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 -- start start 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.760+0000 7f591dc3a740 1 Processor -- start 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584949266f0 0x55849492ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494930a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584946e5890 con 0x558494932fa0 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55849481ed40 con 0x558494927010 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558494935c20 con 0x5584949266f0 2024-09-10T23:14:30.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f591dc3a740 10 client.0 _ll_register_callbacks cb 0x558494882c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:49894/0 (socket says 172.21.15.120:49894) 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 -- 172.21.15.120:0/65200951 learned_addr learned my addr 172.21.15.120:0/65200951 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5919e02640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584949266f0 0x55849492ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5919e02640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494930a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 -- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584949266f0 msgr2=0x55849492ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584949266f0 0x55849492ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 -- 172.21.15.120:0/65200951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 msgr2=0x558494930a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494930a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.761+0000 7f5918e00640 1 -- 172.21.15.120:0/65200951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849493f020 con 0x558494927010 2024-09-10T23:14:30.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f5918e00640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5904006010 tx=0x7f5904031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9561437ff3d3b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:14:30.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904035040 con 0x558494927010 2024-09-10T23:14:30.771 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5904004450 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f590c004d20 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f590404e070 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f591dc3a740 10 client.28862 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584949419d0 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c004d20 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5904004cb0 con 0x558494927010 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.762+0000 7f591dc3a740 10 client.28862 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f591dc3a740 10 client.28862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f591dc3a740 20 client.28862 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558494942ee0 0x5584949452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f591dc3a740 4 client.28862 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558494896db0 con 0x558494942ee0 2024-09-10T23:14:30.772 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f5919e02640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558494942ee0 0x5584949452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f58fcff9640 10 client.28862 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.763+0000 7f5919e02640 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558494942ee0 0x5584949452d0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda5762e9d51585f server_cookie=2028ae0d9cd3dcb0 in_seq=0 out_seq=0 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f58fcff9640 1 -- 172.21.15.120:0/65200951 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558494896db0 con 0x558494942ee0 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f58fcff9640 10 client.28862 handle_command_reply: tid=0 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 client.28862 shutdown 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 msgr2=0x5584949304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5904006010 tx=0x7f5904031810 comp rx=0 tx=0).stop 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 shutdown_connections 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558494942ee0 0x5584949452d0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494930a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494927010 0x5584949304d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f58fcff9640 0 client.28862 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 --2- 172.21.15.120:0/65200951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5584949266f0 0x55849492ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 >> 172.21.15.120:0/65200951 conn(0x558494924520 msgr2=0x5584949298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 shutdown_connections 2024-09-10T23:14:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.765+0000 7f591dc3a740 1 -- 172.21.15.120:0/65200951 wait complete. 2024-09-10T23:14:30.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.766+0000 7f591dc3a740 20 client.28862 trim_cache size 0 max 16384 2024-09-10T23:14:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 Processor -- start 2024-09-10T23:14:30.793 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 -- start start 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494931cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494932fa0 0x55849492d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59141aa0b0 con 0x5584949266f0 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59141ac930 con 0x558494932fa0 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f591dc3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59141aa720 con 0x558494927010 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57734/0 (socket says 172.21.15.120:57734) 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 -- 172.21.15.120:0/3345267168 learned_addr learned my addr 172.21.15.120:0/3345267168 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5918e00640 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494931cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494932fa0 0x55849492d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 -- 172.21.15.120:0/3345267168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 msgr2=0x558494931cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494931cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 -- 172.21.15.120:0/3345267168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494932fa0 msgr2=0x55849492d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494932fa0 0x55849492d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.791+0000 7f5919e02640 1 -- 172.21.15.120:0/3345267168 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558494926e40 con 0x5584949266f0 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f5919e02640 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f590c011010 tx=0x7f590c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fffbc14bd2c968d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f58feffd640 1 -- 172.21.15.120:0/3345267168 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c016040 con 0x5584949266f0 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f58feffd640 1 -- 172.21.15.120:0/3345267168 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c004d10 con 0x5584949266f0 2024-09-10T23:14:30.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f58feffd640 1 -- 172.21.15.120:0/3345267168 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0054b0 con 0x5584949266f0 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f591dc3a740 1 -- 172.21.15.120:0/3345267168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 msgr2=0x55849491af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.792+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f590c011010 tx=0x7f590c009bb0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- 172.21.15.120:0/3345267168 shutdown_connections 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558494932fa0 0x55849492d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494931cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- 172.21.15.120:0/3345267168 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584949266f0 0x55849491af70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- 172.21.15.120:0/3345267168 >> 172.21.15.120:0/3345267168 conn(0x558494924520 msgr2=0x5584949315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- 172.21.15.120:0/3345267168 shutdown_connections 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- 172.21.15.120:0/3345267168 wait complete. 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- start start 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 Processor -- start 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584949266f0 0x55849493b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59141aadf0 con 0x558494932fa0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f59141ac260 con 0x5584949266f0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f59141ab4c0 con 0x558494927010 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.793+0000 7f591dc3a740 10 client.0 _ll_register_callbacks cb 0x7f5904032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5918e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5918e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37396/0 (socket says 172.21.15.120:37396) 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57748/0 (socket says 172.21.15.120:57748) 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5918e00640 1 -- 172.21.15.120:0/1608579349 learned_addr learned my addr 172.21.15.120:0/1608579349 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584949266f0 0x55849493b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 -- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 msgr2=0x558494937e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 -- 172.21.15.120:0/1608579349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584949266f0 msgr2=0x55849493b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:30.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584949266f0 0x55849493b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 -- 172.21.15.120:0/1608579349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558494938c90 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5918e00640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f5919e02640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f590c033040 tx=0x7f590c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93600e39c10e2d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f58fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c02e070 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f590c012510 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558494892860 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f590c012710 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f591dc3a740 10 client.19716 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.794+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584949b0ec0 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558494892860 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f590c028040 con 0x558494932fa0 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 10 client.19716 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:30.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 10 client.19716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 20 client.19716 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584949b1340 0x5584949b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 4 client.19716 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55849488fc90 con 0x5584949b1340 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f5918e00640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584949b1340 0x5584949b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f58fcff9640 10 client.19716 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:30.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.795+0000 7f5918e00640 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584949b1340 0x5584949b3760 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2701b2a5a4e8ffa6 server_cookie=cc177064fd4d8e08 in_seq=0 out_seq=0 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f58fcff9640 1 -- 172.21.15.120:0/1608579349 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55849488fc90 con 0x5584949b1340 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f58fcff9640 10 client.19716 handle_command_reply: tid=0 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 client.19716 shutdown 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 msgr2=0x558494938460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f590c033040 tx=0x7f590c035040 comp rx=0 tx=0).stop 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 shutdown_connections 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5584949b1340 0x5584949b3760 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558494932fa0 0x558494938460 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558494927010 0x558494937e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 --2- 172.21.15.120:0/1608579349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5584949266f0 0x55849493b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.804 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f58fcff9640 0 client.19716 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 >> 172.21.15.120:0/1608579349 conn(0x558494924520 msgr2=0x558494931de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 shutdown_connections 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 1 -- 172.21.15.120:0/1608579349 wait complete. 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.802+0000 7f591dc3a740 20 client.19716 trim_cache size 0 max 16384 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "ab3503c1-c23a-434c-8be0-571ff6479523", 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:14:30.805 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 msgr2=0x7f59000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 0x7f59000b77a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f59100a8460 tx=0x7f5910093040 comp rx=0 tx=0).stop 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 msgr2=0x7f5914197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914197ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f591000d6d0 tx=0x7f591000db90 comp rx=0 tx=0).stop 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 shutdown_connections 2024-09-10T23:14:30.806 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f59000b52e0 0x7f59000b77a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5914108020 0x7f5914198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59141076a0 0x7f59141983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 --2- 172.21.15.120:0/577066690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5914101c90 0x7f5914197ea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 >> 172.21.15.120:0/577066690 conn(0x7f59140fd700 msgr2=0x7f59140ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 shutdown_connections 2024-09-10T23:14:30.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:30.803+0000 7f591b865640 1 -- 172.21.15.120:0/577066690 wait complete. 2024-09-10T23:14:30.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229615 21.64 MB/sec execute 445 sec latency 49.568 ms 2024-09-10T23:14:31.513 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:14:31.535 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:31 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:14:31.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:31.536 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:31 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:14:31.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:31 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:14:31.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:31.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:31 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:14:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:31 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:14:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:31.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:31 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:14:31.773 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:14:31.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229705 21.60 MB/sec execute 446 sec latency 291.314 ms 2024-09-10T23:14:32.474 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:32 smithi120 ceph-mon[31247]: pgmap v1009: 97 pgs: 97 active+clean; 241 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 -- 172.21.15.120:0/913815940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 msgr2=0x7fdd601031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 --2- 172.21.15.120:0/913815940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd601031a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009900 tx=0x7fdd5002a410 comp rx=0 tx=0).stop 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 -- 172.21.15.120:0/913815940 shutdown_connections 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 --2- 172.21.15.120:0/913815940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd6010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 --2- 172.21.15.120:0/913815940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd60108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.482 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 --2- 172.21.15.120:0/913815940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd601031a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 -- 172.21.15.120:0/913815940 >> 172.21.15.120:0/913815940 conn(0x7fdd600fe830 msgr2=0x7fdd60100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 -- 172.21.15.120:0/913815940 shutdown_connections 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.469+0000 7fdd65aed640 1 -- 172.21.15.120:0/913815940 wait complete. 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 Processor -- start 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 -- start start 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd6019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd601a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60112dd0 con 0x7fdd60102dc0 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.470+0000 7fdd65aed640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60112c50 con 0x7fdd60109150 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd65aed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60112f50 con 0x7fdd601087d0 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.483 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd6019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:37420/0 (socket says 172.21.15.120:37420) 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5effd640 1 -- 172.21.15.120:0/1033841365 learned_addr learned my addr 172.21.15.120:0/1033841365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd601a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 -- 172.21.15.120:0/1033841365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 msgr2=0x7fdd6019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 -- 172.21.15.120:0/1033841365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 msgr2=0x7fdd601a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd601a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 -- 172.21.15.120:0/1033841365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601a1570 con 0x7fdd60102dc0 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd601a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5f7fe640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd6019b4c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fdd5002a930 tx=0x7fdd50031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.484 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5effd640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.471+0000 7fdd5cff9640 1 -- 172.21.15.120:0/1033841365 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5003f030 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.472+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd601a1800 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.472+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601a7f20 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.472+0000 7fdd5cff9640 1 -- 172.21.15.120:0/1033841365 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd50004490 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.472+0000 7fdd5cff9640 1 -- 172.21.15.120:0/1033841365 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50035070 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.474+0000 7fdd67ec2740 1 Processor -- start 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.474+0000 7fdd5cff9640 1 -- 172.21.15.120:0/1033841365 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdd5004c060 con 0x7fdd60102dc0 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 -- start start 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea92d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.485 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561b9e9fae50 con 0x561b9ea840a0 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b9e9ea280 con 0x561b9ea849c0 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561b9e9ee090 con 0x561b9ea90950 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57772/0 (socket says 172.21.15.120:57772) 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 -- 172.21.15.120:0/4120165627 learned_addr learned my addr 172.21.15.120:0/4120165627 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5effd640 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea92d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 -- 172.21.15.120:0/4120165627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 msgr2=0x561b9ea92d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea92d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 -- 172.21.15.120:0/4120165627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 msgr2=0x561b9ea90410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea90410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.486 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 -- 172.21.15.120:0/4120165627 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9e748650 con 0x561b9ea840a0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fdd580117e0 tx=0x7fdd58011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ac84c237cfa69f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.475+0000 7fdd5cff9640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 0x7fdd480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd45ffb640 1 -- 172.21.15.120:0/4120165627 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58011310 con 0x561b9ea840a0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd5f7fe640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 0x7fdd480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd5cff9640 1 -- 172.21.15.120:0/1033841365 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdd50105940 con 0x7fdd60102dc0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd45ffb640 1 -- 172.21.15.120:0/4120165627 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd58003e20 con 0x561b9ea840a0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd45ffb640 1 -- 172.21.15.120:0/4120165627 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580044f0 con 0x561b9ea840a0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd67ec2740 1 -- 172.21.15.120:0/4120165627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 msgr2=0x561b9ea84480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fdd580117e0 tx=0x7fdd58011ca0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.476+0000 7fdd5f7fe640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 0x7fdd480b77a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdd501037e0 tx=0x7fdd5010b550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 -- 172.21.15.120:0/4120165627 shutdown_connections 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea92d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.487 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea90410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/4120165627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea84480 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 -- 172.21.15.120:0/4120165627 >> 172.21.15.120:0/4120165627 conn(0x561b9ea81ed0 msgr2=0x561b9ea83070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 -- 172.21.15.120:0/4120165627 shutdown_connections 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 -- 172.21.15.120:0/4120165627 wait complete. 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 -- start start 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 Processor -- start 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.477+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 0x561b9ea90180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea8db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561b9e84b700 con 0x561b9ea849c0 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x561b9e985110 con 0x561b9ea840a0 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561b9ea935d0 con 0x561b9ea90950 2024-09-10T23:14:32.488 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 10 client.0 _ll_register_callbacks cb 0x561b9e99e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57782/0 (socket says 172.21.15.120:57782) 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 0x561b9ea90180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5effd640 1 -- 172.21.15.120:0/2831729342 learned_addr learned my addr 172.21.15.120:0/2831729342 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea8db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 -- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 msgr2=0x561b9ea8db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea8db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 -- 172.21.15.120:0/2831729342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 msgr2=0x561b9ea8d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 -- 172.21.15.120:0/2831729342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9e9b2c40 con 0x561b9ea840a0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 0x561b9ea90180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd580117e0 tx=0x7fdd580040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761997fa3c2cdf4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.489 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd3effd640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5801e070 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdd5800fdd0 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.478+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd58003e20 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd58025040 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 10 client.28880 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b9ea9eed0 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd58003e20 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdd58018020 con 0x561b9ea840a0 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 10 client.28880 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 10 client.28880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 20 client.28880 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:32.490 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eaa0220 0x561b9eaa2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 4 client.28880 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:32.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b9e9fcda0 con 0x561b9eaa0220 2024-09-10T23:14:32.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.479+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eaa0220 0x561b9eaa2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.480+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eaa0220 0x561b9eaa2610 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74549d08ecd50e1 server_cookie=b561f3418dcd9cad in_seq=0 out_seq=0 2024-09-10T23:14:32.491 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.480+0000 7fdd3effd640 10 client.28880 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd3effd640 1 -- 172.21.15.120:0/2831729342 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b9e9fcda0 con 0x561b9eaa0220 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd3effd640 10 client.28880 handle_command_reply: tid=0 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 client.28880 shutdown 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 msgr2=0x561b9ea90180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 0x561b9ea90180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd580117e0 tx=0x7fdd580040d0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 shutdown_connections 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eaa0220 0x561b9eaa2610 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea8db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.492 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2831729342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea840a0 0x561b9ea90180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 >> 172.21.15.120:0/2831729342 conn(0x561b9ea81ed0 msgr2=0x561b9ea87a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd3effd640 0 client.28880 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 shutdown_connections 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2831729342 wait complete. 2024-09-10T23:14:32.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.482+0000 7fdd67ec2740 20 client.28880 trim_cache size 0 max 16384 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 Processor -- start 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 -- start start 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea83460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00c380 con 0x561b9ea840a0 2024-09-10T23:14:32.504 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00ec00 con 0x561b9ea849c0 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00c9f0 con 0x561b9ea90950 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.501+0000 7fdd5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57784/0 (socket says 172.21.15.120:57784) 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 -- 172.21.15.120:0/686827106 learned_addr learned my addr 172.21.15.120:0/686827106 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea83460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 -- 172.21.15.120:0/686827106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 msgr2=0x561b9ea83460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea83460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 -- 172.21.15.120:0/686827106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 msgr2=0x561b9ea82f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea82f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.505 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 -- 172.21.15.120:0/686827106 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9ea82bb0 con 0x561b9ea840a0 2024-09-10T23:14:32.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fdd580039e0 tx=0x7fdd58003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65361bf371f35ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd457fa640 1 -- 172.21.15.120:0/686827106 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5801e070 con 0x561b9ea840a0 2024-09-10T23:14:32.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd457fa640 1 -- 172.21.15.120:0/686827106 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5801aa50 con 0x561b9ea840a0 2024-09-10T23:14:32.506 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.502+0000 7fdd457fa640 1 -- 172.21.15.120:0/686827106 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58005270 con 0x561b9ea840a0 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- 172.21.15.120:0/686827106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 msgr2=0x561b9ea81710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fdd580039e0 tx=0x7fdd58003970 comp rx=0 tx=0).stop 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- 172.21.15.120:0/686827106 shutdown_connections 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea90950 0x561b9ea83460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea849c0 0x561b9ea82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/686827106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea840a0 0x561b9ea81710 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.510 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- 172.21.15.120:0/686827106 >> 172.21.15.120:0/686827106 conn(0x561b9ea81ed0 msgr2=0x561b9ea8be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- 172.21.15.120:0/686827106 shutdown_connections 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- 172.21.15.120:0/686827106 wait complete. 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 -- start start 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.503+0000 7fdd67ec2740 1 Processor -- start 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea840a0 0x561b9ea99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 0x561b9ea94a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00d0c0 con 0x561b9ea849c0 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00e530 con 0x561b9ea90950 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00d790 con 0x561b9ea840a0 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd67ec2740 10 client.0 _ll_register_callbacks cb 0x7fdd5010b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.511 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:57790/0 (socket says 172.21.15.120:57790) 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 -- 172.21.15.120:0/2550738786 learned_addr learned my addr 172.21.15.120:0/2550738786 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea840a0 0x561b9ea99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 0x561b9ea94a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 -- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea840a0 msgr2=0x561b9ea99080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea840a0 0x561b9ea99080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 -- 172.21.15.120:0/2550738786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 msgr2=0x561b9ea94a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 0x561b9ea94a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 -- 172.21.15.120:0/2550738786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9ea952a0 con 0x561b9ea849c0 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 0x561b9ea94a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd5effd640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004090 tx=0x7fdd54004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec030e7dc9502d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.504+0000 7fdd3effd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:32.512 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54029030 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd5801aa50 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdd54003800 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd54003a00 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 10 client.19734 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b9eb0d160 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5801aa50 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdd5402e450 con 0x561b9ea849c0 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 10 client.19734 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 20 client.19734 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:32.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eb0d940 0x561b9eb0fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 4 client.19734 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b9e9f8850 con 0x561b9eb0d940 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.505+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eb0d940 0x561b9eb0fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.506+0000 7fdd5ffff640 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eb0d940 0x561b9eb0fd60 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe83fb289bd7e9f server_cookie=c794348b87bdf008 in_seq=0 out_seq=0 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.506+0000 7fdd3effd640 10 client.19734 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd3effd640 1 -- 172.21.15.120:0/2550738786 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10794 (crc 0 0 0) 0x561b9e9f8850 con 0x561b9eb0d940 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd3effd640 10 client.19734 handle_command_reply: tid=0 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 client.19734 shutdown 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 msgr2=0x561b9ea94550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004090 tx=0x7fdd54004550 comp rx=0 tx=0).stop 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 shutdown_connections 2024-09-10T23:14:32.514 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x561b9eb0d940 0x561b9eb0fd60 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561b9ea90950 0x561b9ea94a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561b9ea849c0 0x561b9ea94550 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 --2- 172.21.15.120:0/2550738786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561b9ea840a0 0x561b9ea99080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 >> 172.21.15.120:0/2550738786 conn(0x561b9ea81ed0 msgr2=0x561b9ea8bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.507+0000 7fdd3effd640 0 client.19734 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 shutdown_connections 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd67ec2740 1 -- 172.21.15.120:0/2550738786 wait complete. 2024-09-10T23:14:32.515 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd67ec2740 20 client.19734 trim_cache size 0 max 16384 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "ab3503c1-c23a-434c-8be0-571ff6479523", 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:14:32.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:14:32.517 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.518 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:14:32.519 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.520 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.521 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.522 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:14:32.524 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.525 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.526 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.527 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:14:32.528 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:14:32.529 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.530 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:14:32.531 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:14:32.532 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.533 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:14:32.534 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.535 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:14:32.536 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.537 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.538 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.539 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.540 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:14:32.541 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:14:32.542 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 msgr2=0x7fdd480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 0x7fdd480b77a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdd501037e0 tx=0x7fdd5010b550 comp rx=0 tx=0).stop 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 msgr2=0x7fdd6019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd6019b4c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fdd5002a930 tx=0x7fdd50031a20 comp rx=0 tx=0).stop 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 shutdown_connections 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdd480b52e0 0x7fdd480b77a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd60109150 0x7fdd601a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdd601087d0 0x7fdd6019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 --2- 172.21.15.120:0/1033841365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd60102dc0 0x7fdd6019b4c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:32.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 >> 172.21.15.120:0/1033841365 conn(0x7fdd600fe830 msgr2=0x7fdd60104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:32.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 shutdown_connections 2024-09-10T23:14:32.544 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:14:32.508+0000 7fdd65aed640 1 -- 172.21.15.120:0/1033841365 wait complete. 2024-09-10T23:14:32.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:32 smithi176 ceph-mon[34982]: pgmap v1009: 97 pgs: 97 active+clean; 241 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-10T23:14:32.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:32 smithi153 ceph-mon[34951]: pgmap v1009: 97 pgs: 97 active+clean; 241 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-10T23:14:32.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 229830 21.58 MB/sec execute 447 sec latency 703.230 ms 2024-09-10T23:14:33.375 INFO:tasks.cephfs.filesystem:scrub status for tag:ab3503c1-c23a-434c-8be0-571ff6479523 - {'path': '/', 'tag': 'ab3503c1-c23a-434c-8be0-571ff6479523', 'options': 'recursive,force'} 2024-09-10T23:14:33.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:33 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:33.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:33 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:33.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:33 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:14:33.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 230312 21.60 MB/sec execute 448 sec latency 92.966 ms 2024-09-10T23:14:34.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:34 smithi176 ceph-mon[34982]: pgmap v1010: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:14:34.599 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:34 smithi153 ceph-mon[34951]: pgmap v1010: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- 172.21.15.120:0/43039208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 msgr2=0x7f1628109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 --2- 172.21.15.120:0/43039208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628109cf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1620009cd0 tx=0x7f162002a8e0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- 172.21.15.120:0/43039208 shutdown_connections 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 --2- 172.21.15.120:0/43039208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628109cf0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 --2- 172.21.15.120:0/43039208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16281029b0 0x7f1628102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 --2- 172.21.15.120:0/43039208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1628101860 0x7f1628101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- 172.21.15.120:0/43039208 >> 172.21.15.120:0/43039208 conn(0x7f16280fd140 msgr2=0x7f16280ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- 172.21.15.120:0/43039208 shutdown_connections 2024-09-10T23:14:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- 172.21.15.120:0/43039208 wait complete. 2024-09-10T23:14:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 Processor -- start 2024-09-10T23:14:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.643+0000 7f162e92b640 1 -- start start 2024-09-10T23:14:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1628101860 0x7f1628198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16281029b0 0x7f1628199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f162810d4b0 con 0x7f1628103370 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f162810d330 con 0x7f1628101860 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162e92b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f162810d630 con 0x7f16281029b0 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f16277fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16281029b0 0x7f1628199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33560/0 (socket says 172.21.15.120:33560) 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f1627fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1628101860 0x7f1628198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 -- 172.21.15.120:0/2301685499 learned_addr learned my addr 172.21.15.120:0/2301685499 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 -- 172.21.15.120:0/2301685499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16281029b0 msgr2=0x7f1628199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16281029b0 0x7f1628199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 -- 172.21.15.120:0/2301685499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1628101860 msgr2=0x7f1628198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1628101860 0x7f1628198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 -- 172.21.15.120:0/2301685499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162819a850 con 0x7f1628103370 2024-09-10T23:14:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f162cec8640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1620033040 tx=0x7f1620003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.644+0000 7f16257fa640 1 -- 172.21.15.120:0/2301685499 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620004240 con 0x7f1628103370 2024-09-10T23:14:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.645+0000 7f16257fa640 1 -- 172.21.15.120:0/2301685499 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16200305a0 con 0x7f1628103370 2024-09-10T23:14:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.645+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162819aae0 con 0x7f1628103370 2024-09-10T23:14:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.645+0000 7f16257fa640 1 -- 172.21.15.120:0/2301685499 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200466e0 con 0x7f1628103370 2024-09-10T23:14:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.645+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16281a1210 con 0x7f1628103370 2024-09-10T23:14:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 Processor -- start 2024-09-10T23:14:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16257fa640 1 -- 172.21.15.120:0/2301685499 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f162004e070 con 0x7f1628103370 2024-09-10T23:14:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 -- start start 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a29c70 0x55fc13a2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 0x55fc13a314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc139976f0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc13986b20 con 0x55fc13a2efe0 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.647+0000 7f16317d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fc1398a930 con 0x55fc13a29c70 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33566/0 (socket says 172.21.15.120:33566) 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 -- 172.21.15.120:0/512754532 learned_addr learned my addr 172.21.15.120:0/512754532 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16277fe640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a29c70 0x55fc13a2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16277fe640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 0x55fc13a314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 -- 172.21.15.120:0/512754532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a29c70 msgr2=0x55fc13a2c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a29c70 0x55fc13a2c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 -- 172.21.15.120:0/512754532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 msgr2=0x55fc13a314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 0x55fc13a314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 -- 172.21.15.120:0/512754532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1394c530 con 0x55fc13a2c5a0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16277fe640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 0x55fc13a314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16257fa640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f1627fff640 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f161800bd10 tx=0x7f16180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f162313d85987e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16277fe640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f160e7fc640 1 -- 172.21.15.120:0/512754532 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004030 con 0x55fc13a2c5a0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f16257fa640 1 -- 172.21.15.120:0/2301685499 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f162002e030 con 0x7f1628103370 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f160e7fc640 1 -- 172.21.15.120:0/512754532 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16180035f0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.648+0000 7f160e7fc640 1 -- 172.21.15.120:0/512754532 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618011770 con 0x55fc13a2c5a0 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- 172.21.15.120:0/512754532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 msgr2=0x55fc13a2e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16277fe640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 0x7f16100b7820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f161c009730 tx=0x7f161c013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f161800bd10 tx=0x7f16180107a0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- 172.21.15.120:0/512754532 shutdown_connections 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2efe0 0x55fc13a314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a2e990 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 --2- 172.21.15.120:0/512754532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a29c70 0x55fc13a2c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- 172.21.15.120:0/512754532 >> 172.21.15.120:0/512754532 conn(0x55fc13a209b0 msgr2=0x55fc13a21740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- 172.21.15.120:0/512754532 shutdown_connections 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- 172.21.15.120:0/512754532 wait complete. 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.649+0000 7f16317d5740 1 -- start start 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 Processor -- start 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a29c70 0x55fc13a33300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fc138beab0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55fc1392c730 con 0x55fc13a29c70 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55fc13a31fc0 con 0x55fc13a2efe0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16317d5740 10 client.0 _ll_register_callbacks cb 0x55fc139806b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f1627fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a29c70 0x55fc13a33300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33572/0 (socket says 172.21.15.120:33572) 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 -- 172.21.15.120:0/3823270303 learned_addr learned my addr 172.21.15.120:0/3823270303 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f162cec8640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 -- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 msgr2=0x55fc13a34570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a34570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 -- 172.21.15.120:0/3823270303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a29c70 msgr2=0x55fc13a33300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a29c70 0x55fc13a33300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 -- 172.21.15.120:0/3823270303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc136e8790 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16277fe640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f161c00fc10 tx=0x7f161c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af23cf79f399984 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.650+0000 7f16077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c00fe50 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f161c01a070 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16180035f0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f161c01f440 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 10 client.19746 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc13761620 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16180035f0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f161c037720 con 0x55fc13a2c5a0 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 10 client.19746 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 10 client.19746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 20 client.19746 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13a41000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 4 client.19746 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc13999640 con 0x55fc13a3ec10 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.651+0000 7f1627fff640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13a41000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.652+0000 7f1627fff640 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13a41000 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d874dab260a31a server_cookie=1a0aaf4dfecbf2e9 in_seq=0 out_seq=0 2024-09-10T23:14:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.652+0000 7f16077fe640 10 client.19746 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16077fe640 1 -- 172.21.15.120:0/3823270303 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc13999640 con 0x55fc13a3ec10 2024-09-10T23:14:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16077fe640 10 client.19746 handle_command_reply: tid=0 2024-09-10T23:14:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 client.19746 shutdown 2024-09-10T23:14:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 msgr2=0x55fc13a339a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f161c00fc10 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 shutdown_connections 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13a41000 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a2c5a0 0x55fc13a339a0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16077fe640 0 client.19746 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 --2- 172.21.15.120:0/3823270303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a29c70 0x55fc13a33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 >> 172.21.15.120:0/3823270303 conn(0x55fc13a209b0 msgr2=0x55fc13a1fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 shutdown_connections 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 1 -- 172.21.15.120:0/3823270303 wait complete. 2024-09-10T23:14:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.654+0000 7f16317d5740 20 client.19746 trim_cache size 0 max 16384 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.671+0000 7f16317d5740 1 Processor -- start 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.671+0000 7f16317d5740 1 -- start start 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.671+0000 7f16317d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.671+0000 7f16317d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a23f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f16317d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f16317d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a5b60 con 0x55fc13a29c70 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f16317d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a83c0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f16317d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a61b0 con 0x55fc13a2efe0 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33576/0 (socket says 172.21.15.120:33576) 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 -- 172.21.15.120:0/2249128694 learned_addr learned my addr 172.21.15.120:0/2249128694 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f16277fe640 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f162cec8640 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 -- 172.21.15.120:0/2249128694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 msgr2=0x55fc13a32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 -- 172.21.15.120:0/2249128694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 msgr2=0x55fc13a23f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a23f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 -- 172.21.15.120:0/2249128694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc13a348e0 con 0x55fc13a29c70 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f1627fff640 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1618000c00 tx=0x7f1618005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465473dd8bb52544 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f160dffb640 1 -- 172.21.15.120:0/2249128694 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180038b0 con 0x55fc13a29c70 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f160dffb640 1 -- 172.21.15.120:0/2249128694 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16180054f0 con 0x55fc13a29c70 2024-09-10T23:14:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.672+0000 7f160dffb640 1 -- 172.21.15.120:0/2249128694 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180119b0 con 0x55fc13a29c70 2024-09-10T23:14:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 -- 172.21.15.120:0/2249128694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 msgr2=0x55fc13a23a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1618000c00 tx=0x7f1618005850 comp rx=0 tx=0).stop 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 -- 172.21.15.120:0/2249128694 shutdown_connections 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a23f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 --2- 172.21.15.120:0/2249128694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a23a00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 -- 172.21.15.120:0/2249128694 >> 172.21.15.120:0/2249128694 conn(0x55fc13a209b0 msgr2=0x55fc13a21650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 -- 172.21.15.120:0/2249128694 shutdown_connections 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.673+0000 7f16317d5740 1 -- 172.21.15.120:0/2249128694 wait complete. 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 -- start start 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 Processor -- start 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a6880 con 0x55fc13a29c70 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a7cf0 con 0x55fc13a2c5a0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f16281a6f50 con 0x55fc13a2efe0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16317d5740 10 client.0 _ll_register_callbacks cb 0x7f161c009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f16277fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f1627fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33578/0 (socket says 172.21.15.120:33578) 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f1627fff640 1 -- 172.21.15.120:0/330464369 learned_addr learned my addr 172.21.15.120:0/330464369 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.674+0000 7f162cec8640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 -- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 msgr2=0x55fc13a398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 -- 172.21.15.120:0/330464369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 msgr2=0x55fc13a3b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a3b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 -- 172.21.15.120:0/330464369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc13a2ed00 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16277fe640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a3b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f1627fff640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1618005c60 tx=0x7f1618005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa0fa7e92ba94d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161803b060 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f16180052c0 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16180054f0 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16180036b0 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 10 client.19752 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc13a38d20 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16180054f0 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1618042ba0 con 0x55fc13a29c70 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 10 client.19752 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 10 client.19752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 20 client.19752 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13aade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 4 client.19752 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.675+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc139950f0 con 0x55fc13a3ec10 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.676+0000 7f16277fe640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13aade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.676+0000 7f16277fe640 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13aade10 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb556848db11540b server_cookie=d5a2396fc44c5600 in_seq=0 out_seq=0 2024-09-10T23:14:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:34.676+0000 7f16077fe640 10 client.19752 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:34.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:34 smithi120 ceph-mon[31247]: pgmap v1010: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:14:34.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 230678 21.61 MB/sec execute 449 sec latency 170.991 ms 2024-09-10T23:14:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16077fe640 1 -- 172.21.15.120:0/330464369 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc139950f0 con 0x55fc13a3ec10 2024-09-10T23:14:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16077fe640 10 client.19752 handle_command_reply: tid=0 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 client.19752 shutdown 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 msgr2=0x55fc13a3aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1618005c60 tx=0x7f1618005730 comp rx=0 tx=0).stop 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 shutdown_connections 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55fc13a3ec10 0x55fc13aade10 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fc13a2efe0 0x55fc13a398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fc13a2c5a0 0x55fc13a3b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16077fe640 0 client.19752 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 --2- 172.21.15.120:0/330464369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fc13a29c70 0x55fc13a3aa30 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 >> 172.21.15.120:0/330464369 conn(0x55fc13a209b0 msgr2=0x55fc13a32540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.530+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 shutdown_connections 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f16317d5740 1 -- 172.21.15.120:0/330464369 wait complete. 2024-09-10T23:14:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f16317d5740 20 client.19752 trim_cache size 0 max 16384 2024-09-10T23:14:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:14:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:14:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:14:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 msgr2=0x7f16100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 0x7f16100b7820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f161c009730 tx=0x7f161c013040 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 msgr2=0x7f1628199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1620033040 tx=0x7f1620003b80 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 shutdown_connections 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1628103370 0x7f1628199f90 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16281029b0 0x7f1628199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 --2- 172.21.15.120:0/2301685499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1628101860 0x7f1628198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 >> 172.21.15.120:0/2301685499 conn(0x7f16280fd140 msgr2=0x7f16280feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 shutdown_connections 2024-09-10T23:14:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:35.531+0000 7f162e92b640 1 -- 172.21.15.120:0/2301685499 wait complete. 2024-09-10T23:14:35.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:14:35.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:14:35.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:14:35.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 230856 21.58 MB/sec execute 450 sec latency 476.642 ms 2024-09-10T23:14:36.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:36 smithi176 ceph-mon[34982]: pgmap v1011: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-09-10T23:14:36.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:36 smithi120 ceph-mon[31247]: pgmap v1011: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-09-10T23:14:36.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:36 smithi153 ceph-mon[34951]: pgmap v1011: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-09-10T23:14:36.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 231078 21.56 MB/sec execute 451 sec latency 320.491 ms 2024-09-10T23:14:37.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 231408 21.54 MB/sec execute 452 sec latency 78.262 ms 2024-09-10T23:14:38.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:38 smithi176 ceph-mon[34982]: pgmap v1012: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 710 op/s 2024-09-10T23:14:38.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:38 smithi120 ceph-mon[31247]: pgmap v1012: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 710 op/s 2024-09-10T23:14:38.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:38 smithi153 ceph-mon[34951]: pgmap v1012: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 710 op/s 2024-09-10T23:14:38.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 231816 21.53 MB/sec execute 453 sec latency 60.804 ms 2024-09-10T23:14:39.890 INFO:tasks.workunit.client.0.smithi120.stdout: 10 232301 21.53 MB/sec execute 454 sec latency 86.038 ms 2024-09-10T23:14:40.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:40 smithi176 ceph-mon[34982]: pgmap v1013: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2024-09-10T23:14:40.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:40 smithi120 ceph-mon[31247]: pgmap v1013: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2024-09-10T23:14:40.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:40 smithi153 ceph-mon[34951]: pgmap v1013: 97 pgs: 97 active+clean; 253 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2024-09-10T23:14:40.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 232904 21.56 MB/sec execute 455 sec latency 69.557 ms 2024-09-10T23:14:41.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 233372 21.56 MB/sec execute 456 sec latency 317.259 ms 2024-09-10T23:14:42.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:42 smithi176 ceph-mon[34982]: pgmap v1014: 97 pgs: 97 active+clean; 240 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-10T23:14:42.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:42 smithi120 ceph-mon[31247]: pgmap v1014: 97 pgs: 97 active+clean; 240 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-10T23:14:42.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:42 smithi153 ceph-mon[34951]: pgmap v1014: 97 pgs: 97 active+clean; 240 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.7 MiB/s wr, 613 op/s 2024-09-10T23:14:42.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 233815 21.58 MB/sec execute 457 sec latency 64.256 ms 2024-09-10T23:14:43.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 234256 21.60 MB/sec execute 458 sec latency 69.033 ms 2024-09-10T23:14:44.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:44 smithi176 ceph-mon[34982]: pgmap v1015: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-09-10T23:14:44.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:44 smithi120 ceph-mon[31247]: pgmap v1015: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-09-10T23:14:44.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:44 smithi153 ceph-mon[34951]: pgmap v1015: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-09-10T23:14:44.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 234614 21.60 MB/sec execute 459 sec latency 79.044 ms 2024-09-10T23:14:45.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 234982 21.59 MB/sec execute 460 sec latency 65.258 ms 2024-09-10T23:14:46.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:46 smithi176 ceph-mon[34982]: pgmap v1016: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-10T23:14:46.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:46 smithi120 ceph-mon[31247]: pgmap v1016: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-10T23:14:46.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:46 smithi153 ceph-mon[34951]: pgmap v1016: 97 pgs: 97 active+clean; 240 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 576 op/s 2024-09-10T23:14:46.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 235199 21.56 MB/sec execute 461 sec latency 281.370 ms 2024-09-10T23:14:47.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 235589 21.54 MB/sec execute 462 sec latency 104.993 ms 2024-09-10T23:14:48.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:48 smithi176 ceph-mon[34982]: pgmap v1017: 97 pgs: 97 active+clean; 246 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-09-10T23:14:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 -- 172.21.15.120:0/3473063779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 msgr2=0x7f6344093d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 --2- 172.21.15.120:0/3473063779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f6344093d10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6340009910 tx=0x7f634002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 -- 172.21.15.120:0/3473063779 shutdown_connections 2024-09-10T23:14:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 --2- 172.21.15.120:0/3473063779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 0x7f634409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 --2- 172.21.15.120:0/3473063779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634409adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 --2- 172.21.15.120:0/3473063779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f6344093d10 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 -- 172.21.15.120:0/3473063779 >> 172.21.15.120:0/3473063779 conn(0x7f634408f450 msgr2=0x7f6344091890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 -- 172.21.15.120:0/3473063779 shutdown_connections 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.624+0000 7f6351b96640 1 -- 172.21.15.120:0/3473063779 wait complete. 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 Processor -- start 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 -- start start 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634412bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 0x7f634412cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63440a11a0 con 0x7f6344093910 2024-09-10T23:14:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63440a1020 con 0x7f634409b300 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351b96640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63440a1320 con 0x7f63440989d0 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f634bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634412bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58798/0 (socket says 172.21.15.120:58798) 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 -- 172.21.15.120:0/3057020570 learned_addr learned my addr 172.21.15.120:0/3057020570 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6351395640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 0x7f634412cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 -- 172.21.15.120:0/3057020570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 msgr2=0x7f634412bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634412bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 -- 172.21.15.120:0/3057020570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 msgr2=0x7f634412cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 0x7f634412cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 -- 172.21.15.120:0/3057020570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634412d430 con 0x7f6344093910 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f634bfff640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634412bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.625+0000 7f6350b94640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f63400309f0 tx=0x7f6340032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.626+0000 7f6349ffb640 1 -- 172.21.15.120:0/3057020570 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340004460 con 0x7f6344093910 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.626+0000 7f6349ffb640 1 -- 172.21.15.120:0/3057020570 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6340005670 con 0x7f6344093910 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.626+0000 7f6349ffb640 1 -- 172.21.15.120:0/3057020570 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340004920 con 0x7f6344093910 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.626+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634412d6c0 con 0x7f6344093910 2024-09-10T23:14:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.626+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6344133df0 con 0x7f6344093910 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 Processor -- start 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 -- start start 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19775b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de197786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de196de6f0 con 0x55de197737a0 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de196cdb20 con 0x55de197761e0 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6355bcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55de196d1930 con 0x55de19770e70 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6351395640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6351395640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34712/0 (socket says 172.21.15.120:34712) 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f6351395640 1 -- 172.21.15.120:0/999267592 learned_addr learned my addr 172.21.15.120:0/999267592 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.629+0000 7f634bfff640 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de197786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6349ffb640 1 -- 172.21.15.120:0/3057020570 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f634004c030 con 0x7f6344093910 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 -- 172.21.15.120:0/999267592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 msgr2=0x55de197786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de197786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 -- 172.21.15.120:0/999267592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 msgr2=0x55de19775b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6350b94640 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19775b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19775b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 -- 172.21.15.120:0/999267592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de19693530 con 0x55de19770e70 2024-09-10T23:14:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6351395640 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f634c068cb0 tx=0x7f634c0694a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e9ca17cc02835f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.630+0000 7f6336ffd640 1 -- 172.21.15.120:0/999267592 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c08a070 con 0x55de19770e70 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6336ffd640 1 -- 172.21.15.120:0/999267592 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634c06c820 con 0x55de19770e70 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6336ffd640 1 -- 172.21.15.120:0/999267592 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c075e60 con 0x55de19770e70 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6349ffb640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 0x7f633c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6350b94640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 0x7f633c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6349ffb640 1 -- 172.21.15.120:0/3057020570 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6340106940 con 0x7f6344093910 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6350b94640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 0x7f633c0b7820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6340032de0 tx=0x7f63400c7640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6355bcd740 1 -- 172.21.15.120:0/999267592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 msgr2=0x55de19773260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6355bcd740 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f634c068cb0 tx=0x7f634c0694a0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6355bcd740 1 -- 172.21.15.120:0/999267592 shutdown_connections 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.631+0000 7f6355bcd740 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de197786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19775b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 --2- 172.21.15.120:0/999267592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19773260 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 -- 172.21.15.120:0/999267592 >> 172.21.15.120:0/999267592 conn(0x55de19767bb0 msgr2=0x55de19768940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 -- 172.21.15.120:0/999267592 shutdown_connections 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 -- 172.21.15.120:0/999267592 wait complete. 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 -- start start 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 Processor -- start 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.632+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de1977b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6355bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55de19605ab0 con 0x55de197737a0 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6355bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55de19673730 con 0x55de197761e0 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6355bcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55de197791c0 con 0x55de19770e70 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6355bcd740 10 client.0 _ll_register_callbacks cb 0x55de196c76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6351395640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58814/0 (socket says 172.21.15.120:58814) 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 -- 172.21.15.120:0/1026787471 learned_addr learned my addr 172.21.15.120:0/1026787471 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6351395640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de1977b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 -- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 msgr2=0x55de1977a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 -- 172.21.15.120:0/1026787471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 msgr2=0x55de1977b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de1977b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 -- 172.21.15.120:0/1026787471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1942f790 con 0x55de197737a0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f634bfff640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6338012d20 tx=0x7f633800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1175565217527a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f632bfff640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380120d0 con 0x55de197737a0 2024-09-10T23:14:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6338010430 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.633+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634c06c820 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6338012270 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 10 client.19761 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de194a8620 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634c06c820 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6338027690 con 0x55de197737a0 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 10 client.19761 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 10 client.19761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 20 client.19761 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de19785e30 0x55de19788220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 4 client.19761 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de196e0640 con 0x55de19785e30 2024-09-10T23:14:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.634+0000 7f6351395640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de19785e30 0x55de19788220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.635+0000 7f632bfff640 10 client.19761 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.635+0000 7f6351395640 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de19785e30 0x55de19788220 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b3e5613c62f7c4 server_cookie=692c89ad31550722 in_seq=0 out_seq=0 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f632bfff640 1 -- 172.21.15.120:0/1026787471 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de196e0640 con 0x55de19785e30 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f632bfff640 10 client.19761 handle_command_reply: tid=0 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 client.19761 shutdown 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 msgr2=0x55de1977ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6338012d20 tx=0x7f633800f790 comp rx=0 tx=0).stop 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 shutdown_connections 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de19785e30 0x55de19788220 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de1977b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977ab50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f632bfff640 0 client.19761 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1026787471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 >> 172.21.15.120:0/1026787471 conn(0x55de19767bb0 msgr2=0x55de19766c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 shutdown_connections 2024-09-10T23:14:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.637+0000 7f6355bcd740 1 -- 172.21.15.120:0/1026787471 wait complete. 2024-09-10T23:14:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.638+0000 7f6355bcd740 20 client.19761 trim_cache size 0 max 16384 2024-09-10T23:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 Processor -- start 2024-09-10T23:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 -- start start 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19779710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63441386a0 con 0x55de197737a0 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f634413af20 con 0x55de197761e0 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.655+0000 7f6355bcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6344138d10 con 0x55de19770e70 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58818/0 (socket says 172.21.15.120:58818) 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 -- 172.21.15.120:0/1225086678 learned_addr learned my addr 172.21.15.120:0/1225086678 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f6351395640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f6351395640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19779710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 -- 172.21.15.120:0/1225086678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 msgr2=0x55de1977a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 -- 172.21.15.120:0/1225086678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 msgr2=0x55de19779710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19779710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 -- 172.21.15.120:0/1225086678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de197632c0 con 0x55de197737a0 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f6351395640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19779710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f634bfff640 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6338012870 tx=0x7f63380092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17121dd8f1f9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f63367fc640 1 -- 172.21.15.120:0/1225086678 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380120d0 con 0x55de197737a0 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f63367fc640 1 -- 172.21.15.120:0/1225086678 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633802c030 con 0x55de197737a0 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.656+0000 7f63367fc640 1 -- 172.21.15.120:0/1225086678 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338005640 con 0x55de197737a0 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 -- 172.21.15.120:0/1225086678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 msgr2=0x55de1977a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6338012870 tx=0x7f63380092a0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 -- 172.21.15.120:0/1225086678 shutdown_connections 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19779710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de1977a9f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1225086678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de1977a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 -- 172.21.15.120:0/1225086678 >> 172.21.15.120:0/1225086678 conn(0x55de19767bb0 msgr2=0x55de197765c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 -- 172.21.15.120:0/1225086678 shutdown_connections 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.657+0000 7f6355bcd740 1 -- 172.21.15.120:0/1225086678 wait complete. 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 -- start start 2024-09-10T23:14:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 Processor -- start 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19781d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19780fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63441393e0 con 0x55de197737a0 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f634413a850 con 0x55de197761e0 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6344139ab0 con 0x55de19770e70 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6355bcd740 10 client.0 _ll_register_callbacks cb 0x7f6338009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6351395640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19781d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f6351395640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19780fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f634bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58832/0 (socket says 172.21.15.120:58832) 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.658+0000 7f634bfff640 1 -- 172.21.15.120:0/1102520247 learned_addr learned my addr 172.21.15.120:0/1102520247 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 -- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 msgr2=0x55de19781d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19781d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 -- 172.21.15.120:0/1102520247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 msgr2=0x55de19780a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 -- 172.21.15.120:0/1102520247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de19788590 con 0x55de197761e0 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f634bfff640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6351395640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19780fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f634c068980 tx=0x7f634c068e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6257c19d559f58b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f632bfff640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c08a040 con 0x55de197761e0 2024-09-10T23:14:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f634c075820 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633802c030 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f634c075a20 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6355bcd740 10 client.28922 fetch_fsmap learned FSMap version 26 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de19775f00 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.659+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633802c030 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f634c090040 con 0x55de197761e0 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 10 client.28922 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 10 client.28922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:14:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 20 client.28922 populate_metadata read hostname 'smithi120' 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de197f3170 0x55de197f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 4 client.28922 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de196dc0f0 con 0x55de197f3170 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f634bfff640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de197f3170 0x55de197f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f634bfff640 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de197f3170 0x55de197f5590 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64fd0356aa14e58 server_cookie=afaa629477b53dce in_seq=0 out_seq=0 2024-09-10T23:14:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:48.660+0000 7f632bfff640 10 client.28922 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:48.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:48 smithi120 ceph-mon[31247]: pgmap v1017: 97 pgs: 97 active+clean; 246 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-09-10T23:14:48.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:48 smithi153 ceph-mon[34951]: pgmap v1017: 97 pgs: 97 active+clean; 246 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-09-10T23:14:48.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 236096 21.56 MB/sec execute 463 sec latency 157.140 ms 2024-09-10T23:14:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.405+0000 7f632bfff640 1 -- 172.21.15.120:0/1102520247 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de196dc0f0 con 0x55de197f3170 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f632bfff640 10 client.28922 handle_command_reply: tid=0 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 client.28922 shutdown 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 msgr2=0x55de19780fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19780fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f634c068980 tx=0x7f634c068e00 comp rx=0 tx=0).stop 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 shutdown_connections 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55de197f3170 0x55de197f5590 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55de197761e0 0x55de19780fb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f632bfff640 0 client.28922 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55de197737a0 0x55de19780a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 --2- 172.21.15.120:0/1102520247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de19770e70 0x55de19781d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 >> 172.21.15.120:0/1102520247 conn(0x55de19767bb0 msgr2=0x55de19779710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 shutdown_connections 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 1 -- 172.21.15.120:0/1102520247 wait complete. 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.406+0000 7f6355bcd740 20 client.28922 trim_cache size 0 max 16384 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:14:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 msgr2=0x7f633c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 0x7f633c0b7820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6340032de0 tx=0x7f63400c7640 comp rx=0 tx=0).stop 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 msgr2=0x7f634412b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f63400309f0 tx=0x7f6340032fb0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 shutdown_connections 2024-09-10T23:14:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f633c0b5360 0x7f633c0b7820 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f634409b300 0x7f634412cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63440989d0 0x7f634412bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 --2- 172.21.15.120:0/3057020570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6344093910 0x7f634412b8f0 secure :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f63400309f0 tx=0x7f6340032fb0 comp rx=0 tx=0).stop 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 >> 172.21.15.120:0/3057020570 conn(0x7f634408f450 msgr2=0x7f63440995d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 shutdown_connections 2024-09-10T23:14:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:14:49.407+0000 7f6351b96640 1 -- 172.21.15.120:0/3057020570 wait complete. 2024-09-10T23:14:49.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 236450 21.55 MB/sec execute 464 sec latency 529.199 ms 2024-09-10T23:14:50.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:50 smithi176 ceph-mon[34982]: pgmap v1018: 97 pgs: 97 active+clean; 248 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 403 op/s 2024-09-10T23:14:50.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:50 smithi120 ceph-mon[31247]: pgmap v1018: 97 pgs: 97 active+clean; 248 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 403 op/s 2024-09-10T23:14:50.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:50 smithi153 ceph-mon[34951]: pgmap v1018: 97 pgs: 97 active+clean; 248 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 403 op/s 2024-09-10T23:14:50.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237079 21.58 MB/sec execute 465 sec latency 40.269 ms 2024-09-10T23:14:51.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237481 21.59 MB/sec execute 466 sec latency 150.207 ms 2024-09-10T23:14:52.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:52 smithi120 ceph-mon[31247]: pgmap v1019: 97 pgs: 97 active+clean; 242 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-09-10T23:14:52.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:52 smithi153 ceph-mon[34951]: pgmap v1019: 97 pgs: 97 active+clean; 242 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-09-10T23:14:52.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:52 smithi176 ceph-mon[34982]: pgmap v1019: 97 pgs: 97 active+clean; 242 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-09-10T23:14:52.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 237858 21.61 MB/sec execute 467 sec latency 63.906 ms 2024-09-10T23:14:53.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 238184 21.60 MB/sec execute 468 sec latency 75.272 ms 2024-09-10T23:14:54.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:54 smithi120 ceph-mon[31247]: pgmap v1020: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 626 op/s 2024-09-10T23:14:54.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:54 smithi153 ceph-mon[34951]: pgmap v1020: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 626 op/s 2024-09-10T23:14:54.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:54 smithi176 ceph-mon[34982]: pgmap v1020: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 626 op/s 2024-09-10T23:14:54.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 238544 21.58 MB/sec execute 469 sec latency 67.355 ms 2024-09-10T23:14:55.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 238923 21.57 MB/sec execute 470 sec latency 94.997 ms 2024-09-10T23:14:56.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:56 smithi120 ceph-mon[31247]: pgmap v1021: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-09-10T23:14:56.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:56 smithi153 ceph-mon[34951]: pgmap v1021: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-09-10T23:14:56.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:56 smithi176 ceph-mon[34982]: pgmap v1021: 97 pgs: 97 active+clean; 237 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-09-10T23:14:56.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 239272 21.55 MB/sec execute 471 sec latency 318.680 ms 2024-09-10T23:14:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:14:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:57 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:14:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:57 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:14:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:57 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:57.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 239670 21.55 MB/sec execute 472 sec latency 225.826 ms 2024-09-10T23:14:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:58 smithi120 ceph-mon[31247]: pgmap v1022: 97 pgs: 97 active+clean; 239 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-09-10T23:14:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:14:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:14:58.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:14:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:58.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:58 smithi153 ceph-mon[34951]: pgmap v1022: 97 pgs: 97 active+clean; 239 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-09-10T23:14:58.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:14:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:14:58.699 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:14:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:58.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:58 smithi176 ceph-mon[34982]: pgmap v1022: 97 pgs: 97 active+clean; 239 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-09-10T23:14:58.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:14:58.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:14:58.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:14:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:14:58.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 240322 21.58 MB/sec execute 473 sec latency 43.961 ms 2024-09-10T23:14:59.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 240887 21.61 MB/sec execute 474 sec latency 95.166 ms 2024-09-10T23:15:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:00 smithi120 ceph-mon[31247]: pgmap v1023: 97 pgs: 97 active+clean; 245 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2024-09-10T23:15:00.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:00 smithi153 ceph-mon[34951]: pgmap v1023: 97 pgs: 97 active+clean; 245 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2024-09-10T23:15:00.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:00 smithi176 ceph-mon[34982]: pgmap v1023: 97 pgs: 97 active+clean; 245 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 542 op/s 2024-09-10T23:15:00.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 241320 21.63 MB/sec execute 475 sec latency 70.729 ms 2024-09-10T23:15:01.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 241599 21.62 MB/sec execute 476 sec latency 249.880 ms 2024-09-10T23:15:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 -- 172.21.15.120:0/636724544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05881027e0 msgr2=0x7f0588102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 --2- 172.21.15.120:0/636724544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05881027e0 0x7f0588102c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0584009900 tx=0x7f058402a410 comp rx=0 tx=0).stop 2024-09-10T23:15:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 -- 172.21.15.120:0/636724544 shutdown_connections 2024-09-10T23:15:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 --2- 172.21.15.120:0/636724544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881031a0 0x7f0588109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 --2- 172.21.15.120:0/636724544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05881027e0 0x7f0588102c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 --2- 172.21.15.120:0/636724544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0588101690 0x7f0588101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 -- 172.21.15.120:0/636724544 >> 172.21.15.120:0/636724544 conn(0x7f05880fcf30 msgr2=0x7f05880ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 -- 172.21.15.120:0/636724544 shutdown_connections 2024-09-10T23:15:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.546+0000 7f058ed96640 1 -- 172.21.15.120:0/636724544 wait complete. 2024-09-10T23:15:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.547+0000 7f058ed96640 1 Processor -- start 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.547+0000 7f058ed96640 1 -- start start 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0588101690 0x7f058819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881027e0 0x7f058819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f058810d3f0 con 0x7f05881031a0 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f058810d270 con 0x7f0588101690 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058ed96640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f058810d570 con 0x7f05881027e0 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058dd94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0588101690 0x7f058819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58378/0 (socket says 172.21.15.120:58378) 2024-09-10T23:15:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 -- 172.21.15.120:0/2997321145 learned_addr learned my addr 172.21.15.120:0/2997321145 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058d593640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881027e0 0x7f058819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 -- 172.21.15.120:0/2997321145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881027e0 msgr2=0x7f058819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881027e0 0x7f058819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 -- 172.21.15.120:0/2997321145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0588101690 msgr2=0x7f058819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0588101690 0x7f058819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.548+0000 7f058e595640 1 -- 172.21.15.120:0/2997321145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058819c950 con 0x7f05881031a0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f058e595640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f058000eb10 tx=0x7f058000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f057effd640 1 -- 172.21.15.120:0/2997321145 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580004b20 con 0x7f05881031a0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f057effd640 1 -- 172.21.15.120:0/2997321145 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0580004cc0 con 0x7f05881031a0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058819cc40 con 0x7f05881031a0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f057effd640 1 -- 172.21.15.120:0/2997321145 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580010690 con 0x7f05881031a0 2024-09-10T23:15:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.549+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05881a3310 con 0x7f05881031a0 2024-09-10T23:15:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.554+0000 7f0592dcd740 1 Processor -- start 2024-09-10T23:15:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f057effd640 1 -- 172.21.15.120:0/2997321145 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0580005020 con 0x7f05881031a0 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 -- start start 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d07780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563402c746f0 con 0x563402d07dd0 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563402c63b20 con 0x563402d02a60 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f0592dcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563402c67930 con 0x563402d05390 2024-09-10T23:15:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.555+0000 7f058d593640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.556+0000 7f058d593640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.556+0000 7f058d593640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44230/0 (socket says 172.21.15.120:44230) 2024-09-10T23:15:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.556+0000 7f058d593640 1 -- 172.21.15.120:0/585832661 learned_addr learned my addr 172.21.15.120:0/585832661 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.556+0000 7f058dd94640 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d07780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f057effd640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 0x7f05740b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f057effd640 1 -- 172.21.15.120:0/2997321145 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f05800e60a0 con 0x7f05881031a0 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f058d593640 1 -- 172.21.15.120:0/585832661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 msgr2=0x563402d07780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f058d593640 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d07780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f058d593640 1 -- 172.21.15.120:0/585832661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 msgr2=0x563402d0a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f058d593640 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d0a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.557+0000 7f058d593640 1 -- 172.21.15.120:0/585832661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563402c29530 con 0x563402d02a60 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.558+0000 7f058d593640 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0584005740 tx=0x7f0584000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1e8a6895ac178 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.559+0000 7f058dd94640 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d07780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:15:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.559+0000 7f0573fff640 1 -- 172.21.15.120:0/585832661 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584050070 con 0x563402d02a60 2024-09-10T23:15:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.559+0000 7f0573fff640 1 -- 172.21.15.120:0/585832661 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0584003830 con 0x563402d02a60 2024-09-10T23:15:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.559+0000 7f0573fff640 1 -- 172.21.15.120:0/585832661 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584035040 con 0x563402d02a60 2024-09-10T23:15:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.560+0000 7f058e595640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 0x7f05740b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- 172.21.15.120:0/585832661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 msgr2=0x563402d04e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0584005740 tx=0x7f0584000bc0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f058e595640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 0x7f05740b78a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f05800ac530 tx=0x7f0580000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- 172.21.15.120:0/585832661 shutdown_connections 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d0a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d07780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 --2- 172.21.15.120:0/585832661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d04e50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- 172.21.15.120:0/585832661 >> 172.21.15.120:0/585832661 conn(0x563402cfdbb0 msgr2=0x563402cfe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- 172.21.15.120:0/585832661 shutdown_connections 2024-09-10T23:15:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- 172.21.15.120:0/585832661 wait complete. 2024-09-10T23:15:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.561+0000 7f0592dcd740 1 -- start start 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.562+0000 7f0592dcd740 1 Processor -- start 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.562+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d07dd0 0x563402d0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563402b9bab0 con 0x563402d02a60 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x563402c09730 con 0x563402d05390 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563402d0ad50 con 0x563402d07dd0 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f0592dcd740 10 client.0 _ll_register_callbacks cb 0x563402c5d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f058dd94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f058dd94640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44244/0 (socket says 172.21.15.120:44244) 2024-09-10T23:15:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.563+0000 7f058dd94640 1 -- 172.21.15.120:0/2696423708 learned_addr learned my addr 172.21.15.120:0/2696423708 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 -- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d07dd0 msgr2=0x563402d0d3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:15:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d07dd0 0x563402d0d3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 -- 172.21.15.120:0/2696423708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 msgr2=0x563402d0c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 -- 172.21.15.120:0/2696423708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634029c5790 con 0x563402d05390 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f058dd94640 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0578010470 tx=0x7f0578010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b0347bd7ce7219 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f05717fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578018070 con 0x563402d05390 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.564+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f0578014800 con 0x563402d05390 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.565+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0584003830 con 0x563402d05390 2024-09-10T23:15:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.565+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0578022050 con 0x563402d05390 2024-09-10T23:15:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 10 client.28934 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563402a3e620 con 0x563402d05390 2024-09-10T23:15:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f057801d440 con 0x563402d05390 2024-09-10T23:15:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f057801c3b0 con 0x563402d05390 2024-09-10T23:15:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 10 client.28934 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 10 client.28934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 20 client.28934 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.567+0000 7f0592dcd740 4 client.28934 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.568+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0584003830 con 0x563402d17b90 2024-09-10T23:15:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.568+0000 7f058d593640 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.573+0000 7f058d593640 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d1a010 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1329e511fa54c283 server_cookie=116b6606fe41668b in_seq=0 out_seq=0 2024-09-10T23:15:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.573+0000 7f05717fa640 10 client.28934 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.589+0000 7f05717fa640 1 -- 172.21.15.120:0/2696423708 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0584003830 con 0x563402d17b90 2024-09-10T23:15:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.589+0000 7f05717fa640 10 client.28934 handle_command_reply: tid=0 2024-09-10T23:15:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.589+0000 7f0592dcd740 1 client.28934 shutdown 2024-09-10T23:15:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.589+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 msgr2=0x563402d0c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.589+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0578010470 tx=0x7f0578010930 comp rx=0 tx=0).stop 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 shutdown_connections 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d1a010 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f05717fa640 0 client.28934 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d07dd0 0x563402d0d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d05390 0x563402d0c820 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2696423708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.590+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 >> 172.21.15.120:0/2696423708 conn(0x563402cfdbb0 msgr2=0x563402cfcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.591+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 shutdown_connections 2024-09-10T23:15:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.591+0000 7f0592dcd740 1 -- 172.21.15.120:0/2696423708 wait complete. 2024-09-10T23:15:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.591+0000 7f0592dcd740 20 client.28934 trim_cache size 0 max 16384 2024-09-10T23:15:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.619+0000 7f0592dcd740 1 Processor -- start 2024-09-10T23:15:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f0592dcd740 1 -- start start 2024-09-10T23:15:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402cfe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f058d593640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f058d593640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58410/0 (socket says 172.21.15.120:58410) 2024-09-10T23:15:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.620+0000 7f058d593640 1 -- 172.21.15.120:0/2307400297 learned_addr learned my addr 172.21.15.120:0/2307400297 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d07dd0 0x563402cfec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f058dd94640 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402cfe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00c330 con 0x563402d02a60 2024-09-10T23:15:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00ebb0 con 0x563402d07dd0 2024-09-10T23:15:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00c9a0 con 0x563402d05390 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.621+0000 7f058d593640 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d07dd0 0x563402cfec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 -- 172.21.15.120:0/2307400297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 msgr2=0x563402cfe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402cfe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 -- 172.21.15.120:0/2307400297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d07dd0 msgr2=0x563402cfec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d07dd0 0x563402cfec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 -- 172.21.15.120:0/2307400297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563402d051c0 con 0x563402d02a60 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f058d593640 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0584003750 tx=0x7f0584005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2ddf332a7b52e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.622+0000 7f05737fe640 1 -- 172.21.15.120:0/2307400297 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584050040 con 0x563402d02a60 2024-09-10T23:15:02.624 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:02 smithi176 ceph-mon[34982]: pgmap v1024: 97 pgs: 97 active+clean; 252 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-09-10T23:15:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.623+0000 7f05737fe640 1 -- 172.21.15.120:0/2307400297 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058404be00 con 0x563402d02a60 2024-09-10T23:15:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.623+0000 7f05737fe640 1 -- 172.21.15.120:0/2307400297 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584055f90 con 0x563402d02a60 2024-09-10T23:15:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 msgr2=0x563402d0d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0584003750 tx=0x7f0584005fd0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 shutdown_connections 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d07dd0 0x563402cfec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402cfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2307400297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d02a60 0x563402d0d3d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 >> 172.21.15.120:0/2307400297 conn(0x563402cfdbb0 msgr2=0x563402d01fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 shutdown_connections 2024-09-10T23:15:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.624+0000 7f0592dcd740 1 -- 172.21.15.120:0/2307400297 wait complete. 2024-09-10T23:15:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 -- start start 2024-09-10T23:15:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 Processor -- start 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d13c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d82b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00d070 con 0x563402d07dd0 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.625+0000 7f0592dcd740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00e4e0 con 0x563402d02a60 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f0592dcd740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00d740 con 0x563402d05390 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f0592dcd740 10 client.0 _ll_register_callbacks cb 0x7f0578009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f058d593640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f058d593640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:44254/0 (socket says 172.21.15.120:44254) 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f058d593640 1 -- 172.21.15.120:0/2869875063 learned_addr learned my addr 172.21.15.120:0/2869875063 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f058d593640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d82b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.626+0000 7f058dd94640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d13c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 -- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 msgr2=0x563402d13c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d13c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 -- 172.21.15.120:0/2869875063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 msgr2=0x563402d82b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d82b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 -- 172.21.15.120:0/2869875063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563402d941e0 con 0x563402d02a60 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f058d593640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f058402a930 tx=0x7f058402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e549be730632e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.627+0000 7f05717fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.628+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584050070 con 0x563402d02a60 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.628+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f058404b950 con 0x563402d02a60 2024-09-10T23:15:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.628+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f058404be00 con 0x563402d02a60 2024-09-10T23:15:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.630+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0584035040 con 0x563402d02a60 2024-09-10T23:15:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.630+0000 7f0592dcd740 10 client.28940 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.630+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563402d954f0 con 0x563402d02a60 2024-09-10T23:15:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058404be00 con 0x563402d02a60 2024-09-10T23:15:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f0584056070 con 0x563402d02a60 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 10 client.28940 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 10 client.28940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 20 client.28940 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d97a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 4 client.28940 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563402c76640 con 0x563402d17b90 2024-09-10T23:15:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.631+0000 7f058dd94640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.642+0000 7f058dd94640 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d97a20 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2e98ec5dbb9533 server_cookie=2d307e97e0cd1197 in_seq=0 out_seq=0 2024-09-10T23:15:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:02.643+0000 7f05717fa640 10 client.28940 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:02 smithi120 ceph-mon[31247]: pgmap v1024: 97 pgs: 97 active+clean; 252 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-09-10T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:02 smithi153 ceph-mon[34951]: pgmap v1024: 97 pgs: 97 active+clean; 252 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-09-10T23:15:02.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 241800 21.60 MB/sec execute 477 sec latency 313.247 ms 2024-09-10T23:15:03.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.419+0000 7f05717fa640 1 -- 172.21.15.120:0/2869875063 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563402c76640 con 0x563402d17b90 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.419+0000 7f05717fa640 10 client.28940 handle_command_reply: tid=0 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.419+0000 7f0592dcd740 1 client.28940 shutdown 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.419+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 msgr2=0x563402d136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.419+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f058402a930 tx=0x7f058402a410 comp rx=0 tx=0).stop 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 shutdown_connections 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x563402d17b90 0x563402d97a20 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563402d07dd0 0x563402d82b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563402d05390 0x563402d13c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 --2- 172.21.15.120:0/2869875063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563402d02a60 0x563402d136a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 >> 172.21.15.120:0/2869875063 conn(0x563402cfdbb0 msgr2=0x563402d0c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f05717fa640 0 client.28940 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 shutdown_connections 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 1 -- 172.21.15.120:0/2869875063 wait complete. 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.420+0000 7f0592dcd740 20 client.28940 trim_cache size 0 max 16384 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:15:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:15:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:15:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:15:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 msgr2=0x7f05740b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 0x7f05740b78a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f05800ac530 tx=0x7f0580000f40 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 msgr2=0x7f058819c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f058000eb10 tx=0x7f058000c2d0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 shutdown_connections 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.422+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f05740b53e0 0x7f05740b78a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05881031a0 0x7f058819c090 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05881027e0 0x7f058819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 --2- 172.21.15.120:0/2997321145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0588101690 0x7f058819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 >> 172.21.15.120:0/2997321145 conn(0x7f05880fcf30 msgr2=0x7f05880fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 shutdown_connections 2024-09-10T23:15:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:03.423+0000 7f058ed96640 1 -- 172.21.15.120:0/2997321145 wait complete. 2024-09-10T23:15:03.663 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:03.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 241955 21.56 MB/sec execute 478 sec latency 410.265 ms 2024-09-10T23:15:04.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 -- 172.21.15.120:0/1348449708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580a50f0 msgr2=0x7ffa580b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 --2- 172.21.15.120:0/1348449708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580a50f0 0x7ffa580b6fb0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009900 tx=0x7ffa5c02a480 comp rx=0 tx=0).stop 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 -- 172.21.15.120:0/1348449708 shutdown_connections 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 --2- 172.21.15.120:0/1348449708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580b74f0 0x7ffa580b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 --2- 172.21.15.120:0/1348449708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580a50f0 0x7ffa580b6fb0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 --2- 172.21.15.120:0/1348449708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a47d0 0x7ffa580a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.212+0000 7ffa67825640 1 -- 172.21.15.120:0/1348449708 >> 172.21.15.120:0/1348449708 conn(0x7ffa5801a080 msgr2=0x7ffa5801c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.213+0000 7ffa67825640 1 -- 172.21.15.120:0/1348449708 shutdown_connections 2024-09-10T23:15:04.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.213+0000 7ffa67825640 1 -- 172.21.15.120:0/1348449708 wait complete. 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.213+0000 7ffa67825640 1 Processor -- start 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 -- start start 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a50f0 0x7ffa58143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 0x7ffa58148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa580bceb0 con 0x7ffa580b74f0 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa580bcd30 con 0x7ffa580a47d0 2024-09-10T23:15:04.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa67825640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffa580bd030 con 0x7ffa580a50f0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46428/0 (socket says 172.21.15.120:46428) 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa655c1640 1 -- 172.21.15.120:0/2693522783 learned_addr learned my addr 172.21.15.120:0/2693522783 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 0x7ffa58148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a50f0 0x7ffa58143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa655c1640 1 -- 172.21.15.120:0/2693522783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a50f0 msgr2=0x7ffa58143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.214+0000 7ffa655c1640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a50f0 0x7ffa58143980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa655c1640 1 -- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 msgr2=0x7ffa58148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa655c1640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 0x7ffa58148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa655c1640 1 -- 172.21.15.120:0/2693522783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa581494f0 con 0x7ffa580a47d0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 0x7ffa58148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa655c1640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa600511b0 tx=0x7ffa60073af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa567fc640 1 -- 172.21.15.120:0/2693522783 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6006cb80 con 0x7ffa580a47d0 2024-09-10T23:15:04.218 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa581497e0 con 0x7ffa580a47d0 2024-09-10T23:15:04.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5814fea0 con 0x7ffa580a47d0 2024-09-10T23:15:04.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa567fc640 1 -- 172.21.15.120:0/2693522783 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa60078070 con 0x7ffa580a47d0 2024-09-10T23:15:04.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.215+0000 7ffa567fc640 1 -- 172.21.15.120:0/2693522783 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa600746a0 con 0x7ffa580a47d0 2024-09-10T23:15:04.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa567fc640 1 -- 172.21.15.120:0/2693522783 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ffa60085070 con 0x7ffa580a47d0 2024-09-10T23:15:04.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 Processor -- start 2024-09-10T23:15:04.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 -- start start 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e956518e50 con 0x55e9565aeb30 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e956508280 con 0x55e9565a2ba0 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.219+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e95650c090 con 0x55e9565a2280 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.220+0000 7ffa567fc640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.220+0000 7ffa65dc2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.220+0000 7ffa65dc2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40488/0 (socket says 172.21.15.120:40488) 2024-09-10T23:15:04.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.220+0000 7ffa567fc640 1 -- 172.21.15.120:0/2693522783 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ffa6006b030 con 0x7ffa580a47d0 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.220+0000 7ffa65dc2640 1 -- 172.21.15.120:0/3285332276 learned_addr learned my addr 172.21.15.120:0/3285332276 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 -- 172.21.15.120:0/3285332276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 msgr2=0x55e9565ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 -- 172.21.15.120:0/3285332276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 msgr2=0x55e9565b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 -- 172.21.15.120:0/3285332276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e956266650 con 0x55e9565a2280 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.221+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cd00 tx=0x7ffa50010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b8ca8e4f11dcef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.222+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffa5001f000 tx=0x7ffa50020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.222+0000 7ffa4b7fe640 1 -- 172.21.15.120:0/3285332276 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50022070 con 0x55e9565a2280 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.222+0000 7ffa4b7fe640 1 -- 172.21.15.120:0/3285332276 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa50018070 con 0x55e9565a2280 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.222+0000 7ffa4b7fe640 1 -- 172.21.15.120:0/3285332276 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001cc20 con 0x55e9565a2280 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- 172.21.15.120:0/3285332276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 msgr2=0x55e9565a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cd00 tx=0x7ffa50010740 comp rx=0 tx=0).stop 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- 172.21.15.120:0/3285332276 shutdown_connections 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/3285332276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565a2660 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- 172.21.15.120:0/3285332276 >> 172.21.15.120:0/3285332276 conn(0x55e9565a00b0 msgr2=0x55e9565a1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- 172.21.15.120:0/3285332276 shutdown_connections 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- 172.21.15.120:0/3285332276 wait complete. 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 -- start start 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.223+0000 7ffa69bfa740 1 Processor -- start 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e956369700 con 0x55e9565aeb30 2024-09-10T23:15:04.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9564a3110 con 0x55e9565a2ba0 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e9565b17b0 con 0x55e9565a2280 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa69bfa740 10 client.0 _ll_register_callbacks cb 0x55e9564bc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40500/0 (socket says 172.21.15.120:40500) 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 -- 172.21.15.120:0/1412842089 learned_addr learned my addr 172.21.15.120:0/1412842089 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:04.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 -- 172.21.15.120:0/1412842089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 msgr2=0x55e9565ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 -- 172.21.15.120:0/1412842089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 msgr2=0x55e9565abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa655c1640 1 -- 172.21.15.120:0/1412842089 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9564d0c40 con 0x55e9565a2280 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.224+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa655c1640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa600669a0 tx=0x7ffa6007de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b78695c4ef171f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6007bdf0 con 0x55e9565a2280 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ffa60110cf0 con 0x55e9565a2280 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa50018070 con 0x55e9565a2280 2024-09-10T23:15:04.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa60112190 con 0x55e9565a2280 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 10 client.29221 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9565bd110 con 0x55e9565a2280 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa50018070 con 0x55e9565a2280 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ffa60110760 con 0x55e9565a2280 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 10 client.29221 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 10 client.29221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 20 client.29221 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9565bea00 0x55e9565c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 4 client.29221 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.225+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95651ada0 con 0x55e9565bea00 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.226+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9565bea00 0x55e9565c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.226+0000 7ffa48ff9640 10 client.29221 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.226+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9565bea00 0x55e9565c0e20 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8bde3f2928bc649 server_cookie=e4ae52f7cedc5ebf in_seq=0 out_seq=0 2024-09-10T23:15:04.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.227+0000 7ffa48ff9640 1 -- 172.21.15.120:0/1412842089 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e95651ada0 con 0x55e9565bea00 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.227+0000 7ffa48ff9640 10 client.29221 handle_command_reply: tid=0 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.227+0000 7ffa69bfa740 1 client.29221 shutdown 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 msgr2=0x55e9565ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa600669a0 tx=0x7ffa6007de40 comp rx=0 tx=0).stop 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 shutdown_connections 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9565bea00 0x55e9565c0e20 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2ba0 0x55e9565ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/1412842089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565ae3c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa48ff9640 0 client.29221 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 >> 172.21.15.120:0/1412842089 conn(0x55e9565a00b0 msgr2=0x55e9565a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 shutdown_connections 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 1 -- 172.21.15.120:0/1412842089 wait complete. 2024-09-10T23:15:04.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.228+0000 7ffa69bfa740 20 client.29221 trim_cache size 0 max 16384 2024-09-10T23:15:04.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.249+0000 7ffa69bfa740 1 Processor -- start 2024-09-10T23:15:04.250 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.249+0000 7ffa69bfa740 1 -- start start 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 0x55e9565ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2ba0 0x55e9565a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400c380 con 0x55e9565aeb30 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400ec00 con 0x55e9565a2280 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400c9f0 con 0x55e9565a2ba0 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa64dc0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2ba0 0x55e9565a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37762/0 (socket says 172.21.15.120:37762) 2024-09-10T23:15:04.251 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 -- 172.21.15.120:0/673226650 learned_addr learned my addr 172.21.15.120:0/673226650 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa655c1640 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 0x55e9565ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 -- 172.21.15.120:0/673226650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2ba0 msgr2=0x55e9565a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2ba0 0x55e9565a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 -- 172.21.15.120:0/673226650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 msgr2=0x55e9565ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 0x55e9565ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 -- 172.21.15.120:0/673226650 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9565c1480 con 0x55e9565aeb30 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa655c1640 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 0x55e9565ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:04.252 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.250+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c4d0 tx=0x7ffa5000cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9840530aedb4b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.251+0000 7ffa4affd640 1 -- 172.21.15.120:0/673226650 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50045070 con 0x55e9565aeb30 2024-09-10T23:15:04.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.251+0000 7ffa4affd640 1 -- 172.21.15.120:0/673226650 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa50018040 con 0x55e9565aeb30 2024-09-10T23:15:04.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.251+0000 7ffa4affd640 1 -- 172.21.15.120:0/673226650 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50022040 con 0x55e9565aeb30 2024-09-10T23:15:04.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- 172.21.15.120:0/673226650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 msgr2=0x55e9565a8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.254 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c4d0 tx=0x7ffa5000cdf0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- 172.21.15.120:0/673226650 shutdown_connections 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565aeb30 0x55e9565a8f60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2ba0 0x55e9565a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/673226650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565a2280 0x55e9565ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- 172.21.15.120:0/673226650 >> 172.21.15.120:0/673226650 conn(0x55e9565a00b0 msgr2=0x55e9565ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- 172.21.15.120:0/673226650 shutdown_connections 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- 172.21.15.120:0/673226650 wait complete. 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.252+0000 7ffa69bfa740 1 -- start start 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 Processor -- start 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 0x55e9565b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565aeb30 0x55e9565b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa655c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40522/0 (socket says 172.21.15.120:40522) 2024-09-10T23:15:04.255 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400d0c0 con 0x55e9565a2ba0 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa655c1640 1 -- 172.21.15.120:0/59973945 learned_addr learned my addr 172.21.15.120:0/59973945 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400e530 con 0x55e9565aeb30 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4400d790 con 0x55e9565a2280 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 0x55e9565b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.253+0000 7ffa69bfa740 10 client.0 _ll_register_callbacks cb 0x7ffa601113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:04.256 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa65dc2640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565aeb30 0x55e9565b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 -- 172.21.15.120:0/59973945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565aeb30 msgr2=0x55e9565b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.257 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565aeb30 0x55e9565b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 -- 172.21.15.120:0/59973945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 msgr2=0x55e9565b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 0x55e9565b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9565b4230 con 0x55e9565a2280 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 0x55e9565b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa655c1640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa6010ed00 tx=0x7ffa60110f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf04a428976e3a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.254+0000 7ffa48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60111a90 con 0x55e9565a2280 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ffa60117700 con 0x55e9565a2280 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa50018040 con 0x55e9565a2280 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa60116570 con 0x55e9565a2280 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 10 client.29227 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:04.258 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e95662c7f0 con 0x55e9565a2280 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6011b020 con 0x55e9565a2280 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7ffa600d63c0 con 0x55e9565a2280 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 10 client.29227 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 10 client.29227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 20 client.29227 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e95662ce60 0x55e95662f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 4 client.29227 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.255+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ffa50018040 con 0x55e95662ce60 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.256+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e95662ce60 0x55e95662f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.256+0000 7ffa48ff9640 10 client.29227 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:04.259 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.256+0000 7ffa64dc0640 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e95662ce60 0x55e95662f280 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb4701975873e15 server_cookie=572c38b9d3d0f3c8 in_seq=0 out_seq=0 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.261+0000 7ffa48ff9640 1 -- 172.21.15.120:0/59973945 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10598 (crc 0 0 0) 0x7ffa50018040 con 0x55e95662ce60 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.261+0000 7ffa48ff9640 10 client.29227 handle_command_reply: tid=0 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.261+0000 7ffa69bfa740 1 client.29227 shutdown 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.261+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 msgr2=0x55e9565b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.261+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa6010ed00 tx=0x7ffa60110f00 comp rx=0 tx=0).stop 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 shutdown_connections 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e95662ce60 0x55e95662f280 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.263 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9565aeb30 0x55e9565b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9565a2ba0 0x55e9565b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 --2- 172.21.15.120:0/59973945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9565a2280 0x55e9565b7120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 >> 172.21.15.120:0/59973945 conn(0x55e9565a00b0 msgr2=0x55e9565ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa48ff9640 0 client.29227 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 shutdown_connections 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 1 -- 172.21.15.120:0/59973945 wait complete. 2024-09-10T23:15:04.264 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.262+0000 7ffa69bfa740 20 client.29227 trim_cache size 0 max 16384 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.265 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.266 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:15:04.267 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.268 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.269 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.270 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:15:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:15:04.274 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:15:04.275 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.276 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.277 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.278 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:15:04.279 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.280 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.281 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.282 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:15:04.283 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.284 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.285 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:04.286 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 msgr2=0x7ffa4c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffa5001f000 tx=0x7ffa50020000 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 msgr2=0x7ffa58143440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa600511b0 tx=0x7ffa60073af0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 shutdown_connections 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffa4c0b52e0 0x7ffa4c0b77a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa580b74f0 0x7ffa58148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa580a50f0 0x7ffa58143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 --2- 172.21.15.120:0/2693522783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa580a47d0 0x7ffa58143440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 >> 172.21.15.120:0/2693522783 conn(0x7ffa5801a080 msgr2=0x7ffa580b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 shutdown_connections 2024-09-10T23:15:04.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:04.263+0000 7ffa67825640 1 -- 172.21.15.120:0/2693522783 wait complete. 2024-09-10T23:15:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:04 smithi120 ceph-mon[31247]: pgmap v1025: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-09-10T23:15:04.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:04 smithi153 ceph-mon[34951]: pgmap v1025: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-09-10T23:15:04.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:04 smithi176 ceph-mon[34982]: pgmap v1025: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-09-10T23:15:04.891 INFO:tasks.workunit.client.0.smithi120.stdout: 10 242322 21.55 MB/sec execute 479 sec latency 103.409 ms 2024-09-10T23:15:05.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:15:05.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:15:05.443 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:05.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 242732 21.54 MB/sec execute 480 sec latency 119.927 ms 2024-09-10T23:15:06.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 -- 172.21.15.120:0/751114337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7681076a0 msgr2=0x7fd768107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 --2- 172.21.15.120:0/751114337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7681076a0 0x7fd768107ae0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd750009900 tx=0x7fd75002a410 comp rx=0 tx=0).stop 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 -- 172.21.15.120:0/751114337 shutdown_connections 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 --2- 172.21.15.120:0/751114337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd768108020 0x7fd76810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 --2- 172.21.15.120:0/751114337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd7681076a0 0x7fd768107ae0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 --2- 172.21.15.120:0/751114337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 0x7fd768102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 -- 172.21.15.120:0/751114337 >> 172.21.15.120:0/751114337 conn(0x7fd7680fd700 msgr2=0x7fd7680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 -- 172.21.15.120:0/751114337 shutdown_connections 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 -- 172.21.15.120:0/751114337 wait complete. 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.073+0000 7fd76d80d640 1 Processor -- start 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 -- start start 2024-09-10T23:15:06.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 0x7fd76819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7681076a0 0x7fd76819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd768111ca0 con 0x7fd768108020 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd768111b20 con 0x7fd7681076a0 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd76d80d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd768111e20 con 0x7fd768101c90 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37800/0 (socket says 172.21.15.120:37800) 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd766ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 0x7fd76819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 -- 172.21.15.120:0/1795655296 learned_addr learned my addr 172.21.15.120:0/1795655296 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7667fc640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7681076a0 0x7fd76819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 -- 172.21.15.120:0/1795655296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 msgr2=0x7fd76819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 0x7fd76819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 -- 172.21.15.120:0/1795655296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7681076a0 msgr2=0x7fd76819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7681076a0 0x7fd76819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 -- 172.21.15.120:0/1795655296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7681a1590 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.074+0000 7fd7677fe640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd760009fd0 tx=0x7fd76000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.075+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760017070 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.075+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7681a1880 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.075+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760003e20 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.075+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760020060 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.075+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a7f40 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.076+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd768102070 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.077+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd760023060 con 0x7fd768108020 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.078+0000 7fd76c80b640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 0x7fd7540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.078+0000 7fd766ffd640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 0x7fd7540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:06.091 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.078+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd7600e5a30 con 0x7fd768108020 2024-09-10T23:15:06.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.078+0000 7fd766ffd640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 0x7fd7540b77a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd758005fd0 tx=0x7fd75800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:06.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.081+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7600ac2d0 con 0x7fd768108020 2024-09-10T23:15:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.323+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7681a82d0 con 0x7fd768108020 2024-09-10T23:15:06.324 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.324+0000 7fd76c80b640 1 -- 172.21.15.120:0/1795655296 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fd7600b0d00 con 0x7fd768108020 2024-09-10T23:15:06.325 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:15:06.325 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:15:06.325 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 msgr2=0x7fd7540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 0x7fd7540b77a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd758005fd0 tx=0x7fd75800b390 comp rx=0 tx=0).stop 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 msgr2=0x7fd7681a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd760009fd0 tx=0x7fd76000b260 comp rx=0 tx=0).stop 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 shutdown_connections 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd7540b52e0 0x7fd7540b77a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.327 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd768108020 0x7fd7681a0dc0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7681076a0 0x7fd76819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 --2- 172.21.15.120:0/1795655296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd768101c90 0x7fd76819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 >> 172.21.15.120:0/1795655296 conn(0x7fd7680fd700 msgr2=0x7fd76806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 shutdown_connections 2024-09-10T23:15:06.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:06.326+0000 7fd76d80d640 1 -- 172.21.15.120:0/1795655296 wait complete. 2024-09-10T23:15:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:06 smithi120 ceph-mon[31247]: pgmap v1026: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-09-10T23:15:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1795655296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:06 smithi153 ceph-mon[34951]: pgmap v1026: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-09-10T23:15:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1795655296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:06.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:06 smithi176 ceph-mon[34982]: pgmap v1026: 97 pgs: 97 active+clean; 249 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-09-10T23:15:06.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1795655296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:06.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 243242 21.55 MB/sec execute 481 sec latency 135.420 ms 2024-09-10T23:15:07.058 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:15:07.359 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:07.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 243776 21.56 MB/sec execute 482 sec latency 148.323 ms 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- 172.21.15.120:0/680535366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d10104600 msgr2=0x7f1d1010caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 --2- 172.21.15.120:0/680535366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d10104600 0x7f1d1010caf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009900 tx=0x7f1d0402a410 comp rx=0 tx=0).stop 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- 172.21.15.120:0/680535366 shutdown_connections 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 --2- 172.21.15.120:0/680535366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d1010d140 0x7f1d1010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 --2- 172.21.15.120:0/680535366 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d10104600 0x7f1d1010caf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 --2- 172.21.15.120:0/680535366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d101040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- 172.21.15.120:0/680535366 >> 172.21.15.120:0/680535366 conn(0x7f1d100fd740 msgr2=0x7f1d100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- 172.21.15.120:0/680535366 shutdown_connections 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- 172.21.15.120:0/680535366 wait complete. 2024-09-10T23:15:08.042 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 Processor -- start 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.978+0000 7f1d16677640 1 -- start start 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d10104600 0x7f1d101998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d1010d140 0x7f1d1019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10112db0 con 0x7f1d1010d140 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10112c30 con 0x7f1d10103ce0 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d16677640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10112f30 con 0x7f1d10104600 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d1010d140 0x7f1d1019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46520/0 (socket says 172.21.15.120:46520) 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 -- 172.21.15.120:0/1484119433 learned_addr learned my addr 172.21.15.120:0/1484119433 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d10104600 0x7f1d101998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.043 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 -- 172.21.15.120:0/1484119433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d10104600 msgr2=0x7f1d101998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d10104600 0x7f1d101998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 -- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d1010d140 msgr2=0x7f1d1019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d1010d140 0x7f1d1019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.979+0000 7f1d0ffff640 1 -- 172.21.15.120:0/1484119433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1019f410 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9f0 tx=0x7f1d0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1484119433 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000cd80 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1019f700 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1484119433 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d000108e0 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1484119433 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00018980 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.980+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d101a5dc0 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 Processor -- start 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 -- start start 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1484119433 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1d00010430 con 0x7f1d10103ce0 2024-09-10T23:15:08.044 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df9768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df982d80 0x55e5df985170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5df8ecea0 con 0x55e5df976df0 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e5df8dc2d0 con 0x55e5df982d80 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5df8e00e0 con 0x55e5df9764d0 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37822/0 (socket says 172.21.15.120:37822) 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.983+0000 7f1d0ffff640 1 -- 172.21.15.120:0/1560367546 learned_addr learned my addr 172.21.15.120:0/1560367546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df9768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df982d80 0x55e5df985170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.045 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 -- 172.21.15.120:0/1560367546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df982d80 msgr2=0x55e5df985170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df982d80 0x55e5df985170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 -- 172.21.15.120:0/1560367546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 msgr2=0x55e5df982840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 -- 172.21.15.120:0/1560367546 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5df63a650 con 0x55e5df9764d0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d0d7fa640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 0x7f1cf80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df9768b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009f00 tx=0x7f1d08011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c74c7513bd44f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 0x7f1cf80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1cf67fc640 1 -- 172.21.15.120:0/1560367546 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004d20 con 0x55e5df9764d0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1d0d7fa640 1 -- 172.21.15.120:0/1484119433 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1d000e5ef0 con 0x7f1d10103ce0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1cf67fc640 1 -- 172.21.15.120:0/1560367546 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08004050 con 0x55e5df9764d0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.984+0000 7f1cf67fc640 1 -- 172.21.15.120:0/1560367546 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08005220 con 0x55e5df9764d0 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1560367546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 msgr2=0x55e5df9768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.046 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d0ffff640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 0x7f1cf80b77a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ac720 tx=0x7f1d000e99d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df9768b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009f00 tx=0x7f1d08011810 comp rx=0 tx=0).stop 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1560367546 shutdown_connections 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df982d80 0x55e5df985170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df976df0 0x55e5df982840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1560367546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df9768b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1560367546 >> 172.21.15.120:0/1560367546 conn(0x55e5df974300 msgr2=0x55e5df9754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1560367546 shutdown_connections 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.985+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1560367546 wait complete. 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 -- start start 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 Processor -- start 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df982630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df97fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.047 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e5df73d700 con 0x55e5df982d80 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e5df877160 con 0x55e5df976df0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e5df985a00 con 0x55e5df9764d0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d18a4c740 10 client.0 _ll_register_callbacks cb 0x55e5df8909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37830/0 (socket says 172.21.15.120:37830) 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d14c14640 1 -- 172.21.15.120:0/797327509 learned_addr learned my addr 172.21.15.120:0/797327509 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.986+0000 7f1d14c14640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df982630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df97fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 -- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 msgr2=0x55e5df982630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df982630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 -- 172.21.15.120:0/797327509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 msgr2=0x55e5df97fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df97fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 -- 172.21.15.120:0/797327509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5df8a4c90 con 0x55e5df982d80 2024-09-10T23:15:08.048 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d14c14640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009b00 tx=0x7f1d08009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513288e454a9d970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1cef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0802d060 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1d08004a40 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d08004050 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d08004c40 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d18a4c740 10 client.19815 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5df9912f0 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.987+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08004050 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1d080348b0 con 0x55e5df982d80 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 10 client.19815 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 20 client.19815 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5df994a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 4 client.19815 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:08.049 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5df8eedf0 con 0x55e5df992640 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5df994a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1cef7fe640 10 client.19815 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.988+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5df994a30 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3460754a8012101 server_cookie=6adca8d6ce30475c in_seq=0 out_seq=0 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1cef7fe640 1 -- 172.21.15.120:0/797327509 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5df8eedf0 con 0x55e5df992640 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1cef7fe640 10 client.19815 handle_command_reply: tid=0 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 client.19815 shutdown 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 msgr2=0x55e5df980040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009b00 tx=0x7f1d08009a50 comp rx=0 tx=0).stop 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 shutdown_connections 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5df994a30 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df980040 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1cef7fe640 0 client.19815 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df97fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/797327509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df982630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.050 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 >> 172.21.15.120:0/797327509 conn(0x55e5df974300 msgr2=0x55e5df979ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 shutdown_connections 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 1 -- 172.21.15.120:0/797327509 wait complete. 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:07.991+0000 7f1d18a4c740 20 client.19815 trim_cache size 0 max 16384 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 Processor -- start 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 -- start start 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df984170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df982d80 0x55e5df9846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101ac380 con 0x55e5df9764d0 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101aec00 con 0x55e5df976df0 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101ac9f0 con 0x55e5df982d80 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.005+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37842/0 (socket says 172.21.15.120:37842) 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 -- 172.21.15.120:0/1764582195 learned_addr learned my addr 172.21.15.120:0/1764582195 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:08.051 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df982d80 0x55e5df9846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 -- 172.21.15.120:0/1764582195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df982d80 msgr2=0x55e5df9846b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df982d80 0x55e5df9846b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 -- 172.21.15.120:0/1764582195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 msgr2=0x55e5df984170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df984170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 -- 172.21.15.120:0/1764582195 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5df976c20 con 0x55e5df9764d0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1d14c14640 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005d70 tx=0x7f1d08009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b343ade8739103c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1cf5ffb640 1 -- 172.21.15.120:0/1764582195 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08015020 con 0x55e5df9764d0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.006+0000 7f1cf5ffb640 1 -- 172.21.15.120:0/1764582195 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0803b030 con 0x55e5df9764d0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1cf5ffb640 1 -- 172.21.15.120:0/1764582195 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0802d040 con 0x55e5df9764d0 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1764582195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 msgr2=0x55e5df98ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005d70 tx=0x7f1d08009e20 comp rx=0 tx=0).stop 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1764582195 shutdown_connections 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df982d80 0x55e5df9846b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.052 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df984170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/1764582195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df9764d0 0x55e5df98ec30 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1764582195 >> 172.21.15.120:0/1764582195 conn(0x55e5df974300 msgr2=0x55e5df9818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1764582195 shutdown_connections 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.007+0000 7f1d18a4c740 1 -- 172.21.15.120:0/1764582195 wait complete. 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 -- start start 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 Processor -- start 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df98b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df985c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101ad0c0 con 0x55e5df982d80 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101ae530 con 0x55e5df976df0 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101ad790 con 0x55e5df9764d0 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d18a4c740 10 client.0 _ll_register_callbacks cb 0x7f1d000e98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:08.053 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.008+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df985c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df98b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40600/0 (socket says 172.21.15.120:40600) 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 -- 172.21.15.120:0/364481151 learned_addr learned my addr 172.21.15.120:0/364481151 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 -- 172.21.15.120:0/364481151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 msgr2=0x55e5df98b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df98b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 -- 172.21.15.120:0/364481151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 msgr2=0x55e5df985c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df985c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 -- 172.21.15.120:0/364481151 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5df986340 con 0x55e5df9764d0 2024-09-10T23:15:08.054 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d14c14640 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f40 tx=0x7f1d08009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f274dc71e3efb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1cef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08015030 con 0x55e5df9764d0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1d08028450 con 0x55e5df9764d0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.009+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0803b030 con 0x55e5df9764d0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d0802d070 con 0x55e5df9764d0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 10 client.29251 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5df9ffd50 con 0x55e5df9764d0 2024-09-10T23:15:08.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0803b030 con 0x55e5df9764d0 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1d080347c0 con 0x55e5df9764d0 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 10 client.29251 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 10 client.29251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 20 client.29251 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5dfa023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 4 client.29251 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5df8ea8a0 con 0x55e5df992640 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.010+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5dfa023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.011+0000 7f1d0f7fe640 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5dfa023f0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24b056ded799fdf server_cookie=ba5ac255cfb0fb34 in_seq=0 out_seq=0 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.011+0000 7f1cef7fe640 10 client.29251 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1cef7fe640 1 -- 172.21.15.120:0/364481151 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5df8ea8a0 con 0x55e5df992640 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1cef7fe640 10 client.29251 handle_command_reply: tid=0 2024-09-10T23:15:08.056 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 client.29251 shutdown 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 msgr2=0x55e5df98ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f40 tx=0x7f1d08009a50 comp rx=0 tx=0).stop 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 shutdown_connections 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e5df992640 0x55e5dfa023f0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e5df982d80 0x55e5df985c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5df976df0 0x55e5df98b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1cef7fe640 0 client.29251 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 --2- 172.21.15.120:0/364481151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e5df9764d0 0x55e5df98ae80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 >> 172.21.15.120:0/364481151 conn(0x55e5df974300 msgr2=0x55e5df972c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 shutdown_connections 2024-09-10T23:15:08.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 1 -- 172.21.15.120:0/364481151 wait complete. 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.015+0000 7f1d18a4c740 20 client.29251 trim_cache size 0 max 16384 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 msgr2=0x7f1cf80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 0x7f1cf80b77a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ac720 tx=0x7f1d000e99d0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 msgr2=0x7f1d10199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9f0 tx=0x7f1d0000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 shutdown_connections 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1cf80b52e0 0x7f1cf80b77a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d1010d140 0x7f1d1019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1d10104600 0x7f1d101998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 --2- 172.21.15.120:0/1484119433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d10103ce0 0x7f1d10199360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 >> 172.21.15.120:0/1484119433 conn(0x7f1d100fd740 msgr2=0x7f1d100fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 shutdown_connections 2024-09-10T23:15:08.058 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:08.016+0000 7f1d16677640 1 -- 172.21.15.120:0/1484119433 wait complete. 2024-09-10T23:15:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:08 smithi120 ceph-mon[31247]: pgmap v1027: 97 pgs: 97 active+clean; 238 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-09-10T23:15:08.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:08 smithi153 ceph-mon[34951]: pgmap v1027: 97 pgs: 97 active+clean; 238 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-09-10T23:15:08.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:08 smithi176 ceph-mon[34982]: pgmap v1027: 97 pgs: 97 active+clean; 238 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.3 MiB/s wr, 551 op/s 2024-09-10T23:15:08.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 244385 21.60 MB/sec execute 483 sec latency 48.618 ms 2024-09-10T23:15:09.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:15:09.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:15:09.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 244866 21.62 MB/sec execute 484 sec latency 63.451 ms 2024-09-10T23:15:10.117 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:10.552 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:10 smithi120 ceph-mon[31247]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-09-10T23:15:10.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.644+0000 7f961caad640 1 -- 172.21.15.120:0/1622959907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 msgr2=0x7f9618107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.669 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.644+0000 7f961caad640 1 --2- 172.21.15.120:0/1622959907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f9618107ae0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f960c009900 tx=0x7f960c02a410 comp rx=0 tx=0).stop 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 -- 172.21.15.120:0/1622959907 shutdown_connections 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 --2- 172.21.15.120:0/1622959907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618108020 0x7f961810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 --2- 172.21.15.120:0/1622959907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f9618107ae0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 --2- 172.21.15.120:0/1622959907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618101c90 0x7f9618102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 -- 172.21.15.120:0/1622959907 >> 172.21.15.120:0/1622959907 conn(0x7f96180fd700 msgr2=0x7f96180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 -- 172.21.15.120:0/1622959907 shutdown_connections 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 -- 172.21.15.120:0/1622959907 wait complete. 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.645+0000 7f961caad640 1 Processor -- start 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 -- start start 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 0x7f961810f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.670 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618108020 0x7f961810fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9618112dd0 con 0x7f96181076a0 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9618112c50 con 0x7f9618101c90 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961caad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9618112f50 con 0x7f9618108020 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f9615d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f9615d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37864/0 (socket says 172.21.15.120:37864) 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f9615d9b640 1 -- 172.21.15.120:0/2515863414 learned_addr learned my addr 172.21.15.120:0/2515863414 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f961659c640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 0x7f961810f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.646+0000 7f9616d9d640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618108020 0x7f961810fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 -- 172.21.15.120:0/2515863414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618108020 msgr2=0x7f961810fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618108020 0x7f961810fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 -- 172.21.15.120:0/2515863414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 msgr2=0x7f961810f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 0x7f961810f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.671 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 -- 172.21.15.120:0/2515863414 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96181a76a0 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f961659c640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 0x7f961810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f960c005940 tx=0x7f960c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f96077fe640 1 -- 172.21.15.120:0/2515863414 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c03f030 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181a7930 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f96077fe640 1 -- 172.21.15.120:0/2515863414 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960c0044c0 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f96077fe640 1 -- 172.21.15.120:0/2515863414 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c035070 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.647+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181a7e10 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.650+0000 7f96077fe640 1 -- 172.21.15.120:0/2515863414 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f960c03ba20 con 0x7f96181076a0 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 Processor -- start 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 -- start start 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96aee0 0x5625fa96b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.672 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa979b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5625fa8e1e60 con 0x5625fa977790 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5625fa8d1290 con 0x5625fa96b800 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961ee82740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625fa8d50a0 con 0x5625fa96aee0 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f9616d9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96aee0 0x5625fa96b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46592/0 (socket says 172.21.15.120:46592) 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f961659c640 1 -- 172.21.15.120:0/3089511301 learned_addr learned my addr 172.21.15.120:0/3089511301 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.651+0000 7f9615d9b640 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa979b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 -- 172.21.15.120:0/3089511301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96aee0 msgr2=0x5625fa96b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96aee0 0x5625fa96b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 -- 172.21.15.120:0/3089511301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 msgr2=0x5625fa979b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa979b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 -- 172.21.15.120:0/3089511301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fa62f930 con 0x5625fa96b800 2024-09-10T23:15:10.673 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f96077fe640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 0x7f96000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f96077fe640 1 -- 172.21.15.120:0/2515863414 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f960c106220 con 0x7f96181076a0 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f9616d9d640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 0x7f96000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f961659c640 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f960800f390 tx=0x7f960800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f04d705d96e61c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f9604ff9640 1 -- 172.21.15.120:0/3089511301 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608003b70 con 0x5625fa96b800 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f9604ff9640 1 -- 172.21.15.120:0/3089511301 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608003d10 con 0x5625fa96b800 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.652+0000 7f9604ff9640 1 -- 172.21.15.120:0/3089511301 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608011a00 con 0x5625fa96b800 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 -- 172.21.15.120:0/3089511301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 msgr2=0x5625fa977250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f9616d9d640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 0x7f96000b77a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f96181110c0 tx=0x7f9610005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f960800f390 tx=0x7f960800f850 comp rx=0 tx=0).stop 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 -- 172.21.15.120:0/3089511301 shutdown_connections 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa979b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96b800 0x5625fa977250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 --2- 172.21.15.120:0/3089511301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96aee0 0x5625fa96b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.674 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 -- 172.21.15.120:0/3089511301 >> 172.21.15.120:0/3089511301 conn(0x5625fa968d10 msgr2=0x5625fa969eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 -- 172.21.15.120:0/3089511301 shutdown_connections 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.653+0000 7f961ee82740 1 -- 172.21.15.120:0/3089511301 wait complete. 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 -- start start 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 Processor -- start 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa9746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5625fa732890 con 0x5625fa977790 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5625fa86bd40 con 0x5625fa96aee0 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5625fa97a440 con 0x5625fa96b800 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961ee82740 10 client.0 _ll_register_callbacks cb 0x5625fa8cfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.675 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f961659c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa9746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37890/0 (socket says 172.21.15.120:37890) 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 -- 172.21.15.120:0/1709264370 learned_addr learned my addr 172.21.15.120:0/1709264370 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9615d9b640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 -- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 msgr2=0x5625fa974c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 -- 172.21.15.120:0/1709264370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 msgr2=0x5625fa9746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.654+0000 7f9616d9d640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa9746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f9616d9d640 1 -- 172.21.15.120:0/1709264370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fa9837f0 con 0x5625fa977790 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f961659c640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa9746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f9616d9d640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9610004640 tx=0x7f9610004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dae7ab2dd6e02d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001f610 con 0x5625fa977790 2024-09-10T23:15:10.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9608003d10 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f961001f7b0 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f961001f9b0 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f961ee82740 10 client.19833 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625fa986200 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608003d10 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.655+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9610027020 con 0x5625fa977790 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 10 client.19833 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 10 client.19833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 20 client.19833 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa989a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 4 client.19833 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625fa8e3db0 con 0x5625fa987660 2024-09-10T23:15:10.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961659c640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa989a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f961659c640 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa989a90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d923ea43a545f2e1 server_cookie=eca8b16f3b92dbf9 in_seq=0 out_seq=0 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.656+0000 7f95fdffb640 10 client.19833 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f95fdffb640 1 -- 172.21.15.120:0/1709264370 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625fa8e3db0 con 0x5625fa987660 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f95fdffb640 10 client.19833 handle_command_reply: tid=0 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 client.19833 shutdown 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 msgr2=0x5625fa975170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9610004640 tx=0x7f9610004730 comp rx=0 tx=0).stop 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 shutdown_connections 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa989a90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f95fdffb640 0 client.19833 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa975170 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 --2- 172.21.15.120:0/1709264370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa9746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 >> 172.21.15.120:0/1709264370 conn(0x5625fa968d10 msgr2=0x5625fa967f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 shutdown_connections 2024-09-10T23:15:10.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.658+0000 7f961ee82740 1 -- 172.21.15.120:0/1709264370 wait complete. 2024-09-10T23:15:10.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.659+0000 7f961ee82740 20 client.19833 trim_cache size 0 max 16384 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.676+0000 7f961ee82740 1 Processor -- start 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.676+0000 7f961ee82740 1 -- start start 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.676+0000 7f961ee82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa973f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961ee82740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961ee82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa971900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961ee82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ae500 con 0x5625fa977790 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961ee82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b0d80 con 0x5625fa96aee0 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961ee82740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aeb70 con 0x5625fa96b800 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9615d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa971900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9615d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40614/0 (socket says 172.21.15.120:40614) 2024-09-10T23:15:10.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9615d9b640 1 -- 172.21.15.120:0/927425131 learned_addr learned my addr 172.21.15.120:0/927425131 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f961659c640 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa973f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 -- 172.21.15.120:0/927425131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 msgr2=0x5625fa974d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 -- 172.21.15.120:0/927425131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 msgr2=0x5625fa973f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa973f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 -- 172.21.15.120:0/927425131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fa969790 con 0x5625fa977790 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9615d9b640 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f9616d9d640 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa971900 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9610009690 tx=0x7f9610012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7864011ef97cf3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f95fffff640 1 -- 172.21.15.120:0/927425131 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100043d0 con 0x5625fa977790 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f95fffff640 1 -- 172.21.15.120:0/927425131 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610038070 con 0x5625fa977790 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.677+0000 7f95fffff640 1 -- 172.21.15.120:0/927425131 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610033790 con 0x5625fa977790 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 -- 172.21.15.120:0/927425131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 msgr2=0x5625fa971900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa971900 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9610009690 tx=0x7f9610012d30 comp rx=0 tx=0).stop 2024-09-10T23:15:10.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 -- 172.21.15.120:0/927425131 shutdown_connections 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa977790 0x5625fa971900 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa974d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 --2- 172.21.15.120:0/927425131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa96aee0 0x5625fa973f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 -- 172.21.15.120:0/927425131 >> 172.21.15.120:0/927425131 conn(0x5625fa968d10 msgr2=0x5625fa974660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 -- 172.21.15.120:0/927425131 shutdown_connections 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.678+0000 7f961ee82740 1 -- 172.21.15.120:0/927425131 wait complete. 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 -- start start 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 Processor -- start 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa97c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 0x5625fa9f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96181af240 con 0x5625fa96aee0 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b06b0 con 0x5625fa977790 2024-09-10T23:15:10.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f96181af910 con 0x5625fa96b800 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961ee82740 10 client.0 _ll_register_callbacks cb 0x7f96100099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37916/0 (socket says 172.21.15.120:37916) 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 -- 172.21.15.120:0/2338716057 learned_addr learned my addr 172.21.15.120:0/2338716057 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa97c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f9616d9d640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 0x5625fa9f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 -- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 msgr2=0x5625fa97c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa97c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 -- 172.21.15.120:0/2338716057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 msgr2=0x5625fa9f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 0x5625fa9f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.679+0000 7f961659c640 1 -- 172.21.15.120:0/2338716057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fa9f3bd0 con 0x5625fa96aee0 2024-09-10T23:15:10.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f9616d9d640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 0x5625fa9f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961659c640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f960800f390 tx=0x7f9608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93563dbbb8aeab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608004c00 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f9608004da0 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9610038070 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9608036070 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 10 client.19839 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625fa9f4ef0 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610038070 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f9608005270 con 0x5625fa96aee0 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 10 client.19839 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 10 client.19839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 20 client.19839 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa9f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:10.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 4 client.19839 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:15:10.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.680+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625fa8df860 con 0x5625fa987660 2024-09-10T23:15:10.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.681+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa9f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:10.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.681+0000 7f9615d9b640 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa9f73a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc9d42839ec13f server_cookie=2f524f994c8742b6 in_seq=0 out_seq=0 2024-09-10T23:15:10.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.681+0000 7f95fdffb640 10 client.19839 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f95fdffb640 1 -- 172.21.15.120:0/2338716057 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625fa8df860 con 0x5625fa987660 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f95fdffb640 10 client.19839 handle_command_reply: tid=0 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 client.19839 shutdown 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 msgr2=0x5625fa97bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f960800f390 tx=0x7f9608009bb0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 shutdown_connections 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5625fa987660 0x5625fa9f73a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625fa977790 0x5625fa9f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5625fa96b800 0x5625fa97c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 --2- 172.21.15.120:0/2338716057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5625fa96aee0 0x5625fa97bea0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f95fdffb640 0 client.19839 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 >> 172.21.15.120:0/2338716057 conn(0x5625fa968d10 msgr2=0x5625fa967f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 shutdown_connections 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 1 -- 172.21.15.120:0/2338716057 wait complete. 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.684+0000 7f961ee82740 20 client.19839 trim_cache size 0 max 16384 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "9d234a42-f4c2-4f54-9479-fa834cfb6378", 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:15:10.690 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:10.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 msgr2=0x7f96000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 0x7f96000b77a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f96181110c0 tx=0x7f9610005f90 comp rx=0 tx=0).stop 2024-09-10T23:15:10.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 msgr2=0x7f961810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:10.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f960c005940 tx=0x7f960c0040c0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 shutdown_connections 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f96000b52e0 0x7f96000b77a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9618108020 0x7f961810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96181076a0 0x7f961810f540 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 --2- 172.21.15.120:0/2515863414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9618101c90 0x7f961810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 >> 172.21.15.120:0/2515863414 conn(0x7f96180fd700 msgr2=0x7f96180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 shutdown_connections 2024-09-10T23:15:10.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:10.685+0000 7f961caad640 1 -- 172.21.15.120:0/2515863414 wait complete. 2024-09-10T23:15:10.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:10 smithi153 ceph-mon[34951]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-09-10T23:15:10.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:10 smithi176 ceph-mon[34982]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 566 op/s 2024-09-10T23:15:10.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245194 21.62 MB/sec execute 485 sec latency 93.627 ms 2024-09-10T23:15:11.450 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:15:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:11 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:15:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:11 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:11.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:11 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:15:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:11 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:15:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:11 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:11.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:11 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:15:11.764 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:11.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:11 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:15:11.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:11 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:11.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:11 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:15:11.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245293 21.59 MB/sec execute 486 sec latency 305.807 ms 2024-09-10T23:15:12.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 -- 172.21.15.120:0/1290877200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075540 msgr2=0x7f3324075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 --2- 172.21.15.120:0/1290877200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075540 0x7f3324075920 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f332002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 -- 172.21.15.120:0/1290877200 shutdown_connections 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 --2- 172.21.15.120:0/1290877200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f332410f3e0 0x7f33241117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 --2- 172.21.15.120:0/1290877200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3324075f40 0x7f332410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 --2- 172.21.15.120:0/1290877200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075540 0x7f3324075920 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 -- 172.21.15.120:0/1290877200 >> 172.21.15.120:0/1290877200 conn(0x7f33240fe830 msgr2=0x7f3324100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 -- 172.21.15.120:0/1290877200 shutdown_connections 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.314+0000 7f332b2a1640 1 -- 172.21.15.120:0/1290877200 wait complete. 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 Processor -- start 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 -- start start 2024-09-10T23:15:12.332 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3324075540 0x7f332419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f332410f3e0 0x7f33241a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3324114f90 con 0x7f3324075f40 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3324114e10 con 0x7f3324075540 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332b2a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3324115110 con 0x7f332410f3e0 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37942/0 (socket says 172.21.15.120:37942) 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 -- 172.21.15.120:0/2237136444 learned_addr learned my addr 172.21.15.120:0/2237136444 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332903d640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3324075540 0x7f332419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332983e640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f332410f3e0 0x7f33241a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 -- 172.21.15.120:0/2237136444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f332410f3e0 msgr2=0x7f33241a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f332410f3e0 0x7f33241a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 -- 172.21.15.120:0/2237136444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3324075540 msgr2=0x7f332419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.333 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3324075540 0x7f332419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 -- 172.21.15.120:0/2237136444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33241a1550 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.315+0000 7f332883c640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f331800e9f0 tx=0x7f331800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.316+0000 7f33167fc640 1 -- 172.21.15.120:0/2237136444 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331800cd80 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.316+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33241a1840 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.316+0000 7f33167fc640 1 -- 172.21.15.120:0/2237136444 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33180054d0 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.316+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33241a7f00 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.316+0000 7f33167fc640 1 -- 172.21.15.120:0/2237136444 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318010690 con 0x7f3324075f40 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 Processor -- start 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 -- start start 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef3e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 0x56216ef4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56216eeb4e50 con 0x56216ef3eda0 2024-09-10T23:15:12.334 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56216eea4280 con 0x56216ef3e480 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332d676740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56216eea8090 con 0x56216ef4ad30 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f33167fc640 1 -- 172.21.15.120:0/2237136444 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3318004e90 con 0x7f3324075f40 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332903d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332903d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37948/0 (socket says 172.21.15.120:37948) 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332903d640 1 -- 172.21.15.120:0/737640353 learned_addr learned my addr 172.21.15.120:0/737640353 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332983e640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef3e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.319+0000 7f332883c640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 0x56216ef4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 -- 172.21.15.120:0/737640353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 msgr2=0x56216ef4d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 0x56216ef4d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 -- 172.21.15.120:0/737640353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 msgr2=0x56216ef3e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef3e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 -- 172.21.15.120:0/737640353 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216ec02650 con 0x56216ef3eda0 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332883c640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 0x56216ef4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:12.335 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332903d640 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f3320032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343f88cc1d5ea7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f330f7fe640 1 -- 172.21.15.120:0/737640353 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320040070 con 0x56216ef3eda0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f330f7fe640 1 -- 172.21.15.120:0/737640353 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3320004030 con 0x56216ef3eda0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f330f7fe640 1 -- 172.21.15.120:0/737640353 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320035040 con 0x56216ef3eda0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f33167fc640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 0x7f33100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f332983e640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 0x7f33100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.320+0000 7f33167fc640 1 -- 172.21.15.120:0/2237136444 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f33180e5fb0 con 0x7f3324075f40 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- 172.21.15.120:0/737640353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 msgr2=0x56216ef4a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332983e640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 0x7f33100b77a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f332406c280 tx=0x7f331c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f3320032b20 comp rx=0 tx=0).stop 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- 172.21.15.120:0/737640353 shutdown_connections 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef4ad30 0x56216ef4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef3eda0 0x56216ef4a7f0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.336 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 --2- 172.21.15.120:0/737640353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef3e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- 172.21.15.120:0/737640353 >> 172.21.15.120:0/737640353 conn(0x56216ef3c2b0 msgr2=0x56216ef3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- 172.21.15.120:0/737640353 shutdown_connections 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- 172.21.15.120:0/737640353 wait complete. 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 -- start start 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.321+0000 7f332d676740 1 Processor -- start 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56216ed05700 con 0x56216ef4ad30 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56216ee3f110 con 0x56216ef3e480 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56216ef4d950 con 0x56216ef3eda0 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332d676740 10 client.0 _ll_register_callbacks cb 0x56216ee58990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.337 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332983e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332883c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46676/0 (socket says 172.21.15.120:46676) 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 -- 172.21.15.120:0/603366681 learned_addr learned my addr 172.21.15.120:0/603366681 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 -- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 msgr2=0x56216ef479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 -- 172.21.15.120:0/603366681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 msgr2=0x56216ef47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332903d640 1 -- 172.21.15.120:0/603366681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216ee6cc40 con 0x56216ef3e480 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.322+0000 7f332983e640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332903d640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3320009610 tx=0x7f3320031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86024563e6632f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320040070 con 0x56216ef3e480 2024-09-10T23:15:12.338 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3320005430 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3320004030 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3320035040 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 10 client.29000 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56216ef59260 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3320004030 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3320004d00 con 0x56216ef3e480 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 10 client.29000 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 10 client.29000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 20 client.29000 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216ef5c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 4 client.29000 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.323+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216eeb6da0 con 0x56216ef5a5b0 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.324+0000 7f332883c640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216ef5c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.339 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.324+0000 7f330cff9640 10 client.29000 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.324+0000 7f332883c640 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216ef5c9a0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0049bf65d274c8f server_cookie=e5746b38668863af in_seq=0 out_seq=0 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f330cff9640 1 -- 172.21.15.120:0/603366681 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56216eeb6da0 con 0x56216ef5a5b0 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f330cff9640 10 client.29000 handle_command_reply: tid=0 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 client.29000 shutdown 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 msgr2=0x56216ef4a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3320009610 tx=0x7f3320031400 comp rx=0 tx=0).stop 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 shutdown_connections 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216ef5c9a0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f330cff9640 0 client.29000 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 --2- 172.21.15.120:0/603366681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4a5a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 >> 172.21.15.120:0/603366681 conn(0x56216ef3c2b0 msgr2=0x56216ef41e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.340 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 shutdown_connections 2024-09-10T23:15:12.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 1 -- 172.21.15.120:0/603366681 wait complete. 2024-09-10T23:15:12.341 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.326+0000 7f332d676740 20 client.29000 trim_cache size 0 max 16384 2024-09-10T23:15:12.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 Processor -- start 2024-09-10T23:15:12.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 -- start start 2024-09-10T23:15:12.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33241ae4d0 con 0x56216ef4ad30 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33241b0d50 con 0x56216ef3e480 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332d676740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f33241aeb40 con 0x56216ef3eda0 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332983e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332903d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332983e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:37960/0 (socket says 172.21.15.120:37960) 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332983e640 1 -- 172.21.15.120:0/2334517218 learned_addr learned my addr 172.21.15.120:0/2334517218 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332903d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:46680/0 (socket says 172.21.15.120:46680) 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 -- 172.21.15.120:0/2334517218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 msgr2=0x56216ef56be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 -- 172.21.15.120:0/2334517218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 msgr2=0x56216ef44c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332883c640 1 -- 172.21.15.120:0/2334517218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216ef3cd30 con 0x56216ef3eda0 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332983e640 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.344+0000 7f332903d640 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:12.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f332883c640 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef3da10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33180e3b30 tx=0x7f33180e9860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7608df30a8c8587c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f330effd640 1 -- 172.21.15.120:0/2334517218 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180b1020 con 0x56216ef3eda0 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f330effd640 1 -- 172.21.15.120:0/2334517218 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33180cf040 con 0x56216ef3eda0 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f330effd640 1 -- 172.21.15.120:0/2334517218 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180ac7a0 con 0x56216ef3eda0 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f332d676740 1 -- 172.21.15.120:0/2334517218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 msgr2=0x56216ef3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.345+0000 7f332d676740 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef3da10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33180e3b30 tx=0x7f33180e9860 comp rx=0 tx=0).stop 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- 172.21.15.120:0/2334517218 shutdown_connections 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef44c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef3da10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- 172.21.15.120:0/2334517218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- 172.21.15.120:0/2334517218 >> 172.21.15.120:0/2334517218 conn(0x56216ef3c2b0 msgr2=0x56216ef3d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- 172.21.15.120:0/2334517218 shutdown_connections 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- 172.21.15.120:0/2334517218 wait complete. 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- start start 2024-09-10T23:15:12.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 Processor -- start 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef4f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33241af210 con 0x56216ef4ad30 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33241b0680 con 0x56216ef3e480 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f33241af8e0 con 0x56216ef3eda0 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.346+0000 7f332d676740 10 client.0 _ll_register_callbacks cb 0x7f331c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:40670/0 (socket says 172.21.15.120:40670) 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332983e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef4f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 -- 172.21.15.120:0/2406662164 learned_addr learned my addr 172.21.15.120:0/2406662164 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332903d640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.352 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 -- 172.21.15.120:0/2406662164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 msgr2=0x56216ef4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 -- 172.21.15.120:0/2406662164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 msgr2=0x56216ef4f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef4f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 -- 172.21.15.120:0/2406662164 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216efc62a0 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332883c640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f331800cfd0 tx=0x7f33180ac2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66835168549f7175 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f330cff9640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180b1040 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f33180a8a10 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33180cf040 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f33180abf10 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332d676740 10 client.29287 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.347+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56216efc7590 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33180cf040 con 0x56216ef3eda0 2024-09-10T23:15:12.353 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f33180e80f0 con 0x56216ef3eda0 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 10 client.29287 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 10 client.29287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 20 client.29287 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216efc9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 4 client.29287 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56216eeb2850 con 0x56216ef5a5b0 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332903d640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216efc9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f330cff9640 10 client.29287 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:12.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.348+0000 7f332903d640 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216efc9c40 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26688f9b9bb6c025 server_cookie=14cbe43a58ba3f65 in_seq=0 out_seq=0 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.350+0000 7f330cff9640 1 -- 172.21.15.120:0/2406662164 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+11084 (crc 0 0 0) 0x56216eeb2850 con 0x56216ef5a5b0 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.350+0000 7f330cff9640 10 client.29287 handle_command_reply: tid=0 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.350+0000 7f332d676740 1 client.29287 shutdown 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.350+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 msgr2=0x56216ef4f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.350+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f331800cfd0 tx=0x7f33180ac2b0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.355 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 shutdown_connections 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56216ef5a5b0 0x56216efc9c40 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56216ef4ad30 0x56216ef4f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56216ef3eda0 0x56216ef4f180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f330cff9640 0 client.29287 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 --2- 172.21.15.120:0/2406662164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56216ef3e480 0x56216ef4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 >> 172.21.15.120:0/2406662164 conn(0x56216ef3c2b0 msgr2=0x56216ef3abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 shutdown_connections 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 1 -- 172.21.15.120:0/2406662164 wait complete. 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332d676740 20 client.29287 trim_cache size 0 max 16384 2024-09-10T23:15:12.356 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "scrub active (7 inodes in the stack)", 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "9d234a42-f4c2-4f54-9479-fa834cfb6378", 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:15:12.357 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.358 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.359 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:15:12.360 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.362 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.363 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.364 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.365 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.366 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:15:12.367 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.368 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.369 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.370 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:15:12.371 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:15:12.372 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.373 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.374 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:15:12.375 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:15:12.376 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.377 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:12.379 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 msgr2=0x7f33100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 0x7f33100b77a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f332406c280 tx=0x7f331c009500 comp rx=0 tx=0).stop 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 msgr2=0x7f332419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f331800e9f0 tx=0x7f331800edf0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 shutdown_connections 2024-09-10T23:15:12.380 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f33100b52e0 0x7f33100b77a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f332410f3e0 0x7f33241a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3324075f40 0x7f332419b9e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 --2- 172.21.15.120:0/2237136444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3324075540 0x7f332419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 >> 172.21.15.120:0/2237136444 conn(0x7f33240fe830 msgr2=0x7f33241115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.351+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 shutdown_connections 2024-09-10T23:15:12.381 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:12.352+0000 7f332b2a1640 1 -- 172.21.15.120:0/2237136444 wait complete. 2024-09-10T23:15:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:12 smithi120 ceph-mon[31247]: pgmap v1029: 97 pgs: 97 active+clean; 237 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 544 op/s 2024-09-10T23:15:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:12 smithi153 ceph-mon[34951]: pgmap v1029: 97 pgs: 97 active+clean; 237 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 544 op/s 2024-09-10T23:15:12.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:12 smithi176 ceph-mon[34982]: pgmap v1029: 97 pgs: 97 active+clean; 237 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 544 op/s 2024-09-10T23:15:12.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245397 21.55 MB/sec execute 487 sec latency 543.668 ms 2024-09-10T23:15:13.231 INFO:tasks.cephfs.filesystem:scrub status for tag:9d234a42-f4c2-4f54-9479-fa834cfb6378 - {'path': '/', 'tag': '9d234a42-f4c2-4f54-9479-fa834cfb6378', 'options': 'recursive,force'} 2024-09-10T23:15:13.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:13 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:13.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:13 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:13.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:13 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:13.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 245708 21.53 MB/sec execute 488 sec latency 103.400 ms 2024-09-10T23:15:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:14 smithi120 ceph-mon[31247]: pgmap v1030: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:15:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:14 smithi153 ceph-mon[34951]: pgmap v1030: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:15:14.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:14 smithi176 ceph-mon[34982]: pgmap v1030: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:15:14.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 246120 21.52 MB/sec execute 489 sec latency 94.241 ms 2024-09-10T23:15:15.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:15 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:15:15.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:15 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:15:15.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:15 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:15:15.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 246589 21.52 MB/sec execute 490 sec latency 94.685 ms 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 -- 172.21.15.120:0/4139553002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec741044a0 msgr2=0x7fec7410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 --2- 172.21.15.120:0/4139553002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec741044a0 0x7fec7410ae20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009cd0 tx=0x7fec6c02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 -- 172.21.15.120:0/4139553002 shutdown_connections 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 --2- 172.21.15.120:0/4139553002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec741044a0 0x7fec7410ae20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 --2- 172.21.15.120:0/4139553002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec74103ae0 0x7fec74103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 --2- 172.21.15.120:0/4139553002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74102990 0x7fec74102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 -- 172.21.15.120:0/4139553002 >> 172.21.15.120:0/4139553002 conn(0x7fec740fe250 msgr2=0x7fec74100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 -- 172.21.15.120:0/4139553002 shutdown_connections 2024-09-10T23:15:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.527+0000 7fec79b0e640 1 -- 172.21.15.120:0/4139553002 wait complete. 2024-09-10T23:15:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 Processor -- start 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 -- start start 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74103ae0 0x7fec74199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec741044a0 0x7fec74199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec7410e650 con 0x7fec74102990 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec7410e4d0 con 0x7fec74103ae0 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec79b0e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec7410e7d0 con 0x7fec741044a0 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55162/0 (socket says 172.21.15.120:55162) 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 -- 172.21.15.120:0/1615687595 learned_addr learned my addr 172.21.15.120:0/1615687595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec73fff640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec741044a0 0x7fec74199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec72ffd640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74103ae0 0x7fec74199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 -- 172.21.15.120:0/1615687595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec741044a0 msgr2=0x7fec74199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec741044a0 0x7fec74199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 -- 172.21.15.120:0/1615687595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74103ae0 msgr2=0x7fec74199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74103ae0 0x7fec74199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.528+0000 7fec737fe640 1 -- 172.21.15.120:0/1615687595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7419a870 con 0x7fec74102990 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec737fe640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9f0 tx=0x7fec6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec70ff9640 1 -- 172.21.15.120:0/1615687595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6400cd80 con 0x7fec74102990 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7419ab60 con 0x7fec74102990 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec70ff9640 1 -- 172.21.15.120:0/1615687595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec640108e0 con 0x7fec74102990 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec70ff9640 1 -- 172.21.15.120:0/1615687595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64018980 con 0x7fec74102990 2024-09-10T23:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.529+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec741a1230 con 0x7fec74102990 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 Processor -- start 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 -- start start 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca165090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9ca0cf6f0 con 0x55e9ca160190 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9ca0beb20 con 0x55e9ca162bd0 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec7c8da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e9ca0c2930 con 0x55e9ca15d860 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec70ff9640 1 -- 172.21.15.120:0/1615687595 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fec64010430 con 0x7fec74102990 2024-09-10T23:15:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:56104/0 (socket says 172.21.15.120:56104) 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 -- 172.21.15.120:0/459469144 learned_addr learned my addr 172.21.15.120:0/459469144 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec737fe640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec72ffd640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 -- 172.21.15.120:0/459469144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 msgr2=0x55e9ca165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 -- 172.21.15.120:0/459469144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 msgr2=0x55e9ca162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec73fff640 1 -- 172.21.15.120:0/459469144 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ca084530 con 0x55e9ca15d860 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.532+0000 7fec737fe640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec73fff640 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec6c02aed0 tx=0x7fec6c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1a9e61d2be4edc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec70ff9640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 0x7fec5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec59ffb640 1 -- 172.21.15.120:0/459469144 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c048070 con 0x55e9ca15d860 2024-09-10T23:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec737fe640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 0x7fec5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec59ffb640 1 -- 172.21.15.120:0/459469144 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c036070 con 0x55e9ca15d860 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec59ffb640 1 -- 172.21.15.120:0/459469144 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c004340 con 0x55e9ca15d860 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.533+0000 7fec70ff9640 1 -- 172.21.15.120:0/1615687595 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec640e5e90 con 0x7fec74102990 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec737fe640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 0x7fec5c0b77a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fec64024920 tx=0x7fec64018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- 172.21.15.120:0/459469144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 msgr2=0x55e9ca15fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec6c02aed0 tx=0x7fec6c009a20 comp rx=0 tx=0).stop 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- 172.21.15.120:0/459469144 shutdown_connections 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 --2- 172.21.15.120:0/459469144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca15fc50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- 172.21.15.120:0/459469144 >> 172.21.15.120:0/459469144 conn(0x55e9ca1589b0 msgr2=0x55e9ca159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- 172.21.15.120:0/459469144 shutdown_connections 2024-09-10T23:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- 172.21.15.120:0/459469144 wait complete. 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 -- start start 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.534+0000 7fec7c8da740 1 Processor -- start 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca166e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca1680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9c9ff6ab0 con 0x55e9ca160190 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55e9ca064730 con 0x55e9ca15d860 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e9ca165b50 con 0x55e9ca162bd0 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 10 client.0 _ll_register_callbacks cb 0x55e9ca0b86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca1680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55178/0 (socket says 172.21.15.120:55178) 2024-09-10T23:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 -- 172.21.15.120:0/2173944104 learned_addr learned my addr 172.21.15.120:0/2173944104 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec73fff640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca166e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 -- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 msgr2=0x55e9ca1680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca1680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 -- 172.21.15.120:0/2173944104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 msgr2=0x55e9ca166e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca166e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 -- 172.21.15.120:0/2173944104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c9e20790 con 0x55e9ca160190 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec72ffd640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fec68012840 tx=0x7fec68012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1910f433ebdd8e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800f8c0 con 0x55e9ca160190 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec6800fa60 con 0x55e9ca160190 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.535+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6c036070 con 0x55e9ca160190 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec6800fc60 con 0x55e9ca160190 2024-09-10T23:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 10 client.19869 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9c9e99620 con 0x55e9ca160190 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c036070 con 0x55e9ca160190 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec68027420 con 0x55e9ca160190 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 10 client.19869 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 10 client.19869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 20 client.19869 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca172810 0x55e9ca174c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 4 client.19869 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ca0d1640 con 0x55e9ca172810 2024-09-10T23:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.536+0000 7fec73fff640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca172810 0x55e9ca174c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.537+0000 7fec52ffd640 10 client.19869 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.537+0000 7fec73fff640 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca172810 0x55e9ca174c00 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6fe1d518e788aaa server_cookie=340fdcc3aa8e93b6 in_seq=0 out_seq=0 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.540+0000 7fec52ffd640 1 -- 172.21.15.120:0/2173944104 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9ca0d1640 con 0x55e9ca172810 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.540+0000 7fec52ffd640 10 client.19869 handle_command_reply: tid=0 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.540+0000 7fec7c8da740 1 client.19869 shutdown 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 msgr2=0x55e9ca167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fec68012840 tx=0x7fec68012d00 comp rx=0 tx=0).stop 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 shutdown_connections 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca172810 0x55e9ca174c00 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca1680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca167500 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec52ffd640 0 client.19869 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 --2- 172.21.15.120:0/2173944104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca166e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 >> 172.21.15.120:0/2173944104 conn(0x55e9ca1589b0 msgr2=0x55e9ca157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 shutdown_connections 2024-09-10T23:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 1 -- 172.21.15.120:0/2173944104 wait complete. 2024-09-10T23:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.541+0000 7fec7c8da740 20 client.19869 trim_cache size 0 max 16384 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.555+0000 7fec7c8da740 1 Processor -- start 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 -- start start 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca162fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca166e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec741a7d80 con 0x55e9ca160190 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec741aa600 con 0x55e9ca162bd0 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec7c8da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec741a83f0 con 0x55e9ca15d860 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55184/0 (socket says 172.21.15.120:55184) 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec73fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca162fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 -- 172.21.15.120:0/3967853075 learned_addr learned my addr 172.21.15.120:0/3967853075 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec73fff640 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca166e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 -- 172.21.15.120:0/3967853075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 msgr2=0x55e9ca162fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca162fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 -- 172.21.15.120:0/3967853075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 msgr2=0x55e9ca166e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca166e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 -- 172.21.15.120:0/3967853075 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ca1628f0 con 0x55e9ca160190 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec72ffd640 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fec680098e0 tx=0x7fec68027f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eb8b109640ee7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.556+0000 7fec597fa640 1 -- 172.21.15.120:0/3967853075 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68027b00 con 0x55e9ca160190 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec597fa640 1 -- 172.21.15.120:0/3967853075 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6802b020 con 0x55e9ca160190 2024-09-10T23:15:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec597fa640 1 -- 172.21.15.120:0/3967853075 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68005220 con 0x55e9ca160190 2024-09-10T23:15:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 -- 172.21.15.120:0/3967853075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 msgr2=0x55e9ca1634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fec680098e0 tx=0x7fec68027f90 comp rx=0 tx=0).stop 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 -- 172.21.15.120:0/3967853075 shutdown_connections 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca162bd0 0x55e9ca166e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca1634f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 --2- 172.21.15.120:0/3967853075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca15d860 0x55e9ca162fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 -- 172.21.15.120:0/3967853075 >> 172.21.15.120:0/3967853075 conn(0x55e9ca1589b0 msgr2=0x55e9ca15ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 -- 172.21.15.120:0/3967853075 shutdown_connections 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.557+0000 7fec7c8da740 1 -- 172.21.15.120:0/3967853075 wait complete. 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 -- start start 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 Processor -- start 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca16e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca16d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec741a8ac0 con 0x55e9ca160190 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fec741a9f30 con 0x55e9ca15d860 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fec741a9190 con 0x55e9ca162bd0 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec7c8da740 10 client.0 _ll_register_callbacks cb 0x7fec68009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:55186/0 (socket says 172.21.15.120:55186) 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec72ffd640 1 -- 172.21.15.120:0/1292516988 learned_addr learned my addr 172.21.15.120:0/1292516988 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec73fff640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca16d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.558+0000 7fec73fff640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca16e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 -- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 msgr2=0x55e9ca16d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca16d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 -- 172.21.15.120:0/1292516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 msgr2=0x55e9ca16e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca16e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 -- 172.21.15.120:0/1292516988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ca16da90 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec73fff640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca16e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec72ffd640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec6800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b507595b16e2d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68034020 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fec68018030 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6802b020 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec68027c90 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec7c8da740 10 client.19875 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9ca16c950 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.559+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68005610 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fec680039b0 con 0x55e9ca160190 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 10 client.19875 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 20 client.19875 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca1dfc70 0x55e9ca1e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 4 client.19875 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fec6802b020 con 0x55e9ca1dfc70 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.560+0000 7fec73fff640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca1dfc70 0x55e9ca1e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.561+0000 7fec73fff640 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca1dfc70 0x55e9ca1e2090 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f444c343cc4122 server_cookie=a720c5d814b9710f in_seq=0 out_seq=0 2024-09-10T23:15:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:16.561+0000 7fec52ffd640 10 client.19875 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:16 smithi120 ceph-mon[31247]: pgmap v1031: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T23:15:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:16 smithi153 ceph-mon[34951]: pgmap v1031: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T23:15:16.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:16 smithi176 ceph-mon[34982]: pgmap v1031: 97 pgs: 97 active+clean; 243 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-09-10T23:15:16.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247035 21.53 MB/sec execute 491 sec latency 201.969 ms 2024-09-10T23:15:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.463+0000 7fec52ffd640 1 -- 172.21.15.120:0/1292516988 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fec6802b020 con 0x55e9ca1dfc70 2024-09-10T23:15:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.463+0000 7fec52ffd640 10 client.19875 handle_command_reply: tid=0 2024-09-10T23:15:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.463+0000 7fec7c8da740 1 client.19875 shutdown 2024-09-10T23:15:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 msgr2=0x55e9ca16eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fec68000c00 tx=0x7fec6800fbe0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 shutdown_connections 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55e9ca1dfc70 0x55e9ca1e2090 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec52ffd640 0 client.19875 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e9ca162bd0 0x55e9ca16d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ca160190 0x55e9ca16eb60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 --2- 172.21.15.120:0/1292516988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e9ca15d860 0x55e9ca16e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 >> 172.21.15.120:0/1292516988 conn(0x55e9ca1589b0 msgr2=0x55e9ca162fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.464+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 shutdown_connections 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.465+0000 7fec7c8da740 1 -- 172.21.15.120:0/1292516988 wait complete. 2024-09-10T23:15:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.465+0000 7fec7c8da740 20 client.19875 trim_cache size 0 max 16384 2024-09-10T23:15:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:15:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:15:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:15:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 msgr2=0x7fec5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 0x7fec5c0b77a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fec64024920 tx=0x7fec64018420 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 msgr2=0x7fec74198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9f0 tx=0x7fec6400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 shutdown_connections 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fec5c0b52e0 0x7fec5c0b77a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec741044a0 0x7fec74199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec74103ae0 0x7fec74199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 --2- 172.21.15.120:0/1615687595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec74102990 0x7fec74198d30 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 >> 172.21.15.120:0/1615687595 conn(0x7fec740fe250 msgr2=0x7fec74106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 shutdown_connections 2024-09-10T23:15:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:17.466+0000 7fec79b0e640 1 -- 172.21.15.120:0/1615687595 wait complete. 2024-09-10T23:15:17.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:17 smithi120 ceph-mon[31247]: pgmap v1032: 97 pgs: 97 active+clean; 249 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 703 op/s 2024-09-10T23:15:17.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:17 smithi153 ceph-mon[34951]: pgmap v1032: 97 pgs: 97 active+clean; 249 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 703 op/s 2024-09-10T23:15:17.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:17 smithi176 ceph-mon[34982]: pgmap v1032: 97 pgs: 97 active+clean; 249 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 703 op/s 2024-09-10T23:15:17.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247249 21.51 MB/sec execute 492 sec latency 724.615 ms 2024-09-10T23:15:18.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 247777 21.53 MB/sec execute 493 sec latency 80.552 ms 2024-09-10T23:15:19.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 248269 21.55 MB/sec execute 494 sec latency 53.959 ms 2024-09-10T23:15:19.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:19 smithi120 ceph-mon[31247]: pgmap v1033: 97 pgs: 97 active+clean; 248 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 501 op/s 2024-09-10T23:15:19.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:19 smithi153 ceph-mon[34951]: pgmap v1033: 97 pgs: 97 active+clean; 248 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 501 op/s 2024-09-10T23:15:20.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:19 smithi176 ceph-mon[34982]: pgmap v1033: 97 pgs: 97 active+clean; 248 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 501 op/s 2024-09-10T23:15:20.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 248656 21.56 MB/sec execute 495 sec latency 90.690 ms 2024-09-10T23:15:21.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 249000 21.55 MB/sec execute 496 sec latency 64.202 ms 2024-09-10T23:15:21.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:21 smithi120 ceph-mon[31247]: pgmap v1034: 97 pgs: 97 active+clean; 244 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 489 op/s 2024-09-10T23:15:21.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:21 smithi153 ceph-mon[34951]: pgmap v1034: 97 pgs: 97 active+clean; 244 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 489 op/s 2024-09-10T23:15:22.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:21 smithi176 ceph-mon[34982]: pgmap v1034: 97 pgs: 97 active+clean; 244 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 489 op/s 2024-09-10T23:15:22.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 249243 21.52 MB/sec execute 497 sec latency 298.187 ms 2024-09-10T23:15:23.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 249578 21.51 MB/sec execute 498 sec latency 95.065 ms 2024-09-10T23:15:23.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:23 smithi120 ceph-mon[31247]: pgmap v1035: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-09-10T23:15:23.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:23 smithi153 ceph-mon[34951]: pgmap v1035: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-09-10T23:15:24.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:23 smithi176 ceph-mon[34982]: pgmap v1035: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-09-10T23:15:24.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 250144 21.52 MB/sec execute 499 sec latency 52.931 ms 2024-09-10T23:15:25.892 INFO:tasks.workunit.client.0.smithi120.stdout: 10 250751 21.54 MB/sec execute 500 sec latency 75.346 ms 2024-09-10T23:15:25.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:25 smithi120 ceph-mon[31247]: pgmap v1036: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 581 op/s 2024-09-10T23:15:25.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:25 smithi153 ceph-mon[34951]: pgmap v1036: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 581 op/s 2024-09-10T23:15:26.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:25 smithi176 ceph-mon[34982]: pgmap v1036: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 581 op/s 2024-09-10T23:15:26.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251326 21.57 MB/sec execute 501 sec latency 65.488 ms 2024-09-10T23:15:27.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 251744 21.58 MB/sec execute 502 sec latency 241.489 ms 2024-09-10T23:15:27.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:27 smithi120 ceph-mon[31247]: pgmap v1037: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-09-10T23:15:27.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:27 smithi153 ceph-mon[34951]: pgmap v1037: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-09-10T23:15:28.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:27 smithi176 ceph-mon[34982]: pgmap v1037: 97 pgs: 97 active+clean; 235 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-09-10T23:15:28.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252101 21.58 MB/sec execute 503 sec latency 140.332 ms 2024-09-10T23:15:29.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252402 21.57 MB/sec execute 504 sec latency 137.108 ms 2024-09-10T23:15:29.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:29 smithi120 ceph-mon[31247]: pgmap v1038: 97 pgs: 97 active+clean; 240 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-09-10T23:15:29.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:29 smithi153 ceph-mon[34951]: pgmap v1038: 97 pgs: 97 active+clean; 240 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-09-10T23:15:30.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:29 smithi176 ceph-mon[34982]: pgmap v1038: 97 pgs: 97 active+clean; 240 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 -- 172.21.15.120:0/684912758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d70076f30 msgr2=0x7f6d70075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 --2- 172.21.15.120:0/684912758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d70076f30 0x7f6d70075580 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a410 comp rx=0 tx=0).stop 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 -- 172.21.15.120:0/684912758 shutdown_connections 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 --2- 172.21.15.120:0/684912758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7010a910 0x7f6d7010cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 --2- 172.21.15.120:0/684912758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70075ac0 0x7f6d70075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 --2- 172.21.15.120:0/684912758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d70076f30 0x7f6d70075580 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 -- 172.21.15.120:0/684912758 >> 172.21.15.120:0/684912758 conn(0x7f6d700fe250 msgr2=0x7f6d70100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 -- 172.21.15.120:0/684912758 shutdown_connections 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.570+0000 7f6d7859b640 1 -- 172.21.15.120:0/684912758 wait complete. 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 Processor -- start 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 -- start start 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d70075ac0 0x7f6d7019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 0x7f6d7019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70110800 con 0x7f6d7010a910 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70110680 con 0x7f6d70075ac0 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d7859b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70110980 con 0x7f6d70076f30 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d76b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d76b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42178/0 (socket says 172.21.15.120:42178) 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d76b38640 1 -- 172.21.15.120:0/1444973421 learned_addr learned my addr 172.21.15.120:0/1444973421 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 0x7f6d7019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.571+0000 7f6d76337640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d70075ac0 0x7f6d7019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 -- 172.21.15.120:0/1444973421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 msgr2=0x7f6d7019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 0x7f6d7019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 -- 172.21.15.120:0/1444973421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d70075ac0 msgr2=0x7f6d7019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d70075ac0 0x7f6d7019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 -- 172.21.15.120:0/1444973421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019caf0 con 0x7f6d7010a910 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 0x7f6d7019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d76b38640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800eaa0 tx=0x7f6d6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d677fe640 1 -- 172.21.15.120:0/1444973421 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800cdb0 con 0x7f6d7010a910 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019cde0 con 0x7f6d7010a910 2024-09-10T23:15:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d677fe640 1 -- 172.21.15.120:0/1444973421 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d68005020 con 0x7f6d7010a910 2024-09-10T23:15:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d677fe640 1 -- 172.21.15.120:0/1444973421 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d680116d0 con 0x7f6d7010a910 2024-09-10T23:15:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.572+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d701a34b0 con 0x7f6d7010a910 2024-09-10T23:15:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.575+0000 7f6d7b443740 1 Processor -- start 2024-09-10T23:15:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d677fe640 1 -- 172.21.15.120:0/1444973421 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6d6800f040 con 0x7f6d7010a910 2024-09-10T23:15:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 -- start start 2024-09-10T23:15:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde69580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556adddd66f0 con 0x556adde67190 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556adddc5b20 con 0x556adde69bd0 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d7b443740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556adddc9930 con 0x556adde64860 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d76337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde69580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d75b36640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d76337640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde69580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42194/0 (socket says 172.21.15.120:42194) 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d76337640 1 -- 172.21.15.120:0/1566361516 learned_addr learned my addr 172.21.15.120:0/1566361516 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55266/0 (socket says 172.21.15.120:55266) 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.576+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 -- 172.21.15.120:0/1566361516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 msgr2=0x556adde6c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 -- 172.21.15.120:0/1566361516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 msgr2=0x556adde69580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde69580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 -- 172.21.15.120:0/1566361516 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556addd8b530 con 0x556adde64860 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d677fe640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 0x7f6d5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d75b36640 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6d60011700 tx=0x7f6d60005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324b8c363296e07f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d76337640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 0x7f6d5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d677fe640 1 -- 172.21.15.120:0/1444973421 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6d680e59c0 con 0x7f6d7010a910 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d64bfe640 1 -- 172.21.15.120:0/1566361516 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60011030 con 0x556adde64860 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d64bfe640 1 -- 172.21.15.120:0/1566361516 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6000fdf0 con 0x556adde64860 2024-09-10T23:15:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.577+0000 7f6d64bfe640 1 -- 172.21.15.120:0/1566361516 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60010ca0 con 0x556adde64860 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d76337640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 0x7f6d5c0b77a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a930 tx=0x7f6d6c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- 172.21.15.120:0/1566361516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 msgr2=0x556adde66c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6d60011700 tx=0x7f6d60005850 comp rx=0 tx=0).stop 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- 172.21.15.120:0/1566361516 shutdown_connections 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde69580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- 172.21.15.120:0/1566361516 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde66c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- 172.21.15.120:0/1566361516 >> 172.21.15.120:0/1566361516 conn(0x556adde5f9b0 msgr2=0x556adde60740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- 172.21.15.120:0/1566361516 shutdown_connections 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- 172.21.15.120:0/1566361516 wait complete. 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 -- start start 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 Processor -- start 2024-09-10T23:15:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde6e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.578+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d7b443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556addcfdab0 con 0x556adde67190 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d7b443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556addd6b730 con 0x556adde69bd0 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d7b443740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556adde6cbb0 con 0x556adde64860 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d7b443740 10 client.0 _ll_register_callbacks cb 0x556adddbf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:55278/0 (socket says 172.21.15.120:55278) 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 -- 172.21.15.120:0/3611731283 learned_addr learned my addr 172.21.15.120:0/3611731283 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde6e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76b38640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 -- 172.21.15.120:0/3611731283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 msgr2=0x556adde6f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 -- 172.21.15.120:0/3611731283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 msgr2=0x556adde6e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde6e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 -- 172.21.15.120:0/3611731283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556addb27790 con 0x556adde64860 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde6e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d76337640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c006010 tx=0x7f6d6c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d3ad0196b98703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c035040 con 0x556adde64860 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6000fdf0 con 0x556adde64860 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.579+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6d6c0035f0 con 0x556adde64860 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c057070 con 0x556adde64860 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 10 client.29317 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556addba0620 con 0x556adde64860 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6000fdf0 con 0x556adde64860 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6d6c044070 con 0x556adde64860 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 10 client.29317 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 10 client.29317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 20 client.29317 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556adde79780 0x556adde7bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 4 client.29317 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556adddd8640 con 0x556adde79780 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.580+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556adde79780 0x556adde7bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.581+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556adde79780 0x556adde7bbb0 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b999225f04af8d server_cookie=7e8fc4aca7352b13 in_seq=0 out_seq=0 2024-09-10T23:15:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.581+0000 7f6d59ffb640 10 client.29317 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3611731283 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556adddd8640 con 0x556adde79780 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d59ffb640 10 client.29317 handle_command_reply: tid=0 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 client.29317 shutdown 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 msgr2=0x556adde6def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c006010 tx=0x7f6d6c009260 comp rx=0 tx=0).stop 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 shutdown_connections 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556adde79780 0x556adde7bbb0 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde69bd0 0x556adde6f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d59ffb640 0 client.29317 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde6e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3611731283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde6def0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 >> 172.21.15.120:0/3611731283 conn(0x556adde5f9b0 msgr2=0x556adde5ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 shutdown_connections 2024-09-10T23:15:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 1 -- 172.21.15.120:0/3611731283 wait complete. 2024-09-10T23:15:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.584+0000 7f6d7b443740 20 client.29317 trim_cache size 0 max 16384 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.599+0000 7f6d7b443740 1 Processor -- start 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.599+0000 7f6d7b443740 1 -- start start 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.599+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.599+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 0x556adde62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.599+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d7b443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701aa060 con 0x556adde69bd0 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d7b443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701ac8e0 con 0x556adde67190 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d7b443740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701aa6d0 con 0x556adde64860 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d76b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d76b38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:42202/0 (socket says 172.21.15.120:42202) 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 0x556adde62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d76b38640 1 -- 172.21.15.120:0/2836219937 learned_addr learned my addr 172.21.15.120:0/2836219937 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d76337640 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 -- 172.21.15.120:0/2836219937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 msgr2=0x556adde60650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde60650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 -- 172.21.15.120:0/2836219937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 msgr2=0x556adde62f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 -- 172.21.15.120:0/2836219937 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adde5b0b0 con 0x556adde67190 2024-09-10T23:15:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d76b38640 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d75b36640 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 0x556adde62a00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006010 tx=0x7f6d600096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab6376f770c7ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d5bfff640 1 -- 172.21.15.120:0/2836219937 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60005040 con 0x556adde67190 2024-09-10T23:15:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d5bfff640 1 -- 172.21.15.120:0/2836219937 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d60004680 con 0x556adde67190 2024-09-10T23:15:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.600+0000 7f6d5bfff640 1 -- 172.21.15.120:0/2836219937 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600296a0 con 0x556adde67190 2024-09-10T23:15:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- 172.21.15.120:0/2836219937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 msgr2=0x556adde62a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 0x556adde62a00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006010 tx=0x7f6d600096a0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- 172.21.15.120:0/2836219937 shutdown_connections 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde69bd0 0x556adde62f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde67190 0x556adde62a00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 --2- 172.21.15.120:0/2836219937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde64860 0x556adde60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- 172.21.15.120:0/2836219937 >> 172.21.15.120:0/2836219937 conn(0x556adde5f9b0 msgr2=0x556adde63d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- 172.21.15.120:0/2836219937 shutdown_connections 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- 172.21.15.120:0/2836219937 wait complete. 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.601+0000 7f6d7b443740 1 -- start start 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 Processor -- start 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 0x556adde74c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701aada0 con 0x556adde67190 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701ac210 con 0x556adde64860 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701ab470 con 0x556adde69bd0 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d7b443740 10 client.0 _ll_register_callbacks cb 0x7f6d6c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d75b36640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:50844/0 (socket says 172.21.15.120:50844) 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 -- 172.21.15.120:0/3142353030 learned_addr learned my addr 172.21.15.120:0/3142353030 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76b38640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 0x556adde74c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 -- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 msgr2=0x556adde74c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 0x556adde74c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 -- 172.21.15.120:0/3142353030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 msgr2=0x556adde746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76337640 1 -- 172.21.15.120:0/3142353030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556adde66fc0 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.602+0000 7f6d76b38640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 0x556adde74c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d76337640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000c00 tx=0x7f6d6c0044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12523147dafa6b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c060030 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d60004680 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f6d6c0035f0 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c035050 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 10 client.29042 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556addee6680 con 0x556adde64860 2024-09-10T23:15:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d60004680 con 0x556adde64860 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f6d6c044040 con 0x556adde64860 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 10 client.29042 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 10 client.29042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 20 client.29042 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556addee6f40 0x556addee9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 4 client.29042 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556adddd40f0 con 0x556addee6f40 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.603+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556addee6f40 0x556addee9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.604+0000 7f6d59ffb640 10 client.29042 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:30.604+0000 7f6d75b36640 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556addee6f40 0x556addee9360 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2564f1a5781703 server_cookie=e91992b3777a03a0 in_seq=0 out_seq=0 2024-09-10T23:15:30.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252687 21.55 MB/sec execute 505 sec latency 182.806 ms 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d59ffb640 1 -- 172.21.15.120:0/3142353030 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556adddd40f0 con 0x556addee6f40 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d59ffb640 10 client.29042 handle_command_reply: tid=0 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 client.29042 shutdown 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 msgr2=0x556adde759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000c00 tx=0x7f6d6c0044c0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 shutdown_connections 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556addee6f40 0x556addee9360 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556adde69bd0 0x556adde74c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556adde67190 0x556adde746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 --2- 172.21.15.120:0/3142353030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556adde64860 0x556adde759f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d59ffb640 0 client.29042 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.377+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 >> 172.21.15.120:0/3142353030 conn(0x556adde5f9b0 msgr2=0x556adde63d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 shutdown_connections 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7b443740 1 -- 172.21.15.120:0/3142353030 wait complete. 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7b443740 20 client.29042 trim_cache size 0 max 16384 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:15:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 msgr2=0x7f6d5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 0x7f6d5c0b77a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a930 tx=0x7f6d6c0314a0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 msgr2=0x7f6d7019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800eaa0 tx=0x7f6d6800ef60 comp rx=0 tx=0).stop 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 shutdown_connections 2024-09-10T23:15:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f6d5c0b52e0 0x7f6d5c0b77a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6d7010a910 0x7f6d7019c230 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d70076f30 0x7f6d7019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 --2- 172.21.15.120:0/1444973421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d70075ac0 0x7f6d7019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 >> 172.21.15.120:0/1444973421 conn(0x7f6d700fe250 msgr2=0x7f6d700fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 shutdown_connections 2024-09-10T23:15:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:31.378+0000 7f6d7859b640 1 -- 172.21.15.120:0/1444973421 wait complete. 2024-09-10T23:15:31.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 252856 21.52 MB/sec execute 506 sec latency 426.814 ms 2024-09-10T23:15:31.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:31 smithi120 ceph-mon[31247]: pgmap v1039: 97 pgs: 97 active+clean; 247 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-10T23:15:31.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:31 smithi153 ceph-mon[34951]: pgmap v1039: 97 pgs: 97 active+clean; 247 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-10T23:15:32.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:31 smithi176 ceph-mon[34982]: pgmap v1039: 97 pgs: 97 active+clean; 247 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-10T23:15:32.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 253106 21.50 MB/sec execute 507 sec latency 248.550 ms 2024-09-10T23:15:33.873 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:33 smithi176 ceph-mon[34982]: pgmap v1040: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-10T23:15:33.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 253610 21.50 MB/sec execute 508 sec latency 71.398 ms 2024-09-10T23:15:33.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:33 smithi120 ceph-mon[31247]: pgmap v1040: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-10T23:15:33.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:33 smithi153 ceph-mon[34951]: pgmap v1040: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-10T23:15:34.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 254242 21.53 MB/sec execute 509 sec latency 56.935 ms 2024-09-10T23:15:35.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 254761 21.54 MB/sec execute 510 sec latency 58.510 ms 2024-09-10T23:15:35.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:35 smithi120 ceph-mon[31247]: pgmap v1041: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-09-10T23:15:35.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:35 smithi153 ceph-mon[34951]: pgmap v1041: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-09-10T23:15:36.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:35 smithi176 ceph-mon[34982]: pgmap v1041: 97 pgs: 97 active+clean; 238 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-09-10T23:15:36.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 255300 21.58 MB/sec execute 511 sec latency 99.523 ms 2024-09-10T23:15:37.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 255597 21.57 MB/sec execute 512 sec latency 206.395 ms 2024-09-10T23:15:37.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:37 smithi153 ceph-mon[34951]: pgmap v1042: 97 pgs: 97 active+clean; 231 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-09-10T23:15:37.949 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:37 smithi120 ceph-mon[31247]: pgmap v1042: 97 pgs: 97 active+clean; 231 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-09-10T23:15:38.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:37 smithi176 ceph-mon[34982]: pgmap v1042: 97 pgs: 97 active+clean; 231 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-09-10T23:15:38.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 255912 21.57 MB/sec execute 513 sec latency 75.147 ms 2024-09-10T23:15:39.893 INFO:tasks.workunit.client.0.smithi120.stdout: 10 256255 21.55 MB/sec execute 514 sec latency 85.256 ms 2024-09-10T23:15:39.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:39 smithi120 ceph-mon[31247]: pgmap v1043: 97 pgs: 97 active+clean; 240 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-09-10T23:15:39.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:39 smithi153 ceph-mon[34951]: pgmap v1043: 97 pgs: 97 active+clean; 240 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-09-10T23:15:40.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:39 smithi176 ceph-mon[34982]: pgmap v1043: 97 pgs: 97 active+clean; 240 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-09-10T23:15:40.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 256621 21.54 MB/sec execute 515 sec latency 88.474 ms 2024-09-10T23:15:41.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 257094 21.54 MB/sec execute 516 sec latency 78.315 ms 2024-09-10T23:15:41.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:41 smithi120 ceph-mon[31247]: pgmap v1044: 97 pgs: 97 active+clean; 244 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 592 op/s 2024-09-10T23:15:41.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:41 smithi153 ceph-mon[34951]: pgmap v1044: 97 pgs: 97 active+clean; 244 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 592 op/s 2024-09-10T23:15:42.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:41 smithi176 ceph-mon[34982]: pgmap v1044: 97 pgs: 97 active+clean; 244 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 592 op/s 2024-09-10T23:15:42.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 257447 21.53 MB/sec execute 517 sec latency 185.006 ms 2024-09-10T23:15:43.233 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:15:43.511 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:43.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 257855 21.53 MB/sec execute 518 sec latency 186.146 ms 2024-09-10T23:15:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:43 smithi120 ceph-mon[31247]: pgmap v1045: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-09-10T23:15:43.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:43 smithi153 ceph-mon[34951]: pgmap v1045: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-09-10T23:15:44.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:43 smithi176 ceph-mon[34982]: pgmap v1045: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-09-10T23:15:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 -- 172.21.15.120:0/1648116107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c101cd0 msgr2=0x7fad5c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 --2- 172.21.15.120:0/1648116107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c101cd0 0x7fad5c1020b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad5802a3f0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 -- 172.21.15.120:0/1648116107 shutdown_connections 2024-09-10T23:15:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 --2- 172.21.15.120:0/1648116107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 0x7fad5c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 --2- 172.21.15.120:0/1648116107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c1076e0 0x7fad5c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 --2- 172.21.15.120:0/1648116107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c101cd0 0x7fad5c1020b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 -- 172.21.15.120:0/1648116107 >> 172.21.15.120:0/1648116107 conn(0x7fad5c0fd740 msgr2=0x7fad5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.087 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 -- 172.21.15.120:0/1648116107 shutdown_connections 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.085+0000 7fad64249640 1 -- 172.21.15.120:0/1648116107 wait complete. 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.086+0000 7fad64249640 1 Processor -- start 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.086+0000 7fad64249640 1 -- start start 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 0x7fad5c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 0x7fad5c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110c20 con 0x7fad5c1076e0 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad617e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 0x7fad5c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39804/0 (socket says 172.21.15.120:39804) 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110aa0 con 0x7fad5c101cd0 2024-09-10T23:15:44.088 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 -- 172.21.15.120:0/3769524439 learned_addr learned my addr 172.21.15.120:0/3769524439 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad617e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 0x7fad5c19e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51540/0 (socket says 172.21.15.120:51540) 2024-09-10T23:15:44.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad627e6640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 0x7fad5c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c110da0 con 0x7fad5c108060 2024-09-10T23:15:44.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 -- 172.21.15.120:0/3769524439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 msgr2=0x7fad5c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 0x7fad5c1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 -- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 msgr2=0x7fad5c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 0x7fad5c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 -- 172.21.15.120:0/3769524439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c197a80 con 0x7fad5c101cd0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.087+0000 7fad61fe5640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad58005940 tx=0x7fad580040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.088+0000 7fad52ffd640 1 -- 172.21.15.120:0/3769524439 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5803f030 con 0x7fad5c101cd0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.088+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c197d10 con 0x7fad5c101cd0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.088+0000 7fad52ffd640 1 -- 172.21.15.120:0/3769524439 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad580044c0 con 0x7fad5c101cd0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.088+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c19cc70 con 0x7fad5c101cd0 2024-09-10T23:15:44.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.088+0000 7fad52ffd640 1 -- 172.21.15.120:0/3769524439 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58035070 con 0x7fad5c101cd0 2024-09-10T23:15:44.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.091+0000 7fad52ffd640 1 -- 172.21.15.120:0/3769524439 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fad5804c060 con 0x7fad5c101cd0 2024-09-10T23:15:44.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.092+0000 7fad6661e740 1 Processor -- start 2024-09-10T23:15:44.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.092+0000 7fad52ffd640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 0x7fad480b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad627e6640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 0x7fad480b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad52ffd640 1 -- 172.21.15.120:0/3769524439 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad58105ac0 con 0x7fad5c101cd0 2024-09-10T23:15:44.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 -- start start 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf460a0 0x5573baf46480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad627e6640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 0x7fad480b7810 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fad54005fd0 tx=0x7fad54009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573baebce50 con 0x5573baf469c0 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5573baeac280 con 0x5573baf52950 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.093+0000 7fad6661e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5573baeb0090 con 0x5573baf460a0 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51542/0 (socket says 172.21.15.120:51542) 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 -- 172.21.15.120:0/4215483846 learned_addr learned my addr 172.21.15.120:0/4215483846 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad61fe5640 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf460a0 0x5573baf46480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 -- 172.21.15.120:0/4215483846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf460a0 msgr2=0x5573baf46480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf460a0 0x5573baf46480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 -- 172.21.15.120:0/4215483846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 msgr2=0x5573baf54d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:15:44.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf54d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 -- 172.21.15.120:0/4215483846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bac0a650 con 0x5573baf469c0 2024-09-10T23:15:44.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.094+0000 7fad617e4640 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009c90 tx=0x7fad4c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18e065ec053dbe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.095+0000 7fad47fff640 1 -- 172.21.15.120:0/4215483846 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c01e070 con 0x5573baf469c0 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.095+0000 7fad47fff640 1 -- 172.21.15.120:0/4215483846 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad4c019ae0 con 0x5573baf469c0 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.095+0000 7fad47fff640 1 -- 172.21.15.120:0/4215483846 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00e9c0 con 0x5573baf469c0 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 -- 172.21.15.120:0/4215483846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 msgr2=0x5573baf52410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009c90 tx=0x7fad4c011810 comp rx=0 tx=0).stop 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 -- 172.21.15.120:0/4215483846 shutdown_connections 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf54d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf469c0 0x5573baf52410 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 --2- 172.21.15.120:0/4215483846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf460a0 0x5573baf46480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 -- 172.21.15.120:0/4215483846 >> 172.21.15.120:0/4215483846 conn(0x5573baf43ed0 msgr2=0x5573baf45070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 -- 172.21.15.120:0/4215483846 shutdown_connections 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.096+0000 7fad6661e740 1 -- 172.21.15.120:0/4215483846 wait complete. 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 -- start start 2024-09-10T23:15:44.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 Processor -- start 2024-09-10T23:15:44.100 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf52320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5573bad0d700 con 0x5573baf460a0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5573bae47110 con 0x5573baf469c0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5573baf555d0 con 0x5573baf52950 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad6661e740 10 client.0 _ll_register_callbacks cb 0x5573bae60990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad627e6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad627e6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf4fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46430/0 (socket says 172.21.15.120:46430) 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad627e6640 1 -- 172.21.15.120:0/1331322302 learned_addr learned my addr 172.21.15.120:0/1331322302 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad61fe5640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf52320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.097+0000 7fad617e4640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.101 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 -- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 msgr2=0x5573baf4fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf4fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 -- 172.21.15.120:0/1331322302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 msgr2=0x5573baf4f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 -- 172.21.15.120:0/1331322302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bae74c40 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad617e4640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad61fe5640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf52320 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fad58103640 tx=0x7fad580cbb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14149bd85a1930 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad457fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58049040 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad580ef0c0 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad4c019ae0 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad580d5840 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad6661e740 10 client.19905 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.098+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573baf60fe0 con 0x5573baf460a0 2024-09-10T23:15:44.102 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad4c019ae0 con 0x5573baf460a0 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fad580d53b0 con 0x5573baf460a0 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 10 client.19905 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 10 client.19905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 20 client.19905 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573baf623f0 0x5573baf64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 4 client.19905 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.099+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573baebeda0 con 0x5573baf623f0 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad617e4640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573baf623f0 0x5573baf64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad617e4640 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573baf623f0 0x5573baf64840 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd81d38fb6cdadcc server_cookie=77a2562ea708b98c in_seq=0 out_seq=0 2024-09-10T23:15:44.103 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad457fa640 10 client.19905 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad457fa640 1 -- 172.21.15.120:0/1331322302 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573baebeda0 con 0x5573baf623f0 2024-09-10T23:15:44.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad457fa640 10 client.19905 handle_command_reply: tid=0 2024-09-10T23:15:44.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.100+0000 7fad6661e740 1 client.19905 shutdown 2024-09-10T23:15:44.104 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 msgr2=0x5573baf52320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf52320 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fad58103640 tx=0x7fad580cbb90 comp rx=0 tx=0).stop 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 shutdown_connections 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573baf623f0 0x5573baf64840 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf4fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 --2- 172.21.15.120:0/1331322302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf52320 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 >> 172.21.15.120:0/1331322302 conn(0x5573baf43ed0 msgr2=0x5573baf49a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad457fa640 0 client.19905 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 shutdown_connections 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 1 -- 172.21.15.120:0/1331322302 wait complete. 2024-09-10T23:15:44.105 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.101+0000 7fad6661e740 20 client.19905 trim_cache size 0 max 16384 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 Processor -- start 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 -- start start 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c072410 con 0x5573baf460a0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c074c90 con 0x5573baf469c0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad6661e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c072a80 con 0x5573baf52950 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51560/0 (socket says 172.21.15.120:51560) 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.123+0000 7fad61fe5640 1 -- 172.21.15.120:0/2464657177 learned_addr learned my addr 172.21.15.120:0/2464657177 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad627e6640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad617e4640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 -- 172.21.15.120:0/2464657177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 msgr2=0x5573baf53d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf53d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 -- 172.21.15.120:0/2464657177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 msgr2=0x5573baf4e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 -- 172.21.15.120:0/2464657177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573baf52690 con 0x5573baf460a0 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad627e6640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf53d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:44.126 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad61fe5640 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fad580d5a00 tx=0x7fad580b3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fd84c5d63a6dea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad477fe640 1 -- 172.21.15.120:0/2464657177 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58049070 con 0x5573baf460a0 2024-09-10T23:15:44.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad477fe640 1 -- 172.21.15.120:0/2464657177 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad580b3040 con 0x5573baf460a0 2024-09-10T23:15:44.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.124+0000 7fad477fe640 1 -- 172.21.15.120:0/2464657177 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580ef0b0 con 0x5573baf460a0 2024-09-10T23:15:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 -- 172.21.15.120:0/2464657177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 msgr2=0x5573baf4dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.130 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fad580d5a00 tx=0x7fad580b3f90 comp rx=0 tx=0).stop 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 -- 172.21.15.120:0/2464657177 shutdown_connections 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf52950 0x5573baf53d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf469c0 0x5573baf4e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 --2- 172.21.15.120:0/2464657177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf4dff0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 -- 172.21.15.120:0/2464657177 >> 172.21.15.120:0/2464657177 conn(0x5573baf43ed0 msgr2=0x5573baf4a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 -- 172.21.15.120:0/2464657177 shutdown_connections 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.125+0000 7fad6661e740 1 -- 172.21.15.120:0/2464657177 wait complete. 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 -- start start 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 Processor -- start 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf5a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf469c0 0x5573baf5ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf5a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c073150 con 0x5573baf460a0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0745c0 con 0x5573baf52950 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c073820 con 0x5573baf469c0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad6661e740 10 client.0 _ll_register_callbacks cb 0x7fad580c8200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad61fe5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf5a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51576/0 (socket says 172.21.15.120:51576) 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad61fe5640 1 -- 172.21.15.120:0/2181258196 learned_addr learned my addr 172.21.15.120:0/2181258196 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.126+0000 7fad627e6640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad617e4640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf469c0 0x5573baf5ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 -- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf469c0 msgr2=0x5573baf5ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf469c0 0x5573baf5ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 -- 172.21.15.120:0/2181258196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 msgr2=0x5573baf5a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf5a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 -- 172.21.15.120:0/2181258196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573baf55d90 con 0x5573baf52950 2024-09-10T23:15:44.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad627e6640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf55850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fad54011ca0 tx=0x7fad54004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b787e40196f77661 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad457fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5402d060 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fad5401fae0 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad580b3040 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad5401fce0 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad6661e740 10 client.29057 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.127+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573bafcfb10 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad580b3040 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fad540333b0 con 0x5573baf52950 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 10 client.29057 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 10 client.29057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 20 client.29057 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573bafd0270 0x5573bafd2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 4 client.29057 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:15:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573baeba850 con 0x5573bafd0270 2024-09-10T23:15:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.128+0000 7fad61fe5640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573bafd0270 0x5573bafd2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.129+0000 7fad457fa640 10 client.29057 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.129+0000 7fad61fe5640 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573bafd0270 0x5573bafd2690 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdead66e91dad5e server_cookie=824c3e3b2de47f40 in_seq=0 out_seq=0 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.130+0000 7fad457fa640 1 -- 172.21.15.120:0/2181258196 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+10882 (crc 0 0 0) 0x5573baeba850 con 0x5573bafd0270 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.130+0000 7fad457fa640 10 client.29057 handle_command_reply: tid=0 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.130+0000 7fad6661e740 1 client.29057 shutdown 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.130+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 msgr2=0x5573baf55850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf55850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fad54011ca0 tx=0x7fad54004ab0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.135 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 shutdown_connections 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5573bafd0270 0x5573bafd2690 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573baf52950 0x5573baf55850 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5573baf469c0 0x5573baf5ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad457fa640 0 client.29057 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 --2- 172.21.15.120:0/2181258196 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5573baf460a0 0x5573baf5a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 >> 172.21.15.120:0/2181258196 conn(0x5573baf43ed0 msgr2=0x5573baf4a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 shutdown_connections 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 1 -- 172.21.15.120:0/2181258196 wait complete. 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.131+0000 7fad6661e740 20 client.29057 trim_cache size 0 max 16384 2024-09-10T23:15:44.136 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:15:44.137 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.138 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.139 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:15:44.140 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.142 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:15:44.143 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.144 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.145 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.146 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:15:44.147 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.148 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.149 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.150 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:15:44.151 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.153 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.154 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:15:44.155 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.156 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.157 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:44.158 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 msgr2=0x7fad480b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 0x7fad480b7810 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fad54005fd0 tx=0x7fad54009500 comp rx=0 tx=0).stop 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 msgr2=0x7fad5c19dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad58005940 tx=0x7fad580040c0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 shutdown_connections 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fad480b5350 0x7fad480b7810 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad5c108060 0x7fad5c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad5c1076e0 0x7fad5c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 --2- 172.21.15.120:0/3769524439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad5c101cd0 0x7fad5c19dd80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 >> 172.21.15.120:0/3769524439 conn(0x7fad5c0fd740 msgr2=0x7fad5c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 shutdown_connections 2024-09-10T23:15:44.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:44.132+0000 7fad64249640 1 -- 172.21.15.120:0/3769524439 wait complete. 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 -- 172.21.15.120:0/2432783031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14103380 msgr2=0x7f4c14109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 --2- 172.21.15.120:0/2432783031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14103380 0x7f4c14109d00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009cd0 tx=0x7f4c0c02a8e0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 -- 172.21.15.120:0/2432783031 shutdown_connections 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 --2- 172.21.15.120:0/2432783031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14103380 0x7f4c14109d00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 --2- 172.21.15.120:0/2432783031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c141029c0 0x7f4c14102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 --2- 172.21.15.120:0/2432783031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c14101870 0x7f4c14101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 -- 172.21.15.120:0/2432783031 >> 172.21.15.120:0/2432783031 conn(0x7f4c140fd130 msgr2=0x7f4c140ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 -- 172.21.15.120:0/2432783031 shutdown_connections 2024-09-10T23:15:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.487+0000 7f4c1c016640 1 -- 172.21.15.120:0/2432783031 wait complete. 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 Processor -- start 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 -- start start 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c141029c0 0x7f4c141993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c14103380 0x7f4c14199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410d4e0 con 0x7f4c14101870 2024-09-10T23:15:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410d360 con 0x7f4c14103380 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1c016640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410d660 con 0x7f4c141029c0 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51586/0 (socket says 172.21.15.120:51586) 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 -- 172.21.15.120:0/1508711518 learned_addr learned my addr 172.21.15.120:0/1508711518 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c195b1640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c141029c0 0x7f4c141993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c14103380 0x7f4c14199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 -- 172.21.15.120:0/1508711518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c141029c0 msgr2=0x7f4c141993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c141029c0 0x7f4c141993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 -- 172.21.15.120:0/1508711518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c14103380 msgr2=0x7f4c14199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c14103380 0x7f4c14199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.488+0000 7f4c19db2640 1 -- 172.21.15.120:0/1508711518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1419a850 con 0x7f4c14101870 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9f0 tx=0x7f4c1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c0affd640 1 -- 172.21.15.120:0/1508711518 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000cd80 con 0x7f4c14101870 2024-09-10T23:15:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c0affd640 1 -- 172.21.15.120:0/1508711518 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c100054d0 con 0x7f4c14101870 2024-09-10T23:15:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1419ab40 con 0x7f4c14101870 2024-09-10T23:15:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c0affd640 1 -- 172.21.15.120:0/1508711518 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10010690 con 0x7f4c14101870 2024-09-10T23:15:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.489+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c14103e90 con 0x7f4c14101870 2024-09-10T23:15:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.492+0000 7f4c1edec740 1 Processor -- start 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.492+0000 7f4c1edec740 1 -- start start 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f141460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 0x56180f1468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56180f0a86f0 con 0x56180f13f070 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56180f097b20 con 0x56180f1419a0 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1edec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56180f09b930 con 0x56180f1443e0 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c0affd640 1 -- 172.21.15.120:0/1508711518 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4c10004e90 con 0x7f4c14101870 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c1a5b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39856/0 (socket says 172.21.15.120:39856) 2024-09-10T23:15:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c19db2640 1 -- 172.21.15.120:0/3657026632 learned_addr learned my addr 172.21.15.120:0/3657026632 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.493+0000 7f4c195b1640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 0x56180f1468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c0affd640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 0x7f4c000b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3657026632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 msgr2=0x56180f1468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 0x56180f1468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3657026632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 msgr2=0x56180f143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3657026632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180f05d530 con 0x56180f13f070 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c195b1640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 0x56180f1468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c0affd640 1 -- 172.21.15.120:0/1508711518 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c100e5f80 con 0x7f4c14101870 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c19db2640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.494+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 0x7f4c000b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.495+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f141460 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009ca0 tx=0x7f4c0c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4aec1262771d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.495+0000 7f4bfffff640 1 -- 172.21.15.120:0/3657026632 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c042040 con 0x56180f13f070 2024-09-10T23:15:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.495+0000 7f4bfffff640 1 -- 172.21.15.120:0/3657026632 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c04d070 con 0x56180f13f070 2024-09-10T23:15:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.495+0000 7f4bfffff640 1 -- 172.21.15.120:0/3657026632 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c030ce0 con 0x56180f13f070 2024-09-10T23:15:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.495+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 0x7f4c000b7820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56180f146f90 tx=0x7f4c0c04d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 -- 172.21.15.120:0/3657026632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 msgr2=0x56180f141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f141460 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009ca0 tx=0x7f4c0c005ef0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 -- 172.21.15.120:0/3657026632 shutdown_connections 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1443e0 0x56180f1468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1419a0 0x56180f143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3657026632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f141460 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 -- 172.21.15.120:0/3657026632 >> 172.21.15.120:0/3657026632 conn(0x56180f13a1c0 msgr2=0x56180f13af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 -- 172.21.15.120:0/3657026632 shutdown_connections 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.496+0000 7f4c1edec740 1 -- 172.21.15.120:0/3657026632 wait complete. 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.497+0000 7f4c1edec740 1 -- start start 2024-09-10T23:15:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.497+0000 7f4c1edec740 1 Processor -- start 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 0x56180f149a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56180efcfab0 con 0x56180f1443e0 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x56180f03d730 con 0x56180f13f070 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56180f1473c0 con 0x56180f1419a0 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1edec740 10 client.0 _ll_register_callbacks cb 0x56180f0916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39864/0 (socket says 172.21.15.120:39864) 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c19db2640 1 -- 172.21.15.120:0/1129052256 learned_addr learned my addr 172.21.15.120:0/1129052256 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.498+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 0x56180f149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 -- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 msgr2=0x56180f148ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f148ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 -- 172.21.15.120:0/1129052256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 msgr2=0x56180f149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 0x56180f149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 -- 172.21.15.120:0/1129052256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180edf9790 con 0x56180f13f070 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 0x56180f149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c19db2640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9c0 tx=0x7f4c10009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e9455075e6e1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4bfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100ac0c0 con 0x56180f13f070 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c100b5870 con 0x56180f13f070 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c0c04d070 con 0x56180f13f070 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c100a8610 con 0x56180f13f070 2024-09-10T23:15:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c1edec740 10 client.29069 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56180ee72620 con 0x56180f13f070 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.499+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c04d070 con 0x56180f13f070 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4c100a80c0 con 0x56180f13f070 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 10 client.29069 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 10 client.29069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 20 client.29069 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f154140 0x56180f156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 4 client.29069 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56180f0aa640 con 0x56180f154140 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.500+0000 7f4c195b1640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f154140 0x56180f156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.501+0000 7f4bfd7fa640 10 client.29069 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.501+0000 7f4c195b1640 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f154140 0x56180f156570 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac72edeaa8fa835d server_cookie=fd946896be6f56e5 in_seq=0 out_seq=0 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.502+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/1129052256 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56180f0aa640 con 0x56180f154140 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.502+0000 7f4bfd7fa640 10 client.29069 handle_command_reply: tid=0 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 client.29069 shutdown 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 msgr2=0x56180f148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9c0 tx=0x7f4c10009740 comp rx=0 tx=0).stop 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 shutdown_connections 2024-09-10T23:15:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f154140 0x56180f156570 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1443e0 0x56180f149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4bfd7fa640 0 client.29069 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 --2- 172.21.15.120:0/1129052256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f13f070 0x56180f148800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 >> 172.21.15.120:0/1129052256 conn(0x56180f13a1c0 msgr2=0x56180f141d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 shutdown_connections 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 1 -- 172.21.15.120:0/1129052256 wait complete. 2024-09-10T23:15:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.503+0000 7f4c1edec740 20 client.29069 trim_cache size 0 max 16384 2024-09-10T23:15:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.522+0000 7f4c1edec740 1 Processor -- start 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.522+0000 7f4c1edec740 1 -- start start 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f13aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f13af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a5b50 con 0x56180f13f070 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a83d0 con 0x56180f1443e0 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1edec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a61c0 con 0x56180f1419a0 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51614/0 (socket says 172.21.15.120:51614) 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c19db2640 1 -- 172.21.15.120:0/3000299184 learned_addr learned my addr 172.21.15.120:0/3000299184 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f13af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c195b1640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f13aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3000299184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 msgr2=0x56180f13aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f13aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3000299184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 msgr2=0x56180f12cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/3000299184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180f1417d0 con 0x56180f1443e0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c19db2640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f13af60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009cd0 tx=0x7f4c0c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aea630b386fe116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.523+0000 7f4bff7fe640 1 -- 172.21.15.120:0/3000299184 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c059070 con 0x56180f1443e0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4bff7fe640 1 -- 172.21.15.120:0/3000299184 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c044070 con 0x56180f1443e0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4bff7fe640 1 -- 172.21.15.120:0/3000299184 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c042040 con 0x56180f1443e0 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 -- 172.21.15.120:0/3000299184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 msgr2=0x56180f13af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f13af60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009cd0 tx=0x7f4c0c004ab0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 -- 172.21.15.120:0/3000299184 shutdown_connections 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f13af60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f1419a0 0x56180f13aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 --2- 172.21.15.120:0/3000299184 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f13f070 0x56180f12cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 -- 172.21.15.120:0/3000299184 >> 172.21.15.120:0/3000299184 conn(0x56180f13a1c0 msgr2=0x56180f147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 -- 172.21.15.120:0/3000299184 shutdown_connections 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.524+0000 7f4c1edec740 1 -- 172.21.15.120:0/3000299184 wait complete. 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 -- start start 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 Processor -- start 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 0x56180f14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 0x56180f150300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f1bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a6890 con 0x56180f1419a0 2024-09-10T23:15:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a7d00 con 0x56180f1443e0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141a6f60 con 0x56180f13f070 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1edec740 10 client.0 _ll_register_callbacks cb 0x7f4c10004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 0x56180f14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c19db2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 0x56180f14fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46510/0 (socket says 172.21.15.120:46510) 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c19db2640 1 -- 172.21.15.120:0/2735326047 learned_addr learned my addr 172.21.15.120:0/2735326047 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c195b1640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 0x56180f150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.525+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f1bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 msgr2=0x56180f14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 0x56180f14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/2735326047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 msgr2=0x56180f150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 0x56180f150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 -- 172.21.15.120:0/2735326047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180f1bf760 con 0x56180f1443e0 2024-09-10T23:15:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c195b1640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 0x56180f150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1a5b3640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f1bf070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c02ae00 tx=0x7f4c0c004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a2500243df83d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4bfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c062060 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c0c044070 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.526+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4c0c030a20 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c0c059040 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 10 client.29075 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56180f1c0a50 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c044070 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4c0c030070 con 0x56180f1443e0 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 10 client.29075 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 10 client.29075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 20 client.29075 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f1c0f50 0x56180f1c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 4 client.29075 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:15:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56180f0a60f0 con 0x56180f1c0f50 2024-09-10T23:15:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.527+0000 7f4c19db2640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f1c0f50 0x56180f1c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.528+0000 7f4bfd7fa640 10 client.29075 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:44.528+0000 7f4c19db2640 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f1c0f50 0x56180f1c3340 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578e02a0fa6685f2 server_cookie=fb32d4e11fa4c17d in_seq=0 out_seq=0 2024-09-10T23:15:44.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 258233 21.53 MB/sec execute 519 sec latency 202.708 ms 2024-09-10T23:15:45.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:15:45.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4bfd7fa640 1 -- 172.21.15.120:0/2735326047 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56180f0a60f0 con 0x56180f1c0f50 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4bfd7fa640 10 client.29075 handle_command_reply: tid=0 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 client.29075 shutdown 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 msgr2=0x56180f1bf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f1bf070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c02ae00 tx=0x7f4c0c004920 comp rx=0 tx=0).stop 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 shutdown_connections 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x56180f1c0f50 0x56180f1c3340 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56180f1443e0 0x56180f1bf070 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56180f1419a0 0x56180f150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 --2- 172.21.15.120:0/2735326047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56180f13f070 0x56180f14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4bfd7fa640 0 client.29075 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 >> 172.21.15.120:0/2735326047 conn(0x56180f13a1c0 msgr2=0x56180f147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 shutdown_connections 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 1 -- 172.21.15.120:0/2735326047 wait complete. 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.428+0000 7f4c1edec740 20 client.29075 trim_cache size 0 max 16384 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:15:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:15:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:15:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 msgr2=0x7f4c000b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 0x7f4c000b7820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56180f146f90 tx=0x7f4c0c04d000 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 msgr2=0x7f4c14198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9f0 tx=0x7f4c1000edf0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 shutdown_connections 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f4c000b5360 0x7f4c000b7820 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c14103380 0x7f4c14199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c141029c0 0x7f4c141993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 --2- 172.21.15.120:0/1508711518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c14101870 0x7f4c14198d80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 >> 172.21.15.120:0/1508711518 conn(0x7f4c140fd130 msgr2=0x7f4c140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 shutdown_connections 2024-09-10T23:15:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:45.429+0000 7f4c1c016640 1 -- 172.21.15.120:0/1508711518 wait complete. 2024-09-10T23:15:45.465 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:45.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 258472 21.52 MB/sec execute 520 sec latency 448.336 ms 2024-09-10T23:15:45.896 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:45 smithi120 ceph-mon[31247]: pgmap v1046: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 402 op/s 2024-09-10T23:15:45.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:45 smithi153 ceph-mon[34951]: pgmap v1046: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 402 op/s 2024-09-10T23:15:45.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 -- 172.21.15.120:0/2717948689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 msgr2=0x7f61e0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/2717948689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 0x7f61e0102070 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009900 tx=0x7f61d002a410 comp rx=0 tx=0).stop 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 -- 172.21.15.120:0/2717948689 shutdown_connections 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/2717948689 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/2717948689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/2717948689 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 0x7f61e0102070 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 -- 172.21.15.120:0/2717948689 >> 172.21.15.120:0/2717948689 conn(0x7f61e00fd700 msgr2=0x7f61e00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 -- 172.21.15.120:0/2717948689 shutdown_connections 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.994+0000 7f61e5dd4640 1 -- 172.21.15.120:0/2717948689 wait complete. 2024-09-10T23:15:45.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.996+0000 7f61e5dd4640 1 Processor -- start 2024-09-10T23:15:45.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.996+0000 7f61e5dd4640 1 -- start start 2024-09-10T23:15:45.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.996+0000 7f61e5dd4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 0x7f61e01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.996+0000 7f61e5dd4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.998 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.996+0000 7f61e5dd4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61deffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61deffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39898/0 (socket says 172.21.15.120:39898) 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46530/0 (socket says 172.21.15.120:46530) 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61deffd640 1 -- 172.21.15.120:0/3949204991 learned_addr learned my addr 172.21.15.120:0/3949204991 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006b9b0 con 0x7f61e0101c90 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006b830 con 0x7f61e01076a0 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0113000 con 0x7f61e0108020 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 -- 172.21.15.120:0/3949204991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 msgr2=0x7f61e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 -- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 msgr2=0x7f61e01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 0x7f61e01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:45.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.997+0000 7f61dffff640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019f350 con 0x7f61e0108020 2024-09-10T23:15:46.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.998+0000 7f61dffff640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009fd0 tx=0x7f61d800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:46.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.998+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d800ce20 con 0x7f61e0108020 2024-09-10T23:15:46.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.998+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d8005020 con 0x7f61e0108020 2024-09-10T23:15:46.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.998+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019f640 con 0x7f61e0108020 2024-09-10T23:15:46.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.999+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8010690 con 0x7f61e0108020 2024-09-10T23:15:46.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:45.999+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a5d00 con 0x7f61e0108020 2024-09-10T23:15:46.001 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.000+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e0102070 con 0x7f61e0108020 2024-09-10T23:15:46.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.002+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f61d80054d0 con 0x7f61e0108020 2024-09-10T23:15:46.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.003+0000 7f61dcff9640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 0x7f61c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:46.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.003+0000 7f61df7fe640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 0x7f61c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:46.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.003+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f61d80e6320 con 0x7f61e0108020 2024-09-10T23:15:46.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.004+0000 7f61df7fe640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 0x7f61c80b77a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f61d002a930 tx=0x7f61d0005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:46.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.011+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61d80ad210 con 0x7f61e0108020 2024-09-10T23:15:46.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:45 smithi176 ceph-mon[34982]: pgmap v1046: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 402 op/s 2024-09-10T23:15:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.300+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e0109e00 con 0x7f61e0108020 2024-09-10T23:15:46.301 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.301+0000 7f61dcff9640 1 -- 172.21.15.120:0/3949204991 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f61d80acb10 con 0x7f61e0108020 2024-09-10T23:15:46.302 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:15:46.302 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:15:46.302 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 msgr2=0x7f61c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 0x7f61c80b77a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f61d002a930 tx=0x7f61d0005d50 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 msgr2=0x7f61e019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009fd0 tx=0x7f61d800ef10 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 shutdown_connections 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f61c80b52e0 0x7f61c80b77a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61e0108020 0x7f61e019eb80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61e01076a0 0x7f61e01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 --2- 172.21.15.120:0/3949204991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61e0101c90 0x7f61e01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 >> 172.21.15.120:0/3949204991 conn(0x7f61e00fd700 msgr2=0x7f61e00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:46.305 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 shutdown_connections 2024-09-10T23:15:46.306 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:46.304+0000 7f61e5dd4640 1 -- 172.21.15.120:0/3949204991 wait complete. 2024-09-10T23:15:46.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 258923 21.54 MB/sec execute 521 sec latency 88.619 ms 2024-09-10T23:15:46.945 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:46 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3949204991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:46.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:46 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3949204991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:46.972 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:15:47.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:46 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3949204991' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:15:47.260 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:47.691 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:47 smithi120 ceph-mon[31247]: pgmap v1047: 97 pgs: 97 active+clean; 251 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-09-10T23:15:47.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 259241 21.54 MB/sec execute 522 sec latency 154.939 ms 2024-09-10T23:15:47.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 -- 172.21.15.120:0/2056063032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec10a150 msgr2=0x7f82ec1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 --2- 172.21.15.120:0/2056063032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec10a150 0x7f82ec1111b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009860 tx=0x7f82e802a410 comp rx=0 tx=0).stop 2024-09-10T23:15:47.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 -- 172.21.15.120:0/2056063032 shutdown_connections 2024-09-10T23:15:47.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 --2- 172.21.15.120:0/2056063032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec11c0a0 0x7f82ec11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 --2- 172.21.15.120:0/2056063032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 0x7f82ec10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 --2- 172.21.15.120:0/2056063032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec10a150 0x7f82ec1111b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.821+0000 7f82f459c640 1 -- 172.21.15.120:0/2056063032 >> 172.21.15.120:0/2056063032 conn(0x7f82ec06dfc0 msgr2=0x7f82ec06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.823+0000 7f82f459c640 1 -- 172.21.15.120:0/2056063032 shutdown_connections 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.823+0000 7f82f459c640 1 -- 172.21.15.120:0/2056063032 wait complete. 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.823+0000 7f82f459c640 1 Processor -- start 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.823+0000 7f82f459c640 1 -- start start 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 0x7f82ec117d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec11c0a0 0x7f82ec1a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f2338640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f2338640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39914/0 (socket says 172.21.15.120:39914) 2024-09-10T23:15:47.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f2338640 1 -- 172.21.15.120:0/888150030 learned_addr learned my addr 172.21.15.120:0/888150030 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec114110 con 0x7f82ec10a530 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec113f90 con 0x7f82ec10a150 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec114290 con 0x7f82ec11c0a0 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f2b39640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec11c0a0 0x7f82ec1a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.824+0000 7f82f1b37640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 0x7f82ec117d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 -- 172.21.15.120:0/888150030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec11c0a0 msgr2=0x7f82ec1a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec11c0a0 0x7f82ec1a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 -- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 msgr2=0x7f82ec117d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 0x7f82ec117d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec1a5880 con 0x7f82ec10a150 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.825+0000 7f82f2338640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82e8031000 tx=0x7f82e80053d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.826+0000 7f82e37fe640 1 -- 172.21.15.120:0/888150030 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004250 con 0x7f82ec10a150 2024-09-10T23:15:47.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.826+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec1a5b10 con 0x7f82ec10a150 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.826+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec07b3b0 con 0x7f82ec10a150 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.826+0000 7f82e37fe640 1 -- 172.21.15.120:0/888150030 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e80043f0 con 0x7f82ec10a150 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.826+0000 7f82e37fe640 1 -- 172.21.15.120:0/888150030 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004db0 con 0x7f82ec10a150 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.831+0000 7f82f6971740 1 Processor -- start 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.840+0000 7f82e37fe640 1 -- 172.21.15.120:0/888150030 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f82e804c030 con 0x7f82ec10a150 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.842+0000 7f82f6971740 1 -- start start 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a187a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a18710ea0 con 0x557a1879aa10 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a187002d0 con 0x557a187a69a0 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f6971740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a187040e0 con 0x557a1879a0f0 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46558/0 (socket says 172.21.15.120:46558) 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 -- 172.21.15.120:0/2582202018 learned_addr learned my addr 172.21.15.120:0/2582202018 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f1b37640 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 -- 172.21.15.120:0/2582202018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 msgr2=0x557a187a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 -- 172.21.15.120:0/2582202018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 msgr2=0x557a187a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a187a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.843+0000 7f82f2b39640 1 -- 172.21.15.120:0/2582202018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1845e650 con 0x557a1879a0f0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.844+0000 7f82f2b39640 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82e4008dc0 tx=0x7f82e400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ab8d5678eaf7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.854+0000 7f82e37fe640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 0x7f82d80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.854+0000 7f82e37fe640 1 -- 172.21.15.120:0/888150030 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f82e8106820 con 0x7f82ec10a150 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.854+0000 7f82e0ff9640 1 -- 172.21.15.120:0/2582202018 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4014070 con 0x557a1879a0f0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.854+0000 7f82e0ff9640 1 -- 172.21.15.120:0/2582202018 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e4003b70 con 0x557a1879a0f0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.854+0000 7f82e0ff9640 1 -- 172.21.15.120:0/2582202018 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4010790 con 0x557a1879a0f0 2024-09-10T23:15:47.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.856+0000 7f82f6971740 1 -- 172.21.15.120:0/2582202018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 msgr2=0x557a1879a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.856+0000 7f82f6971740 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82e4008dc0 tx=0x7f82e400e720 comp rx=0 tx=0).stop 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 -- 172.21.15.120:0/2582202018 shutdown_connections 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a187a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 --2- 172.21.15.120:0/2582202018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a1879a4d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 -- 172.21.15.120:0/2582202018 >> 172.21.15.120:0/2582202018 conn(0x557a18797f20 msgr2=0x557a187990c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f2338640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 0x7f82d80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 -- 172.21.15.120:0/2582202018 shutdown_connections 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 -- 172.21.15.120:0/2582202018 wait complete. 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.857+0000 7f82f6971740 1 -- start start 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 Processor -- start 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879aa10 0x557a187a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557a18561700 con 0x557a1879a0f0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x557a1869b160 con 0x557a187a69a0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a187a9620 con 0x557a1879aa10 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f6971740 10 client.0 _ll_register_callbacks cb 0x557a186b49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.858+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39938/0 (socket says 172.21.15.120:39938) 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 -- 172.21.15.120:0/1523744557 learned_addr learned my addr 172.21.15.120:0/1523744557 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879aa10 0x557a187a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2338640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 0x7f82d80b79a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82e80ccf40 tx=0x7f82e80c8f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 -- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879aa10 msgr2=0x557a187a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879aa10 0x557a187a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 -- 172.21.15.120:0/1523744557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 msgr2=0x557a187a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 -- 172.21.15.120:0/1523744557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a186c8c90 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.859+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82e4008260 tx=0x7f82e4008290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68c1f0e5868e2aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.860+0000 7f82d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.860+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4014040 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.860+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e4003b70 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.860+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f82e4003970 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.860+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82e402b070 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.863+0000 7f82f6971740 10 client.29090 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.863+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a187d7040 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e4003b70 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f82e4010bc0 con 0x557a187a69a0 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 10 client.29090 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 10 client.29090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:47.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 20 client.29090 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a187d87c0 0x557a187dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 4 client.29090 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a18712df0 con 0x557a187d87c0 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.865+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a187d87c0 0x557a187dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.866+0000 7f82d5ffb640 10 client.29090 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.867+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a187d87c0 0x557a187dabe0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a69be617f22995 server_cookie=2411415f41712ac1 in_seq=0 out_seq=0 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.876+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1523744557 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a18712df0 con 0x557a187d87c0 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.876+0000 7f82d5ffb640 10 client.29090 handle_command_reply: tid=0 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.876+0000 7f82f6971740 1 client.29090 shutdown 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 msgr2=0x557a187a3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f82e4008260 tx=0x7f82e4008290 comp rx=0 tx=0).stop 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 shutdown_connections 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a187d87c0 0x557a187dabe0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a187a3db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879aa10 0x557a187a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 --2- 172.21.15.120:0/1523744557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 >> 172.21.15.120:0/1523744557 conn(0x557a18797f20 msgr2=0x557a1879da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82d5ffb640 0 client.29090 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 shutdown_connections 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.877+0000 7f82f6971740 1 -- 172.21.15.120:0/1523744557 wait complete. 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.882+0000 7f82f6971740 20 client.29090 trim_cache size 0 max 16384 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 Processor -- start 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 -- start start 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a187a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a18799520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000c4b0 con 0x557a1879aa10 2024-09-10T23:15:47.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000ed30 con 0x557a187a69a0 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f6971740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000cb20 con 0x557a1879a0f0 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f1b37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f1b37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51676/0 (socket says 172.21.15.120:51676) 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.910+0000 7f82f1b37640 1 -- 172.21.15.120:0/1776070436 learned_addr learned my addr 172.21.15.120:0/1776070436 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 -- 172.21.15.120:0/1776070436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 msgr2=0x557a187a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a187a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 -- 172.21.15.120:0/1776070436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 msgr2=0x557a18799520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a18799520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 -- 172.21.15.120:0/1776070436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a187989a0 con 0x557a1879aa10 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005fd0 tx=0x7f82dc01a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c368a1f1113f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82d7fff640 1 -- 172.21.15.120:0/1776070436 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc027070 con 0x557a1879aa10 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82d7fff640 1 -- 172.21.15.120:0/1776070436 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82dc00fd60 con 0x557a1879aa10 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.911+0000 7f82d7fff640 1 -- 172.21.15.120:0/1776070436 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc003d60 con 0x557a1879aa10 2024-09-10T23:15:47.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- 172.21.15.120:0/1776070436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 msgr2=0x557a18798fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005fd0 tx=0x7f82dc01a270 comp rx=0 tx=0).stop 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- 172.21.15.120:0/1776070436 shutdown_connections 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a187a69a0 0x557a18799520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879aa10 0x557a18798fe0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 --2- 172.21.15.120:0/1776070436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1879a0f0 0x557a187a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- 172.21.15.120:0/1776070436 >> 172.21.15.120:0/1776070436 conn(0x557a18797f20 msgr2=0x557a187a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- 172.21.15.120:0/1776070436 shutdown_connections 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- 172.21.15.120:0/1776070436 wait complete. 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.912+0000 7f82f6971740 1 -- start start 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 Processor -- start 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 0x557a187dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.930 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a187a69a0 0x557a187e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000d1f0 con 0x557a1879a0f0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000e660 con 0x557a1879aa10 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000d8c0 con 0x557a187a69a0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f6971740 10 client.0 _ll_register_callbacks cb 0x7f82e4008c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51684/0 (socket says 172.21.15.120:51684) 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 -- 172.21.15.120:0/1028214999 learned_addr learned my addr 172.21.15.120:0/1028214999 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 0x557a187dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a187a69a0 0x557a187e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 -- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a187a69a0 msgr2=0x557a187e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a187a69a0 0x557a187e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 -- 172.21.15.120:0/1028214999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 msgr2=0x557a187dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 0x557a187dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.931 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 -- 172.21.15.120:0/1028214999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a187e0af0 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 0x557a187dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82f2b39640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005170 tx=0x7f82e4008510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97f5e9a4c3d33a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.913+0000 7f82d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4014070 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f82e4003970 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc00fd60 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82e4003b70 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 10 client.19944 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a18845670 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82dc00fd60 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f82e4030480 con 0x557a1879a0f0 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 10 client.19944 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 10 client.19944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:47.932 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 20 client.19944 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a18845b80 0x557a18847fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 4 client.19944 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a1870e8a0 con 0x557a18845b80 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.914+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a18845b80 0x557a18847fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.915+0000 7f82d5ffb640 10 client.19944 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.915+0000 7f82f1b37640 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a18845b80 0x557a18847fa0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dbb97cd36e87a77 server_cookie=16d9cc780e9ab49 in_seq=0 out_seq=0 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.916+0000 7f82d5ffb640 1 -- 172.21.15.120:0/1028214999 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a1870e8a0 con 0x557a18845b80 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.916+0000 7f82d5ffb640 10 client.19944 handle_command_reply: tid=0 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.916+0000 7f82f6971740 1 client.19944 shutdown 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 msgr2=0x557a187aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005170 tx=0x7f82e4008510 comp rx=0 tx=0).stop 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 shutdown_connections 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x557a18845b80 0x557a18847fa0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.933 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a187a69a0 0x557a187e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557a1879aa10 0x557a187dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 --2- 172.21.15.120:0/1028214999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557a1879a0f0 0x557a187aefa0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82d5ffb640 0 client.19944 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 >> 172.21.15.120:0/1028214999 conn(0x557a18797f20 msgr2=0x557a18798fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 shutdown_connections 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 1 -- 172.21.15.120:0/1028214999 wait complete. 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f6971740 20 client.19944 trim_cache size 0 max 16384 2024-09-10T23:15:47.934 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:15:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 msgr2=0x7f82d80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 0x7f82d80b79a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82e80ccf40 tx=0x7f82e80c8f60 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 msgr2=0x7f82ec117840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82e8031000 tx=0x7f82e80053d0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 shutdown_connections 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f82d80b54e0 0x7f82d80b79a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f82ec11c0a0 0x7f82ec1a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82ec10a530 0x7f82ec117d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 --2- 172.21.15.120:0/888150030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec10a150 0x7f82ec117840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.917+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 >> 172.21.15.120:0/888150030 conn(0x7f82ec06dfc0 msgr2=0x7f82ec0729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.918+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 shutdown_connections 2024-09-10T23:15:47.936 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:47.918+0000 7f82f459c640 1 -- 172.21.15.120:0/888150030 wait complete. 2024-09-10T23:15:47.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:47 smithi153 ceph-mon[34951]: pgmap v1047: 97 pgs: 97 active+clean; 251 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-09-10T23:15:48.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:47 smithi176 ceph-mon[34982]: pgmap v1047: 97 pgs: 97 active+clean; 251 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-09-10T23:15:48.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 259570 21.52 MB/sec execute 523 sec latency 201.352 ms 2024-09-10T23:15:49.737 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:15:49.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:15:49.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 259949 21.51 MB/sec execute 524 sec latency 59.263 ms 2024-09-10T23:15:49.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:49 smithi120 ceph-mon[31247]: pgmap v1048: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-09-10T23:15:49.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:49 smithi153 ceph-mon[34951]: pgmap v1048: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-09-10T23:15:50.015 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:50.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:49 smithi176 ceph-mon[34982]: pgmap v1048: 97 pgs: 97 active+clean; 243 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.532+0000 7f1ab6ffd640 1 -- 172.21.15.120:0/2310252947 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8030d80 con 0x7f1ab811c0a0 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.533+0000 7f1abe688640 1 -- 172.21.15.120:0/2310252947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 msgr2=0x7f1ab811e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.533+0000 7f1abe688640 1 --2- 172.21.15.120:0/2310252947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab811e490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009860 tx=0x7f1aa802a410 comp rx=0 tx=0).stop 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.533+0000 7f1abe688640 1 -- 172.21.15.120:0/2310252947 shutdown_connections 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.533+0000 7f1abe688640 1 --2- 172.21.15.120:0/2310252947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab811e490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.534 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.534+0000 7f1abe688640 1 --2- 172.21.15.120:0/2310252947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.534+0000 7f1abe688640 1 --2- 172.21.15.120:0/2310252947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 0x7f1ab81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.534+0000 7f1abe688640 1 -- 172.21.15.120:0/2310252947 >> 172.21.15.120:0/2310252947 conn(0x7f1ab806dfc0 msgr2=0x7f1ab806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.534+0000 7f1abe688640 1 -- 172.21.15.120:0/2310252947 shutdown_connections 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.534+0000 7f1abe688640 1 -- 172.21.15.120:0/2310252947 wait complete. 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 Processor -- start 2024-09-10T23:15:50.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 -- start start 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 0x7f1ab81a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab81ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8121c10 con 0x7f1ab810a150 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8121a90 con 0x7f1ab810a530 2024-09-10T23:15:50.536 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.535+0000 7f1abe688640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8121d90 con 0x7f1ab811c0a0 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39980/0 (socket says 172.21.15.120:39980) 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 -- 172.21.15.120:0/1499396464 learned_addr learned my addr 172.21.15.120:0/1499396464 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1abcc25640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab81ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 0x7f1ab81a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 -- 172.21.15.120:0/1499396464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 msgr2=0x7f1ab81ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab81ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 -- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 msgr2=0x7f1ab81a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.537 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 0x7f1ab81a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.536+0000 7f1ab77fe640 1 -- 172.21.15.120:0/1499396464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81ae140 con 0x7f1ab810a530 2024-09-10T23:15:50.538 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.537+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9c0 tx=0x7f1aac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.538+0000 7f1ab57fa640 1 -- 172.21.15.120:0/1499396464 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00ccc0 con 0x7f1ab810a530 2024-09-10T23:15:50.539 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.538+0000 7f1ab57fa640 1 -- 172.21.15.120:0/1499396464 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac0054d0 con 0x7f1ab810a530 2024-09-10T23:15:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.539+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab81ae430 con 0x7f1ab810a530 2024-09-10T23:15:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.539+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81b4b70 con 0x7f1ab810a530 2024-09-10T23:15:50.540 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.539+0000 7f1ab57fa640 1 -- 172.21.15.120:0/1499396464 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac010430 con 0x7f1ab810a530 2024-09-10T23:15:50.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.544+0000 7f1ac0a5d740 1 Processor -- start 2024-09-10T23:15:50.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.545+0000 7f1ab57fa640 1 -- 172.21.15.120:0/1499396464 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1aac004f30 con 0x7f1ab810a530 2024-09-10T23:15:50.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 -- start start 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55992cea5e60 con 0x55992cf2f800 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55992ce95290 con 0x55992cf2eee0 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55992ce990a0 con 0x55992cf3b790 2024-09-10T23:15:50.547 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1abcc25640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1abcc25640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:39996/0 (socket says 172.21.15.120:39996) 2024-09-10T23:15:50.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1abcc25640 1 -- 172.21.15.120:0/551484197 learned_addr learned my addr 172.21.15.120:0/551484197 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:50.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.546+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.547+0000 7f1abcc25640 1 -- 172.21.15.120:0/551484197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 msgr2=0x55992cf3db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.548 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1ab57fa640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 0x7f1aa00b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 -- 172.21.15.120:0/551484197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 msgr2=0x55992cf3b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf3b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 -- 172.21.15.120:0/551484197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992cbf3930 con 0x55992cf2eee0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 0x7f1aa00b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.548+0000 7f1abcc25640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80058c0 tx=0x7f1aa80305c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca044e5a996b009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.549+0000 7f1ab57fa640 1 -- 172.21.15.120:0/1499396464 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1aac0e5e10 con 0x7f1ab810a530 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.549+0000 7f1a9e7fc640 1 -- 172.21.15.120:0/551484197 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa804e070 con 0x55992cf2eee0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.549+0000 7f1abcc25640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 0x7f1aa00b7800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1aa804b000 tx=0x7f1aa804c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.550+0000 7f1a9e7fc640 1 -- 172.21.15.120:0/551484197 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa8041070 con 0x55992cf2eee0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.550+0000 7f1a9e7fc640 1 -- 172.21.15.120:0/551484197 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80046c0 con 0x55992cf2eee0 2024-09-10T23:15:50.551 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.550+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf3b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:50.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/551484197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 msgr2=0x55992cf2f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80058c0 tx=0x7f1aa80305c0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/551484197 shutdown_connections 2024-09-10T23:15:50.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf3db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf3b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/551484197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf2f2c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.552+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/551484197 >> 172.21.15.120:0/551484197 conn(0x55992cf2cd10 msgr2=0x55992cf2deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.553+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/551484197 shutdown_connections 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.553+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/551484197 wait complete. 2024-09-10T23:15:50.554 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.553+0000 7f1ac0a5d740 1 -- start start 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.553+0000 7f1ac0a5d740 1 Processor -- start 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf38750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf38c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55992ccf6890 con 0x55992cf2f800 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55992ce2fd40 con 0x55992cf2eee0 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55992cf3e440 con 0x55992cf3b790 2024-09-10T23:15:50.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.554+0000 7f1ac0a5d740 10 client.0 _ll_register_callbacks cb 0x55992ce93c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:50.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46620/0 (socket says 172.21.15.120:46620) 2024-09-10T23:15:50.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 -- 172.21.15.120:0/3550686365 learned_addr learned my addr 172.21.15.120:0/3550686365 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:50.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf38c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 -- 172.21.15.120:0/3550686365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 msgr2=0x55992cf38750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf38750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 -- 172.21.15.120:0/3550686365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 msgr2=0x55992cf38c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf38c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.555+0000 7f1abcc25640 1 -- 172.21.15.120:0/3550686365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992cf58830 con 0x55992cf3b790 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.556+0000 7f1abcc25640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80048f0 tx=0x7f1aa8005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58172167ccc94105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.556+0000 7f1a977fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.556+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa804e050 con 0x55992cf3b790 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.556+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1aa8004e00 con 0x55992cf3b790 2024-09-10T23:15:50.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.557+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa8041070 con 0x55992cf3b790 2024-09-10T23:15:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.558+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8005530 con 0x55992cf3b790 2024-09-10T23:15:50.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.559+0000 7f1ac0a5d740 10 client.29383 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:50.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.559+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55992cf5b1e0 con 0x55992cf3b790 2024-09-10T23:15:50.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.560+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa8041070 con 0x55992cf3b790 2024-09-10T23:15:50.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1aa8003760 con 0x55992cf3b790 2024-09-10T23:15:50.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 10 client.29383 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:50.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 10 client.29383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:50.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 20 client.29383 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:50.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cf5c530 0x55992cf5e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 4 client.29383 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:50.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.561+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55992cea7db0 con 0x55992cf5c530 2024-09-10T23:15:50.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.565+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cf5c530 0x55992cf5e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.567+0000 7f1a977fe640 10 client.29383 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:50.568 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.567+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cf5c530 0x55992cf5e920 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772271010fcc3ea server_cookie=37c75edaa52c38d2 in_seq=0 out_seq=0 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.577+0000 7f1a977fe640 1 -- 172.21.15.120:0/3550686365 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55992cea7db0 con 0x55992cf5c530 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.577+0000 7f1a977fe640 10 client.29383 handle_command_reply: tid=0 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 client.29383 shutdown 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 msgr2=0x55992cf391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80048f0 tx=0x7f1aa8005d70 comp rx=0 tx=0).stop 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 shutdown_connections 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cf5c530 0x55992cf5e920 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf3b790 0x55992cf391d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf38c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3550686365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2eee0 0x55992cf38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 >> 172.21.15.120:0/3550686365 conn(0x55992cf2cd10 msgr2=0x55992cf2bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1a977fe640 0 client.29383 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 shutdown_connections 2024-09-10T23:15:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.578+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3550686365 wait complete. 2024-09-10T23:15:50.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.579+0000 7f1ac0a5d740 20 client.29383 trim_cache size 0 max 16384 2024-09-10T23:15:50.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 Processor -- start 2024-09-10T23:15:50.600 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 -- start start 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 0x55992cf35980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8074940 con 0x55992cf2f800 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b4e90 con 0x55992cf3b790 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81a9210 con 0x55992cf2eee0 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51732/0 (socket says 172.21.15.120:51732) 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 -- 172.21.15.120:0/3757347345 learned_addr learned my addr 172.21.15.120:0/3757347345 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1abcc25640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 0x55992cf35980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 -- 172.21.15.120:0/3757347345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 msgr2=0x55992cf38c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf38c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.601 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 -- 172.21.15.120:0/3757347345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 msgr2=0x55992cf35980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 0x55992cf35980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1ab77fe640 1 -- 172.21.15.120:0/3757347345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992cf2d790 con 0x55992cf2f800 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.599+0000 7f1abcc25640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 0x55992cf35980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.600+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0a9870 tx=0x7f1aac0ac490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0a4a2192ec587d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.600+0000 7f1a9dffb640 1 -- 172.21.15.120:0/3757347345 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0ac110 con 0x55992cf2f800 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.600+0000 7f1a9dffb640 1 -- 172.21.15.120:0/3757347345 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac005680 con 0x55992cf2f800 2024-09-10T23:15:50.602 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.600+0000 7f1a9dffb640 1 -- 172.21.15.120:0/3757347345 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004680 con 0x55992cf2f800 2024-09-10T23:15:50.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3757347345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 msgr2=0x55992cf39190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0a9870 tx=0x7f1aac0ac490 comp rx=0 tx=0).stop 2024-09-10T23:15:50.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3757347345 shutdown_connections 2024-09-10T23:15:50.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf3b790 0x55992cf35980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.605 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf2f800 0x55992cf39190 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/3757347345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf38c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3757347345 >> 172.21.15.120:0/3757347345 conn(0x55992cf2cd10 msgr2=0x55992cf3a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3757347345 shutdown_connections 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/3757347345 wait complete. 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 -- start start 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.601+0000 7f1ac0a5d740 1 Processor -- start 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2f800 0x55992cf60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf3b790 0x55992cfc22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118450 con 0x55992cf3b790 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81198c0 con 0x55992cf2f800 2024-09-10T23:15:50.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118b20 con 0x55992cf2eee0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ac0a5d740 10 client.0 _ll_register_callbacks cb 0x7f1aa804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46650/0 (socket says 172.21.15.120:46650) 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 -- 172.21.15.120:0/1896867667 learned_addr learned my addr 172.21.15.120:0/1896867667 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2f800 0x55992cf60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1abcc25640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf3b790 0x55992cfc22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 -- 172.21.15.120:0/1896867667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2f800 msgr2=0x55992cf60710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2f800 0x55992cf60710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 -- 172.21.15.120:0/1896867667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf3b790 msgr2=0x55992cfc22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf3b790 0x55992cfc22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 -- 172.21.15.120:0/1896867667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992cfc29e0 con 0x55992cf2eee0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1ab7fff640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009540 tx=0x7f1ab000ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc498804ae0f787 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.602+0000 7f1a977fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:50.607 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0023020 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f1ab0019d50 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aac005680 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ab001f480 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 10 client.29389 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55992cfc9ea0 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab001fad0 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f1ab001e8f0 con 0x55992cf2eee0 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 10 client.29389 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 10 client.29389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 20 client.29389 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cfca650 0x55992cfcca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 4 client.29389 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:15:50.608 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.603+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1aac005680 con 0x55992cfca650 2024-09-10T23:15:50.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.604+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cfca650 0x55992cfcca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:50.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.604+0000 7f1a977fe640 10 client.29389 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:50.609 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.604+0000 7f1ab77fe640 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cfca650 0x55992cfcca70 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bb7fd93ca04ac6 server_cookie=3524c9fbafff10a8 in_seq=0 out_seq=0 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.607+0000 7f1a977fe640 1 -- 172.21.15.120:0/1896867667 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1aac005680 con 0x55992cfca650 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.607+0000 7f1a977fe640 10 client.29389 handle_command_reply: tid=0 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.607+0000 7f1ac0a5d740 1 client.29389 shutdown 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 msgr2=0x55992cf60170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009540 tx=0x7f1ab000ace0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 shutdown_connections 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55992cfca650 0x55992cfcca70 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55992cf3b790 0x55992cfc22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55992cf2f800 0x55992cf60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1a977fe640 0 client.29389 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:50.610 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 --2- 172.21.15.120:0/1896867667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55992cf2eee0 0x55992cf60170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 >> 172.21.15.120:0/1896867667 conn(0x55992cf2cd10 msgr2=0x55992cf466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 shutdown_connections 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 1 -- 172.21.15.120:0/1896867667 wait complete. 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.608+0000 7f1ac0a5d740 20 client.29389 trim_cache size 0 max 16384 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "2dbbc85c-b952-4f9d-a994-746a3e80926f", 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:15:50.611 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 msgr2=0x7f1aa00b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 0x7f1aa00b7800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1aa804b000 tx=0x7f1aa804c000 comp rx=0 tx=0).stop 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 msgr2=0x7f1ab81a85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9c0 tx=0x7f1aac00ee80 comp rx=0 tx=0).stop 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 shutdown_connections 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f1aa00b5340 0x7f1aa00b7800 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.612 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ab811c0a0 0x7f1ab81ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab810a530 0x7f1ab81a85d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 --2- 172.21.15.120:0/1499396464 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab810a150 0x7f1ab81a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 >> 172.21.15.120:0/1499396464 conn(0x7f1ab806dfc0 msgr2=0x7f1ab811c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 shutdown_connections 2024-09-10T23:15:50.613 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:50.609+0000 7f1abe688640 1 -- 172.21.15.120:0/1499396464 wait complete. 2024-09-10T23:15:50.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 260287 21.50 MB/sec execute 525 sec latency 89.921 ms 2024-09-10T23:15:51.326 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:15:51.623 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:15:51.875 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:51 smithi120 ceph-mon[31247]: pgmap v1049: 97 pgs: 97 active+clean; 231 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T23:15:51.876 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:51 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:15:51.876 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:51 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:51.876 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:51 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:15:51.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 260425 21.48 MB/sec execute 526 sec latency 354.796 ms 2024-09-10T23:15:51.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:51 smithi153 ceph-mon[34951]: pgmap v1049: 97 pgs: 97 active+clean; 231 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T23:15:51.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:51 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:15:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:51 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:51.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:51 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:15:52.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:51 smithi176 ceph-mon[34982]: pgmap v1049: 97 pgs: 97 active+clean; 231 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T23:15:52.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:51 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:15:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:51 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:52.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:51 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:15:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 -- 172.21.15.120:0/853984796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4954109120 msgr2=0x7f495410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 --2- 172.21.15.120:0/853984796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4954109120 0x7f495410f920 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f494c009ca0 tx=0x7f494c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 -- 172.21.15.120:0/853984796 shutdown_connections 2024-09-10T23:15:52.131 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 --2- 172.21.15.120:0/853984796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4954109120 0x7f495410f920 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 --2- 172.21.15.120:0/853984796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49541087a0 0x7f4954108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 --2- 172.21.15.120:0/853984796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954102d90 0x7f4954103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 -- 172.21.15.120:0/853984796 >> 172.21.15.120:0/853984796 conn(0x7f49540fe820 msgr2=0x7f4954100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 -- 172.21.15.120:0/853984796 shutdown_connections 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 -- 172.21.15.120:0/853984796 wait complete. 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.128+0000 7f4959814640 1 Processor -- start 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 -- start start 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4954102d90 0x7f49541992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 0x7f49541997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4954112da0 con 0x7f49541087a0 2024-09-10T23:15:52.132 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4954112c20 con 0x7f4954109120 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4959814640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4954112f20 con 0x7f4954102d90 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49527fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 0x7f49541997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:40044/0 (socket says 172.21.15.120:40044) 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 -- 172.21.15.120:0/2428356958 learned_addr learned my addr 172.21.15.120:0/2428356958 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4954102d90 0x7f49541992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 -- 172.21.15.120:0/2428356958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4954102d90 msgr2=0x7f49541992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4954102d90 0x7f49541992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 -- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 msgr2=0x7f49541997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 0x7f49541997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49537fe640 1 -- 172.21.15.120:0/2428356958 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495419f360 con 0x7f4954109120 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.129+0000 7f49527fc640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 0x7f49541997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f49537fe640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f494c005cc0 tx=0x7f494c031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.133 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f4958812640 1 -- 172.21.15.120:0/2428356958 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c003b70 con 0x7f4954109120 2024-09-10T23:15:52.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495419f5f0 con 0x7f4954109120 2024-09-10T23:15:52.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f4958812640 1 -- 172.21.15.120:0/2428356958 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f494c0318f0 con 0x7f4954109120 2024-09-10T23:15:52.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f4958812640 1 -- 172.21.15.120:0/2428356958 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c02f950 con 0x7f4954109120 2024-09-10T23:15:52.134 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.130+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49541a5d10 con 0x7f4954109120 2024-09-10T23:15:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.132+0000 7f4958812640 1 -- 172.21.15.120:0/2428356958 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f494c02fb50 con 0x7f4954109120 2024-09-10T23:15:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 Processor -- start 2024-09-10T23:15:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 -- start start 2024-09-10T23:15:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 0x5638f314e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.136 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f315d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638f30c0e50 con 0x5638f315ad60 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638f30b0280 con 0x5638f314e4b0 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f495bbe9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5638f30b4090 con 0x5638f314edd0 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46692/0 (socket says 172.21.15.120:46692) 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 -- 172.21.15.120:0/1958034399 learned_addr learned my addr 172.21.15.120:0/1958034399 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f49527fc640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f315d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f49527fc640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 0x5638f314e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 -- 172.21.15.120:0/1958034399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 msgr2=0x5638f314e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 0x5638f314e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 -- 172.21.15.120:0/1958034399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 msgr2=0x5638f315d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.137 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f315d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f4952ffd640 1 -- 172.21.15.120:0/1958034399 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f2e0e650 con 0x5638f314edd0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.133+0000 7f49527fc640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 0x5638f314e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f4958812640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f4952ffd640 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4944009cf0 tx=0x7f4944011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1751c4c89555d507 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f4958812640 1 -- 172.21.15.120:0/2428356958 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f494c106cc0 con 0x7f4954109120 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f4939ffb640 1 -- 172.21.15.120:0/1958034399 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944004230 con 0x5638f314edd0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f4939ffb640 1 -- 172.21.15.120:0/1958034399 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49440043d0 con 0x5638f314edd0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.134+0000 7f49527fc640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f4939ffb640 1 -- 172.21.15.120:0/1958034399 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944019880 con 0x5638f314edd0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 -- 172.21.15.120:0/1958034399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 msgr2=0x5638f315a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f49527fc640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4948009fd0 tx=0x7f4948005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4944009cf0 tx=0x7f4944011810 comp rx=0 tx=0).stop 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 -- 172.21.15.120:0/1958034399 shutdown_connections 2024-09-10T23:15:52.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f315d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314edd0 0x5638f315a820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 --2- 172.21.15.120:0/1958034399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f314e4b0 0x5638f314e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 -- 172.21.15.120:0/1958034399 >> 172.21.15.120:0/1958034399 conn(0x5638f314c2e0 msgr2=0x5638f314d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 -- 172.21.15.120:0/1958034399 shutdown_connections 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.135+0000 7f495bbe9740 1 -- 172.21.15.120:0/1958034399 wait complete. 2024-09-10T23:15:52.139 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 -- start start 2024-09-10T23:15:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 Processor -- start 2024-09-10T23:15:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.142 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f3157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f3158650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5638f2f11700 con 0x5638f315ad60 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5638f304b110 con 0x5638f31581d0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.136+0000 7f495bbe9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5638f315da80 con 0x5638f314e4b0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f495bbe9740 10 client.0 _ll_register_callbacks cb 0x5638f3064990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:46706/0 (socket says 172.21.15.120:46706) 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 -- 172.21.15.120:0/744916425 learned_addr learned my addr 172.21.15.120:0/744916425 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f49537fe640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f3158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f49527fc640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f3157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 -- 172.21.15.120:0/744916425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 msgr2=0x5638f3158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f3158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.143 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 -- 172.21.15.120:0/744916425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 msgr2=0x5638f3157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f3157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 -- 172.21.15.120:0/744916425 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f3078c40 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f49537fe640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f3158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4952ffd640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4944019520 tx=0x7f4944003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74609bf513e990a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.137+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944027070 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4944005240 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49440043d0 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4944033050 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 10 client.29401 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638f3169870 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49440043d0 con 0x5638f314e4b0 2024-09-10T23:15:52.144 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4944004b70 con 0x5638f314e4b0 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 10 client.29401 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 10 client.29401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 20 client.29401 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f316d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 4 client.29401 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638f30c2da0 con 0x5638f316ac20 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.138+0000 7f49527fc640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f316d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.139+0000 7f4932ffd640 10 client.29401 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.139+0000 7f49527fc640 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f316d010 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba06b981b6e1f196 server_cookie=6fe80d4c17e37700 in_seq=0 out_seq=0 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f4932ffd640 1 -- 172.21.15.120:0/744916425 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638f30c2da0 con 0x5638f316ac20 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f4932ffd640 10 client.29401 handle_command_reply: tid=0 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 client.29401 shutdown 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 msgr2=0x5638f315a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4944019520 tx=0x7f4944003820 comp rx=0 tx=0).stop 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 shutdown_connections 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f316d010 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f4932ffd640 0 client.29401 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f3158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f315ad60 0x5638f3157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 --2- 172.21.15.120:0/744916425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f314e4b0 0x5638f315a730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 >> 172.21.15.120:0/744916425 conn(0x5638f314c2e0 msgr2=0x5638f3151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 shutdown_connections 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 1 -- 172.21.15.120:0/744916425 wait complete. 2024-09-10T23:15:52.146 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.140+0000 7f495bbe9740 20 client.29401 trim_cache size 0 max 16384 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.159+0000 7f495bbe9740 1 Processor -- start 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.159+0000 7f495bbe9740 1 -- start start 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f315c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f3154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49541aa0d0 con 0x5638f314e4b0 2024-09-10T23:15:52.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49541ac930 con 0x5638f31581d0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f495bbe9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49541aa720 con 0x5638f315ad60 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51790/0 (socket says 172.21.15.120:51790) 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 -- 172.21.15.120:0/3740965474 learned_addr learned my addr 172.21.15.120:0/3740965474 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f49527fc640 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f315c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f49537fe640 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f3154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 -- 172.21.15.120:0/3740965474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 msgr2=0x5638f3154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f3154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 -- 172.21.15.120:0/3740965474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 msgr2=0x5638f315c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f315c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 -- 172.21.15.120:0/3740965474 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f316d010 con 0x5638f314e4b0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f4952ffd640 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f49440059d0 tx=0x7f4944009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57be3e6636edddbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f49397fa640 1 -- 172.21.15.120:0/3740965474 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944030070 con 0x5638f314e4b0 2024-09-10T23:15:52.162 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f49397fa640 1 -- 172.21.15.120:0/3740965474 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4944009910 con 0x5638f314e4b0 2024-09-10T23:15:52.163 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.160+0000 7f49397fa640 1 -- 172.21.15.120:0/3740965474 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944027070 con 0x5638f314e4b0 2024-09-10T23:15:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 -- 172.21.15.120:0/3740965474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 msgr2=0x5638f315c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f49440059d0 tx=0x7f4944009640 comp rx=0 tx=0).stop 2024-09-10T23:15:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 -- 172.21.15.120:0/3740965474 shutdown_connections 2024-09-10T23:15:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f3154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.165 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f315c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 --2- 172.21.15.120:0/3740965474 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f315c150 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 -- 172.21.15.120:0/3740965474 >> 172.21.15.120:0/3740965474 conn(0x5638f314c2e0 msgr2=0x5638f314d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 -- 172.21.15.120:0/3740965474 shutdown_connections 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.161+0000 7f495bbe9740 1 -- 172.21.15.120:0/3740965474 wait complete. 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 -- start start 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 Processor -- start 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f31632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f315dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49541aadf0 con 0x5638f314e4b0 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49541ac260 con 0x5638f31581d0 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49541ab4c0 con 0x5638f315ad60 2024-09-10T23:15:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f495bbe9740 10 client.0 _ll_register_callbacks cb 0x7f4944009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:51804/0 (socket says 172.21.15.120:51804) 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f4952ffd640 1 -- 172.21.15.120:0/2652145173 learned_addr learned my addr 172.21.15.120:0/2652145173 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f49537fe640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f315dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.162+0000 7f49527fc640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f31632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 -- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 msgr2=0x5638f315dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f315dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 -- 172.21.15.120:0/2652145173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 msgr2=0x5638f31632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f31632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 -- 172.21.15.120:0/2652145173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f315e290 con 0x5638f314e4b0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f49527fc640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f31632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4952ffd640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4944004f40 tx=0x7f4944003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56f0f28e6df7636 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944030070 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4944009910 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f49440051c0 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4944027040 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f495bbe9740 10 client.19974 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.163+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638f31d80b0 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4944009910 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f4944004970 con 0x5638f314e4b0 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 10 client.19974 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 10 client.19974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 20 client.19974 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f31da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 4 client.19974 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:15:52.168 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638f30be850 con 0x5638f316ac20 2024-09-10T23:15:52.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.164+0000 7f49527fc640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f31da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:52.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.165+0000 7f4932ffd640 10 client.19974 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:52.169 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.165+0000 7f49527fc640 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f31da760 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a484a69f703069 server_cookie=c50246611cb05c4 in_seq=0 out_seq=0 2024-09-10T23:15:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f4932ffd640 1 -- 172.21.15.120:0/2652145173 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+11362 (crc 0 0 0) 0x5638f30be850 con 0x5638f316ac20 2024-09-10T23:15:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f4932ffd640 10 client.19974 handle_command_reply: tid=0 2024-09-10T23:15:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 client.19974 shutdown 2024-09-10T23:15:52.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 msgr2=0x5638f3162da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4944004f40 tx=0x7f4944003f20 comp rx=0 tx=0).stop 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 shutdown_connections 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5638f316ac20 0x5638f31da760 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5638f315ad60 0x5638f315dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5638f31581d0 0x5638f31632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f4932ffd640 0 client.19974 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 --2- 172.21.15.120:0/2652145173 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5638f314e4b0 0x5638f3162da0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.166+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 >> 172.21.15.120:0/2652145173 conn(0x5638f314c2e0 msgr2=0x5638f3165d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 shutdown_connections 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f495bbe9740 1 -- 172.21.15.120:0/2652145173 wait complete. 2024-09-10T23:15:52.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f495bbe9740 20 client.19974 trim_cache size 0 max 16384 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "2dbbc85c-b952-4f9d-a994-746a3e80926f": { 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "2dbbc85c-b952-4f9d-a994-746a3e80926f", 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:15:52.172 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.173 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:15:52.174 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.176 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.177 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.177 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:15:52.178 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.179 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:15:52.180 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:15:52.180 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.180 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.181 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.181 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:15:52.182 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.183 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.184 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.185 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:15:52.186 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.187 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.188 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "2dbbc85c-b952-4f9d-a994-746a3e80926f": { 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:50.606606+0000", 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.189 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.190 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.191 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.192 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:15:52.193 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.194 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.195 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:15:52.196 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.197 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:52.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 msgr2=0x7f493c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4948009fd0 tx=0x7f4948005f90 comp rx=0 tx=0).stop 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 msgr2=0x7f495419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f494c005cc0 tx=0x7f494c031700 comp rx=0 tx=0).stop 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 shutdown_connections 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f493c0b52e0 0x7f493c0b77a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.199 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.167+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4954109120 0x7f495419eb90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.168+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49541087a0 0x7f49541997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.168+0000 7f4959814640 1 --2- 172.21.15.120:0/2428356958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4954102d90 0x7f49541992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.168+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 >> 172.21.15.120:0/2428356958 conn(0x7f49540fe820 msgr2=0x7f4954100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.168+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 shutdown_connections 2024-09-10T23:15:52.200 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:15:52.168+0000 7f4959814640 1 -- 172.21.15.120:0/2428356958 wait complete. 2024-09-10T23:15:52.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 260777 21.47 MB/sec execute 527 sec latency 452.872 ms 2024-09-10T23:15:52.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:52 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:52.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:52 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:53.032 INFO:tasks.cephfs.filesystem:scrub status for tag:2dbbc85c-b952-4f9d-a994-746a3e80926f - {'path': '/', 'tag': '2dbbc85c-b952-4f9d-a994-746a3e80926f', 'options': 'recursive,force'} 2024-09-10T23:15:53.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:52 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:15:53.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 261092 21.45 MB/sec execute 528 sec latency 196.421 ms 2024-09-10T23:15:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:53 smithi120 ceph-mon[31247]: pgmap v1050: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-10T23:15:53.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:53 smithi153 ceph-mon[34951]: pgmap v1050: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-10T23:15:54.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:53 smithi176 ceph-mon[34982]: pgmap v1050: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-10T23:15:54.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 261583 21.46 MB/sec execute 529 sec latency 90.134 ms 2024-09-10T23:15:55.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 262034 21.47 MB/sec execute 530 sec latency 97.091 ms 2024-09-10T23:15:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:55 smithi120 ceph-mon[31247]: pgmap v1051: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 565 op/s 2024-09-10T23:15:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:55 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:15:55.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:55 smithi153 ceph-mon[34951]: pgmap v1051: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 565 op/s 2024-09-10T23:15:55.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:55 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:15:56.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:55 smithi176 ceph-mon[34982]: pgmap v1051: 97 pgs: 97 active+clean; 237 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 565 op/s 2024-09-10T23:15:56.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:55 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:15:56.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 262534 21.50 MB/sec execute 531 sec latency 74.021 ms 2024-09-10T23:15:57.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 262984 21.51 MB/sec execute 532 sec latency 80.204 ms 2024-09-10T23:15:57.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:57 smithi120 ceph-mon[31247]: pgmap v1052: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 692 op/s 2024-09-10T23:15:57.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:57 smithi153 ceph-mon[34951]: pgmap v1052: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 692 op/s 2024-09-10T23:15:58.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:57 smithi176 ceph-mon[34982]: pgmap v1052: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.2 MiB/s wr, 692 op/s 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 -- 172.21.15.120:0/3733090180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 msgr2=0x7f42b8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/3733090180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b8102c90 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009930 tx=0x7f42ac02a420 comp rx=0 tx=0).stop 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 -- 172.21.15.120:0/3733090180 shutdown_connections 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/3733090180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81031d0 0x7f42b8109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/3733090180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b8102c90 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/3733090180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81016c0 0x7f42b8101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 -- 172.21.15.120:0/3733090180 >> 172.21.15.120:0/3733090180 conn(0x7f42b80fcf60 msgr2=0x7f42b80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 -- 172.21.15.120:0/3733090180 shutdown_connections 2024-09-10T23:15:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.523+0000 7f42bd5a1640 1 -- 172.21.15.120:0/3733090180 wait complete. 2024-09-10T23:15:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 Processor -- start 2024-09-10T23:15:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 -- start start 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 0x7f42b8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42b810d330 con 0x7f42b8102810 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42b810d1b0 con 0x7f42b81016c0 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42bd5a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42b810d4b0 con 0x7f42b81031d0 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.524+0000 7f42b7fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58730/0 (socket says 172.21.15.120:58730) 2024-09-10T23:15:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b77fe640 1 -- 172.21.15.120:0/4241706745 learned_addr learned my addr 172.21.15.120:0/4241706745 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:54372/0 (socket says 172.21.15.120:54372) 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42bcda0640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 0x7f42b8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 -- 172.21.15.120:0/4241706745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 msgr2=0x7f42b8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 0x7f42b8198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 -- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 msgr2=0x7f42b81981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 -- 172.21.15.120:0/4241706745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b81996b0 con 0x7f42b81016c0 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b77fe640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42bcda0640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 0x7f42b8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42a800e9f0 tx=0x7f42a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b57fa640 1 -- 172.21.15.120:0/4241706745 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a800ccc0 con 0x7f42b81016c0 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b81999a0 con 0x7f42b81016c0 2024-09-10T23:15:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b57fa640 1 -- 172.21.15.120:0/4241706745 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a8005020 con 0x7f42b81016c0 2024-09-10T23:15:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42b57fa640 1 -- 172.21.15.120:0/4241706745 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8004a00 con 0x7f42b81016c0 2024-09-10T23:15:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.525+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b81a0070 con 0x7f42b81016c0 2024-09-10T23:15:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 Processor -- start 2024-09-10T23:15:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 -- start start 2024-09-10T23:15:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42b57fa640 1 -- 172.21.15.120:0/4241706745 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f42a8004ba0 con 0x7f42b81016c0 2024-09-10T23:15:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004db780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004dbdd0 0x55f9004de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f9004486f0 con 0x55f9004d6a60 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f900437b20 con 0x55f9004dbdd0 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42c15d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f90043b930 con 0x55f9004d9390 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58744/0 (socket says 172.21.15.120:58744) 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42bcda0640 1 -- 172.21.15.120:0/1008437766 learned_addr learned my addr 172.21.15.120:0/1008437766 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42b7fff640 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.528+0000 7f42b77fe640 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004dbdd0 0x55f9004de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 -- 172.21.15.120:0/1008437766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 msgr2=0x55f9004db780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004db780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 -- 172.21.15.120:0/1008437766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004dbdd0 msgr2=0x55f9004de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004dbdd0 0x55f9004de290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 -- 172.21.15.120:0/1008437766 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9003fd530 con 0x55f9004d6a60 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42bcda0640 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f42b00107a0 tx=0x7f42b0010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2529f7333834ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f429e7fc640 1 -- 172.21.15.120:0/1008437766 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0003b70 con 0x55f9004d6a60 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42b57fa640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 0x7f42a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f429e7fc640 1 -- 172.21.15.120:0/1008437766 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b000ede0 con 0x55f9004d6a60 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f429e7fc640 1 -- 172.21.15.120:0/1008437766 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0004650 con 0x55f9004d6a60 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 0x7f42a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.529+0000 7f42b57fa640 1 -- 172.21.15.120:0/4241706745 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42a80e5b50 con 0x7f42b81016c0 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- 172.21.15.120:0/1008437766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 msgr2=0x55f9004d8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42b7fff640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 0x7f42a00b77a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f42a80a95b0 tx=0x7f42a80cf040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f42b00107a0 tx=0x7f42b0010c60 comp rx=0 tx=0).stop 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- 172.21.15.120:0/1008437766 shutdown_connections 2024-09-10T23:15:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004dbdd0 0x55f9004de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004db780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- 172.21.15.120:0/1008437766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d6a60 0x55f9004d8e50 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- 172.21.15.120:0/1008437766 >> 172.21.15.120:0/1008437766 conn(0x55f9004d1bb0 msgr2=0x55f9004d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- 172.21.15.120:0/1008437766 shutdown_connections 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- 172.21.15.120:0/1008437766 wait complete. 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 -- start start 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 Processor -- start 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.530+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004e0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f90036fab0 con 0x55f9004dbdd0 2024-09-10T23:15:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f9003dd730 con 0x55f9004d6a60 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f9004dedb0 con 0x55f9004d9390 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 10 client.0 _ll_register_callbacks cb 0x55f9004316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58752/0 (socket says 172.21.15.120:58752) 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 -- 172.21.15.120:0/3215461900 learned_addr learned my addr 172.21.15.120:0/3215461900 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42b77fe640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004e0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 -- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 msgr2=0x55f9004e0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004e0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 -- 172.21.15.120:0/3215461900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 msgr2=0x55f9004e00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 -- 172.21.15.120:0/3215461900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f900199790 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42b77fe640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004e0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ed80 tx=0x7f42b0010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a75d5eeb28ae9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42977fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0024040 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42b0004730 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b000ede0 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42b0032050 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 10 client.19986 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.531+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f900212620 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b000ede0 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f42b001a740 con 0x55f9004dbdd0 2024-09-10T23:15:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 10 client.19986 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 10 client.19986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 20 client.19986 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f9004edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 4 client.19986 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f90044a640 con 0x55f9004eb720 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42b77fe640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f9004edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42b77fe640 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f9004edb10 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00fccd1da642106 server_cookie=46f6ff6168160613 in_seq=0 out_seq=0 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.532+0000 7f42977fe640 10 client.19986 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42977fe640 1 -- 172.21.15.120:0/3215461900 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f90044a640 con 0x55f9004eb720 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42977fe640 10 client.19986 handle_command_reply: tid=0 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 client.19986 shutdown 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 msgr2=0x55f9004e1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ed80 tx=0x7f42b0010eb0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 shutdown_connections 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f9004edb10 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004e1360 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42977fe640 0 client.19986 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004e0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3215461900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.534+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 >> 172.21.15.120:0/3215461900 conn(0x55f9004d1bb0 msgr2=0x55f9004d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.535+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 shutdown_connections 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.535+0000 7f42c15d6740 1 -- 172.21.15.120:0/3215461900 wait complete. 2024-09-10T23:15:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.535+0000 7f42c15d6740 20 client.19986 trim_cache size 0 max 16384 2024-09-10T23:15:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 Processor -- start 2024-09-10T23:15:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 -- start start 2024-09-10T23:15:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a6c40 con 0x55f9004dbdd0 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a94c0 con 0x55f9004d6a60 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42c15d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a72b0 con 0x55f9004d9390 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42b77fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58768/0 (socket says 172.21.15.120:58768) 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 -- 172.21.15.120:0/3053124285 learned_addr learned my addr 172.21.15.120:0/3053124285 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 -- 172.21.15.120:0/3053124285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 msgr2=0x55f9004d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004d6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 -- 172.21.15.120:0/3053124285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 msgr2=0x55f9004d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.551+0000 7f42bcda0640 1 -- 172.21.15.120:0/3053124285 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9004ede80 con 0x55f9004dbdd0 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.552+0000 7f42bcda0640 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f42b00107a0 tx=0x7f42b00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f836258174dbecac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.552+0000 7f429dffb640 1 -- 172.21.15.120:0/3053124285 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b001ad30 con 0x55f9004dbdd0 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.552+0000 7f429dffb640 1 -- 172.21.15.120:0/3053124285 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b0003970 con 0x55f9004dbdd0 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.552+0000 7f429dffb640 1 -- 172.21.15.120:0/3053124285 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b001a720 con 0x55f9004dbdd0 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- 172.21.15.120:0/3053124285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 msgr2=0x55f9004dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f42b00107a0 tx=0x7f42b00096a0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- 172.21.15.120:0/3053124285 shutdown_connections 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004dbdd0 0x55f9004dc1b0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004d9390 0x55f9004d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- 172.21.15.120:0/3053124285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- 172.21.15.120:0/3053124285 >> 172.21.15.120:0/3053124285 conn(0x55f9004d1bb0 msgr2=0x55f9004d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- 172.21.15.120:0/3053124285 shutdown_connections 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- 172.21.15.120:0/3053124285 wait complete. 2024-09-10T23:15:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- start start 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 Processor -- start 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 0x55f900556630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a7980 con 0x55f9004d9390 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a8df0 con 0x55f9004d6a60 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f42b81a8050 con 0x55f9004dbdd0 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.553+0000 7f42c15d6740 10 client.0 _ll_register_callbacks cb 0x7f42ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 0x55f900556630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42b77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58774/0 (socket says 172.21.15.120:58774) 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42b77fe640 1 -- 172.21.15.120:0/582351621 learned_addr learned my addr 172.21.15.120:0/582351621 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:15:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 -- 172.21.15.120:0/582351621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 msgr2=0x55f9004e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 -- 172.21.15.120:0/582351621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 msgr2=0x55f9004e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 -- 172.21.15.120:0/582351621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f900556e60 con 0x55f9004dbdd0 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42b77fe640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42bcda0640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 0x55f900556630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42b0002230 tx=0x7f42b0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d08efdc5755878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42977fe640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000ee20 con 0x55f9004dbdd0 2024-09-10T23:15:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b0003970 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f42b001a4e0 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42b001a6e0 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42c15d6740 10 client.29425 fetch_fsmap learned FSMap version 26 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.554+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f900558170 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b0003970 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f42b002d910 con 0x55f9004dbdd0 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 10 client.29425 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 10 client.29425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 20 client.29425 populate_metadata read hostname 'smithi120' 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f90055aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 4 client.29425 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9004460f0 con 0x55f9004eb720 2024-09-10T23:15:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42b77fe640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f90055aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:15:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42b77fe640 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f90055aa70 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd07639a0c1dfb50 server_cookie=30b4e9d6a7f510c3 in_seq=0 out_seq=0 2024-09-10T23:15:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:58.555+0000 7f42977fe640 10 client.29425 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:58.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 263082 21.48 MB/sec execute 533 sec latency 359.981 ms 2024-09-10T23:15:58.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:15:58.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:58.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:58.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:15:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:15:58.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:15:58.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:58.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:58.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:15:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:15:59.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:15:59.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:15:59.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:15:59.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:15:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:15:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42977fe640 1 -- 172.21.15.120:0/582351621 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9004460f0 con 0x55f9004eb720 2024-09-10T23:15:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42977fe640 10 client.29425 handle_command_reply: tid=0 2024-09-10T23:15:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 client.29425 shutdown 2024-09-10T23:15:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 msgr2=0x55f900556630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 0x55f900556630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42b0002230 tx=0x7f42b0009920 comp rx=0 tx=0).stop 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 shutdown_connections 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f9004eb720 0x55f90055aa70 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9004dbdd0 0x55f900556630 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f9004d9390 0x55f9004e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42977fe640 0 client.29425 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 --2- 172.21.15.120:0/582351621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f9004d6a60 0x55f9004e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 >> 172.21.15.120:0/582351621 conn(0x55f9004d1bb0 msgr2=0x55f9004dc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 shutdown_connections 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 1 -- 172.21.15.120:0/582351621 wait complete. 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.419+0000 7f42c15d6740 20 client.29425 trim_cache size 0 max 16384 2024-09-10T23:15:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:15:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:15:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:15:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:15:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 msgr2=0x7f42a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 0x7f42a00b77a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f42a80a95b0 tx=0x7f42a80cf040 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 msgr2=0x7f42b8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42a800e9f0 tx=0x7f42a800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 shutdown_connections 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f42a00b52e0 0x7f42a00b77a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42b81031d0 0x7f42b8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42b8102810 0x7f42b81981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 --2- 172.21.15.120:0/4241706745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42b81016c0 0x7f42b8197b70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 >> 172.21.15.120:0/4241706745 conn(0x7f42b80fcf60 msgr2=0x7f42b8103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 shutdown_connections 2024-09-10T23:15:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:15:59.420+0000 7f42bd5a1640 1 -- 172.21.15.120:0/4241706745 wait complete. 2024-09-10T23:15:59.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 263229 21.45 MB/sec execute 534 sec latency 490.422 ms 2024-09-10T23:16:00.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:00 smithi120 ceph-mon[31247]: pgmap v1053: 97 pgs: 97 active+clean; 243 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 482 op/s 2024-09-10T23:16:00.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:00 smithi153 ceph-mon[34951]: pgmap v1053: 97 pgs: 97 active+clean; 243 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 482 op/s 2024-09-10T23:16:00.823 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:00 smithi176 ceph-mon[34982]: pgmap v1053: 97 pgs: 97 active+clean; 243 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 482 op/s 2024-09-10T23:16:00.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 263614 21.44 MB/sec execute 535 sec latency 97.417 ms 2024-09-10T23:16:01.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 264020 21.44 MB/sec execute 536 sec latency 63.730 ms 2024-09-10T23:16:02.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:02 smithi120 ceph-mon[31247]: pgmap v1054: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-10T23:16:02.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:02 smithi153 ceph-mon[34951]: pgmap v1054: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-10T23:16:02.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:02 smithi176 ceph-mon[34982]: pgmap v1054: 97 pgs: 97 active+clean; 240 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.5 MiB/s wr, 513 op/s 2024-09-10T23:16:02.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 264505 21.43 MB/sec execute 537 sec latency 91.450 ms 2024-09-10T23:16:03.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 264867 21.44 MB/sec execute 538 sec latency 183.737 ms 2024-09-10T23:16:04.624 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:04 smithi176 ceph-mon[34982]: pgmap v1055: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-10T23:16:04.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:04 smithi120 ceph-mon[31247]: pgmap v1055: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-10T23:16:04.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:04 smithi153 ceph-mon[34951]: pgmap v1055: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-09-10T23:16:04.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 265453 21.46 MB/sec execute 539 sec latency 86.393 ms 2024-09-10T23:16:05.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 265963 21.48 MB/sec execute 540 sec latency 60.691 ms 2024-09-10T23:16:06.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:06 smithi120 ceph-mon[31247]: pgmap v1056: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:16:06.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:06 smithi153 ceph-mon[34951]: pgmap v1056: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:16:06.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:06 smithi176 ceph-mon[34982]: pgmap v1056: 97 pgs: 97 active+clean; 233 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 566 op/s 2024-09-10T23:16:06.894 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266388 21.49 MB/sec execute 541 sec latency 75.337 ms 2024-09-10T23:16:07.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266764 21.49 MB/sec execute 542 sec latency 109.402 ms 2024-09-10T23:16:08.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:08 smithi120 ceph-mon[31247]: pgmap v1057: 97 pgs: 97 active+clean; 236 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 624 op/s 2024-09-10T23:16:08.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:08 smithi153 ceph-mon[34951]: pgmap v1057: 97 pgs: 97 active+clean; 236 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 624 op/s 2024-09-10T23:16:08.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:08 smithi176 ceph-mon[34982]: pgmap v1057: 97 pgs: 97 active+clean; 236 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 624 op/s 2024-09-10T23:16:08.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 266961 21.46 MB/sec execute 543 sec latency 399.777 ms 2024-09-10T23:16:09.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 267279 21.45 MB/sec execute 544 sec latency 128.330 ms 2024-09-10T23:16:10.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:10 smithi120 ceph-mon[31247]: pgmap v1058: 97 pgs: 97 active+clean; 240 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-09-10T23:16:10.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:10 smithi153 ceph-mon[34951]: pgmap v1058: 97 pgs: 97 active+clean; 240 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-09-10T23:16:10.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:10 smithi176 ceph-mon[34982]: pgmap v1058: 97 pgs: 97 active+clean; 240 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-09-10T23:16:10.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 267688 21.45 MB/sec execute 545 sec latency 130.248 ms 2024-09-10T23:16:11.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268202 21.45 MB/sec execute 546 sec latency 143.891 ms 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 -- 172.21.15.120:0/267500176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 msgr2=0x7fed3c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 --2- 172.21.15.120:0/267500176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c109cf0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fed34009cd0 tx=0x7fed3402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 -- 172.21.15.120:0/267500176 shutdown_connections 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 --2- 172.21.15.120:0/267500176 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c109cf0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 --2- 172.21.15.120:0/267500176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c1029b0 0x7fed3c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.518 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 --2- 172.21.15.120:0/267500176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c101860 0x7fed3c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 -- 172.21.15.120:0/267500176 >> 172.21.15.120:0/267500176 conn(0x7fed3c0fd140 msgr2=0x7fed3c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 -- 172.21.15.120:0/267500176 shutdown_connections 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 -- 172.21.15.120:0/267500176 wait complete. 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.514+0000 7fed4249b640 1 Processor -- start 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 -- start start 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c101860 0x7fed3c105600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c1029b0 0x7fed3c103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.519 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c10d4d0 con 0x7fed3c103370 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c10d350 con 0x7fed3c101860 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed4249b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c10d650 con 0x7fed3c1029b0 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41394/0 (socket says 172.21.15.120:41394) 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 -- 172.21.15.120:0/1875311995 learned_addr learned my addr 172.21.15.120:0/1875311995 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed33fff640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c1029b0 0x7fed3c103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed3bfff640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c101860 0x7fed3c105600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 -- 172.21.15.120:0/1875311995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c1029b0 msgr2=0x7fed3c103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c1029b0 0x7fed3c103e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 -- 172.21.15.120:0/1875311995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c101860 msgr2=0x7fed3c105600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c101860 0x7fed3c105600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 -- 172.21.15.120:0/1875311995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c104c10 con 0x7fed3c103370 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed40a38640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fed34033040 tx=0x7fed34003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed39ffb640 1 -- 172.21.15.120:0/1875311995 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34004240 con 0x7fed3c103370 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.515+0000 7fed39ffb640 1 -- 172.21.15.120:0/1875311995 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed340305a0 con 0x7fed3c103370 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.516+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a2db0 con 0x7fed3c103370 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.516+0000 7fed39ffb640 1 -- 172.21.15.120:0/1875311995 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340466e0 con 0x7fed3c103370 2024-09-10T23:16:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.516+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a3270 con 0x7fed3c103370 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed39ffb640 1 -- 172.21.15.120:0/1875311995 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fed3404e070 con 0x7fed3c103370 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 Processor -- start 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 -- start start 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb5d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e8baca6f0 con 0x556e8bb5dbd0 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e8bab9b20 con 0x556e8bb5b190 2024-09-10T23:16:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.518+0000 7fed4526d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556e8babd930 con 0x556e8bb58860 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:34974/0 (socket says 172.21.15.120:34974) 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 -- 172.21.15.120:0/655318870 learned_addr learned my addr 172.21.15.120:0/655318870 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed3bfff640 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb5d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 -- 172.21.15.120:0/655318870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 msgr2=0x556e8bb5d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb5d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 -- 172.21.15.120:0/655318870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 msgr2=0x556e8bb60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 -- 172.21.15.120:0/655318870 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8ba7f530 con 0x556e8bb58860 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed39ffb640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 0x7fed280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed33fff640 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed24010990 tx=0x7fed24005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91db19cf47c1d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed3bfff640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 0x7fed280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed39ffb640 1 -- 172.21.15.120:0/1875311995 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed3402e030 con 0x7fed3c103370 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.519+0000 7fed327fc640 1 -- 172.21.15.120:0/655318870 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24016070 con 0x556e8bb58860 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed327fc640 1 -- 172.21.15.120:0/655318870 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2400f7c0 con 0x556e8bb58860 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed327fc640 1 -- 172.21.15.120:0/655318870 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24019040 con 0x556e8bb58860 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed3bfff640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 0x7fed280b77a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009cf0 tx=0x7fed2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 -- 172.21.15.120:0/655318870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 msgr2=0x556e8bb5ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fed24010990 tx=0x7fed24005f90 comp rx=0 tx=0).stop 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 -- 172.21.15.120:0/655318870 shutdown_connections 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb5d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 --2- 172.21.15.120:0/655318870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb58860 0x556e8bb5ac50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 -- 172.21.15.120:0/655318870 >> 172.21.15.120:0/655318870 conn(0x556e8bb539b0 msgr2=0x556e8bb54740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 -- 172.21.15.120:0/655318870 shutdown_connections 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.520+0000 7fed4526d740 1 -- 172.21.15.120:0/655318870 wait complete. 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 -- start start 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 Processor -- start 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb62560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5dbd0 0x556e8bb63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e8b9f1ab0 con 0x556e8bb58860 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556e8ba5f730 con 0x556e8bb5dbd0 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556e8bb60bb0 con 0x556e8bb5b190 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed4526d740 10 client.0 _ll_register_callbacks cb 0x556e8bab36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41406/0 (socket says 172.21.15.120:41406) 2024-09-10T23:16:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed3bfff640 1 -- 172.21.15.120:0/2472247115 learned_addr learned my addr 172.21.15.120:0/2472247115 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed33fff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb62560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.521+0000 7fed40a38640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5dbd0 0x556e8bb63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 -- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 msgr2=0x556e8bb62560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb62560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 -- 172.21.15.120:0/2472247115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5dbd0 msgr2=0x556e8bb63130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5dbd0 0x556e8bb63130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 -- 172.21.15.120:0/2472247115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8b81b790 con 0x556e8bb58860 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed3bfff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fed2c020a10 tx=0x7fed2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a6bc592a965056 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c02d060 con 0x556e8bb58860 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2400f7c0 con 0x556e8bb58860 2024-09-10T23:16:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed2c018030 con 0x556e8bb58860 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed2c011940 con 0x556e8bb58860 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed4526d740 10 client.20004 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e8b894620 con 0x556e8bb58860 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2400f7c0 con 0x556e8bb58860 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.522+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fed2c01dbc0 con 0x556e8bb58860 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 10 client.20004 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 10 client.20004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 20 client.20004 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bb6d7b0 0x556e8bb6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 4 client.20004 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8bacc640 con 0x556e8bb6d7b0 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed33fff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bb6d7b0 0x556e8bb6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed1b7fe640 10 client.20004 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.523+0000 7fed33fff640 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bb6d7b0 0x556e8bb6fc30 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96cc63222df9b6e server_cookie=745115761d6d55aa in_seq=0 out_seq=0 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed1b7fe640 1 -- 172.21.15.120:0/2472247115 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e8bacc640 con 0x556e8bb6d7b0 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed1b7fe640 10 client.20004 handle_command_reply: tid=0 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 client.20004 shutdown 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 msgr2=0x556e8bb61ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fed2c020a10 tx=0x7fed2c004730 comp rx=0 tx=0).stop 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 shutdown_connections 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bb6d7b0 0x556e8bb6fc30 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5dbd0 0x556e8bb63130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed1b7fe640 0 client.20004 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 --2- 172.21.15.120:0/2472247115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb61ec0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 >> 172.21.15.120:0/2472247115 conn(0x556e8bb539b0 msgr2=0x556e8bb52bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 shutdown_connections 2024-09-10T23:16:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 1 -- 172.21.15.120:0/2472247115 wait complete. 2024-09-10T23:16:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.528+0000 7fed4526d740 20 client.20004 trim_cache size 0 max 16384 2024-09-10T23:16:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.552+0000 7fed4526d740 1 Processor -- start 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.552+0000 7fed4526d740 1 -- start start 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 0x556e8bb582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1a9f80 con 0x556e8bb5dbd0 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ac800 con 0x556e8bb58860 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed4526d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1aa5f0 con 0x556e8bb5b190 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 0x556e8bb582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41408/0 (socket says 172.21.15.120:41408) 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 -- 172.21.15.120:0/733761733 learned_addr learned my addr 172.21.15.120:0/733761733 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed33fff640 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 -- 172.21.15.120:0/733761733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 msgr2=0x556e8bb744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 -- 172.21.15.120:0/733761733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 msgr2=0x556e8bb582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 0x556e8bb582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed40a38640 1 -- 172.21.15.120:0/733761733 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8bb62230 con 0x556e8bb5dbd0 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.553+0000 7fed3bfff640 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 0x556e8bb582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.554+0000 7fed40a38640 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fed340cd4e0 tx=0x7fed340095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b62a377e47903 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.554+0000 7fed31ffb640 1 -- 172.21.15.120:0/733761733 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340ccf50 con 0x556e8bb5dbd0 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.554+0000 7fed31ffb640 1 -- 172.21.15.120:0/733761733 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34036030 con 0x556e8bb5dbd0 2024-09-10T23:16:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.554+0000 7fed31ffb640 1 -- 172.21.15.120:0/733761733 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340d6d70 con 0x556e8bb5dbd0 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- 172.21.15.120:0/733761733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 msgr2=0x556e8bb74a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fed340cd4e0 tx=0x7fed340095e0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- 172.21.15.120:0/733761733 shutdown_connections 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb5dbd0 0x556e8bb74a30 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5b190 0x556e8bb744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- 172.21.15.120:0/733761733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb58860 0x556e8bb582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- 172.21.15.120:0/733761733 >> 172.21.15.120:0/733761733 conn(0x556e8bb539b0 msgr2=0x556e8bb57b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- 172.21.15.120:0/733761733 shutdown_connections 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- 172.21.15.120:0/733761733 wait complete. 2024-09-10T23:16:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- start start 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 Processor -- start 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb68860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5dbd0 0x556e8bb68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1aacc0 con 0x556e8bb58860 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ac130 con 0x556e8bb5b190 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1ab390 con 0x556e8bb5dbd0 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.555+0000 7fed4526d740 10 client.0 _ll_register_callbacks cb 0x7fed2c0208a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:41412/0 (socket says 172.21.15.120:41412) 2024-09-10T23:16:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 -- 172.21.15.120:0/834541548 learned_addr learned my addr 172.21.15.120:0/834541548 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed40a38640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5dbd0 0x556e8bb68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed33fff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb68860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 -- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5dbd0 msgr2=0x556e8bb68f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5dbd0 0x556e8bb68f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 -- 172.21.15.120:0/834541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 msgr2=0x556e8bb68860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb68860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 -- 172.21.15.120:0/834541548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8bb673d0 con 0x556e8bb58860 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed3bfff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fed2c020870 tx=0x7fed2c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8594bf87bb3f036 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c038070 con 0x556e8bb58860 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fed2c00fdd0 con 0x556e8bb58860 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34036030 con 0x556e8bb58860 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed2c046050 con 0x556e8bb58860 2024-09-10T23:16:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.556+0000 7fed4526d740 10 client.20010 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e8bbda980 con 0x556e8bb58860 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34036030 con 0x556e8bb58860 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fed2c003a30 con 0x556e8bb58860 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 10 client.20010 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 10 client.20010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 20 client.20010 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bbdb360 0x556e8bbdd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 4 client.20010 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e8bac80f0 con 0x556e8bbdb360 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.557+0000 7fed33fff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bbdb360 0x556e8bbdd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.558+0000 7fed1b7fe640 10 client.20010 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:12.558+0000 7fed33fff640 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bbdb360 0x556e8bbdd780 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c34c25b86b655b7 server_cookie=f42f6236d4f85582 in_seq=0 out_seq=0 2024-09-10T23:16:12.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:12 smithi120 ceph-mon[31247]: pgmap v1059: 97 pgs: 97 active+clean; 242 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 543 op/s 2024-09-10T23:16:12.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:12 smithi153 ceph-mon[34951]: pgmap v1059: 97 pgs: 97 active+clean; 242 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 543 op/s 2024-09-10T23:16:12.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:12 smithi176 ceph-mon[34982]: pgmap v1059: 97 pgs: 97 active+clean; 242 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 543 op/s 2024-09-10T23:16:12.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268680 21.45 MB/sec execute 547 sec latency 208.356 ms 2024-09-10T23:16:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed1b7fe640 1 -- 172.21.15.120:0/834541548 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e8bac80f0 con 0x556e8bbdb360 2024-09-10T23:16:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed1b7fe640 10 client.20010 handle_command_reply: tid=0 2024-09-10T23:16:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed4526d740 1 client.20010 shutdown 2024-09-10T23:16:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 msgr2=0x556e8bb69b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fed2c020870 tx=0x7fed2c011d10 comp rx=0 tx=0).stop 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.661+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 shutdown_connections 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556e8bbdb360 0x556e8bbdd780 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed1b7fe640 0 client.20010 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556e8bb5dbd0 0x556e8bb68f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556e8bb5b190 0x556e8bb68860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 --2- 172.21.15.120:0/834541548 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e8bb58860 0x556e8bb69b60 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 >> 172.21.15.120:0/834541548 conn(0x556e8bb539b0 msgr2=0x556e8bb523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 shutdown_connections 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 1 -- 172.21.15.120:0/834541548 wait complete. 2024-09-10T23:16:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.662+0000 7fed4526d740 20 client.20010 trim_cache size 0 max 16384 2024-09-10T23:16:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:16:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:16:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:16:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:16:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 msgr2=0x7fed280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 0x7fed280b77a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009cf0 tx=0x7fed2c005f90 comp rx=0 tx=0).stop 2024-09-10T23:16:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 msgr2=0x7fed3c1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fed34033040 tx=0x7fed34003b80 comp rx=0 tx=0).stop 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 shutdown_connections 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fed280b52e0 0x7fed280b77a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed3c103370 0x7fed3c1043a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed3c1029b0 0x7fed3c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 --2- 172.21.15.120:0/1875311995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed3c101860 0x7fed3c105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 >> 172.21.15.120:0/1875311995 conn(0x7fed3c0fd140 msgr2=0x7fed3c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 shutdown_connections 2024-09-10T23:16:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:13.663+0000 7fed4249b640 1 -- 172.21.15.120:0/1875311995 wait complete. 2024-09-10T23:16:13.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 268837 21.43 MB/sec execute 548 sec latency 484.383 ms 2024-09-10T23:16:14.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:14 smithi120 ceph-mon[31247]: pgmap v1060: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 516 op/s 2024-09-10T23:16:14.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:14 smithi153 ceph-mon[34951]: pgmap v1060: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 516 op/s 2024-09-10T23:16:14.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:14 smithi176 ceph-mon[34982]: pgmap v1060: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 516 op/s 2024-09-10T23:16:14.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 269273 21.45 MB/sec execute 549 sec latency 90.013 ms 2024-09-10T23:16:15.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 269723 21.46 MB/sec execute 550 sec latency 87.774 ms 2024-09-10T23:16:16.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:16 smithi120 ceph-mon[31247]: pgmap v1061: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 262 op/s 2024-09-10T23:16:16.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:16 smithi153 ceph-mon[34951]: pgmap v1061: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 262 op/s 2024-09-10T23:16:16.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:16 smithi176 ceph-mon[34982]: pgmap v1061: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 262 op/s 2024-09-10T23:16:16.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 270100 21.47 MB/sec execute 551 sec latency 81.417 ms 2024-09-10T23:16:17.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 270440 21.45 MB/sec execute 552 sec latency 117.623 ms 2024-09-10T23:16:18.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:18 smithi120 ceph-mon[31247]: pgmap v1062: 97 pgs: 97 active+clean; 238 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 533 op/s 2024-09-10T23:16:18.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:18 smithi153 ceph-mon[34951]: pgmap v1062: 97 pgs: 97 active+clean; 238 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 533 op/s 2024-09-10T23:16:18.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:18 smithi176 ceph-mon[34982]: pgmap v1062: 97 pgs: 97 active+clean; 238 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.1 MiB/s wr, 533 op/s 2024-09-10T23:16:18.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 270732 21.44 MB/sec execute 553 sec latency 258.195 ms 2024-09-10T23:16:19.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 271076 21.42 MB/sec execute 554 sec latency 139.321 ms 2024-09-10T23:16:20.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:20 smithi120 ceph-mon[31247]: pgmap v1063: 97 pgs: 97 active+clean; 226 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-09-10T23:16:20.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:20 smithi153 ceph-mon[34951]: pgmap v1063: 97 pgs: 97 active+clean; 226 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-09-10T23:16:20.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:20 smithi176 ceph-mon[34982]: pgmap v1063: 97 pgs: 97 active+clean; 226 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-09-10T23:16:20.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 271661 21.44 MB/sec execute 555 sec latency 48.600 ms 2024-09-10T23:16:21.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 272273 21.45 MB/sec execute 556 sec latency 81.243 ms 2024-09-10T23:16:22.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:22 smithi120 ceph-mon[31247]: pgmap v1064: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:16:22.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:22 smithi153 ceph-mon[34951]: pgmap v1064: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:16:22.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:22 smithi176 ceph-mon[34982]: pgmap v1064: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 576 op/s 2024-09-10T23:16:22.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 272816 21.49 MB/sec execute 557 sec latency 37.276 ms 2024-09-10T23:16:23.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:16:23.321 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:23.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 273200 21.49 MB/sec execute 558 sec latency 230.806 ms 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 -- 172.21.15.120:0/4100854715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 msgr2=0x7f3fb410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 --2- 172.21.15.120:0/4100854715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb410f560 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009ca0 tx=0x7f3fac02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 -- 172.21.15.120:0/4100854715 shutdown_connections 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 --2- 172.21.15.120:0/4100854715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb410f560 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 --2- 172.21.15.120:0/4100854715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 --2- 172.21.15.120:0/4100854715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 0x7f3fb4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 -- 172.21.15.120:0/4100854715 >> 172.21.15.120:0/4100854715 conn(0x7f3fb40fd700 msgr2=0x7f3fb40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.907 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 -- 172.21.15.120:0/4100854715 shutdown_connections 2024-09-10T23:16:23.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 -- 172.21.15.120:0/4100854715 wait complete. 2024-09-10T23:16:23.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.906+0000 7f3fba809640 1 Processor -- start 2024-09-10T23:16:23.908 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 -- start start 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 0x7f3fb4101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4112d70 con 0x7f3fb410d100 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4112bf0 con 0x7f3fb4103ca0 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fba809640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4112ef0 con 0x7f3fb41045c0 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53502/0 (socket says 172.21.15.120:53502) 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 -- 172.21.15.120:0/290623895 learned_addr learned my addr 172.21.15.120:0/290623895 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 0x7f3fb4101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 -- 172.21.15.120:0/290623895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 msgr2=0x7f3fb4101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.910 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 0x7f3fb4101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 -- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 msgr2=0x7f3fb41023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb41023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb37fe640 1 -- 172.21.15.120:0/290623895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb41a76a0 con 0x7f3fb41045c0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.907+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e9c0 tx=0x7f3fa800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fb17fa640 1 -- 172.21.15.120:0/290623895 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800ccc0 con 0x7f3fb41045c0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb41a7990 con 0x7f3fb41045c0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fb17fa640 1 -- 172.21.15.120:0/290623895 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fa80035f0 con 0x7f3fb41045c0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fb17fa640 1 -- 172.21.15.120:0/290623895 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800ce60 con 0x7f3fb41045c0 2024-09-10T23:16:23.911 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.908+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb41a7ed0 con 0x7f3fb41045c0 2024-09-10T23:16:23.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.910+0000 7f3fbcbde740 1 Processor -- start 2024-09-10T23:16:23.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 -- start start 2024-09-10T23:16:23.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb17fa640 1 -- 172.21.15.120:0/290623895 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3fa8004000 con 0x7f3fb41045c0 2024-09-10T23:16:23.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.917 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x556707143410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x556707145d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567070ade50 con 0x556707143950 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55670709d280 con 0x5567071379c0 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5567070a1090 con 0x5567071370a0 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x556707145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53510/0 (socket says 172.21.15.120:53510) 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 -- 172.21.15.120:0/3372804964 learned_addr learned my addr 172.21.15.120:0/3372804964 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x556707143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 -- 172.21.15.120:0/3372804964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 msgr2=0x556707143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x556707143410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 -- 172.21.15.120:0/3372804964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 msgr2=0x556707145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x556707145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb8da6640 1 -- 172.21.15.120:0/3372804964 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556706dfb650 con 0x5567071370a0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.911+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x556707145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3fac035000 tx=0x7f3fac02f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57bbe33481d2354 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3fb17fa640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 0x7f3f9c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3f9a7fc640 1 -- 172.21.15.120:0/3372804964 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac031450 con 0x5567071370a0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 0x7f3f9c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3f9a7fc640 1 -- 172.21.15.120:0/3372804964 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fac036040 con 0x5567071370a0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3fb17fa640 1 -- 172.21.15.120:0/290623895 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3fa80e5dc0 con 0x7f3fb41045c0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.912+0000 7f3f9a7fc640 1 -- 172.21.15.120:0/3372804964 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac004380 con 0x5567071370a0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- 172.21.15.120:0/3372804964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 msgr2=0x556707137480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 0x7f3f9c0b7820 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x556707135db0 tx=0x7f3fa4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3fac035000 tx=0x7f3fac02f310 comp rx=0 tx=0).stop 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- 172.21.15.120:0/3372804964 shutdown_connections 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x556707145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x556707143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/3372804964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707137480 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- 172.21.15.120:0/3372804964 >> 172.21.15.120:0/3372804964 conn(0x556707134ed0 msgr2=0x556707136070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- 172.21.15.120:0/3372804964 shutdown_connections 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- 172.21.15.120:0/3372804964 wait complete. 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.913+0000 7f3fbcbde740 1 -- start start 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 Processor -- start 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707143140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 0x556707140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556707143950 0x556707140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556706efe700 con 0x5567071379c0 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556707038110 con 0x556707143950 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556707146440 con 0x5567071370a0 2024-09-10T23:16:23.920 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fbcbde740 10 client.0 _ll_register_callbacks cb 0x556707051990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 0x556707140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53522/0 (socket says 172.21.15.120:53522) 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb3fff640 1 -- 172.21.15.120:0/2555822546 learned_addr learned my addr 172.21.15.120:0/2555822546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556707143950 0x556707140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 -- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 msgr2=0x556707143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 -- 172.21.15.120:0/2555822546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556707143950 msgr2=0x556707140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556707143950 0x556707140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 -- 172.21.15.120:0/2555822546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556707065c40 con 0x5567071379c0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 0x556707140590 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80e3940 tx=0x7f3fa8003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cd0b80282a22a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.914+0000 7f3f937fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80ac380 con 0x5567071379c0 2024-09-10T23:16:23.921 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3fa8004320 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fac036040 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3fa80abd20 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 10 client.20022 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556707151d50 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fac036040 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3fa80e9ae0 con 0x5567071379c0 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 10 client.20022 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:23.922 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 20 client.20022 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071530f0 0x5567071554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 4 client.20022 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.915+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567070afda0 con 0x5567071530f0 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.916+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071530f0 0x5567071554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.916+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071530f0 0x5567071554e0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36be7f2e4e264a65 server_cookie=b3ec5a6df49b338b in_seq=0 out_seq=0 2024-09-10T23:16:23.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.916+0000 7f3f937fe640 10 client.20022 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:23.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.918+0000 7f3f937fe640 1 -- 172.21.15.120:0/2555822546 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567070afda0 con 0x5567071530f0 2024-09-10T23:16:23.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.918+0000 7f3f937fe640 10 client.20022 handle_command_reply: tid=0 2024-09-10T23:16:23.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.918+0000 7f3fbcbde740 1 client.20022 shutdown 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.918+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 msgr2=0x556707140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.918+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 0x556707140590 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80e3940 tx=0x7f3fa8003e20 comp rx=0 tx=0).stop 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 shutdown_connections 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071530f0 0x5567071554e0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556707143950 0x556707140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071379c0 0x556707140590 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3f937fe640 0 client.20022 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2555822546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x556707143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 >> 172.21.15.120:0/2555822546 conn(0x556707134ed0 msgr2=0x55670713aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 shutdown_connections 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2555822546 wait complete. 2024-09-10T23:16:23.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.919+0000 7f3fbcbde740 20 client.20022 trim_cache size 0 max 16384 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 Processor -- start 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 -- start start 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x55670713d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x55670713dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41ae450 con 0x556707143950 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41b0cd0 con 0x5567071379c0 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41aeac0 con 0x5567071370a0 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36078/0 (socket says 172.21.15.120:36078) 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fb8da6640 1 -- 172.21.15.120:0/1621242665 learned_addr learned my addr 172.21.15.120:0/1621242665 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.936+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x55670713d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x55670713dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 -- 172.21.15.120:0/1621242665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 msgr2=0x55670713d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x55670713d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 -- 172.21.15.120:0/1621242665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 msgr2=0x55670713dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x55670713dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 -- 172.21.15.120:0/1621242665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567071365b0 con 0x556707143950 2024-09-10T23:16:23.941 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3fac003f10 tx=0x7f3fac02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942bc744bba24c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3f99ffb640 1 -- 172.21.15.120:0/1621242665 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac009800 con 0x556707143950 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3f99ffb640 1 -- 172.21.15.120:0/1621242665 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fac036040 con 0x556707143950 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.937+0000 7f3f99ffb640 1 -- 172.21.15.120:0/1621242665 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac02f8c0 con 0x556707143950 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- 172.21.15.120:0/1621242665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 msgr2=0x55670715ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3fac003f10 tx=0x7f3fac02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- 172.21.15.120:0/1621242665 shutdown_connections 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556707143950 0x55670715ad70 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x55670713dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/1621242665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5567071370a0 0x55670713d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- 172.21.15.120:0/1621242665 >> 172.21.15.120:0/1621242665 conn(0x556707134ed0 msgr2=0x5567071423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- 172.21.15.120:0/1621242665 shutdown_connections 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- 172.21.15.120:0/1621242665 wait complete. 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 -- start start 2024-09-10T23:16:23.942 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.938+0000 7f3fbcbde740 1 Processor -- start 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x5567071485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41af190 con 0x5567071370a0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41b0600 con 0x5567071379c0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb41af860 con 0x556707143950 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 10 client.0 _ll_register_callbacks cb 0x7f3fa80e9590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36090/0 (socket says 172.21.15.120:36090) 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb8da6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53546/0 (socket says 172.21.15.120:53546) 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 -- 172.21.15.120:0/2705413764 learned_addr learned my addr 172.21.15.120:0/2705413764 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x5567071485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 -- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 msgr2=0x5567071b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 -- 172.21.15.120:0/2705413764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 msgr2=0x5567071485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x5567071485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 -- 172.21.15.120:0/2705413764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567071b96c0 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb8da6640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fb3fff640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4012cb0 tx=0x7f3fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17b8139e0d679f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3f937fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4036060 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f3fa4018030 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.939+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fac036040 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3fa4003b00 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 10 client.20028 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567071c0b10 con 0x5567071370a0 2024-09-10T23:16:23.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fac036040 con 0x5567071370a0 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f3fa4031410 con 0x5567071370a0 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 10 client.20028 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 10 client.20028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 20 client.20028 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071c1290 0x5567071c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 4 client.20028 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567070ab850 con 0x5567071c1290 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.940+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071c1290 0x5567071c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.941+0000 7f3fb37fe640 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071c1290 0x5567071c36b0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d182077a197deb8 server_cookie=3f8eb556e60cb4a4 in_seq=0 out_seq=0 2024-09-10T23:16:23.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.941+0000 7f3f937fe640 10 client.20028 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3f937fe640 1 -- 172.21.15.120:0/2705413764 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+11166 (crc 0 0 0) 0x5567070ab850 con 0x5567071c1290 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3f937fe640 10 client.20028 handle_command_reply: tid=0 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 client.20028 shutdown 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 msgr2=0x556707148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4012cb0 tx=0x7f3fa40046c0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 shutdown_connections 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5567071c1290 0x5567071c36b0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556707143950 0x5567071b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567071379c0 0x5567071485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3f937fe640 0 client.20028 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 --2- 172.21.15.120:0/2705413764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567071370a0 0x556707148070 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 >> 172.21.15.120:0/2705413764 conn(0x556707134ed0 msgr2=0x556707141bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 shutdown_connections 2024-09-10T23:16:23.947 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 1 -- 172.21.15.120:0/2705413764 wait complete. 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.943+0000 7f3fbcbde740 20 client.20028 trim_cache size 0 max 16384 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:16:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:16:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.955 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:16:23.956 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.958 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.959 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.960 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.961 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "2dbbc85c-b952-4f9d-a994-746a3e80926f": { 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:50.606606+0000", 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:16:23.962 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.963 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.964 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.965 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.966 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:16:23.967 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.968 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.969 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:23.970 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 msgr2=0x7f3f9c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 0x7f3f9c0b7820 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x556707135db0 tx=0x7f3fa4005f90 comp rx=0 tx=0).stop 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 msgr2=0x7f3fb4101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e9c0 tx=0x7f3fa800ee80 comp rx=0 tx=0).stop 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 shutdown_connections 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f3f9c0b5360 0x7f3f9c0b7820 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.971 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fb410d100 0x7f3fb41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3fb41045c0 0x7f3fb4101e90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 --2- 172.21.15.120:0/290623895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fb4103ca0 0x7f3fb4101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:23.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 >> 172.21.15.120:0/290623895 conn(0x7f3fb40fd700 msgr2=0x7f3fb40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:23.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 shutdown_connections 2024-09-10T23:16:23.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:23.944+0000 7f3fba809640 1 -- 172.21.15.120:0/290623895 wait complete. 2024-09-10T23:16:24.517 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:24 smithi120 ceph-mon[31247]: pgmap v1065: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 609 op/s 2024-09-10T23:16:24.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:24 smithi153 ceph-mon[34951]: pgmap v1065: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 609 op/s 2024-09-10T23:16:24.824 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:24 smithi176 ceph-mon[34982]: pgmap v1065: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 609 op/s 2024-09-10T23:16:24.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 273527 21.49 MB/sec execute 559 sec latency 175.060 ms 2024-09-10T23:16:24.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:16:24.988 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:16:25.277 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:25.665 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:25 smithi120 ceph-mon[31247]: pgmap v1066: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-10T23:16:25.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:25 smithi153 ceph-mon[34951]: pgmap v1066: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-10T23:16:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 -- 172.21.15.120:0/2642542130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a8075540 msgr2=0x7fe5a8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 --2- 172.21.15.120:0/2642542130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a8075540 0x7fe5a8075920 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a410 comp rx=0 tx=0).stop 2024-09-10T23:16:25.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 -- 172.21.15.120:0/2642542130 shutdown_connections 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 --2- 172.21.15.120:0/2642542130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8113b40 0x7fe5a8115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 --2- 172.21.15.120:0/2642542130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8075e60 0x7fe5a8113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 --2- 172.21.15.120:0/2642542130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a8075540 0x7fe5a8075920 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 -- 172.21.15.120:0/2642542130 >> 172.21.15.120:0/2642542130 conn(0x7fe5a80fe830 msgr2=0x7fe5a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 -- 172.21.15.120:0/2642542130 shutdown_connections 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.768+0000 7fe5ae039640 1 -- 172.21.15.120:0/2642542130 wait complete. 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 Processor -- start 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 -- start start 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8075e60 0x7fe5a819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 0x7fe5a81a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 0x7fe5a81a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119410 con 0x7fe5a81a53c0 2024-09-10T23:16:25.774 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119290 con 0x7fe5a8075e60 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5ae039640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119590 con 0x7fe5a8113b40 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 0x7fe5a81a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 0x7fe5a81a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36106/0 (socket says 172.21.15.120:36106) 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5a7fff640 1 -- 172.21.15.120:0/469267159 learned_addr learned my addr 172.21.15.120:0/469267159 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5a6ffd640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 0x7fe5a81a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.769+0000 7fe5a77fe640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8075e60 0x7fe5a819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 -- 172.21.15.120:0/469267159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8075e60 msgr2=0x7fe5a819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8075e60 0x7fe5a819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 -- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 msgr2=0x7fe5a81a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 0x7fe5a81a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 -- 172.21.15.120:0/469267159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a5f50 con 0x7fe5a8113b40 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a6ffd640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 0x7fe5a81a0020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00ccc0 con 0x7fe5a8113b40 2024-09-10T23:16:25.775 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81ac540 con 0x7fe5a8113b40 2024-09-10T23:16:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0108e0 con 0x7fe5a8113b40 2024-09-10T23:16:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c018880 con 0x7fe5a8113b40 2024-09-10T23:16:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.770+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81ac930 con 0x7fe5a8113b40 2024-09-10T23:16:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.771+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a8075920 con 0x7fe5a8113b40 2024-09-10T23:16:25.776 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.773+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe59c010430 con 0x7fe5a8113b40 2024-09-10T23:16:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.773+0000 7fe5a4ff9640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 0x7fe5900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.774+0000 7fe5a77fe640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 0x7fe5900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.774+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe59c0e63d0 con 0x7fe5a8113b40 2024-09-10T23:16:25.777 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.774+0000 7fe5a77fe640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 0x7fe5900b77a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe598006010 tx=0x7fe598005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:25.778 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:25.778+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe59c0acc10 con 0x7fe5a8113b40 2024-09-10T23:16:25.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:25 smithi176 ceph-mon[34982]: pgmap v1066: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-10T23:16:25.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 273842 21.48 MB/sec execute 560 sec latency 104.137 ms 2024-09-10T23:16:26.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.004+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5a81a0c10 con 0x7fe5a8113b40 2024-09-10T23:16:26.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.005+0000 7fe5a4ff9640 1 -- 172.21.15.120:0/469267159 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fe5a81a0c10 con 0x7fe5a8113b40 2024-09-10T23:16:26.006 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:16:26.006 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:16:26.006 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:16:26.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 msgr2=0x7fe5900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 0x7fe5900b77a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe598006010 tx=0x7fe598005a20 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 msgr2=0x7fe5a81a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 0x7fe5a81a0020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 shutdown_connections 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe5900b52e0 0x7fe5900b77a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5a81a53c0 0x7fe5a81a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5a8113b40 0x7fe5a81a0020 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 --2- 172.21.15.120:0/469267159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5a8075e60 0x7fe5a819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 >> 172.21.15.120:0/469267159 conn(0x7fe5a80fe830 msgr2=0x7fe5a80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 shutdown_connections 2024-09-10T23:16:26.008 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:26.007+0000 7fe5ae039640 1 -- 172.21.15.120:0/469267159 wait complete. 2024-09-10T23:16:26.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/469267159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:26.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/469267159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:26.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:16:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 -- 172.21.15.120:0/3608554164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 msgr2=0x7f5374101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3608554164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f5374101c70 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5364009930 tx=0x7f536402a420 comp rx=0 tx=0).stop 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 -- 172.21.15.120:0/3608554164 shutdown_connections 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3608554164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5374103380 0x7f5374109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3608554164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53741029c0 0x7f5374102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3608554164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f5374101c70 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 -- 172.21.15.120:0/3608554164 >> 172.21.15.120:0/3608554164 conn(0x7f53740fd150 msgr2=0x7f53740ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 -- 172.21.15.120:0/3608554164 shutdown_connections 2024-09-10T23:16:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 -- 172.21.15.120:0/3608554164 wait complete. 2024-09-10T23:16:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.761+0000 7f5379cd6640 1 Processor -- start 2024-09-10T23:16:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 -- start start 2024-09-10T23:16:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53741029c0 0x7f537419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 0x7f53741a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d4e0 con 0x7f5374101870 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d360 con 0x7f5374103380 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5379cd6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d660 con 0x7f53741029c0 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36120/0 (socket says 172.21.15.120:36120) 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 -- 172.21.15.120:0/3035186699 learned_addr learned my addr 172.21.15.120:0/3035186699 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5373fff640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 0x7f53741a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5372ffd640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53741029c0 0x7f537419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 -- 172.21.15.120:0/3035186699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53741029c0 msgr2=0x7f537419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53741029c0 0x7f537419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 -- 172.21.15.120:0/3035186699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 msgr2=0x7f53741a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 0x7f53741a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 -- 172.21.15.120:0/3035186699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53741a0950 con 0x7f5374101870 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f5373fff640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 0x7f53741a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.762+0000 7f53737fe640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5364009a50 tx=0x7f53640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.763+0000 7f5370ff9640 1 -- 172.21.15.120:0/3035186699 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536403f030 con 0x7f5374101870 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.763+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a0be0 con 0x7f5374101870 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.763+0000 7f5370ff9640 1 -- 172.21.15.120:0/3035186699 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364031910 con 0x7f5374101870 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.763+0000 7f5370ff9640 1 -- 172.21.15.120:0/3035186699 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364035070 con 0x7f5374101870 2024-09-10T23:16:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.763+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a1120 con 0x7f5374101870 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.765+0000 7f5370ff9640 1 -- 172.21.15.120:0/3035186699 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f536403ba60 con 0x7f5374101870 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.765+0000 7f537caa3740 1 Processor -- start 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 -- start start 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee03c6f0 con 0x55f3ee0ca860 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee02bb20 con 0x55f3ee0cfbd0 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f537caa3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee02f930 con 0x55f3ee0cd190 2024-09-10T23:16:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36126/0 (socket says 172.21.15.120:36126) 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 -- 172.21.15.120:0/920771777 learned_addr learned my addr 172.21.15.120:0/920771777 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f53737fe640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5372ffd640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 -- 172.21.15.120:0/920771777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 msgr2=0x55f3ee0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 -- 172.21.15.120:0/920771777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 msgr2=0x55f3ee0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5373fff640 1 -- 172.21.15.120:0/920771777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3edff1530 con 0x55f3ee0ca860 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5372ffd640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f5370ff9640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 0x7f535c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.766+0000 7f53737fe640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 0x7f535c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f5373fff640 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f536c0117e0 tx=0x7f536c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2fb6f3106cef3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f5370ff9640 1 -- 172.21.15.120:0/3035186699 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53641063d0 con 0x7f5374101870 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f5359ffb640 1 -- 172.21.15.120:0/920771777 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c011310 con 0x55f3ee0ca860 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f5359ffb640 1 -- 172.21.15.120:0/920771777 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c003e20 con 0x55f3ee0ca860 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f5359ffb640 1 -- 172.21.15.120:0/920771777 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0044f0 con 0x55f3ee0ca860 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f537caa3740 1 -- 172.21.15.120:0/920771777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 msgr2=0x55f3ee0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f53737fe640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 0x7f535c0b77a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53640c9ee0 tx=0x7f536410b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.767+0000 7f537caa3740 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f536c0117e0 tx=0x7f536c011ca0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- 172.21.15.120:0/920771777 shutdown_connections 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- 172.21.15.120:0/920771777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0ccc50 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- 172.21.15.120:0/920771777 >> 172.21.15.120:0/920771777 conn(0x55f3ee0c59b0 msgr2=0x55f3ee0c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- 172.21.15.120:0/920771777 shutdown_connections 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- 172.21.15.120:0/920771777 wait complete. 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- start start 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 Processor -- start 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f3edf63ab0 con 0x55f3ee0ca860 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f3edfd1730 con 0x55f3ee0cfbd0 2024-09-10T23:16:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee0d2bb0 con 0x55f3ee0cd190 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.768+0000 7f537caa3740 10 client.0 _ll_register_callbacks cb 0x55f3ee0256b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5372ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36138/0 (socket says 172.21.15.120:36138) 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 -- 172.21.15.120:0/1443377596 learned_addr learned my addr 172.21.15.120:0/1443377596 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 -- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 msgr2=0x55f3ee0d4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0d4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 -- 172.21.15.120:0/1443377596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 msgr2=0x55f3ee0d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 -- 172.21.15.120:0/1443377596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3edd8d790 con 0x55f3ee0ca860 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5373fff640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f536c006010 tx=0x7f536c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1fc9ba236de2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c02b020 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f536c004c40 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.769+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c003e20 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f536c022040 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 10 client.20046 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3ede06620 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c003e20 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f536c018020 con 0x55f3ee0ca860 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 10 client.20046 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 10 client.20046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 20 client.20046 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee0df4c0 0x55f3ee0e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 4 client.20046 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ee03e640 con 0x55f3ee0df4c0 2024-09-10T23:16:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.770+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee0df4c0 0x55f3ee0e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.771+0000 7f5352ffd640 10 client.20046 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.771+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee0df4c0 0x55f3ee0e18b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7297d1ead04facb8 server_cookie=13bd1374759c9313 in_seq=0 out_seq=0 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f5352ffd640 1 -- 172.21.15.120:0/1443377596 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3ee03e640 con 0x55f3ee0df4c0 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f5352ffd640 10 client.20046 handle_command_reply: tid=0 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f537caa3740 1 client.20046 shutdown 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 msgr2=0x55f3ee0d3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f536c006010 tx=0x7f536c005a90 comp rx=0 tx=0).stop 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.774+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 shutdown_connections 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee0df4c0 0x55f3ee0e18b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0d4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 --2- 172.21.15.120:0/1443377596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0d3ef0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f5352ffd640 0 client.20046 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 >> 172.21.15.120:0/1443377596 conn(0x55f3ee0c59b0 msgr2=0x55f3ee0c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 shutdown_connections 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 1 -- 172.21.15.120:0/1443377596 wait complete. 2024-09-10T23:16:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.775+0000 7f537caa3740 20 client.20046 trim_cache size 0 max 16384 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 Processor -- start 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 -- start start 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a7db0 con 0x55f3ee0ca860 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53741aa630 con 0x55f3ee0cfbd0 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f537caa3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a8420 con 0x55f3ee0cd190 2024-09-10T23:16:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f5373fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36144/0 (socket says 172.21.15.120:36144) 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f5373fff640 1 -- 172.21.15.120:0/1924570543 learned_addr learned my addr 172.21.15.120:0/1924570543 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.789+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 -- 172.21.15.120:0/1924570543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 msgr2=0x55f3ee0e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 -- 172.21.15.120:0/1924570543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 msgr2=0x55f3ee0e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 -- 172.21.15.120:0/1924570543 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ee0d4260 con 0x55f3ee0ca860 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f5373fff640 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f536c005d70 tx=0x7f536c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee764ed5f88e1299 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f53597fa640 1 -- 172.21.15.120:0/1924570543 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c02b040 con 0x55f3ee0ca860 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f53597fa640 1 -- 172.21.15.120:0/1924570543 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c00fd70 con 0x55f3ee0ca860 2024-09-10T23:16:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.790+0000 7f53597fa640 1 -- 172.21.15.120:0/1924570543 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c027b80 con 0x55f3ee0ca860 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- 172.21.15.120:0/1924570543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 msgr2=0x55f3ee0c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f536c005d70 tx=0x7f536c009500 comp rx=0 tx=0).stop 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- 172.21.15.120:0/1924570543 shutdown_connections 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cd190 0x55f3ee0e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 --2- 172.21.15.120:0/1924570543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0ca860 0x55f3ee0c8f00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- 172.21.15.120:0/1924570543 >> 172.21.15.120:0/1924570543 conn(0x55f3ee0c59b0 msgr2=0x55f3ee0c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- 172.21.15.120:0/1924570543 shutdown_connections 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- 172.21.15.120:0/1924570543 wait complete. 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 -- start start 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.791+0000 7f537caa3740 1 Processor -- start 2024-09-10T23:16:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 0x55f3ee0dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a8af0 con 0x55f3ee0cd190 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a9f60 con 0x55f3ee0ca860 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a91c0 con 0x55f3ee0cfbd0 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 10 client.0 _ll_register_callbacks cb 0x7f53680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36150/0 (socket says 172.21.15.120:36150) 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 -- 172.21.15.120:0/1973379658 learned_addr learned my addr 172.21.15.120:0/1973379658 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5373fff640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5373fff640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 0x55f3ee0dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 -- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cfbd0 msgr2=0x55f3ee0dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0dae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 -- 172.21.15.120:0/1973379658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 msgr2=0x55f3ee0dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 0x55f3ee0dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 -- 172.21.15.120:0/1973379658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ee0e1c20 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5373fff640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 0x55f3ee0dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5372ffd640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x55f3ee0e20e0 tx=0x7f53680039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa50210a3c3b616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368010630 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c00fd70 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f53680107d0 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.792+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53680109d0 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 10 client.20052 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3ee0d54d0 con 0x55f3ee0cd190 2024-09-10T23:16:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c00fd70 con 0x55f3ee0cd190 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5368023030 con 0x55f3ee0cd190 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 10 client.20052 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 10 client.20052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 20 client.20052 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee14cca0 0x55f3ee14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 4 client.20052 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3ee03a0f0 con 0x55f3ee14cca0 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f5373fff640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee14cca0 0x55f3ee14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f5352ffd640 10 client.20052 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:26.793+0000 7f5373fff640 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee14cca0 0x55f3ee14f0c0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af02cdb3062add4 server_cookie=1be53bbd5dd59b8b in_seq=0 out_seq=0 2024-09-10T23:16:26.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/469267159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:16:26.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 274188 21.46 MB/sec execute 561 sec latency 101.018 ms 2024-09-10T23:16:27.046 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:27.567 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:27 smithi120 ceph-mon[31247]: pgmap v1067: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 -- 172.21.15.120:0/3745555840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe014104580 msgr2=0x7fe01410cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3745555840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe014104580 0x7fe01410cb60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009900 tx=0x7fdffc02a410 comp rx=0 tx=0).stop 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 -- 172.21.15.120:0/3745555840 shutdown_connections 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3745555840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01410d130 0x7fe01410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3745555840 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe014104580 0x7fe01410cb60 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3745555840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe014103b80 0x7fe014103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 -- 172.21.15.120:0/3745555840 >> 172.21.15.120:0/3745555840 conn(0x7fe0140fd6f0 msgr2=0x7fe0140ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 -- 172.21.15.120:0/3745555840 shutdown_connections 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.663+0000 7fe018b0f640 1 -- 172.21.15.120:0/3745555840 wait complete. 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.664+0000 7fe018b0f640 1 Processor -- start 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.664+0000 7fe018b0f640 1 -- start start 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe014103b80 0x7fe014199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe014112d70 con 0x7fe01410d130 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe014112bf0 con 0x7fe014104580 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe018b0f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe014112ef0 con 0x7fe014103b80 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35256/0 (socket says 172.21.15.120:35256) 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36178/0 (socket says 172.21.15.120:36178) 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe009d9b640 1 -- 172.21.15.120:0/3413161763 learned_addr learned my addr 172.21.15.120:0/3413161763 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 -- 172.21.15.120:0/3413161763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe014103b80 msgr2=0x7fe014199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe014103b80 0x7fe014199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 -- 172.21.15.120:0/3413161763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 msgr2=0x7fe0141997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 -- 172.21.15.120:0/3413161763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01419f330 con 0x7fe01410d130 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe009d9b640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe012d9d640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009fd0 tx=0x7fe00c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.665+0000 7fe00bfff640 1 -- 172.21.15.120:0/3413161763 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00ce20 con 0x7fe01410d130 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.666+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01419f620 con 0x7fe01410d130 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.666+0000 7fe00bfff640 1 -- 172.21.15.120:0/3413161763 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00c005020 con 0x7fe01410d130 2024-09-10T23:16:27.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.666+0000 7fe00bfff640 1 -- 172.21.15.120:0/3413161763 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c010690 con 0x7fe01410d130 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.666+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a5ce0 con 0x7fe01410d130 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.668+0000 7fe01aee4740 1 Processor -- start 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.668+0000 7fe00bfff640 1 -- 172.21.15.120:0/3413161763 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe00c0054d0 con 0x7fe01410d130 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 -- start start 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 0x5637ec165d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637ec0cdea0 con 0x5637ec1570f0 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637ec0bd2d0 con 0x5637ec157a10 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01aee4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5637ec0c10e0 con 0x5637ec1639a0 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35260/0 (socket says 172.21.15.120:35260) 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec1574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36186/0 (socket says 172.21.15.120:36186) 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 -- 172.21.15.120:0/3283239144 learned_addr learned my addr 172.21.15.120:0/3283239144 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe009d9b640 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 0x5637ec165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 -- 172.21.15.120:0/3283239144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 msgr2=0x5637ec165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 0x5637ec165d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 -- 172.21.15.120:0/3283239144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 msgr2=0x5637ec1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe01259c640 1 -- 172.21.15.120:0/3283239144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ebe1b650 con 0x5637ec157a10 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.669+0000 7fe009d9b640 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 0x5637ec165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe00bfff640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 0x7fe0000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe01259c640 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe00400bb10 tx=0x7fe00400d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd3ed5d11b9d5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe009d9b640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 0x7fe0000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe00bfff640 1 -- 172.21.15.120:0/3413161763 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe00c0e62e0 con 0x7fe01410d130 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe008d99640 1 -- 172.21.15.120:0/3283239144 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004016070 con 0x5637ec157a10 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe008d99640 1 -- 172.21.15.120:0/3283239144 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00400dce0 con 0x5637ec157a10 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.670+0000 7fe008d99640 1 -- 172.21.15.120:0/3283239144 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004010460 con 0x5637ec157a10 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- 172.21.15.120:0/3283239144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 msgr2=0x5637ec163460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe00400bb10 tx=0x7fe00400d2e0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe009d9b640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 0x7fe0000b77a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031440 tx=0x7fdffc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- 172.21.15.120:0/3283239144 shutdown_connections 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1639a0 0x5637ec165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.681 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec163460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 --2- 172.21.15.120:0/3283239144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- 172.21.15.120:0/3283239144 >> 172.21.15.120:0/3283239144 conn(0x5637ec154f20 msgr2=0x5637ec1560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- 172.21.15.120:0/3283239144 shutdown_connections 2024-09-10T23:16:27.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- 172.21.15.120:0/3283239144 wait complete. 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 -- start start 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.671+0000 7fe01aee4740 1 Processor -- start 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec163200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5637ebf1e700 con 0x5637ec157a10 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5637ec058160 con 0x5637ec1639a0 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5637ec166620 con 0x5637ec1570f0 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01aee4740 10 client.0 _ll_register_callbacks cb 0x5637ec0719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36202/0 (socket says 172.21.15.120:36202) 2024-09-10T23:16:27.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 -- 172.21.15.120:0/1576799434 learned_addr learned my addr 172.21.15.120:0/1576799434 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe012d9d640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe01259c640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 -- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 msgr2=0x5637ec163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 -- 172.21.15.120:0/1576799434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 msgr2=0x5637ec160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 -- 172.21.15.120:0/1576799434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ec085c90 con 0x5637ec157a10 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe012d9d640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fe009d9b640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009a20 tx=0x7fdffc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519b81caaa5f7e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fdff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.672+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc04e070 con 0x5637ec157a10 2024-09-10T23:16:27.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00400dce0 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdffc003ac0 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdffc044070 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 10 client.20064 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637ec171ec0 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00400dce0 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdffc04a420 con 0x5637ec157a10 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 10 client.20064 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 20 client.20064 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 4 client.20064 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ec0cfdf0 con 0x5637ec173340 2024-09-10T23:16:27.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.673+0000 7fe01259c640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.674+0000 7fdff9ffb640 10 client.20064 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:27.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.674+0000 7fe01259c640 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1757c0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0db1bf1bea6b326 server_cookie=3359404203af110c in_seq=0 out_seq=0 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1576799434 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637ec0cfdf0 con 0x5637ec173340 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fdff9ffb640 10 client.20064 handle_command_reply: tid=0 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 client.20064 shutdown 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 msgr2=0x5637ec160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009a20 tx=0x7fdffc02a410 comp rx=0 tx=0).stop 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 shutdown_connections 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1757c0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec157a10 0x5637ec160650 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1576799434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.689 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fdff9ffb640 0 client.20064 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 >> 172.21.15.120:0/1576799434 conn(0x5637ec154f20 msgr2=0x5637ec15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 shutdown_connections 2024-09-10T23:16:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 1 -- 172.21.15.120:0/1576799434 wait complete. 2024-09-10T23:16:27.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.679+0000 7fe01aee4740 20 client.20064 trim_cache size 0 max 16384 2024-09-10T23:16:27.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 Processor -- start 2024-09-10T23:16:27.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 -- start start 2024-09-10T23:16:27.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec157a10 0x5637ec15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141ac230 con 0x5637ec1570f0 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141aeab0 con 0x5637ec1639a0 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01aee4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141ac8a0 con 0x5637ec157a10 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36214/0 (socket says 172.21.15.120:36214) 2024-09-10T23:16:27.699 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 -- 172.21.15.120:0/113510601 learned_addr learned my addr 172.21.15.120:0/113510601 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe012d9d640 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe009d9b640 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec157a10 0x5637ec15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 -- 172.21.15.120:0/113510601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec157a10 msgr2=0x5637ec15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec157a10 0x5637ec15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 -- 172.21.15.120:0/113510601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 msgr2=0x5637ec15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 -- 172.21.15.120:0/113510601 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ec15b3a0 con 0x5637ec1570f0 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.693+0000 7fe01259c640 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe00400db00 tx=0x7fe0040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f88706c0ef87746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fdffbfff640 1 -- 172.21.15.120:0/113510601 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004016040 con 0x5637ec1570f0 2024-09-10T23:16:27.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fdffbfff640 1 -- 172.21.15.120:0/113510601 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004004500 con 0x5637ec1570f0 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fdffbfff640 1 -- 172.21.15.120:0/113510601 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004003e50 con 0x5637ec1570f0 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 -- 172.21.15.120:0/113510601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 msgr2=0x5637ec16f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe00400db00 tx=0x7fe0040046c0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 -- 172.21.15.120:0/113510601 shutdown_connections 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec1639a0 0x5637ec15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec157a10 0x5637ec15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 --2- 172.21.15.120:0/113510601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1570f0 0x5637ec16f880 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 -- 172.21.15.120:0/113510601 >> 172.21.15.120:0/113510601 conn(0x5637ec154f20 msgr2=0x5637ec155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.694+0000 7fe01aee4740 1 -- 172.21.15.120:0/113510601 shutdown_connections 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 -- 172.21.15.120:0/113510601 wait complete. 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 -- start start 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 Processor -- start 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec167a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec167fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 0x5637ec168500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141acf70 con 0x5637ec1639a0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141ae3e0 con 0x5637ec157a10 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe0141ad640 con 0x5637ec1570f0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01aee4740 10 client.0 _ll_register_callbacks cb 0x7fdffc032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe012d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 0x5637ec168500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec167fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01259c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53670/0 (socket says 172.21.15.120:53670) 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.695+0000 7fe01259c640 1 -- 172.21.15.120:0/1423099525 learned_addr learned my addr 172.21.15.120:0/1423099525 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:27.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 -- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 msgr2=0x5637ec167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 -- 172.21.15.120:0/1423099525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 msgr2=0x5637ec167fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec167fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 -- 172.21.15.120:0/1423099525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ec1559a0 con 0x5637ec1639a0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe009d9b640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec167fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe012d9d640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 0x5637ec168500 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0e4180 tx=0x7fe00c0a8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798d7cca89c376f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0d3070 con 0x5637ec1639a0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fe00c0a8040 con 0x5637ec1639a0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5637ec0cb8a0 con 0x5637ec1639a0 2024-09-10T23:16:27.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe00c0b15b0 con 0x5637ec1639a0 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 10 client.20070 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637ec1e0640 con 0x5637ec1639a0 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5637ec0cb8a0 con 0x5637ec1639a0 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fe00c0a8420 con 0x5637ec1639a0 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 10 client.20070 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 10 client.20070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 20 client.20070 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 4 client.20070 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.696+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637ec0c8cd0 con 0x5637ec173340 2024-09-10T23:16:27.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.697+0000 7fe01259c640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.697+0000 7fdff9ffb640 10 client.20070 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:27.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.697+0000 7fe01259c640 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1e2dd0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0d45b0fc8318ef server_cookie=5470e2efacc384f1 in_seq=0 out_seq=0 2024-09-10T23:16:27.706 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:27 smithi153 ceph-mon[34951]: pgmap v1067: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-09-10T23:16:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fdff9ffb640 1 -- 172.21.15.120:0/1423099525 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637ec0c8cd0 con 0x5637ec173340 2024-09-10T23:16:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fdff9ffb640 10 client.20070 handle_command_reply: tid=0 2024-09-10T23:16:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 client.20070 shutdown 2024-09-10T23:16:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 msgr2=0x5637ec168500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.707 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 0x5637ec168500 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0e4180 tx=0x7fe00c0a8360 comp rx=0 tx=0).stop 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 shutdown_connections 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x5637ec173340 0x5637ec1e2dd0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5637ec1639a0 0x5637ec168500 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637ec157a10 0x5637ec167fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 --2- 172.21.15.120:0/1423099525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5637ec1570f0 0x5637ec167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 >> 172.21.15.120:0/1423099525 conn(0x5637ec154f20 msgr2=0x5637ec161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fdff9ffb640 0 client.20070 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 shutdown_connections 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.702+0000 7fe01aee4740 1 -- 172.21.15.120:0/1423099525 wait complete. 2024-09-10T23:16:27.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe01aee4740 20 client.20070 trim_cache size 0 max 16384 2024-09-10T23:16:27.709 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:16:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 msgr2=0x7fe0000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 0x7fe0000b77a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031440 tx=0x7fdffc005b50 comp rx=0 tx=0).stop 2024-09-10T23:16:27.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 msgr2=0x7fe01419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009fd0 tx=0x7fe00c00ef10 comp rx=0 tx=0).stop 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 shutdown_connections 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fe0000b52e0 0x7fe0000b77a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe01410d130 0x7fe01419eb60 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe014104580 0x7fe0141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 --2- 172.21.15.120:0/3413161763 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe014103b80 0x7fe014199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 >> 172.21.15.120:0/3413161763 conn(0x7fe0140fd6f0 msgr2=0x7fe0140fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 shutdown_connections 2024-09-10T23:16:27.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:27.703+0000 7fe018b0f640 1 -- 172.21.15.120:0/3413161763 wait complete. 2024-09-10T23:16:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.751+0000 7f5352ffd640 1 -- 172.21.15.120:0/1973379658 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3ee03a0f0 con 0x55f3ee14cca0 2024-09-10T23:16:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.751+0000 7f5352ffd640 10 client.20052 handle_command_reply: tid=0 2024-09-10T23:16:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.751+0000 7f537caa3740 1 client.20052 shutdown 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 msgr2=0x55f3ee0da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x55f3ee0e20e0 tx=0x7f53680039c0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 shutdown_connections 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f3ee14cca0 0x55f3ee14f0c0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f3ee0cfbd0 0x55f3ee0dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f3ee0cd190 0x55f3ee0da800 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 --2- 172.21.15.120:0/1973379658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f3ee0ca860 0x55f3ee0dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 >> 172.21.15.120:0/1973379658 conn(0x55f3ee0c59b0 msgr2=0x55f3ee0c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f5352ffd640 0 client.20052 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 shutdown_connections 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 1 -- 172.21.15.120:0/1973379658 wait complete. 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.752+0000 7f537caa3740 20 client.20052 trim_cache size 0 max 16384 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:16:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:16:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 msgr2=0x7f535c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 0x7f535c0b77a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f53640c9ee0 tx=0x7f536410b040 comp rx=0 tx=0).stop 2024-09-10T23:16:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 msgr2=0x7f537406bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5364009a50 tx=0x7f53640042f0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 shutdown_connections 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f535c0b52e0 0x7f535c0b77a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5374103380 0x7f53741a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53741029c0 0x7f537419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 --2- 172.21.15.120:0/3035186699 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5374101870 0x7f537406bcc0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 >> 172.21.15.120:0/3035186699 conn(0x7f53740fd150 msgr2=0x7f5374105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 shutdown_connections 2024-09-10T23:16:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:27.753+0000 7f5379cd6640 1 -- 172.21.15.120:0/3035186699 wait complete. 2024-09-10T23:16:27.822 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:27 smithi176 ceph-mon[34982]: pgmap v1067: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-09-10T23:16:27.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 274282 21.43 MB/sec execute 562 sec latency 550.284 ms 2024-09-10T23:16:28.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 274584 21.41 MB/sec execute 563 sec latency 205.209 ms 2024-09-10T23:16:29.697 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:16:29.697 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:16:29.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275011 21.42 MB/sec execute 564 sec latency 233.319 ms 2024-09-10T23:16:29.947 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:29 smithi120 ceph-mon[31247]: pgmap v1068: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 383 op/s 2024-09-10T23:16:29.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:29 smithi153 ceph-mon[34951]: pgmap v1068: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 383 op/s 2024-09-10T23:16:30.005 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:30.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:29 smithi176 ceph-mon[34982]: pgmap v1068: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 383 op/s 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 -- 172.21.15.120:0/2783131639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b4108020 msgr2=0x7f52b410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/2783131639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b4108020 0x7f52b410e820 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009ca0 tx=0x7f52ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 -- 172.21.15.120:0/2783131639 shutdown_connections 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/2783131639 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b4108020 0x7f52b410e820 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/2783131639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b41076a0 0x7f52b4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/2783131639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 -- 172.21.15.120:0/2783131639 >> 172.21.15.120:0/2783131639 conn(0x7f52b40fd700 msgr2=0x7f52b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 -- 172.21.15.120:0/2783131639 shutdown_connections 2024-09-10T23:16:30.555 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.532+0000 7f52bbaa8640 1 -- 172.21.15.120:0/2783131639 wait complete. 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 Processor -- start 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 -- start start 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b4108020 0x7f52b40fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111ce0 con 0x7f52b41076a0 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111b60 con 0x7f52b4108020 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52bbaa8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111e60 con 0x7f52b4101c90 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.533+0000 7f52b9844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53678/0 (socket says 172.21.15.120:53678) 2024-09-10T23:16:30.556 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36246/0 (socket says 172.21.15.120:36246) 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9844640 1 -- 172.21.15.120:0/1616933481 learned_addr learned my addr 172.21.15.120:0/1616933481 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52ba045640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b4108020 0x7f52b40fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 -- 172.21.15.120:0/1616933481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 msgr2=0x7f52b40fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 -- 172.21.15.120:0/1616933481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b4108020 msgr2=0x7f52b40fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b4108020 0x7f52b40fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 -- 172.21.15.120:0/1616933481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41a7620 con 0x7f52b41076a0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9844640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52b9043640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f52a400e9f0 tx=0x7f52a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52aaffd640 1 -- 172.21.15.120:0/1616933481 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400ccc0 con 0x7f52b41076a0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b41a7910 con 0x7f52b41076a0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52aaffd640 1 -- 172.21.15.120:0/1616933481 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a40108e0 con 0x7f52b41076a0 2024-09-10T23:16:30.557 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52aaffd640 1 -- 172.21.15.120:0/1616933481 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4018880 con 0x7f52b41076a0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.534+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a7ea0 con 0x7f52b41076a0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52aaffd640 1 -- 172.21.15.120:0/1616933481 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f52a4010430 con 0x7f52b41076a0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 Processor -- start 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 -- start start 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c40250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f668baae60 con 0x55f668c40790 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f668b9a290 con 0x55f668c33ee0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.537+0000 7f52bde7d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f668b9e0a0 con 0x55f668c34800 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c40250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.558 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c42b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36256/0 (socket says 172.21.15.120:36256) 2024-09-10T23:16:30.559 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9043640 1 -- 172.21.15.120:0/3174788417 learned_addr learned my addr 172.21.15.120:0/3174788417 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52ba045640 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 -- 172.21.15.120:0/3174788417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 msgr2=0x55f668c342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 -- 172.21.15.120:0/3174788417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 msgr2=0x55f668c42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.560 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 -- 172.21.15.120:0/3174788417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6688f8930 con 0x55f668c34800 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52aaffd640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 0x7f52a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52ba045640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 0x7f52a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52b9844640 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c40250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52b0011520 tx=0x7f52b00119e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457f16d11f6ad94c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f52aaffd640 1 -- 172.21.15.120:0/1616933481 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f52a40e5ec0 con 0x7f52b41076a0 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f529ffff640 1 -- 172.21.15.120:0/3174788417 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b001e070 con 0x55f668c34800 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f529ffff640 1 -- 172.21.15.120:0/3174788417 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b00035f0 con 0x55f668c34800 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.538+0000 7f529ffff640 1 -- 172.21.15.120:0/3174788417 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b000f840 con 0x55f668c34800 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 -- 172.21.15.120:0/3174788417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 msgr2=0x55f668c40250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c40250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52b0011520 tx=0x7f52b00119e0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.561 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52ba045640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 0x7f52a00b77a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac0099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 -- 172.21.15.120:0/3174788417 shutdown_connections 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c40250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 --2- 172.21.15.120:0/3174788417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 -- 172.21.15.120:0/3174788417 >> 172.21.15.120:0/3174788417 conn(0x55f668c31d10 msgr2=0x55f668c32eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 -- 172.21.15.120:0/3174788417 shutdown_connections 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.539+0000 7f52bde7d740 1 -- 172.21.15.120:0/3174788417 wait complete. 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 -- start start 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 Processor -- start 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.562 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c3e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f6689fb890 con 0x55f668c40790 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55f668b34d40 con 0x55f668c33ee0 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f668c43440 con 0x55f668c34800 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52bde7d740 10 client.0 _ll_register_callbacks cb 0x55f668b98c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52b9844640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52ba045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c3e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52b9844640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:35316/0 (socket says 172.21.15.120:35316) 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52b9844640 1 -- 172.21.15.120:0/846676944 learned_addr learned my addr 172.21.15.120:0/846676944 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.540+0000 7f52b9043640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.563 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 -- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 msgr2=0x55f668c3dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 -- 172.21.15.120:0/846676944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 msgr2=0x55f668c3e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c3e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 -- 172.21.15.120:0/846676944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f668c4c820 con 0x55f668c33ee0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52ba045640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c3e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52b9844640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f52b0011440 tx=0x7f52b00039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d2ce7a62990b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f529d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0027060 con 0x55f668c33ee0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f52b0003bc0 con 0x55f668c33ee0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b00035f0 con 0x55f668c33ee0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52b001e040 con 0x55f668c33ee0 2024-09-10T23:16:30.564 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52bde7d740 10 client.29222 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.541+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f668c4f1b0 con 0x55f668c33ee0 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b00035f0 con 0x55f668c33ee0 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f52b0021030 con 0x55f668c33ee0 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 10 client.29222 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 10 client.29222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 20 client.29222 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668c50610 0x55f668c52a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 4 client.29222 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f668bacdb0 con 0x55f668c50610 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52b9043640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668c50610 0x55f668c52a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f529d7fa640 10 client.29222 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.542+0000 7f52b9043640 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668c50610 0x55f668c52a00 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e707aad5b4a66154 server_cookie=f69af64f14d96eed in_seq=0 out_seq=0 2024-09-10T23:16:30.565 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f529d7fa640 1 -- 172.21.15.120:0/846676944 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f668bacdb0 con 0x55f668c50610 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f529d7fa640 10 client.29222 handle_command_reply: tid=0 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 client.29222 shutdown 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 msgr2=0x55f668c3d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f52b0011440 tx=0x7f52b00039d0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 shutdown_connections 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668c50610 0x55f668c52a00 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c3e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 --2- 172.21.15.120:0/846676944 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3d710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f529d7fa640 0 client.29222 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 >> 172.21.15.120:0/846676944 conn(0x55f668c31d10 msgr2=0x55f668c30f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 shutdown_connections 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 1 -- 172.21.15.120:0/846676944 wait complete. 2024-09-10T23:16:30.566 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.545+0000 7f52bde7d740 20 client.29222 trim_cache size 0 max 16384 2024-09-10T23:16:30.570 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 Processor -- start 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 -- start start 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ae4d0 con 0x55f668c40790 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b0d50 con 0x55f668c33ee0 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52bde7d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41aeb40 con 0x55f668c34800 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52ba045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52ba045640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36276/0 (socket says 172.21.15.120:36276) 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52ba045640 1 -- 172.21.15.120:0/1375576595 learned_addr learned my addr 172.21.15.120:0/1375576595 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:30.571 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52b9043640 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.563+0000 7f52b9844640 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 -- 172.21.15.120:0/1375576595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 msgr2=0x55f668c3ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 -- 172.21.15.120:0/1375576595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 msgr2=0x55f668c3a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 -- 172.21.15.120:0/1375576595 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f668c32790 con 0x55f668c40790 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f52ba045640 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f52ac035000 tx=0x7f52ac03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12feca5fe10e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f529f7fe640 1 -- 172.21.15.120:0/1375576595 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac056070 con 0x55f668c40790 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f529f7fe640 1 -- 172.21.15.120:0/1375576595 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac036050 con 0x55f668c40790 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.564+0000 7f529f7fe640 1 -- 172.21.15.120:0/1375576595 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac02f8c0 con 0x55f668c40790 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- 172.21.15.120:0/1375576595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 msgr2=0x55f668c58170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.572 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f52ac035000 tx=0x7f52ac03f000 comp rx=0 tx=0).stop 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- 172.21.15.120:0/1375576595 shutdown_connections 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c40790 0x55f668c58170 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c34800 0x55f668c3ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1375576595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c33ee0 0x55f668c3a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- 172.21.15.120:0/1375576595 >> 172.21.15.120:0/1375576595 conn(0x55f668c31d10 msgr2=0x55f668c3f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- 172.21.15.120:0/1375576595 shutdown_connections 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- 172.21.15.120:0/1375576595 wait complete. 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 -- start start 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.565+0000 7f52bde7d740 1 Processor -- start 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 0x55f668c47170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c40790 0x55f668cb5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41af210 con 0x55f668c34800 2024-09-10T23:16:30.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b0680 con 0x55f668c40790 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41af8e0 con 0x55f668c33ee0 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52bde7d740 10 client.0 _ll_register_callbacks cb 0x7f52b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 0x55f668c47170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36284/0 (socket says 172.21.15.120:36284) 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9844640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 0x55f668c47170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:53720/0 (socket says 172.21.15.120:53720) 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 -- 172.21.15.120:0/1627172174 learned_addr learned my addr 172.21.15.120:0/1627172174 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52ba045640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c40790 0x55f668cb5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 -- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 msgr2=0x55f668c47170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 0x55f668c47170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.574 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 -- 172.21.15.120:0/1627172174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c40790 msgr2=0x55f668cb5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c40790 0x55f668cb5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.566+0000 7f52b9043640 1 -- 172.21.15.120:0/1627172174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f668cb5a80 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52b9043640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f52a40e3a40 tx=0x7f52a40e99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3100f7f9a9f0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40a6e60 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f52a40a89d0 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac036050 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a40ac0b0 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 10 client.20088 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f668cbe030 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac036050 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f52a40a80b0 con 0x55f668c34800 2024-09-10T23:16:30.575 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 10 client.20088 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 10 client.20088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 20 client.20088 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668cbe790 0x55f668cc0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 4 client.20088 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.567+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f668ba8860 con 0x55f668cbe790 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.568+0000 7f52b9844640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668cbe790 0x55f668cc0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.568+0000 7f529d7fa640 10 client.20088 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:30.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.568+0000 7f52b9844640 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668cbe790 0x55f668cc0bb0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8126a8101d2f14a5 server_cookie=67992aecd33f205f in_seq=0 out_seq=0 2024-09-10T23:16:30.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f529d7fa640 1 -- 172.21.15.120:0/1627172174 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f668ba8860 con 0x55f668cbe790 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f529d7fa640 10 client.20088 handle_command_reply: tid=0 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f52bde7d740 1 client.20088 shutdown 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 msgr2=0x55f668c47710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f52a40e3a40 tx=0x7f52a40e99a0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 shutdown_connections 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.577+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55f668cbe790 0x55f668cc0bb0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f668c40790 0x55f668cb5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f668c34800 0x55f668c47710 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 --2- 172.21.15.120:0/1627172174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f668c33ee0 0x55f668c47170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f529d7fa640 0 client.20088 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 >> 172.21.15.120:0/1627172174 conn(0x55f668c31d10 msgr2=0x55f668c30750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 shutdown_connections 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 1 -- 172.21.15.120:0/1627172174 wait complete. 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bde7d740 20 client.20088 trim_cache size 0 max 16384 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stdout: "return_code": 0, 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_tag": "20493467-6325-4ca0-9248-5c9e4611e3a4", 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": "asynchronous" 2024-09-10T23:16:30.580 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 msgr2=0x7f52a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 0x7f52a00b77a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac0099a0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 msgr2=0x7f52b40fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f52a400e9f0 tx=0x7f52a400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 shutdown_connections 2024-09-10T23:16:30.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f52a00b52e0 0x7f52a00b77a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52b4108020 0x7f52b40fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52b41076a0 0x7f52b40fe140 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 --2- 172.21.15.120:0/1616933481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52b4101c90 0x7f52b40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.578+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 >> 172.21.15.120:0/1616933481 conn(0x7f52b40fd700 msgr2=0x7f52b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.579+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 shutdown_connections 2024-09-10T23:16:30.582 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:30.579+0000 7f52bbaa8640 1 -- 172.21.15.120:0/1616933481 wait complete. 2024-09-10T23:16:30.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275513 21.42 MB/sec execute 565 sec latency 95.594 ms 2024-09-10T23:16:31.389 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:16:31.674 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:16:31.846 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:31 smithi120 ceph-mon[31247]: pgmap v1069: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-10T23:16:31.846 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:31 smithi120 ceph-mon[31247]: scrub queued for path: / 2024-09-10T23:16:31.846 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:31 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:31.846 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:31 smithi120 ceph-mon[31247]: scrub summary: active paths [/] 2024-09-10T23:16:31.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275630 21.39 MB/sec execute 566 sec latency 361.630 ms 2024-09-10T23:16:31.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:31 smithi153 ceph-mon[34951]: pgmap v1069: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-10T23:16:31.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:31 smithi153 ceph-mon[34951]: scrub queued for path: / 2024-09-10T23:16:31.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:31 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:31.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:31 smithi153 ceph-mon[34951]: scrub summary: active paths [/] 2024-09-10T23:16:32.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:31 smithi176 ceph-mon[34982]: pgmap v1069: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-10T23:16:32.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:31 smithi176 ceph-mon[34982]: scrub queued for path: / 2024-09-10T23:16:32.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:31 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:32.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:31 smithi176 ceph-mon[34982]: scrub summary: active paths [/] 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 -- 172.21.15.120:0/1694308251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc1076e0 msgr2=0x7fd0cc107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/1694308251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc107b20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009900 tx=0x7fd0bc02a410 comp rx=0 tx=0).stop 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 -- 172.21.15.120:0/1694308251 shutdown_connections 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/1694308251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc108060 0x7fd0cc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/1694308251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc107b20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/1694308251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.219 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 -- 172.21.15.120:0/1694308251 >> 172.21.15.120:0/1694308251 conn(0x7fd0cc0fd740 msgr2=0x7fd0cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 -- 172.21.15.120:0/1694308251 shutdown_connections 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.186+0000 7fd0d46aa640 1 -- 172.21.15.120:0/1694308251 wait complete. 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 Processor -- start 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 -- start start 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc108060 0x7fd0cc1a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc111ce0 con 0x7fd0cc101cd0 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc111b60 con 0x7fd0cc108060 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d46aa640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc111e60 con 0x7fd0cc1076e0 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36302/0 (socket says 172.21.15.120:36302) 2024-09-10T23:16:32.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 -- 172.21.15.120:0/775083318 learned_addr learned my addr 172.21.15.120:0/775083318 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc108060 0x7fd0cc1a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 -- 172.21.15.120:0/775083318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc1076e0 msgr2=0x7fd0cc19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 -- 172.21.15.120:0/775083318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc108060 msgr2=0x7fd0cc1a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc108060 0x7fd0cc1a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.187+0000 7fd0d2446640 1 -- 172.21.15.120:0/775083318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc1a5630 con 0x7fd0cc101cd0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0d2446640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800ea60 tx=0x7fd0c800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0c37fe640 1 -- 172.21.15.120:0/775083318 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800cde0 con 0x7fd0cc101cd0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0c37fe640 1 -- 172.21.15.120:0/775083318 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c80035f0 con 0x7fd0cc101cd0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc1a5800 con 0x7fd0cc101cd0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0c37fe640 1 -- 172.21.15.120:0/775083318 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8010690 con 0x7fd0cc101cd0 2024-09-10T23:16:32.221 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.188+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a5cb0 con 0x7fd0cc101cd0 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.190+0000 7fd0c37fe640 1 -- 172.21.15.120:0/775083318 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd0c8004000 con 0x7fd0cc101cd0 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.190+0000 7fd0d6a7f740 1 Processor -- start 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 -- start start 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3aeda0 0x560bff3bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560bff324e50 con 0x560bff3ae480 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560bff314280 con 0x560bff3bf140 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560bff318090 con 0x560bff3aeda0 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36312/0 (socket says 172.21.15.120:36312) 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 -- 172.21.15.120:0/4224347748 learned_addr learned my addr 172.21.15.120:0/4224347748 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2446640 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3aeda0 0x560bff3bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.222 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0c37fe640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 0x7fd0b80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 -- 172.21.15.120:0/4224347748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3aeda0 msgr2=0x560bff3bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3aeda0 0x560bff3bec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 -- 172.21.15.120:0/4224347748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 msgr2=0x560bff3c1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.191+0000 7fd0d2c47640 1 -- 172.21.15.120:0/4224347748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bff072650 con 0x560bff3ae480 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0c37fe640 1 -- 172.21.15.120:0/775083318 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd0c80e5e10 con 0x7fd0cc101cd0 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4011440 tx=0x7fd0c4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cb2a432b5e9460 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 0x7fd0b80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0c0ff9640 1 -- 172.21.15.120:0/4224347748 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c401e070 con 0x560bff3ae480 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0c0ff9640 1 -- 172.21.15.120:0/4224347748 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c40045c0 con 0x560bff3ae480 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.192+0000 7fd0c0ff9640 1 -- 172.21.15.120:0/4224347748 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4019430 con 0x560bff3ae480 2024-09-10T23:16:32.223 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4224347748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 msgr2=0x560bff3ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 0x7fd0b80b77a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4010010 tx=0x7fd0c4019a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4011440 tx=0x7fd0c4011900 comp rx=0 tx=0).stop 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4224347748 shutdown_connections 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3aeda0 0x560bff3bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4224347748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3ae480 0x560bff3ae860 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4224347748 >> 172.21.15.120:0/4224347748 conn(0x560bff3ac2b0 msgr2=0x560bff3ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4224347748 shutdown_connections 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4224347748 wait complete. 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- start start 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 Processor -- start 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 0x560bff3bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.224 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3bf140 0x560bff3bc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560bff175700 con 0x560bff3aeda0 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x560bff2af110 con 0x560bff3ae480 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560bff3c1df0 con 0x560bff3bf140 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.193+0000 7fd0d6a7f740 10 client.0 _ll_register_callbacks cb 0x560bff2c8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36314/0 (socket says 172.21.15.120:36314) 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d2446640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 0x560bff3bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 -- 172.21.15.120:0/314736188 learned_addr learned my addr 172.21.15.120:0/314736188 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3bf140 0x560bff3bc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 -- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3bf140 msgr2=0x560bff3bc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3bf140 0x560bff3bc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 -- 172.21.15.120:0/314736188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 msgr2=0x560bff3bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.225 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 0x560bff3bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 -- 172.21.15.120:0/314736188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bff2dcc40 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d2446640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 0x560bff3bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc02a930 tx=0x7fd0bc032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9f4cfc2ab0575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc035040 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd0bc031720 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c40045c0 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc04d070 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d6a7f740 10 client.20100 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.194+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560bff3cd770 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c40045c0 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd0bc004750 con 0x560bff3aeda0 2024-09-10T23:16:32.226 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 10 client.20100 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 10 client.20100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 20 client.20100 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff3cec30 0x560bff3d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 4 client.20100 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bff326da0 con 0x560bff3cec30 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d2446640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff3cec30 0x560bff3d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0b5ffb640 10 client.20100 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.195+0000 7fd0d2446640 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff3cec30 0x560bff3d1060 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a1b56f2898e9dd server_cookie=98ce81a6d80c578f in_seq=0 out_seq=0 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/314736188 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560bff326da0 con 0x560bff3cec30 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0b5ffb640 10 client.20100 handle_command_reply: tid=0 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 client.20100 shutdown 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 msgr2=0x560bff3bbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc02a930 tx=0x7fd0bc032f00 comp rx=0 tx=0).stop 2024-09-10T23:16:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 shutdown_connections 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff3cec30 0x560bff3d1060 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3bf140 0x560bff3bc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3bbf20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0b5ffb640 0 client.20100 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/314736188 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3ae480 0x560bff3bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 >> 172.21.15.120:0/314736188 conn(0x560bff3ac2b0 msgr2=0x560bff3b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 shutdown_connections 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/314736188 wait complete. 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.196+0000 7fd0d6a7f740 20 client.20100 trim_cache size 0 max 16384 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 Processor -- start 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 -- start start 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.228 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1aa0a0 con 0x560bff3aeda0 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ac920 con 0x560bff3bf140 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1aa710 con 0x560bff3ae480 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36328/0 (socket says 172.21.15.120:36328) 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 -- 172.21.15.120:0/4117386222 learned_addr learned my addr 172.21.15.120:0/4117386222 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d2446640 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 -- 172.21.15.120:0/4117386222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 msgr2=0x560bff3cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 -- 172.21.15.120:0/4117386222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 msgr2=0x560bff3ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.229 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 -- 172.21.15.120:0/4117386222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bff3acd30 con 0x560bff3aeda0 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc004ba0 tx=0x7fd0bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bf960df32b3411 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.214+0000 7fd0b7fff640 1 -- 172.21.15.120:0/4117386222 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc035030 con 0x560bff3aeda0 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0b7fff640 1 -- 172.21.15.120:0/4117386222 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc005850 con 0x560bff3aeda0 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0b7fff640 1 -- 172.21.15.120:0/4117386222 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0326f0 con 0x560bff3aeda0 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4117386222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 msgr2=0x560bff3ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc004ba0 tx=0x7fd0bc02a410 comp rx=0 tx=0).stop 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4117386222 shutdown_connections 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3ad3e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/4117386222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4117386222 >> 172.21.15.120:0/4117386222 conn(0x560bff3ac2b0 msgr2=0x560bff3bff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4117386222 shutdown_connections 2024-09-10T23:16:32.230 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.215+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/4117386222 wait complete. 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 -- start start 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 Processor -- start 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1aade0 con 0x560bff3aeda0 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ac250 con 0x560bff3bf140 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ab4b0 con 0x560bff3ae480 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d6a7f740 10 client.0 _ll_register_callbacks cb 0x7fd0bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d2446640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36342/0 (socket says 172.21.15.120:36342) 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 -- 172.21.15.120:0/3835438979 learned_addr learned my addr 172.21.15.120:0/3835438979 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:32.231 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d2c47640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 -- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 msgr2=0x560bff3c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 -- 172.21.15.120:0/3835438979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 msgr2=0x560bff3c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d1c45640 1 -- 172.21.15.120:0/3835438979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bff3c4590 con 0x560bff3aeda0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.216+0000 7fd0d2446640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d1c45640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc004ba0 tx=0x7fd0bc0325a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d413f00501995a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc035070 con 0x560bff3aeda0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd0bc004f90 con 0x560bff3aeda0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc005850 con 0x560bff3aeda0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc0051b0 con 0x560bff3aeda0 2024-09-10T23:16:32.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 10 client.20106 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560bff43c330 con 0x560bff3aeda0 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc005850 con 0x560bff3aeda0 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fd0bc052930 con 0x560bff3aeda0 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 10 client.20106 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 10 client.20106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 20 client.20106 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff43c9b0 0x560bff43edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 4 client.20106 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.217+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bff322850 con 0x560bff43c9b0 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.218+0000 7fd0d2446640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff43c9b0 0x560bff43edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.218+0000 7fd0b5ffb640 10 client.20106 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:32.233 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.218+0000 7fd0d2446640 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff43c9b0 0x560bff43edd0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77fda04d7859c23a server_cookie=1a821e5c3f267915 in_seq=0 out_seq=0 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.220+0000 7fd0b5ffb640 1 -- 172.21.15.120:0/3835438979 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+11646 (crc 0 0 0) 0x560bff322850 con 0x560bff43c9b0 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.220+0000 7fd0b5ffb640 10 client.20106 handle_command_reply: tid=0 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.220+0000 7fd0d6a7f740 1 client.20106 shutdown 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 msgr2=0x560bff3c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc004ba0 tx=0x7fd0bc0325a0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 shutdown_connections 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x560bff43c9b0 0x560bff43edd0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560bff3bf140 0x560bff3c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560bff3aeda0 0x560bff3c75c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 --2- 172.21.15.120:0/3835438979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560bff3ae480 0x560bff3c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 >> 172.21.15.120:0/3835438979 conn(0x560bff3ac2b0 msgr2=0x560bff3bd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.235 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0b5ffb640 0 client.20106 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 shutdown_connections 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 1 -- 172.21.15.120:0/3835438979 wait complete. 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d6a7f740 20 client.20106 trim_cache size 0 max 16384 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": { 2024-09-10T23:16:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "20493467-6325-4ca0-9248-5c9e4611e3a4": { 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "tag": "20493467-6325-4ca0-9248-5c9e4611e3a4", 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "options": "recursive,force" 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:16:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:16:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "20493467-6325-4ca0-9248-5c9e4611e3a4": { 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:16:30.576286+0000", 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:16:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "2dbbc85c-b952-4f9d-a994-746a3e80926f": { 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:50.606606+0000", 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:16:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:16:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 msgr2=0x7fd0b80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 0x7fd0b80b77a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4010010 tx=0x7fd0c4019a80 comp rx=0 tx=0).stop 2024-09-10T23:16:32.260 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 msgr2=0x7fd0cc19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800ea60 tx=0x7fd0c800ef20 comp rx=0 tx=0).stop 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 shutdown_connections 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd0b80b52e0 0x7fd0b80b77a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0cc108060 0x7fd0cc1a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0cc1076e0 0x7fd0cc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 --2- 172.21.15.120:0/775083318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc101cd0 0x7fd0cc19a070 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.221+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 >> 172.21.15.120:0/775083318 conn(0x7fd0cc0fd740 msgr2=0x7fd0cc104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.222+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 shutdown_connections 2024-09-10T23:16:32.261 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:16:32.222+0000 7fd0d46aa640 1 -- 172.21.15.120:0/775083318 wait complete. 2024-09-10T23:16:32.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 275968 21.39 MB/sec execute 567 sec latency 351.517 ms 2024-09-10T23:16:32.904 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:32 smithi120 ceph-mon[31247]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:32.934 INFO:tasks.cephfs.filesystem:scrub status for tag:20493467-6325-4ca0-9248-5c9e4611e3a4 - {'path': '/', 'tag': '20493467-6325-4ca0-9248-5c9e4611e3a4', 'options': 'recursive,force'} 2024-09-10T23:16:32.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:32 smithi153 ceph-mon[34951]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:33.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:32 smithi176 ceph-mon[34982]: scrub summary: idle+waiting paths [/] 2024-09-10T23:16:33.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 276446 21.41 MB/sec execute 568 sec latency 88.093 ms 2024-09-10T23:16:33.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:33 smithi120 ceph-mon[31247]: pgmap v1070: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T23:16:33.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:33 smithi153 ceph-mon[34951]: pgmap v1070: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T23:16:34.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:33 smithi176 ceph-mon[34982]: pgmap v1070: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T23:16:34.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 276820 21.42 MB/sec execute 569 sec latency 156.464 ms 2024-09-10T23:16:35.692 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:35 smithi120 ceph-mon[31247]: pgmap v1071: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-10T23:16:35.692 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:35 smithi120 ceph-mon[31247]: scrub summary: idle 2024-09-10T23:16:35.874 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:35 smithi176 ceph-mon[34982]: pgmap v1071: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-10T23:16:35.875 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:35 smithi176 ceph-mon[34982]: scrub summary: idle 2024-09-10T23:16:35.895 INFO:tasks.workunit.client.0.smithi120.stdout: 10 277130 21.42 MB/sec execute 570 sec latency 82.624 ms 2024-09-10T23:16:35.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:35 smithi153 ceph-mon[34951]: pgmap v1071: 97 pgs: 97 active+clean; 230 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-10T23:16:35.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:35 smithi153 ceph-mon[34951]: scrub summary: idle 2024-09-10T23:16:36.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 277481 21.41 MB/sec execute 571 sec latency 68.253 ms 2024-09-10T23:16:37.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 277841 21.39 MB/sec execute 572 sec latency 85.304 ms 2024-09-10T23:16:37.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:37 smithi120 ceph-mon[31247]: pgmap v1072: 97 pgs: 97 active+clean; 232 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-09-10T23:16:37.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:37 smithi153 ceph-mon[34951]: pgmap v1072: 97 pgs: 97 active+clean; 232 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-09-10T23:16:38.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:37 smithi176 ceph-mon[34982]: pgmap v1072: 97 pgs: 97 active+clean; 232 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2024-09-10T23:16:38.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 278387 21.41 MB/sec execute 573 sec latency 114.862 ms 2024-09-10T23:16:39.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 278751 21.41 MB/sec execute 574 sec latency 312.142 ms 2024-09-10T23:16:39.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:39 smithi120 ceph-mon[31247]: pgmap v1073: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 618 op/s 2024-09-10T23:16:39.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:39 smithi153 ceph-mon[34951]: pgmap v1073: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 618 op/s 2024-09-10T23:16:40.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:39 smithi176 ceph-mon[34982]: pgmap v1073: 97 pgs: 97 active+clean; 235 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 618 op/s 2024-09-10T23:16:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- 172.21.15.120:0/466982103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 msgr2=0x7fdb50109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- 172.21.15.120:0/466982103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50109cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009ca0 tx=0x7fdb4802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- 172.21.15.120:0/466982103 shutdown_connections 2024-09-10T23:16:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- 172.21.15.120:0/466982103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50109cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- 172.21.15.120:0/466982103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb50102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- 172.21.15.120:0/466982103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 0x7fdb50101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- 172.21.15.120:0/466982103 >> 172.21.15.120:0/466982103 conn(0x7fdb500fd140 msgr2=0x7fdb500ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- 172.21.15.120:0/466982103 shutdown_connections 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- 172.21.15.120:0/466982103 wait complete. 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 Processor -- start 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- start start 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 0x7fdb50197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d4d0 con 0x7fdb501029b0 2024-09-10T23:16:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.851+0000 7fdb553bc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d350 con 0x7fdb50103370 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb553bc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d650 con 0x7fdb50101860 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45098/0 (socket says 172.21.15.120:45098) 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 -- 172.21.15.120:0/2591721562 learned_addr learned my addr 172.21.15.120:0/2591721562 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb4effd640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 0x7fdb50197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 -- 172.21.15.120:0/2591721562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 msgr2=0x7fdb50197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 0x7fdb50197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 -- 172.21.15.120:0/2591721562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 msgr2=0x7fdb50198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 -- 172.21.15.120:0/2591721562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50199790 con 0x7fdb501029b0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb47fff640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800eb00 tx=0x7fdb38009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb4cff9640 1 -- 172.21.15.120:0/2591721562 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3801c070 con 0x7fdb501029b0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb50199a80 con 0x7fdb501029b0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb4cff9640 1 -- 172.21.15.120:0/2591721562 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3800bc30 con 0x7fdb501029b0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb4cff9640 1 -- 172.21.15.120:0/2591721562 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38016610 con 0x7fdb501029b0 2024-09-10T23:16:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.852+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5006aa10 con 0x7fdb501029b0 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 Processor -- start 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 -- start start 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e52c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e55580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556bb9dc26f0 con 0x556bb9e53190 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556bb9db1b20 con 0x556bb9e50860 2024-09-10T23:16:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb58264740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556bb9db5930 con 0x556bb9e55bd0 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4cff9640 1 -- 172.21.15.120:0/2591721562 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdb38017410 con 0x7fdb501029b0 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e55580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e52c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e55580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45106/0 (socket says 172.21.15.120:45106) 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4effd640 1 -- 172.21.15.120:0/2307842180 learned_addr learned my addr 172.21.15.120:0/2307842180 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb47fff640 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4f7fe640 1 -- 172.21.15.120:0/2307842180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 msgr2=0x556bb9e58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4f7fe640 1 -- 172.21.15.120:0/2307842180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 msgr2=0x556bb9e55580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.855+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e55580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb4f7fe640 1 -- 172.21.15.120:0/2307842180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb9d77530 con 0x556bb9e50860 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e52c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005e30 tx=0x7fdb48005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b228230e09b0f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb457fa640 1 -- 172.21.15.120:0/2307842180 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48031de0 con 0x556bb9e50860 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb457fa640 1 -- 172.21.15.120:0/2307842180 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48036030 con 0x556bb9e50860 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb457fa640 1 -- 172.21.15.120:0/2307842180 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480043c0 con 0x556bb9e50860 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.856+0000 7fdb4cff9640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 0x7fdb3c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb4effd640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 0x7fdb3c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb4cff9640 1 -- 172.21.15.120:0/2591721562 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdb380e7490 con 0x7fdb501029b0 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb4effd640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 0x7fdb3c0b77a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ee0 tx=0x7fdb40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 -- 172.21.15.120:0/2307842180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 msgr2=0x556bb9e52c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e52c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005e30 tx=0x7fdb48005ef0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 -- 172.21.15.120:0/2307842180 shutdown_connections 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e55580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 --2- 172.21.15.120:0/2307842180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e52c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 -- 172.21.15.120:0/2307842180 >> 172.21.15.120:0/2307842180 conn(0x556bb9e4b9b0 msgr2=0x556bb9e4c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 -- 172.21.15.120:0/2307842180 shutdown_connections 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.857+0000 7fdb58264740 1 -- 172.21.15.120:0/2307842180 wait complete. 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 -- start start 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 Processor -- start 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e50860 0x556bb9e59cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e5af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556bb9ce9ab0 con 0x556bb9e53190 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556bb9d57730 con 0x556bb9e55bd0 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556bb9e589d0 con 0x556bb9e50860 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb58264740 10 client.0 _ll_register_callbacks cb 0x556bb9dab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e50860 0x556bb9e59cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45122/0 (socket says 172.21.15.120:45122) 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 -- 172.21.15.120:0/1417571212 learned_addr learned my addr 172.21.15.120:0/1417571212 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e5af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 -- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e50860 msgr2=0x556bb9e59cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e50860 0x556bb9e59cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 -- 172.21.15.120:0/1417571212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 msgr2=0x556bb9e5af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e5af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb47fff640 1 -- 172.21.15.120:0/1417571212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb9b13790 con 0x556bb9e53190 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.858+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e5af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ada30 tx=0x7fdb380aa380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e75269c80b7168 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380ad680 con 0x556bb9e53190 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdb380b6d80 con 0x556bb9e53190 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb48036030 con 0x556bb9e53190 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb380ad4a0 con 0x556bb9e53190 2024-09-10T23:16:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 10 client.20118 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556bb9b8c620 con 0x556bb9e53190 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48036030 con 0x556bb9e53190 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdb380acf90 con 0x556bb9e53190 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 10 client.20118 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 10 client.20118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 20 client.20118 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9e65640 0x556bb9e67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 4 client.20118 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bb9dc4640 con 0x556bb9e65640 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.859+0000 7fdb4effd640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9e65640 0x556bb9e67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.860+0000 7fdb367fc640 10 client.20118 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.860+0000 7fdb4effd640 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9e65640 0x556bb9e67a30 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d3461e15f4edeb server_cookie=8d34d9695c8a3162 in_seq=0 out_seq=0 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb367fc640 1 -- 172.21.15.120:0/1417571212 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bb9dc4640 con 0x556bb9e65640 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb367fc640 10 client.20118 handle_command_reply: tid=0 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 client.20118 shutdown 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 msgr2=0x556bb9e5a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ada30 tx=0x7fdb380aa380 comp rx=0 tx=0).stop 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 shutdown_connections 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9e65640 0x556bb9e67a30 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e5af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb367fc640 0 client.20118 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e5a390 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 --2- 172.21.15.120:0/1417571212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e50860 0x556bb9e59cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 >> 172.21.15.120:0/1417571212 conn(0x556bb9e4b9b0 msgr2=0x556bb9e4abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 shutdown_connections 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 1 -- 172.21.15.120:0/1417571212 wait complete. 2024-09-10T23:16:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.864+0000 7fdb58264740 20 client.20118 trim_cache size 0 max 16384 2024-09-10T23:16:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 Processor -- start 2024-09-10T23:16:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 -- start start 2024-09-10T23:16:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e58f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a5b60 con 0x556bb9e53190 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a83e0 con 0x556bb9e50860 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb58264740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a61d0 con 0x556bb9e55bd0 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45132/0 (socket says 172.21.15.120:45132) 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e58f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 -- 172.21.15.120:0/1335860311 learned_addr learned my addr 172.21.15.120:0/1335860311 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 -- 172.21.15.120:0/1335860311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 msgr2=0x556bb9e4d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e4d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 -- 172.21.15.120:0/1335860311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 msgr2=0x556bb9e58f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e58f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb47fff640 1 -- 172.21.15.120:0/1335860311 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb9e5b2d0 con 0x556bb9e53190 2024-09-10T23:16:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.880+0000 7fdb4effd640 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb47fff640 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800b750 tx=0x7fdb3801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baafab795b898a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb44ff9640 1 -- 172.21.15.120:0/1335860311 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380a9bd0 con 0x556bb9e53190 2024-09-10T23:16:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb44ff9640 1 -- 172.21.15.120:0/1335860311 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb380ad840 con 0x556bb9e53190 2024-09-10T23:16:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb44ff9640 1 -- 172.21.15.120:0/1335860311 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380e72d0 con 0x556bb9e53190 2024-09-10T23:16:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb58264740 1 -- 172.21.15.120:0/1335860311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 msgr2=0x556bb9e59490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.881+0000 7fdb58264740 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800b750 tx=0x7fdb3801f000 comp rx=0 tx=0).stop 2024-09-10T23:16:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- 172.21.15.120:0/1335860311 shutdown_connections 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e55bd0 0x556bb9e4d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e53190 0x556bb9e59490 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- 172.21.15.120:0/1335860311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e50860 0x556bb9e58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- 172.21.15.120:0/1335860311 >> 172.21.15.120:0/1335860311 conn(0x556bb9e4b9b0 msgr2=0x556bb9e55fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- 172.21.15.120:0/1335860311 shutdown_connections 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- 172.21.15.120:0/1335860311 wait complete. 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- start start 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 Processor -- start 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e53190 0x556bb9e60e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e61370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a68a0 con 0x556bb9e50860 2024-09-10T23:16:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a7d10 con 0x556bb9e55bd0 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a6f70 con 0x556bb9e53190 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb58264740 10 client.0 _ll_register_callbacks cb 0x7fdb40009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb4effd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:45148/0 (socket says 172.21.15.120:45148) 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb4effd640 1 -- 172.21.15.120:0/4128936068 learned_addr learned my addr 172.21.15.120:0/4128936068 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb47fff640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e53190 0x556bb9e60e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.882+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e61370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 -- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e53190 msgr2=0x556bb9e60e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e53190 0x556bb9e60e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 -- 172.21.15.120:0/4128936068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 msgr2=0x556bb9e61370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e61370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 -- 172.21.15.120:0/4128936068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb9e5f570 con 0x556bb9e50860 2024-09-10T23:16:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4f7fe640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e61370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb4effd640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009710 tx=0x7fdb40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a7bcb7041094e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb367fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40005020 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdb400051c0 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb380ad840 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb400053c0 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb58264740 10 client.20124 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556bb9ed2a80 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.883+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb380ad840 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fdb4002b790 con 0x556bb9e50860 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 10 client.20124 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 10 client.20124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 20 client.20124 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9ed33c0 0x556bb9ed57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 4 client.20124 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bb9dc00f0 con 0x556bb9ed33c0 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb47fff640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9ed33c0 0x556bb9ed57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb367fc640 10 client.20124 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:40.884+0000 7fdb47fff640 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9ed33c0 0x556bb9ed57e0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91845dbacc734c5 server_cookie=2b7eccb37567b64d in_seq=0 out_seq=0 2024-09-10T23:16:40.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 279386 21.41 MB/sec execute 575 sec latency 102.130 ms 2024-09-10T23:16:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.650+0000 7fdb367fc640 1 -- 172.21.15.120:0/4128936068 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bb9dc00f0 con 0x556bb9ed33c0 2024-09-10T23:16:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.650+0000 7fdb367fc640 10 client.20124 handle_command_reply: tid=0 2024-09-10T23:16:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.650+0000 7fdb58264740 1 client.20124 shutdown 2024-09-10T23:16:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 msgr2=0x556bb9e608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009710 tx=0x7fdb40017000 comp rx=0 tx=0).stop 2024-09-10T23:16:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 shutdown_connections 2024-09-10T23:16:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x556bb9ed33c0 0x556bb9ed57e0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556bb9e55bd0 0x556bb9e61370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb367fc640 0 client.20124 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556bb9e53190 0x556bb9e60e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 --2- 172.21.15.120:0/4128936068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556bb9e50860 0x556bb9e608f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 >> 172.21.15.120:0/4128936068 conn(0x556bb9e4b9b0 msgr2=0x556bb9e4f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 shutdown_connections 2024-09-10T23:16:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 1 -- 172.21.15.120:0/4128936068 wait complete. 2024-09-10T23:16:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb58264740 20 client.20124 trim_cache size 0 max 16384 2024-09-10T23:16:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:16:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:16:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:16:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:16:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 msgr2=0x7fdb3c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 0x7fdb3c0b77a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ee0 tx=0x7fdb40005f90 comp rx=0 tx=0).stop 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 msgr2=0x7fdb501982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.651+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800eb00 tx=0x7fdb38009f20 comp rx=0 tx=0).stop 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 shutdown_connections 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdb3c0b52e0 0x7fdb3c0b77a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb50103370 0x7fdb50198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb501029b0 0x7fdb501982a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 --2- 172.21.15.120:0/2591721562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb50101860 0x7fdb50197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 >> 172.21.15.120:0/2591721562 conn(0x7fdb500fd140 msgr2=0x7fdb50105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 shutdown_connections 2024-09-10T23:16:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:41.652+0000 7fdb553bc640 1 -- 172.21.15.120:0/2591721562 wait complete. 2024-09-10T23:16:41.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 279528 21.39 MB/sec execute 576 sec latency 420.532 ms 2024-09-10T23:16:41.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:41 smithi120 ceph-mon[31247]: pgmap v1074: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 654 op/s 2024-09-10T23:16:41.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:41 smithi153 ceph-mon[34951]: pgmap v1074: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 654 op/s 2024-09-10T23:16:42.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:41 smithi176 ceph-mon[34982]: pgmap v1074: 97 pgs: 97 active+clean; 241 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.0 MiB/s wr, 654 op/s 2024-09-10T23:16:42.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 280094 21.41 MB/sec execute 577 sec latency 76.318 ms 2024-09-10T23:16:43.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 280548 21.44 MB/sec execute 578 sec latency 105.113 ms 2024-09-10T23:16:43.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:43 smithi120 ceph-mon[31247]: pgmap v1075: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-09-10T23:16:43.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:43 smithi153 ceph-mon[34951]: pgmap v1075: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-09-10T23:16:44.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:43 smithi176 ceph-mon[34982]: pgmap v1075: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-09-10T23:16:44.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 280776 21.42 MB/sec execute 579 sec latency 269.642 ms 2024-09-10T23:16:45.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 281105 21.41 MB/sec execute 580 sec latency 99.488 ms 2024-09-10T23:16:45.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:45 smithi120 ceph-mon[31247]: pgmap v1076: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-10T23:16:45.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:45 smithi153 ceph-mon[34951]: pgmap v1076: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-10T23:16:46.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:45 smithi176 ceph-mon[34982]: pgmap v1076: 97 pgs: 97 active+clean; 241 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-10T23:16:46.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 281436 21.39 MB/sec execute 581 sec latency 78.054 ms 2024-09-10T23:16:47.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 281908 21.39 MB/sec execute 582 sec latency 95.808 ms 2024-09-10T23:16:47.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:47 smithi120 ceph-mon[31247]: pgmap v1077: 97 pgs: 97 active+clean; 225 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-10T23:16:47.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:47 smithi153 ceph-mon[34951]: pgmap v1077: 97 pgs: 97 active+clean; 225 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-10T23:16:48.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:47 smithi176 ceph-mon[34982]: pgmap v1077: 97 pgs: 97 active+clean; 225 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-10T23:16:48.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 282507 21.42 MB/sec execute 583 sec latency 81.598 ms 2024-09-10T23:16:49.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 282978 21.41 MB/sec execute 584 sec latency 130.897 ms 2024-09-10T23:16:49.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:49 smithi120 ceph-mon[31247]: pgmap v1078: 97 pgs: 97 active+clean; 230 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 557 op/s 2024-09-10T23:16:49.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:49 smithi153 ceph-mon[34951]: pgmap v1078: 97 pgs: 97 active+clean; 230 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 557 op/s 2024-09-10T23:16:50.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:49 smithi176 ceph-mon[34982]: pgmap v1078: 97 pgs: 97 active+clean; 230 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.6 MiB/s wr, 557 op/s 2024-09-10T23:16:50.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 283420 21.43 MB/sec execute 585 sec latency 79.124 ms 2024-09-10T23:16:51.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 283916 21.45 MB/sec execute 586 sec latency 85.883 ms 2024-09-10T23:16:51.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:51 smithi120 ceph-mon[31247]: pgmap v1079: 97 pgs: 97 active+clean; 238 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-09-10T23:16:51.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:51 smithi153 ceph-mon[34951]: pgmap v1079: 97 pgs: 97 active+clean; 238 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-09-10T23:16:52.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:51 smithi176 ceph-mon[34982]: pgmap v1079: 97 pgs: 97 active+clean; 238 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 583 op/s 2024-09-10T23:16:52.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 284360 21.46 MB/sec execute 587 sec latency 74.264 ms 2024-09-10T23:16:53.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 284766 21.45 MB/sec execute 588 sec latency 46.256 ms 2024-09-10T23:16:53.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:53 smithi120 ceph-mon[31247]: pgmap v1080: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 567 op/s 2024-09-10T23:16:53.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:53 smithi153 ceph-mon[34951]: pgmap v1080: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 567 op/s 2024-09-10T23:16:54.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:53 smithi176 ceph-mon[34982]: pgmap v1080: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.5 MiB/s wr, 567 op/s 2024-09-10T23:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/1962373521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48101870 msgr2=0x7f5a48101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/1962373521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48101870 0x7f5a48101c70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009900 tx=0x7f5a3802a410 comp rx=0 tx=0).stop 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/1962373521 shutdown_connections 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/1962373521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48103380 0x7f5a48109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/1962373521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a48102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/1962373521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48101870 0x7f5a48101c70 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/1962373521 >> 172.21.15.120:0/1962373521 conn(0x7f5a480fd130 msgr2=0x7f5a480ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/1962373521 shutdown_connections 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/1962373521 wait complete. 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 Processor -- start 2024-09-10T23:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.739+0000 7f5a4d9f3640 1 -- start start 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48101870 0x7f5a48198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4810e620 con 0x7f5a48103380 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4810e4a0 con 0x7f5a48101870 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4810e7a0 con 0x7f5a481029c0 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58328/0 (socket says 172.21.15.120:58328) 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57830/0 (socket says 172.21.15.120:57830) 2024-09-10T23:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 -- 172.21.15.120:0/2819580638 learned_addr learned my addr 172.21.15.120:0/2819580638 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48101870 0x7f5a48198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 -- 172.21.15.120:0/2819580638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 msgr2=0x7f5a481993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 -- 172.21.15.120:0/2819580638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48101870 msgr2=0x7f5a48198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48101870 0x7f5a48198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 -- 172.21.15.120:0/2819580638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4819a8c0 con 0x7f5a48103380 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a477fe640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000cad0 tx=0x7f5a4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4c9f1640 1 -- 172.21.15.120:0/2819580638 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000ae20 con 0x7f5a48103380 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4819abb0 con 0x7f5a48103380 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.740+0000 7f5a4c9f1640 1 -- 172.21.15.120:0/2819580638 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a40005020 con 0x7f5a48103380 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.741+0000 7f5a4c9f1640 1 -- 172.21.15.120:0/2819580638 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000f6b0 con 0x7f5a48103380 2024-09-10T23:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.741+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481a1280 con 0x7f5a48103380 2024-09-10T23:16:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.743+0000 7f5a507c7740 1 Processor -- start 2024-09-10T23:16:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a4c9f1640 1 -- 172.21.15.120:0/2819580638 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5a400054f0 con 0x7f5a48103380 2024-09-10T23:16:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 -- start start 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a7a2176f0 con 0x558a7a2aefe0 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a7a206b20 con 0x558a7a2a9c70 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a507c7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a7a20a930 con 0x558a7a2ac5a0 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a46ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58338/0 (socket says 172.21.15.120:58338) 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 -- 172.21.15.120:0/4031811907 learned_addr learned my addr 172.21.15.120:0/4031811907 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a4c9f1640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 0x7f5a340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 -- 172.21.15.120:0/4031811907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 msgr2=0x558a7a2ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 -- 172.21.15.120:0/4031811907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 msgr2=0x558a7a2ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.744+0000 7f5a3ffff640 1 -- 172.21.15.120:0/4031811907 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7a1cc530 con 0x558a7a2aefe0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a477fe640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 0x7f5a340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5a30011c40 tx=0x7f5a30009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa097d940224977 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a4c9f1640 1 -- 172.21.15.120:0/2819580638 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5a400e5b40 con 0x7f5a48103380 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a3dffb640 1 -- 172.21.15.120:0/4031811907 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000fc50 con 0x558a7a2aefe0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a3dffb640 1 -- 172.21.15.120:0/4031811907 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a3000fdf0 con 0x558a7a2aefe0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.745+0000 7f5a3dffb640 1 -- 172.21.15.120:0/4031811907 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000e610 con 0x558a7a2aefe0 2024-09-10T23:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- 172.21.15.120:0/4031811907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 msgr2=0x558a7a2b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a477fe640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 0x7f5a340b77a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000caa0 tx=0x7f5a40070b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5a30011c40 tx=0x7f5a30009bc0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- 172.21.15.120:0/4031811907 shutdown_connections 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b14a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 --2- 172.21.15.120:0/4031811907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- 172.21.15.120:0/4031811907 >> 172.21.15.120:0/4031811907 conn(0x558a7a2a09b0 msgr2=0x558a7a2a1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- 172.21.15.120:0/4031811907 shutdown_connections 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- 172.21.15.120:0/4031811907 wait complete. 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.746+0000 7f5a507c7740 1 -- start start 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 Processor -- start 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a7a13eab0 con 0x558a7a2aefe0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x558a7a1ac730 con 0x558a7a2a9c70 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558a7a2b1fc0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a507c7740 10 client.0 _ll_register_callbacks cb 0x558a7a2006b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:57860/0 (socket says 172.21.15.120:57860) 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58340/0 (socket says 172.21.15.120:58340) 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 -- 172.21.15.120:0/1994628319 learned_addr learned my addr 172.21.15.120:0/1994628319 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 -- 172.21.15.120:0/1994628319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 msgr2=0x558a7a2b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 -- 172.21.15.120:0/1994628319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 msgr2=0x558a7a2b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a46ffd640 1 -- 172.21.15.120:0/1994628319 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a79f68790 con 0x558a7a2ac5a0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.747+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009230 tx=0x7f5a38004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd963bc0ee047c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38035040 con 0x558a7a2ac5a0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3000fdf0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5a38003ac0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a38003ce0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a507c7740 10 client.29563 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.748+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a79fe1620 con 0x558a7a2ac5a0 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a3000fdf0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5a3803c5e0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 10 client.29563 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 10 client.29563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 20 client.29563 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a2c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 4 client.29563 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7a219640 con 0x558a7a2becd0 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.749+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a2c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.750+0000 7f5a26ffd640 10 client.29563 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.750+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a2c1110 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ccd8197fcdd8a9 server_cookie=75e9ce893a47d171 in_seq=0 out_seq=0 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a26ffd640 1 -- 172.21.15.120:0/1994628319 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a7a219640 con 0x558a7a2becd0 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a26ffd640 10 client.29563 handle_command_reply: tid=0 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 client.29563 shutdown 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 msgr2=0x558a7a2b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009230 tx=0x7f5a38004440 comp rx=0 tx=0).stop 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 shutdown_connections 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a2c1110 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3a00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a26ffd640 0 client.29563 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 --2- 172.21.15.120:0/1994628319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2a9c70 0x558a7a2b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.755+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 >> 172.21.15.120:0/1994628319 conn(0x558a7a2a09b0 msgr2=0x558a7a29fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.756+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 shutdown_connections 2024-09-10T23:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.756+0000 7f5a507c7740 1 -- 172.21.15.120:0/1994628319 wait complete. 2024-09-10T23:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.756+0000 7f5a507c7740 20 client.29563 trim_cache size 0 max 16384 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 Processor -- start 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 -- start start 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2a9c70 0x558a7a2c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a5c50 con 0x558a7a2aefe0 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a84d0 con 0x558a7a2ac5a0 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a507c7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a62c0 con 0x558a7a2a9c70 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58354/0 (socket says 172.21.15.120:58354) 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 -- 172.21.15.120:0/617303725 learned_addr learned my addr 172.21.15.120:0/617303725 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2a9c70 0x558a7a2c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 -- 172.21.15.120:0/617303725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2a9c70 msgr2=0x558a7a2c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2a9c70 0x558a7a2c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 -- 172.21.15.120:0/617303725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2ac5a0 msgr2=0x558a7a2b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 -- 172.21.15.120:0/617303725 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7a2ac3d0 con 0x558a7a2aefe0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009b90 tx=0x7f5a30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bf92a5a76dca85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.771+0000 7f5a3d7fa640 1 -- 172.21.15.120:0/617303725 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300055d0 con 0x558a7a2aefe0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a3d7fa640 1 -- 172.21.15.120:0/617303725 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30004af0 con 0x558a7a2aefe0 2024-09-10T23:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a3d7fa640 1 -- 172.21.15.120:0/617303725 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30031ad0 con 0x558a7a2aefe0 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- 172.21.15.120:0/617303725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 msgr2=0x558a7a2b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009b90 tx=0x7f5a30005f90 comp rx=0 tx=0).stop 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- 172.21.15.120:0/617303725 shutdown_connections 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2aefe0 0x558a7a2b38a0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 --2- 172.21.15.120:0/617303725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2a9c70 0x558a7a2c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- 172.21.15.120:0/617303725 >> 172.21.15.120:0/617303725 conn(0x558a7a2a09b0 msgr2=0x558a7a2a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- 172.21.15.120:0/617303725 shutdown_connections 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- 172.21.15.120:0/617303725 wait complete. 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.772+0000 7f5a507c7740 1 -- start start 2024-09-10T23:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 Processor -- start 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2aefe0 0x558a7a2b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a6990 con 0x558a7a2a9c70 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a7e00 con 0x558a7a2aefe0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481a7060 con 0x558a7a2ac5a0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a507c7740 10 client.0 _ll_register_callbacks cb 0x7f5a38009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:58366/0 (socket says 172.21.15.120:58366) 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 -- 172.21.15.120:0/3145560973 learned_addr learned my addr 172.21.15.120:0/3145560973 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2aefe0 0x558a7a2b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 -- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 msgr2=0x558a7a2b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 -- 172.21.15.120:0/3145560973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2aefe0 msgr2=0x558a7a2b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2aefe0 0x558a7a2b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.773+0000 7f5a3ffff640 1 -- 172.21.15.120:0/3145560973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7a2b9a30 con 0x558a7a2a9c70 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a3ffff640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009c40 tx=0x7f5a30009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369aa6126bb03428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30026040 con 0x558a7a2a9c70 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f5a30004ca0 con 0x558a7a2a9c70 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a30004af0 con 0x558a7a2a9c70 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a30004ea0 con 0x558a7a2a9c70 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 10 client.20142 fetch_fsmap learned FSMap version 26 2024-09-10T23:16:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a7a2badd0 con 0x558a7a2a9c70 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a30004af0 con 0x558a7a2a9c70 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f5a3001d020 con 0x558a7a2a9c70 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 10 client.20142 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 10 client.20142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 20 client.20142 populate_metadata read hostname 'smithi120' 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a32dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 4 client.20142 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a7a2150f0 con 0x558a7a2becd0 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.774+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a32dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.775+0000 7f5a46ffd640 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a32dc40 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2afae696d0ad96a server_cookie=c922b30e65ab8c7f in_seq=0 out_seq=0 2024-09-10T23:16:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:54.775+0000 7f5a26ffd640 10 client.20142 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:54.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 284932 21.42 MB/sec execute 589 sec latency 273.091 ms 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.692+0000 7f5a26ffd640 1 -- 172.21.15.120:0/3145560973 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a7a2150f0 con 0x558a7a2becd0 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.692+0000 7f5a26ffd640 10 client.20142 handle_command_reply: tid=0 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.692+0000 7f5a507c7740 1 client.20142 shutdown 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 msgr2=0x558a7a2b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009c40 tx=0x7f5a30009ed0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 shutdown_connections 2024-09-10T23:16:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x558a7a2becd0 0x558a7a32dc40 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558a7a2aefe0 0x558a7a2b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a26ffd640 0 client.20142 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558a7a2ac5a0 0x558a7a2b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 --2- 172.21.15.120:0/3145560973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a7a2a9c70 0x558a7a2b7f90 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 >> 172.21.15.120:0/3145560973 conn(0x558a7a2a09b0 msgr2=0x558a7a2b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 shutdown_connections 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.693+0000 7f5a507c7740 1 -- 172.21.15.120:0/3145560973 wait complete. 2024-09-10T23:16:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.694+0000 7f5a507c7740 20 client.20142 trim_cache size 0 max 16384 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:{ 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "message": "", 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout: "return_code": 0 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stdout:} 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.694+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 msgr2=0x7f5a340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.694+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 0x7f5a340b77a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000caa0 tx=0x7f5a40070b50 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.694+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 msgr2=0x7f5a4819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.694+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000cad0 tx=0x7f5a4000cf90 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 shutdown_connections 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f5a340b52e0 0x7f5a340b77a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a48103380 0x7f5a4819a000 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a481029c0 0x7f5a481993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 --2- 172.21.15.120:0/2819580638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a48101870 0x7f5a48198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 >> 172.21.15.120:0/2819580638 conn(0x7f5a480fd130 msgr2=0x7f5a48105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 shutdown_connections 2024-09-10T23:16:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi120.front.sepia.ceph.com.smithi120.stderr:2024-09-10T23:16:55.695+0000 7f5a4d9f3640 1 -- 172.21.15.120:0/2819580638 wait complete. 2024-09-10T23:16:55.896 INFO:tasks.workunit.client.0.smithi120.stdout: 10 285027 21.39 MB/sec execute 590 sec latency 583.792 ms 2024-09-10T23:16:55.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:55 smithi120 ceph-mon[31247]: pgmap v1081: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 408 op/s 2024-09-10T23:16:55.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:55 smithi153 ceph-mon[34951]: pgmap v1081: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 408 op/s 2024-09-10T23:16:56.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:55 smithi176 ceph-mon[34982]: pgmap v1081: 97 pgs: 97 active+clean; 240 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 408 op/s 2024-09-10T23:16:56.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 285355 21.38 MB/sec execute 591 sec latency 112.441 ms 2024-09-10T23:16:57.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 285948 21.41 MB/sec execute 592 sec latency 50.733 ms 2024-09-10T23:16:57.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:57 smithi120 ceph-mon[31247]: pgmap v1082: 97 pgs: 97 active+clean; 256 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-09-10T23:16:57.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:57 smithi153 ceph-mon[34951]: pgmap v1082: 97 pgs: 97 active+clean; 256 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-09-10T23:16:58.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:57 smithi176 ceph-mon[34982]: pgmap v1082: 97 pgs: 97 active+clean; 256 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-09-10T23:16:58.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 286539 21.41 MB/sec execute 593 sec latency 89.600 ms 2024-09-10T23:16:58.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:58 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:58.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:58 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:59.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:58 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:16:59.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 286973 21.43 MB/sec execute 594 sec latency 195.745 ms 2024-09-10T23:16:59.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:59 smithi120 ceph-mon[31247]: pgmap v1083: 97 pgs: 97 active+clean; 247 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-09-10T23:16:59.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:16:59.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:16:59.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:16:59 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:16:59.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:59 smithi153 ceph-mon[34951]: pgmap v1083: 97 pgs: 97 active+clean; 247 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-09-10T23:16:59.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:16:59.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:16:59.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:16:59 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:17:00.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:59 smithi176 ceph-mon[34982]: pgmap v1083: 97 pgs: 97 active+clean; 247 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-09-10T23:17:00.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:17:00.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:17:00.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:16:59 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' 2024-09-10T23:17:00.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 287461 21.45 MB/sec execute 595 sec latency 80.660 ms 2024-09-10T23:17:01.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 287828 21.45 MB/sec execute 596 sec latency 64.847 ms 2024-09-10T23:17:01.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:01 smithi120 ceph-mon[31247]: pgmap v1084: 97 pgs: 97 active+clean; 230 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-09-10T23:17:01.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:01 smithi153 ceph-mon[34951]: pgmap v1084: 97 pgs: 97 active+clean; 230 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-09-10T23:17:02.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:01 smithi176 ceph-mon[34982]: pgmap v1084: 97 pgs: 97 active+clean; 230 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2024-09-10T23:17:02.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 288221 21.44 MB/sec execute 597 sec latency 48.563 ms 2024-09-10T23:17:02.935 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:17:03.236 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.816+0000 7fda5c309640 1 -- 172.21.15.120:0/2200162222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda54108060 msgr2=0x7fda5410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.816+0000 7fda5c309640 1 --2- 172.21.15.120:0/2200162222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda54108060 0x7fda5410e860 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009ca0 tx=0x7fda4c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 -- 172.21.15.120:0/2200162222 shutdown_connections 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 --2- 172.21.15.120:0/2200162222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda54108060 0x7fda5410e860 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 --2- 172.21.15.120:0/2200162222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda541076e0 0x7fda54107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 --2- 172.21.15.120:0/2200162222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda541020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 -- 172.21.15.120:0/2200162222 >> 172.21.15.120:0/2200162222 conn(0x7fda540fd740 msgr2=0x7fda540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 -- 172.21.15.120:0/2200162222 shutdown_connections 2024-09-10T23:17:03.818 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.817+0000 7fda5c309640 1 -- 172.21.15.120:0/2200162222 wait complete. 2024-09-10T23:17:03.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 Processor -- start 2024-09-10T23:17:03.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 -- start start 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda54197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 0x7fda5419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda54110c20 con 0x7fda541076e0 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda54110aa0 con 0x7fda54108060 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.818+0000 7fda5c309640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fda54110da0 con 0x7fda54101cd0 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda54197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48614/0 (socket says 172.21.15.120:48614) 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 -- 172.21.15.120:0/170933262 learned_addr learned my addr 172.21.15.120:0/170933262 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 0x7fda5419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 -- 172.21.15.120:0/170933262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 msgr2=0x7fda54197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda54197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 -- 172.21.15.120:0/170933262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 msgr2=0x7fda5419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 0x7fda5419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda598a4640 1 -- 172.21.15.120:0/170933262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5419d0e0 con 0x7fda541076e0 2024-09-10T23:17:03.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda54197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:03.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.819+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 0x7fda5419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.821+0000 7fda598a4640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fda44007bd0 tx=0x7fda4400f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.821+0000 7fda4b7fe640 1 -- 172.21.15.120:0/170933262 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4400fc10 con 0x7fda541076e0 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.822+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5419d3d0 con 0x7fda541076e0 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.822+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda541a3c60 con 0x7fda541076e0 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.822+0000 7fda4b7fe640 1 -- 172.21.15.120:0/170933262 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4400fdb0 con 0x7fda541076e0 2024-09-10T23:17:03.823 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.823+0000 7fda4b7fe640 1 -- 172.21.15.120:0/170933262 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44003a60 con 0x7fda541076e0 2024-09-10T23:17:03.830 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.826+0000 7fda5e6de740 1 Processor -- start 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 -- start start 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x55676014b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567600b5e50 con 0x55676013f9c0 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5567600a5280 con 0x55676013f0a0 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5e6de740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5567600a9090 con 0x55676014b950 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5a8a6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5a8a6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58844/0 (socket says 172.21.15.120:58844) 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5a8a6640 1 -- 172.21.15.120:0/4241591590 learned_addr learned my addr 172.21.15.120:0/4241591590 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda4b7fe640 1 -- 172.21.15.120:0/170933262 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fda44003ce0 con 0x7fda541076e0 2024-09-10T23:17:03.831 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x55676014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.827+0000 7fda598a4640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 -- 172.21.15.120:0/4241591590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 msgr2=0x55676014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 -- 172.21.15.120:0/4241591590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 msgr2=0x55676014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x55676014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 -- 172.21.15.120:0/4241591590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675fe03650 con 0x55676013f0a0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda598a4640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda4c033040 tx=0x7fda4c02f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9130e967ffcaa28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda48de6640 1 -- 172.21.15.120:0/4241591590 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c031450 con 0x55676013f0a0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda48de6640 1 -- 172.21.15.120:0/4241591590 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4c036040 con 0x55676013f0a0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.828+0000 7fda48de6640 1 -- 172.21.15.120:0/4241591590 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0042a0 con 0x55676013f0a0 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda4b7fe640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 0x7fda400b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.832 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 0x7fda400b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda4b7fe640 1 -- 172.21.15.120:0/170933262 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fda440e5540 con 0x7fda541076e0 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 -- 172.21.15.120:0/4241591590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 msgr2=0x55676013f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda4c033040 tx=0x7fda4c02f310 comp rx=0 tx=0).stop 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 -- 172.21.15.120:0/4241591590 shutdown_connections 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x55676014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4241591590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676013f480 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.829+0000 7fda5e6de740 1 -- 172.21.15.120:0/4241591590 >> 172.21.15.120:0/4241591590 conn(0x55676013ced0 msgr2=0x55676013e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.830+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 0x7fda400b76f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55676013ddb0 tx=0x7fda5000f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.830+0000 7fda5e6de740 1 -- 172.21.15.120:0/4241591590 shutdown_connections 2024-09-10T23:17:03.833 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.830+0000 7fda5e6de740 1 -- 172.21.15.120:0/4241591590 wait complete. 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.831+0000 7fda5e6de740 1 -- start start 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.831+0000 7fda5e6de740 1 Processor -- start 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.831+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.831+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x5567601486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x556760148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5e6de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55675ff06700 con 0x55676013f9c0 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.837 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58858/0 (socket says 172.21.15.120:58858) 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 -- 172.21.15.120:0/3538222929 learned_addr learned my addr 172.21.15.120:0/3538222929 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda598a4640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x5567601486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x556760148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x556760040110 con 0x55676013f0a0 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55676014e3f0 con 0x55676014b950 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5e6de740 10 client.0 _ll_register_callbacks cb 0x556760059990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 -- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 msgr2=0x556760148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x556760148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 -- 172.21.15.120:0/3538222929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 msgr2=0x5567601486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.838 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x5567601486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.832+0000 7fda5a0a5640 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55676006dc40 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c036040 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda3dffb640 10 client.29291 ms_handle_connect on v2:172.21.15.153:3300/0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda5001aeb0 tx=0x7fda50004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697197d11b8789d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda3dffb640 1 -- 172.21.15.120:0/3538222929 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50030630 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda3dffb640 1 -- 172.21.15.120:0/3538222929 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fda500307d0 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda3dffb640 1 -- 172.21.15.120:0/3538222929 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda500309f0 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda5e6de740 10 client.29291 fetch_fsmap learned FSMap version 26 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.833+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556760159ce0 con 0x55676013f0a0 2024-09-10T23:17:03.839 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda3dffb640 1 -- 172.21.15.120:0/3538222929 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fda50038020 con 0x55676013f0a0 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 10 client.29291 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 10 client.29291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 20 client.29291 populate_metadata read hostname 'smithi120' 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x55676015d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 4 client.29291 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567600b7da0 con 0x55676015b0e0 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.834+0000 7fda598a4640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x55676015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.835+0000 7fda3dffb640 10 client.29291 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:03.840 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.835+0000 7fda598a4640 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x55676015d510 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1a0758bd49bbb0 server_cookie=1185ef2e1a1c00ec in_seq=0 out_seq=0 2024-09-10T23:17:03.841 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda3dffb640 1 -- 172.21.15.120:0/3538222929 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567600b7da0 con 0x55676015b0e0 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda3dffb640 10 client.29291 handle_command_reply: tid=0 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 client.29291 shutdown 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 msgr2=0x55676014b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda5001aeb0 tx=0x7fda50004730 comp rx=0 tx=0).stop 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 shutdown_connections 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x55676015d510 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x556760148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda3dffb640 0 client.29291 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x5567601486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 --2- 172.21.15.120:0/3538222929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013f0a0 0x55676014b250 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 >> 172.21.15.120:0/3538222929 conn(0x55676013ced0 msgr2=0x556760142a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 shutdown_connections 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 1 -- 172.21.15.120:0/3538222929 wait complete. 2024-09-10T23:17:03.842 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.839+0000 7fda5e6de740 20 client.29291 trim_cache size 0 max 16384 2024-09-10T23:17:03.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 Processor -- start 2024-09-10T23:17:03.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 -- start start 2024-09-10T23:17:03.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760157650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.864 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda541a9d40 con 0x55676013f9c0 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda541ac5c0 con 0x55676013ced0 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.862+0000 7fda5e6de740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fda541aa3b0 con 0x55676014b950 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48644/0 (socket says 172.21.15.120:48644) 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 -- 172.21.15.120:0/4290851029 learned_addr learned my addr 172.21.15.120:0/4290851029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda5a8a6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 -- 172.21.15.120:0/4290851029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 msgr2=0x55676014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 -- 172.21.15.120:0/4290851029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 msgr2=0x556760157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.865 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 -- 172.21.15.120:0/4290851029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556760149020 con 0x55676013f9c0 2024-09-10T23:17:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda598a4640 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fda440a8330 tx=0x7fda440e8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eaec6e40e4d2f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda3ffff640 1 -- 172.21.15.120:0/4290851029 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440ab870 con 0x55676013f9c0 2024-09-10T23:17:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda3ffff640 1 -- 172.21.15.120:0/4290851029 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440ac460 con 0x55676013f9c0 2024-09-10T23:17:03.866 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.863+0000 7fda3ffff640 1 -- 172.21.15.120:0/4290851029 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440e50b0 con 0x55676013f9c0 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.864+0000 7fda5e6de740 1 -- 172.21.15.120:0/4290851029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 msgr2=0x556760147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.864+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fda440a8330 tx=0x7fda440e8a80 comp rx=0 tx=0).stop 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.865+0000 7fda5e6de740 1 -- 172.21.15.120:0/4290851029 shutdown_connections 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.865+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676014b950 0x55676014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.865+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676013f9c0 0x556760147650 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.865+0000 7fda5e6de740 1 --2- 172.21.15.120:0/4290851029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.865+0000 7fda5e6de740 1 -- 172.21.15.120:0/4290851029 >> 172.21.15.120:0/4290851029 conn(0x55676013caf0 msgr2=0x556760146f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- 172.21.15.120:0/4290851029 shutdown_connections 2024-09-10T23:17:03.869 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- 172.21.15.120:0/4290851029 wait complete. 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- start start 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 Processor -- start 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 0x5567601536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676014b950 0x556760153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda541aaa80 con 0x55676014b950 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fda541abef0 con 0x55676013ced0 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.866+0000 7fda5e6de740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fda541ab150 con 0x55676013f9c0 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda5e6de740 10 client.0 _ll_register_callbacks cb 0x7fda5001e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda5a0a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.120:58878/0 (socket says 172.21.15.120:58878) 2024-09-10T23:17:03.870 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda5a0a5640 1 -- 172.21.15.120:0/1985037314 learned_addr learned my addr 172.21.15.120:0/1985037314 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 0x5567601536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda5a8a6640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676014b950 0x556760153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 -- 172.21.15.120:0/1985037314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 msgr2=0x556760145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 -- 172.21.15.120:0/1985037314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676014b950 msgr2=0x556760153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676014b950 0x556760153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 -- 172.21.15.120:0/1985037314 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556760154230 con 0x55676013f9c0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda598a4640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 0x5567601536b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b410 tx=0x7fda440ab950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ca37f80f19ca72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.867+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440b5260 con 0x55676013f9c0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda440ac460 con 0x55676013f9c0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mon.1 v2:172.21.15.176:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fda440e5110 con 0x55676013f9c0 2024-09-10T23:17:03.871 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda440e5310 con 0x55676013f9c0 2024-09-10T23:17:03.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 10 client.29587 fetch_fsmap learned FSMap version 26 2024-09-10T23:17:03.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567601c8ff0 con 0x55676013f9c0 2024-09-10T23:17:03.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mon.1 v2:172.21.15.176:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440ac460 con 0x55676013f9c0 2024-09-10T23:17:03.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mon.1 v2:172.21.15.176:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7fda440a76b0 con 0x55676013f9c0 2024-09-10T23:17:03.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 10 client.29587 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:17:03.873 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 10 client.29587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 20 client.29587 populate_metadata read hostname 'smithi120' 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x5567601cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 4 client.29587 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.868+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567600b3850 con 0x55676015b0e0 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x5567601cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda3dffb640 10 client.29587 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda5a0a5640 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x5567601cb510 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f940242ddf11d11a server_cookie=ceb614ec6d5a7663 in_seq=0 out_seq=0 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda3dffb640 1 -- 172.21.15.120:0/1985037314 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+11450 (crc 0 0 0) 0x5567600b3850 con 0x55676015b0e0 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda3dffb640 10 client.29587 handle_command_reply: tid=0 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda5e6de740 1 client.29587 shutdown 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 msgr2=0x5567601536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.869+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 0x5567601536b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b410 tx=0x7fda440ab950 comp rx=0 tx=0).stop 2024-09-10T23:17:03.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 shutdown_connections 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55676015b0e0 0x5567601cb510 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55676014b950 0x556760153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55676013f9c0 0x5567601536b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 --2- 172.21.15.120:0/1985037314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55676013ced0 0x556760145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda3dffb640 0 client.29587 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 >> 172.21.15.120:0/1985037314 conn(0x55676013caf0 msgr2=0x556760147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 shutdown_connections 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 1 -- 172.21.15.120:0/1985037314 wait complete. 2024-09-10T23:17:03.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5e6de740 20 client.29587 trim_cache size 0 max 16384 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "status": "no active scrubs running", 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "scrubs": {}, 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_stats": { 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "9d234a42-f4c2-4f54-9479-fa834cfb6378": { 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:10.682926+0000", 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.876 INFO:teuthology.orchestra.run.smithi120.stdout: "ab3503c1-c23a-434c-8be0-571ff6479523": { 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:14:30.802244+0000", 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "15d1876e-977a-45d9-8082-947af36d1a52": { 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:12:30.721211+0000", 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.877 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "7b4feb37-814c-4b4e-8d92-89df57f17c70": { 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:50.837534+0000", 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "1e09b8ea-07fb-48f0-a34b-c815ff45daaa": { 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:10.948886+0000", 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.878 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "a1afbd89-a833-4db7-b7a9-e5dfcbb0a8c5": { 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:10:30.062192+0000", 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "513af67f-ac28-4e63-863d-e11a2008f95c": { 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:49.440524+0000", 2024-09-10T23:17:03.879 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "76fd4309-53dc-472d-bac3-a65eb1ac4495": { 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:06.641730+0000", 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.880 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "c6719e3d-696f-4c8f-92f0-e7df480d8da9": { 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:13.423740+0000", 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "ff0a0b63-8f51-43d9-943f-ad59d73316ad": { 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:06:29.824137+0000", 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.881 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "13b316f4-2fb8-4178-b82e-6e02b551a316": { 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:40.674708+0000", 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.882 INFO:teuthology.orchestra.run.smithi120.stdout: "1fb84b3b-20ba-4294-8ac5-e3b5ce0da1c0": { 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:27.364093+0000", 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "da14d37b-643f-43a0-a252-93f0ac5bb9b1": { 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:08:29.544180+0000", 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.883 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "5b13ee9d-06c7-493a-b472-734d0fb7db89": { 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:55:00.470103+0000", 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "1108fa36-9551-4546-98e9-d41c113b5b06": { 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:00:25.754053+0000", 2024-09-10T23:17:03.884 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "20493467-6325-4ca0-9248-5c9e4611e3a4": { 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:16:30.576286+0000", 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "ec85941b-87cc-4645-b788-d806c41ade31": { 2024-09-10T23:17:03.885 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:54:20.846491+0000", 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "fafad612-5cd2-4024-9a30-621cf6373623": { 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:13:50.906564+0000", 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.886 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "7cd29d46-2281-4b94-a993-80e3b8f9b9bd": { 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:52:20.931667+0000", 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "291470de-ce79-4054-bccb-d0d766773893": { 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:40.418539+0000", 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.887 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "1b9690c2-c976-40be-b599-edc68acd7a29": { 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:44.430408+0000", 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "93126434-a545-4e80-8433-4566deba8ef8": { 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:10.312762+0000", 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.888 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "4777f81f-6a19-4258-a6b1-74ce729177eb": { 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:56:21.019313+0000", 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "9ef6bbf7-89a6-4969-ac4a-672295c39b5e": { 2024-09-10T23:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:00.669277+0000", 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "2dbbc85c-b952-4f9d-a994-746a3e80926f": { 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:15:50.606606+0000", 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.890 INFO:teuthology.orchestra.run.smithi120.stdout: "0c391c7a-36ff-4d6f-a7b1-2e76368cdd48": { 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:58:19.724165+0000", 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "363bfa6b-82dd-4066-8117-7d10a3d87602": { 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:53:40.648885+0000", 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.891 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "f966826f-4efc-4867-9127-7696da6e341a": { 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:11:10.257864+0000", 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "2626e072-8186-435c-bd3f-c6171dd25846": { 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:05.018725+0000", 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.892 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "c589f7ad-e5a8-40d9-9f72-c5c57b87d9a0": { 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:01:52.414344+0000", 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "0cdd6cde-fe76-4a81-bb3f-57bd7ef71cc4": { 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:02:32.386023+0000", 2024-09-10T23:17:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "2b3a1bd2-bd10-4048-b05f-00fd520d6700": { 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:59:46.313387+0000", 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "8888eb7b-edf6-49c6-ab37-3c5f565cc85f": { 2024-09-10T23:17:03.894 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:10.301815+0000", 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "bed6772e-b2a1-4157-8568-445845a41147": { 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:11.886705+0000", 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.895 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "756a95b7-b141-4232-b641-f7ca49c6ddcd": { 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:03:51.701385+0000", 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "0b9d307e-5656-41f6-b079-f9e7a8dd8606": { 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:09:09.542853+0000", 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.896 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "840e4494-7018-402f-96de-4461c5098aae": { 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:04:30.813072+0000", 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "010375e9-ff31-4fd9-8442-d91c1ce255e0": { 2024-09-10T23:17:03.897 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:05:50.287448+0000", 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "374bdd63-bd62-4996-bb70-48d17f31fda6": { 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T22:57:00.691925+0000", 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "2e936509-53bd-4750-adb6-e64711626a8e": { 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "start_time": "2024-09-10T23:07:49.942499+0000", 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "path": "/", 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_started": 0, 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_passed": 0, 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_failed": 0, 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: "uninline_skipped": 0 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:03.899 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.870+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 msgr2=0x7fda400b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 0x7fda400b76f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55676013ddb0 tx=0x7fda5000f040 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 msgr2=0x7fda54197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fda44007bd0 tx=0x7fda4400f500 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 shutdown_connections 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fda400b5230 0x7fda400b76f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda54108060 0x7fda5419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda541076e0 0x7fda54197570 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 --2- 172.21.15.120:0/170933262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda54101cd0 0x7fda54197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 >> 172.21.15.120:0/170933262 conn(0x7fda540fd740 msgr2=0x7fda540ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 shutdown_connections 2024-09-10T23:17:03.900 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:03.871+0000 7fda5c309640 1 -- 172.21.15.120:0/170933262 wait complete. 2024-09-10T23:17:03.901 INFO:tasks.workunit.client.0.smithi120.stdout: 10 288538 21.42 MB/sec execute 598 sec latency 101.515 ms 2024-09-10T23:17:03.912 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:03 smithi120 ceph-mon[31247]: pgmap v1085: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-10T23:17:03.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:03 smithi153 ceph-mon[34951]: pgmap v1085: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-10T23:17:04.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:03 smithi176 ceph-mon[34982]: pgmap v1085: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-10T23:17:04.814 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:17:04.814 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:04.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 288893 21.42 MB/sec execute 599 sec latency 266.771 ms 2024-09-10T23:17:05.176 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 -- 172.21.15.120:0/335701615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60010d120 msgr2=0x7fd60010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 --2- 172.21.15.120:0/335701615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60010d120 0x7fd60010f550 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ca0 tx=0x7fd5f802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 -- 172.21.15.120:0/335701615 shutdown_connections 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 --2- 172.21.15.120:0/335701615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd60010d120 0x7fd60010f550 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 --2- 172.21.15.120:0/335701615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd600104570 0x7fd60010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 --2- 172.21.15.120:0/335701615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 0x7fd600103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 -- 172.21.15.120:0/335701615 >> 172.21.15.120:0/335701615 conn(0x7fd6000fd700 msgr2=0x7fd6000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 -- 172.21.15.120:0/335701615 shutdown_connections 2024-09-10T23:17:05.740 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.716+0000 7fd607d7b640 1 -- 172.21.15.120:0/335701615 wait complete. 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 Processor -- start 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 -- start start 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 0x7fd60019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 0x7fd6001a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd600112d60 con 0x7fd600104570 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd600112be0 con 0x7fd60010d120 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd607d7b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd600112ee0 con 0x7fd600103b70 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48670/0 (socket says 172.21.15.120:48670) 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 -- 172.21.15.120:0/826767235 learned_addr learned my addr 172.21.15.120:0/826767235 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd606318640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 0x7fd6001a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:05.741 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605b17640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 0x7fd60019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 -- 172.21.15.120:0/826767235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 msgr2=0x7fd60019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 0x7fd60019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 -- 172.21.15.120:0/826767235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 msgr2=0x7fd6001a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 0x7fd6001a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd605316640 1 -- 172.21.15.120:0/826767235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6001a14e0 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.717+0000 7fd606318640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 0x7fd6001a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd605316640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000cd80 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f00108e0 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001a17d0 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0018980 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.718+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001a7e90 con 0x7fd600104570 2024-09-10T23:17:05.742 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.719+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd600103f50 con 0x7fd600104570 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.721+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd5f0010430 con 0x7fd600104570 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.722+0000 7fd5f6ffd640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 0x7fd5ec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.722+0000 7fd605b17640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 0x7fd5ec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.722+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fd5f00e62f0 con 0x7fd600104570 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.722+0000 7fd605b17640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 0x7fd5ec0b7820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009bd0 tx=0x7fd5fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:05.743 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.725+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd5f00acb90 con 0x7fd600104570 2024-09-10T23:17:05.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 cleanup 600 sec 2024-09-10T23:17:05.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:05 smithi120 ceph-mon[31247]: pgmap v1086: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T23:17:05.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:05 smithi153 ceph-mon[34951]: pgmap v1086: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T23:17:05.953 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.953+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60019c560 con 0x7fd600104570 2024-09-10T23:17:05.954 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.954+0000 7fd5f6ffd640 1 -- 172.21.15.120:0/826767235 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fd60019c560 con 0x7fd600104570 2024-09-10T23:17:05.954 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:05.955 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:17:05.955 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:17:05.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 msgr2=0x7fd5ec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:05.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 0x7fd5ec0b7820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009bd0 tx=0x7fd5fc005f90 comp rx=0 tx=0).stop 2024-09-10T23:17:05.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 msgr2=0x7fd60019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 shutdown_connections 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fd5ec0b5360 0x7fd5ec0b7820 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd60010d120 0x7fd6001a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd600104570 0x7fd60019b970 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 --2- 172.21.15.120:0/826767235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd600103b70 0x7fd60019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 >> 172.21.15.120:0/826767235 conn(0x7fd6000fd700 msgr2=0x7fd60010f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 shutdown_connections 2024-09-10T23:17:05.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:05.956+0000 7fd607d7b640 1 -- 172.21.15.120:0/826767235 wait complete. 2024-09-10T23:17:06.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:05 smithi176 ceph-mon[34982]: pgmap v1086: 97 pgs: 97 active+clean; 231 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T23:17:06.635 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:17:06.890 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:06.897 INFO:tasks.workunit.client.0.smithi120.stdout: 10 cleanup 601 sec 2024-09-10T23:17:06.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:06 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/826767235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:06.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:06 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/826767235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:07.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:06 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/826767235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: 0 cleanup 601 sec 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: Operation Count AvgLat MaxLat 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: ---------------------------------------- 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: NTCreateX 412584 6.052 724.612 2024-09-10T23:17:07.139 INFO:tasks.workunit.client.0.smithi120.stdout: Close 303061 0.002 0.099 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Rename 17493 25.121 692.588 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Unlink 83330 9.232 569.462 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Qpathinfo 374337 4.617 724.373 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Qfileinfo 65284 0.004 182.234 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Qfsinfo 68582 0.165 23.688 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Sfileinfo 33659 0.912 654.927 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Find 144628 0.523 290.726 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: WriteX 203899 0.088 647.399 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: ReadX 647493 0.029 361.541 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: LockX 1346 8.898 176.020 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: UnlockX 1346 7.874 147.068 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: Flush 28931 13.275 720.820 2024-09-10T23:17:07.140 INFO:tasks.workunit.client.0.smithi120.stdout: 2024-09-10T23:17:07.141 INFO:tasks.workunit.client.0.smithi120.stdout:Throughput 21.416 MB/sec 10 clients 10 procs max_latency=724.615 ms 2024-09-10T23:17:07.141 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T23:17:07.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- 172.21.15.120:0/1630720870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 msgr2=0x7f4910107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- 172.21.15.120:0/1630720870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f4910107ae0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f4900009900 tx=0x7f490002a410 comp rx=0 tx=0).stop 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- 172.21.15.120:0/1630720870 shutdown_connections 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- 172.21.15.120:0/1630720870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910108020 0x7f491010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- 172.21.15.120:0/1630720870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f4910107ae0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- 172.21.15.120:0/1630720870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910101c90 0x7f4910102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- 172.21.15.120:0/1630720870 >> 172.21.15.120:0/1630720870 conn(0x7f49100fd700 msgr2=0x7f49100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.441 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- 172.21.15.120:0/1630720870 shutdown_connections 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- 172.21.15.120:0/1630720870 wait complete. 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 Processor -- start 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- start start 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910101c90 0x7f49100fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 0x7f49100fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4910111ce0 con 0x7f49101076a0 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4910111b60 con 0x7f4910108020 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.431+0000 7f49183e4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4910111e60 con 0x7f4910101c90 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48686/0 (socket says 172.21.15.120:48686) 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 -- 172.21.15.120:0/3882737167 learned_addr learned my addr 172.21.15.120:0/3882737167 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:07.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f4916180640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910101c90 0x7f49100fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f4916981640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 0x7f49100fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 -- 172.21.15.120:0/3882737167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910101c90 msgr2=0x7f49100fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910101c90 0x7f49100fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 -- 172.21.15.120:0/3882737167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 msgr2=0x7f49100fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 0x7f49100fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 -- 172.21.15.120:0/3882737167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101a76a0 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f4916981640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 0x7f49100fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f491597f640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4900005940 tx=0x7f4900004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f49077fe640 1 -- 172.21.15.120:0/3882737167 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490003f030 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101a7930 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f49077fe640 1 -- 172.21.15.120:0/3882737167 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900003aa0 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f49077fe640 1 -- 172.21.15.120:0/3882737167 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900035070 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.432+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a7e60 con 0x7f49101076a0 2024-09-10T23:17:07.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f49077fe640 1 -- 172.21.15.120:0/3882737167 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f490003bcc0 con 0x7f49101076a0 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 Processor -- start 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 -- start start 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfc54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfd3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55efabf3bea0 con 0x55efabfc5a10 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efabf2b2d0 con 0x55efabfc50f0 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491a7b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55efabf2f0e0 con 0x55efabfd19a0 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f4916180640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f491597f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfd3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f4916180640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48688/0 (socket says 172.21.15.120:48688) 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f4916180640 1 -- 172.21.15.120:0/3514890506 learned_addr learned my addr 172.21.15.120:0/3514890506 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:07.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.435+0000 7f4916981640 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfc54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 -- 172.21.15.120:0/3514890506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 msgr2=0x55efabfd3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfd3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 -- 172.21.15.120:0/3514890506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 msgr2=0x55efabfc54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfc54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 -- 172.21.15.120:0/3514890506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efabc89650 con 0x55efabfc5a10 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f49077fe640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 0x7f48fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916180640 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f490c00bd10 tx=0x7f490c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f717285281b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4916981640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 0x7f48fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f49077fe640 1 -- 172.21.15.120:0/3882737167 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f49001065a0 con 0x7f49101076a0 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4904de6640 1 -- 172.21.15.120:0/3514890506 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c004030 con 0x55efabfc5a10 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4904de6640 1 -- 172.21.15.120:0/3514890506 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490c0035f0 con 0x55efabfc5a10 2024-09-10T23:17:07.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.436+0000 7f4904de6640 1 -- 172.21.15.120:0/3514890506 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0117b0 con 0x55efabfc5a10 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- 172.21.15.120:0/3514890506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 msgr2=0x55efabfd1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f4916981640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 0x7f48fc0b77a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f49100ffcc0 tx=0x7f4908009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f490c00bd10 tx=0x7f490c0107a0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- 172.21.15.120:0/3514890506 shutdown_connections 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfd3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfd1460 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3514890506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfc54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- 172.21.15.120:0/3514890506 >> 172.21.15.120:0/3514890506 conn(0x55efabfc2f20 msgr2=0x55efabfc40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- 172.21.15.120:0/3514890506 shutdown_connections 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- 172.21.15.120:0/3514890506 wait complete. 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.437+0000 7f491a7b9740 1 -- start start 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 Processor -- start 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc5a10 0x55efabfce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55efabd8c700 con 0x55efabfd19a0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55efabec6160 con 0x55efabfc5a10 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55efabfd4620 con 0x55efabfc50f0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491a7b9740 10 client.0 _ll_register_callbacks cb 0x55efabedf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916180640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48696/0 (socket says 172.21.15.120:48696) 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916180640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:52208/0 (socket says 172.21.15.120:52208) 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 -- 172.21.15.120:0/427627368 learned_addr learned my addr 172.21.15.120:0/427627368 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f491597f640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc5a10 0x55efabfce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 -- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 msgr2=0x55efabfd12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.447 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 -- 172.21.15.120:0/427627368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc5a10 msgr2=0x55efabfce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc5a10 0x55efabfce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 -- 172.21.15.120:0/427627368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efabef3c90 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916180640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f4916981640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f4908009910 tx=0x7f4908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d7caffb146d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.438+0000 7f48f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49080109c0 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f4908010b60 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c0035f0 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4908010d60 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 10 client.20178 fetch_fsmap learned FSMap version 26 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efabfe0020 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490c0035f0 con 0x55efabfd19a0 2024-09-10T23:17:07.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f490801f480 con 0x55efabfd19a0 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 10 client.20178 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 20 client.20178 populate_metadata read hostname 'smithi120' 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efabfe1530 0x55efabfe3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 4 client.20178 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efabf3ddf0 con 0x55efabfe1530 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.439+0000 7f4916180640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efabfe1530 0x55efabfe3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.440+0000 7f48f9ffb640 10 client.20178 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:07.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.440+0000 7f4916180640 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efabfe1530 0x55efabfe3970 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d2fbdcb1e04177 server_cookie=733db1dd61db80fb in_seq=0 out_seq=0 2024-09-10T23:17:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f48f9ffb640 1 -- 172.21.15.120:0/427627368 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efabf3ddf0 con 0x55efabfe1530 2024-09-10T23:17:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f48f9ffb640 10 client.20178 handle_command_reply: tid=0 2024-09-10T23:17:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 client.20178 shutdown 2024-09-10T23:17:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 msgr2=0x55efabfceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f4908009910 tx=0x7f4908004a40 comp rx=0 tx=0).stop 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 shutdown_connections 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efabfe1530 0x55efabfe3970 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfd19a0 0x55efabfceda0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc5a10 0x55efabfce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f48f9ffb640 0 client.20178 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 --2- 172.21.15.120:0/427627368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc50f0 0x55efabfd12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 >> 172.21.15.120:0/427627368 conn(0x55efabfc2f20 msgr2=0x55efabfc8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 shutdown_connections 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 1 -- 172.21.15.120:0/427627368 wait complete. 2024-09-10T23:17:07.451 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.442+0000 7f491a7b9740 20 client.20178 trim_cache size 0 max 16384 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.455+0000 7f491a7b9740 1 Processor -- start 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.455+0000 7f491a7b9740 1 -- start start 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfcdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfc3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49101ae4f0 con 0x55efabfc5a10 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b0d70 con 0x55efabfc50f0 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491a7b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49101aeb60 con 0x55efabfd19a0 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48708/0 (socket says 172.21.15.120:48708) 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 -- 172.21.15.120:0/1285439756 learned_addr learned my addr 172.21.15.120:0/1285439756 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:07.457 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f4916981640 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfc3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f4916180640 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfcdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 -- 172.21.15.120:0/1285439756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 msgr2=0x55efabfc3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfc3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 -- 172.21.15.120:0/1285439756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 msgr2=0x55efabfcdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfcdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 -- 172.21.15.120:0/1285439756 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efabfbe7e0 con 0x55efabfc5a10 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f491597f640 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f49000ccd20 tx=0x7f4900005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e83bc741c470b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f48fbfff640 1 -- 172.21.15.120:0/1285439756 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900049040 con 0x55efabfc5a10 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f48fbfff640 1 -- 172.21.15.120:0/1285439756 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49000cc8a0 con 0x55efabfc5a10 2024-09-10T23:17:07.458 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.456+0000 7f48fbfff640 1 -- 172.21.15.120:0/1285439756 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000ccd90 con 0x55efabfc5a10 2024-09-10T23:17:07.461 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- 172.21.15.120:0/1285439756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 msgr2=0x55efabfc2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f49000ccd20 tx=0x7f4900005c00 comp rx=0 tx=0).stop 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- 172.21.15.120:0/1285439756 shutdown_connections 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfd19a0 0x55efabfc3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc5a10 0x55efabfc2760 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 --2- 172.21.15.120:0/1285439756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfc50f0 0x55efabfcdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- 172.21.15.120:0/1285439756 >> 172.21.15.120:0/1285439756 conn(0x55efabfc2f20 msgr2=0x55efabfccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- 172.21.15.120:0/1285439756 shutdown_connections 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- 172.21.15.120:0/1285439756 wait complete. 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.457+0000 7f491a7b9740 1 -- start start 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 Processor -- start 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.462 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 0x55efabfd9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 0x55efabfd48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49101af230 con 0x55efabfc50f0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b06a0 con 0x55efabfd19a0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f49101af900 con 0x55efabfc5a10 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491a7b9740 10 client.0 _ll_register_callbacks cb 0x7f4908009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491597f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 0x55efabfd9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48722/0 (socket says 172.21.15.120:48722) 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 -- 172.21.15.120:0/3185736698 learned_addr learned my addr 172.21.15.120:0/3185736698 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916981640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 0x55efabfd48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 -- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 msgr2=0x55efabfd9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 0x55efabfd9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.463 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 -- 172.21.15.120:0/3185736698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 msgr2=0x55efabfd48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 0x55efabfd48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 -- 172.21.15.120:0/3185736698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efabfda570 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f491597f640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 0x55efabfd9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916981640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 0x55efabfd48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f4916180640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f490c011010 tx=0x7f490c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7937d77a9d9c83f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.458+0000 7f48f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c005b90 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mon.0 v2:172.21.15.120:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f490c005240 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49000cc8a0 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f490c005460 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 10 client.20184 fetch_fsmap learned FSMap version 26 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efac04e960 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mon.0 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49000cc8a0 con 0x55efabfc50f0 2024-09-10T23:17:07.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mon.0 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8330+0+0 (secure 0 0 0) 0x7f490c02e5e0 con 0x55efabfc50f0 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 10 client.20184 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 10 client.20184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 20 client.20184 populate_metadata read hostname 'smithi120' 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efac04ee60 0x55efac051280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 4 client.20184 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 --> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efabf398a0 con 0x55efac04ee60 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.459+0000 7f491597f640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efac04ee60 0x55efac051280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.460+0000 7f48f9ffb640 10 client.20184 ms_handle_connect on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.460+0000 7f491597f640 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efac04ee60 0x55efac051280 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64b7bc3d229dad6 server_cookie=28c0fd13894acf3d in_seq=0 out_seq=0 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f48f9ffb640 1 -- 172.21.15.120:0/3185736698 <== mds.0 v2:172.21.15.176:6838/3141658633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efabf398a0 con 0x55efac04ee60 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f48f9ffb640 10 client.20184 handle_command_reply: tid=0 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 client.20184 shutdown 2024-09-10T23:17:07.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 msgr2=0x55efabfd98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f490c011010 tx=0x7f490c0058e0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 shutdown_connections 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:6838/3141658633,v1:172.21.15.176:6839/3141658633] conn(0x55efac04ee60 0x55efac051280 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55efabfd19a0 0x55efabfd48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55efabfc5a10 0x55efabfd9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f48f9ffb640 0 client.20184 ms_handle_reset on v2:172.21.15.176:6838/3141658633 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 --2- 172.21.15.120:0/3185736698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55efabfc50f0 0x55efabfd98f0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 >> 172.21.15.120:0/3185736698 conn(0x55efabfc2f20 msgr2=0x55efabfc3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 shutdown_connections 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 1 -- 172.21.15.120:0/3185736698 wait complete. 2024-09-10T23:17:07.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f491a7b9740 20 client.20184 trim_cache size 0 max 16384 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stdout:[] 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 msgr2=0x7f48fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 0x7f48fc0b77a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f49100ffcc0 tx=0x7f4908009500 comp rx=0 tx=0).stop 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 msgr2=0x7f49100fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.461+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4900005940 tx=0x7f4900004270 comp rx=0 tx=0).stop 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 shutdown_connections 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f48fc0b52e0 0x7f48fc0b77a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4910108020 0x7f49100fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49101076a0 0x7f49100fe140 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 --2- 172.21.15.120:0/3882737167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4910101c90 0x7f49100fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:07.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 >> 172.21.15.120:0/3882737167 conn(0x7f49100fd700 msgr2=0x7f491006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:07.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 shutdown_connections 2024-09-10T23:17:07.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:07.462+0000 7f49183e4640 1 -- 172.21.15.120:0/3882737167 wait complete. 2024-09-10T23:17:07.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:07 smithi120 ceph-mon[31247]: pgmap v1087: 97 pgs: 97 active+clean; 244 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-09-10T23:17:07.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:07 smithi153 ceph-mon[34951]: pgmap v1087: 97 pgs: 97 active+clean; 244 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-09-10T23:17:08.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:07 smithi176 ceph-mon[34982]: pgmap v1087: 97 pgs: 97 active+clean; 244 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-09-10T23:17:08.255 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-10T23:17:08.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T23:17:08.537 DEBUG:teuthology.parallel:result is None 2024-09-10T23:17:08.537 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T23:17:08.570 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T23:17:08.570 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T23:17:08.582 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T23:17:08.619 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T23:17:08.628 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T23:17:08.628 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T23:17:08.629 INFO:tasks.fwd_scrub:done joining 2024-09-10T23:17:08.629 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T23:17:08.638 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:17:08.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.758+0000 7fdaf4d89640 1 -- 172.21.15.120:0/4199642737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01029e0 msgr2=0x7fdaf0102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.758+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/4199642737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01029e0 0x7fdaf0102e60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009930 tx=0x7fdad802a420 comp rx=0 tx=0).stop 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 -- 172.21.15.120:0/4199642737 shutdown_connections 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/4199642737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01033a0 0x7fdaf0109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/4199642737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01029e0 0x7fdaf0102e60 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/4199642737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 0x7fdaf0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 -- 172.21.15.120:0/4199642737 >> 172.21.15.120:0/4199642737 conn(0x7fdaf00fd150 msgr2=0x7fdaf00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 -- 172.21.15.120:0/4199642737 shutdown_connections 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 -- 172.21.15.120:0/4199642737 wait complete. 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 Processor -- start 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 -- start start 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 0x7fdaf0105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.759+0000 7fdaf4d89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01029e0 0x7fdaf0103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:08.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d500 con 0x7fdaf01033a0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d380 con 0x7fdaf01029e0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d680 con 0x7fdaf0101890 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48740/0 (socket says 172.21.15.120:48740) 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 -- 172.21.15.120:0/2899261009 learned_addr learned my addr 172.21.15.120:0/2899261009 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaee59c640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 0x7fdaf0105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdae7fff640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01029e0 0x7fdaf0103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 -- 172.21.15.120:0/2899261009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 msgr2=0x7fdaf0105630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 0x7fdaf0105630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 -- 172.21.15.120:0/2899261009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01029e0 msgr2=0x7fdaf0103e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01029e0 0x7fdaf0103e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 -- 172.21.15.120:0/2899261009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0104c40 con 0x7fdaf01033a0 2024-09-10T23:17:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaeed9d640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fdae800eaa0 tx=0x7fdae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:08.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800cdb0 con 0x7fdaf01033a0 2024-09-10T23:17:08.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8005020 con 0x7fdaf01033a0 2024-09-10T23:17:08.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80116d0 con 0x7fdaf01033a0 2024-09-10T23:17:08.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf01a2e50 con 0x7fdaf01033a0 2024-09-10T23:17:08.764 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.760+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a32c0 con 0x7fdaf01033a0 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.761+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf01a3550 con 0x7fdaf01033a0 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.763+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdae800f040 con 0x7fdaf01033a0 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.764+0000 7fdae77fe640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 0x7fdadc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.764+0000 7fdaee59c640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 0x7fdadc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.764+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdae80e5a10 con 0x7fdaf01033a0 2024-09-10T23:17:08.765 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.764+0000 7fdaee59c640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 0x7fdadc0b77a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005fd0 tx=0x7fdae000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.768+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdae8016030 con 0x7fdaf01033a0 2024-09-10T23:17:08.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.945+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaf01a38e0 con 0x7fdaf01033a0 2024-09-10T23:17:08.946 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.946+0000 7fdae77fe640 1 -- 172.21.15.120:0/2899261009 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22746 (secure 0 0 0) 0x7fdae80ac900 con 0x7fdaf01033a0 2024-09-10T23:17:08.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:08.947 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:57:58.168156+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 msgr2=0x7fdadc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 0x7fdadc0b77a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005fd0 tx=0x7fdae000b390 comp rx=0 tx=0).stop 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 msgr2=0x7fdaf01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fdae800eaa0 tx=0x7fdae800ef60 comp rx=0 tx=0).stop 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 shutdown_connections 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdadc0b52e0 0x7fdadc0b77a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.949 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdaf01033a0 0x7fdaf01043d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdaf01029e0 0x7fdaf0103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 --2- 172.21.15.120:0/2899261009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdaf0101890 0x7fdaf0105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:08.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 >> 172.21.15.120:0/2899261009 conn(0x7fdaf00fd150 msgr2=0x7fdaf00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:08.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 shutdown_connections 2024-09-10T23:17:08.950 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:08.948+0000 7fdaf4d89640 1 -- 172.21.15.120:0/2899261009 wait complete. 2024-09-10T23:17:08.960 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:45:56.375659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:50:23.901262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:50:25.937324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:17:08.961 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:17:09.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 -- 172.21.15.120:0/568671388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80103ab0 msgr2=0x7fcd80103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 --2- 172.21.15.120:0/568671388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80103ab0 0x7fcd80103f30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009930 tx=0x7fcd7402a420 comp rx=0 tx=0).stop 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 -- 172.21.15.120:0/568671388 shutdown_connections 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 --2- 172.21.15.120:0/568671388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80104470 0x7fcd8010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 --2- 172.21.15.120:0/568671388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80103ab0 0x7fcd80103f30 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 --2- 172.21.15.120:0/568671388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80102960 0x7fcd80102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 -- 172.21.15.120:0/568671388 >> 172.21.15.120:0/568671388 conn(0x7fcd800fe220 msgr2=0x7fcd80100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 -- 172.21.15.120:0/568671388 shutdown_connections 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.061+0000 7fcd86bec640 1 -- 172.21.15.120:0/568671388 wait complete. 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 Processor -- start 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 -- start start 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 0x7fcd801993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.065 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80104470 0x7fcd8019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8010e5d0 con 0x7fcd80102960 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8010e450 con 0x7fcd80103ab0 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd86bec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8010e750 con 0x7fcd80104470 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd84988640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd84988640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48758/0 (socket says 172.21.15.120:48758) 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd84988640 1 -- 172.21.15.120:0/2284062540 learned_addr learned my addr 172.21.15.120:0/2284062540 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd85189640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80104470 0x7fcd8019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd7ffff640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 0x7fcd801993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.062+0000 7fcd84988640 1 -- 172.21.15.120:0/2284062540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80104470 msgr2=0x7fcd8019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd84988640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80104470 0x7fcd8019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd84988640 1 -- 172.21.15.120:0/2284062540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 msgr2=0x7fcd801993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd84988640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 0x7fcd801993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.066 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd84988640 1 -- 172.21.15.120:0/2284062540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8019a8c0 con 0x7fcd80102960 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd7ffff640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 0x7fcd801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd84988640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e9f0 tx=0x7fcd7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000cd80 con 0x7fcd80102960 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd700054d0 con 0x7fcd80102960 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8019abb0 con 0x7fcd80102960 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70010690 con 0x7fcd80102960 2024-09-10T23:17:09.067 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.063+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd801a1280 con 0x7fcd80102960 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.064+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80109cb0 con 0x7fcd80102960 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.065+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcd70004e90 con 0x7fcd80102960 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.066+0000 7fcd7dffb640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 0x7fcd680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.066+0000 7fcd7ffff640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 0x7fcd680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.066+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fcd700e60c0 con 0x7fcd80102960 2024-09-10T23:17:09.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.067+0000 7fcd7ffff640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 0x7fcd680b7820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006010 tx=0x7fcd74000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.070+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcd700acfb0 con 0x7fcd80102960 2024-09-10T23:17:09.244 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.244+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd80063d50 con 0x7fcd80102960 2024-09-10T23:17:09.245 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.244+0000 7fcd7dffb640 1 -- 172.21.15.120:0/2284062540 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fcd700ac8b0 con 0x7fcd80102960 2024-09-10T23:17:09.245 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 1 2024-09-10T23:17:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 msgr2=0x7fcd680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 0x7fcd680b7820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd74006010 tx=0x7fcd74000bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.247 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 msgr2=0x7fcd80198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000e9f0 tx=0x7fcd7000edf0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 shutdown_connections 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fcd680b5360 0x7fcd680b7820 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd80104470 0x7fcd8019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd80103ab0 0x7fcd801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 --2- 172.21.15.120:0/2284062540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd80102960 0x7fcd80198d80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 >> 172.21.15.120:0/2284062540 conn(0x7fcd800fe220 msgr2=0x7fcd80106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 shutdown_connections 2024-09-10T23:17:09.248 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.247+0000 7fcd86bec640 1 -- 172.21.15.120:0/2284062540 wait complete. 2024-09-10T23:17:09.258 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 -- 172.21.15.120:0/688036224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01086a0 msgr2=0x7fdae010ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 --2- 172.21.15.120:0/688036224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01086a0 0x7fdae010ab60 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 -- 172.21.15.120:0/688036224 shutdown_connections 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 --2- 172.21.15.120:0/688036224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae01086a0 0x7fdae010ab60 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 --2- 172.21.15.120:0/688036224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae0105af0 0x7fdae0107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.362 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 --2- 172.21.15.120:0/688036224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae006bee0 0x7fdae0105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 -- 172.21.15.120:0/688036224 >> 172.21.15.120:0/688036224 conn(0x7fdae00fd150 msgr2=0x7fdae00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 -- 172.21.15.120:0/688036224 shutdown_connections 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.359+0000 7fdae687f640 1 -- 172.21.15.120:0/688036224 wait complete. 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 Processor -- start 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 -- start start 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 0x7fdae01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 0x7fdae0199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e5d0 con 0x7fdae006bee0 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e450 con 0x7fdae0105af0 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae687f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e750 con 0x7fdae01086a0 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.363 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48780/0 (socket says 172.21.15.120:48780) 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 -- 172.21.15.120:0/1506843604 learned_addr learned my addr 172.21.15.120:0/1506843604 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadf7fe640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 0x7fdae01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae4e1c640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 0x7fdae0199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 -- 172.21.15.120:0/1506843604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 msgr2=0x7fdae0199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 0x7fdae0199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 -- 172.21.15.120:0/1506843604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 msgr2=0x7fdae01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 0x7fdae01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadffff640 1 -- 172.21.15.120:0/1506843604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019a7e0 con 0x7fdae006bee0 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdadf7fe640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 0x7fdae01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.360+0000 7fdae4e1c640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 0x7fdae0199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdadffff640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d950 tx=0x7fdad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0017070 con 0x7fdae006bee0 2024-09-10T23:17:09.364 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdad0009c30 con 0x7fdae006bee0 2024-09-10T23:17:09.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000f620 con 0x7fdae006bee0 2024-09-10T23:17:09.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae019aad0 con 0x7fdae006bee0 2024-09-10T23:17:09.365 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.361+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a11a0 con 0x7fdae006bee0 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.362+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae0105f50 con 0x7fdae006bee0 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.363+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdad0010460 con 0x7fdae006bee0 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.364+0000 7fdadd7fa640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 0x7fdac80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.364+0000 7fdadf7fe640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 0x7fdac80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.364+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdad00e5870 con 0x7fdae006bee0 2024-09-10T23:17:09.366 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.364+0000 7fdadf7fe640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 0x7fdac80b7820 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdad400abe0 tx=0x7fdad400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.368 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.368+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdad00ac760 con 0x7fdae006bee0 2024-09-10T23:17:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.542+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdae0063d50 con 0x7fdae006bee0 2024-09-10T23:17:09.543 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.543+0000 7fdadd7fa640 1 -- 172.21.15.120:0/1506843604 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fdad0009de0 con 0x7fdae006bee0 2024-09-10T23:17:09.543 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 32 2024-09-10T23:17:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 msgr2=0x7fdac80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 0x7fdac80b7820 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdad400abe0 tx=0x7fdad400a5c0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 msgr2=0x7fdae0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d950 tx=0x7fdad000de10 comp rx=0 tx=0).stop 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 shutdown_connections 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdac80b5360 0x7fdac80b7820 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae01086a0 0x7fdae0199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae0105af0 0x7fdae01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 --2- 172.21.15.120:0/1506843604 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae006bee0 0x7fdae0198ca0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 >> 172.21.15.120:0/1506843604 conn(0x7fdae00fd150 msgr2=0x7fdae0103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 shutdown_connections 2024-09-10T23:17:09.546 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.545+0000 7fdae687f640 1 -- 172.21.15.120:0/1506843604 wait complete. 2024-09-10T23:17:09.556 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T23:17:09.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.687+0000 7f22de110640 1 -- 172.21.15.120:0/3180523715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 msgr2=0x7f22d80746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.687+0000 7f22de110640 1 --2- 172.21.15.120:0/3180523715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 0x7f22d80746a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009be0 tx=0x7f22d002a710 comp rx=0 tx=0).stop 2024-09-10T23:17:09.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.694+0000 7f22de110640 1 -- 172.21.15.120:0/3180523715 shutdown_connections 2024-09-10T23:17:09.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.694+0000 7f22de110640 1 --2- 172.21.15.120:0/3180523715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d810ab10 0x7f22d811a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.694+0000 7f22de110640 1 --2- 172.21.15.120:0/3180523715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810a150 0x7f22d810a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.694 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.694+0000 7f22de110640 1 --2- 172.21.15.120:0/3180523715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 0x7f22d80746a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.695 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.694+0000 7f22de110640 1 -- 172.21.15.120:0/3180523715 >> 172.21.15.120:0/3180523715 conn(0x7f22d806fb20 msgr2=0x7f22d8071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.696+0000 7f22de110640 1 -- 172.21.15.120:0/3180523715 shutdown_connections 2024-09-10T23:17:09.696 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.696+0000 7f22de110640 1 -- 172.21.15.120:0/3180523715 wait complete. 2024-09-10T23:17:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.696+0000 7f22de110640 1 Processor -- start 2024-09-10T23:17:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 -- start start 2024-09-10T23:17:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 0x7f22d8119590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d810a150 0x7f22d8119ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112180 con 0x7f22d810ab10 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112000 con 0x7f22d810a150 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.697+0000 7f22de110640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8112300 con 0x7f22d80742a0 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48806/0 (socket says 172.21.15.120:48806) 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 -- 172.21.15.120:0/235923607 learned_addr learned my addr 172.21.15.120:0/235923607 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:09.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 -- 172.21.15.120:0/235923607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 msgr2=0x7f22d8119590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:17:09.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 0x7f22d8119590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 -- 172.21.15.120:0/235923607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d810a150 msgr2=0x7f22d8119ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:17:09.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d810a150 0x7f22d8119ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 -- 172.21.15.120:0/235923607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d81185d0 con 0x7f22d810ab10 2024-09-10T23:17:09.702 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.698+0000 7f22d7fff640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b3f0 tx=0x7f22c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.701+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8015020 con 0x7f22d810ab10 2024-09-10T23:17:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.701+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d81188c0 con 0x7f22d810ab10 2024-09-10T23:17:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.701+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d8122b50 con 0x7f22d810ab10 2024-09-10T23:17:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.701+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c8011ce0 con 0x7f22d810ab10 2024-09-10T23:17:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.701+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8010e10 con 0x7f22d810ab10 2024-09-10T23:17:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.705+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d81102c0 con 0x7f22d810ab10 2024-09-10T23:17:09.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.707+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f22c8011830 con 0x7f22d810ab10 2024-09-10T23:17:09.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.708+0000 7f22d4ff9640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 0x7f22c40b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:09.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.708+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f22c80e5660 con 0x7f22d810ab10 2024-09-10T23:17:09.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.709+0000 7f22d77fe640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 0x7f22c40b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:09.712 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.709+0000 7f22d77fe640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 0x7f22c40b79b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22d0031040 tx=0x7f22d0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:09.718 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.717+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22c801f020 con 0x7f22d810ab10 2024-09-10T23:17:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:09 smithi120 ceph-mon[31247]: pgmap v1088: 97 pgs: 97 active+clean; 244 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 435 op/s 2024-09-10T23:17:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:09 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2899261009' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:09 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2284062540' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:17:09.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:09 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1506843604' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:17:09.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:09 smithi153 ceph-mon[34951]: pgmap v1088: 97 pgs: 97 active+clean; 244 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 435 op/s 2024-09-10T23:17:09.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:09 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2899261009' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:09.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:09 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2284062540' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:17:09.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:09 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1506843604' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:17:09.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.978+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22d802e980 con 0x7f22d810ab10 2024-09-10T23:17:09.978 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.978+0000 7f22d4ff9640 1 -- 172.21.15.120:0/235923607 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f22c80ac4f0 con 0x7f22d810ab10 2024-09-10T23:17:09.979 INFO:teuthology.orchestra.run.smithi120.stdout:pg_num: 64 2024-09-10T23:17:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 msgr2=0x7f22c40b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 0x7f22c40b79b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22d0031040 tx=0x7f22d0005fd0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 msgr2=0x7f22d8117ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b3f0 tx=0x7f22c800b8b0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.981 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 shutdown_connections 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f22c40b54f0 0x7f22c40b79b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d810ab10 0x7f22d8117ed0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d810a150 0x7f22d8119ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 --2- 172.21.15.120:0/235923607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d80742a0 0x7f22d8119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.980+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 >> 172.21.15.120:0/235923607 conn(0x7f22d806fb20 msgr2=0x7f22d810d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.981+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 shutdown_connections 2024-09-10T23:17:09.982 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:09.981+0000 7f22de110640 1 -- 172.21.15.120:0/235923607 wait complete. 2024-09-10T23:17:09.992 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T23:17:10.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:09 smithi176 ceph-mon[34982]: pgmap v1088: 97 pgs: 97 active+clean; 244 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 435 op/s 2024-09-10T23:17:10.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:09 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2899261009' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:10.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:09 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2284062540' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:17:10.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:09 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1506843604' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- 172.21.15.120:0/38951883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 msgr2=0x7fdae4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/38951883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae4101c90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009930 tx=0x7fdae002a420 comp rx=0 tx=0).stop 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- 172.21.15.120:0/38951883 shutdown_connections 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/38951883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41033a0 0x7fdae4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/38951883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41029e0 0x7fdae4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/38951883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae4101c90 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- 172.21.15.120:0/38951883 >> 172.21.15.120:0/38951883 conn(0x7fdae40fd150 msgr2=0x7fdae40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- 172.21.15.120:0/38951883 shutdown_connections 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- 172.21.15.120:0/38951883 wait complete. 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 Processor -- start 2024-09-10T23:17:10.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.090+0000 7fdaeb2be640 1 -- start start 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 0x7fdae419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 0x7fdae41a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410d4e0 con 0x7fdae4101890 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410d360 con 0x7fdae41033a0 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410d660 con 0x7fdae41029e0 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48824/0 (socket says 172.21.15.120:48824) 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 -- 172.21.15.120:0/932170801 learned_addr learned my addr 172.21.15.120:0/932170801 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae8859640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 0x7fdae419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae985b640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 0x7fdae41a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 -- 172.21.15.120:0/932170801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 msgr2=0x7fdae419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 0x7fdae419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:10.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 -- 172.21.15.120:0/932170801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 msgr2=0x7fdae41a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 0x7fdae41a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 -- 172.21.15.120:0/932170801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae41a0920 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae8859640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 0x7fdae419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae985b640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 0x7fdae41a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdae905a640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009a50 tx=0x7fdae00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae003f030 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae0003cb0 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae41a0bb0 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0035070 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.091+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a10a0 con 0x7fdae4101890 2024-09-10T23:17:10.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.092+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae4101c90 con 0x7fdae4101890 2024-09-10T23:17:10.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.094+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdae003bb90 con 0x7fdae4101890 2024-09-10T23:17:10.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.094+0000 7fdada7fc640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 0x7fdad00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:10.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.094+0000 7fdae8859640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 0x7fdad00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:10.096 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.094+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fdae0106900 con 0x7fdae4101890 2024-09-10T23:17:10.097 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.095+0000 7fdae8859640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 0x7fdad00b77a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdad4005d00 tx=0x7fdad4005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:10.098 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.097+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdae00cd7f0 con 0x7fdae4101890 2024-09-10T23:17:10.321 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.320+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fdae402e5c0 con 0x7fdae4101890 2024-09-10T23:17:10.328 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.327+0000 7fdada7fc640 1 -- 172.21.15.120:0/932170801 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+827948 (secure 0 0 0) 0x7fdae402e5c0 con 0x7fdae4101890 2024-09-10T23:17:10.338 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:10.338 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "epoch": 18, 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "active_gid": 14150, 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "active_name": "x", 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "active_addrs": { 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6800", 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 578504839 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v1", 2024-09-10T23:17:10.339 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:6801", 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 578504839 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "active_addr": "172.21.15.120:6801/578504839", 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "active_change": "2024-09-10T22:43:14.065938+0000", 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "available": true, 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "standbys": [ 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "gid": 24112, 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "z", 2024-09-10T23:17:10.340 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.341 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.342 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.343 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.345 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.346 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.347 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-10T23:17:10.348 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.349 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.350 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.351 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.352 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.354 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.357 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.358 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.359 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.360 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.361 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.362 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:10.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.364 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.366 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.366 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.366 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.366 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.367 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.367 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.368 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.369 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-10T23:17:10.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:10.372 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.373 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.374 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:17:10.375 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.376 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.377 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.378 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.379 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.381 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.382 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.383 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:17:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-10T23:17:10.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.386 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.387 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-10T23:17:10.388 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-10T23:17:10.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.390 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.391 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-10T23:17:10.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-10T23:17:10.393 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:17:10.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:17:10.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.397 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.398 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.399 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nginx": { 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nginx", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nginx container image", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.401 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.402 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.405 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.406 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba_metrics": { 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-10T23:17:10.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-10T23:17:10.408 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.409 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:17:10.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-10T23:17:10.411 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-10T23:17:10.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-10T23:17:10.413 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-10T23:17:10.414 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-10T23:17:10.415 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.416 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.417 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-10T23:17:10.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.420 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.421 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:17:10.422 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.423 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.424 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.428 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.429 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.430 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.434 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.435 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.436 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.437 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.438 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.439 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.440 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.441 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.442 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.443 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.444 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.445 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.446 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.447 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.448 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.449 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.450 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.454 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.455 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.456 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.457 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.458 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.459 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.460 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.464 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.466 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.469 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.472 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.474 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.475 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "[]", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "{}", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.487 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:17:10.488 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:17:10.489 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.490 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:17:10.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:17:10.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:17:10.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.495 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:17:10.496 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.497 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:17:10.498 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:17:10.499 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-10T23:17:10.500 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-10T23:17:10.501 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-10T23:17:10.502 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-10T23:17:10.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:10.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.506 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.507 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.508 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.509 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.514 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.515 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.519 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.520 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-10T23:17:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.522 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.525 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.526 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.529 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.531 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.532 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.533 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.534 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.535 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.536 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.537 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.538 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.541 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.542 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.543 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.545 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.546 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.547 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.548 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.549 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.552 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.553 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.554 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.556 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.557 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:17:10.558 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.559 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.560 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.561 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.562 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.563 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.564 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.568 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.570 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.571 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.573 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.574 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.577 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.578 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.579 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.580 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.581 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.582 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.584 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.585 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.586 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.587 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.588 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.589 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.590 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-10T23:17:10.591 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-10T23:17:10.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.594 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.595 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.596 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.597 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.601 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.602 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.603 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.604 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.605 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.606 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.607 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.608 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.609 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.610 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.611 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.612 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.614 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.615 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.616 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.617 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.618 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.619 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.620 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.621 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.623 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.624 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.625 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.626 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.628 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.629 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.630 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.631 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.633 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.634 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.635 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.636 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.637 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.638 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.639 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.640 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.641 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.644 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.645 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.646 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.647 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.648 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.650 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.652 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.653 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.655 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.656 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.657 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.658 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.660 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.661 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.662 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-10T23:17:10.663 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:17:10.664 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.667 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.670 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.671 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.672 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-10T23:17:10.673 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.680 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.681 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.682 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.683 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.684 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.685 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.686 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.687 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.688 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.690 INFO:teuthology.orchestra.run.smithi120.stdout: "max_requests": { 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_requests", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.691 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.692 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.693 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.694 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.695 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.696 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.697 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.698 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.699 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.701 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.702 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.703 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.705 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.706 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.707 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.708 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.709 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.710 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.711 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.713 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.714 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.715 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.716 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.720 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.722 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.723 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.724 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.725 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.726 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.727 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.728 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smb", 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "internal_store_backend": { 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "internal_store_backend", 2024-09-10T23:17:10.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.731 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.732 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.733 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.734 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.735 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "update_orchestration": { 2024-09-10T23:17:10.736 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "update_orchestration", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.738 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.739 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.740 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.741 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.742 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.743 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.744 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.745 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.746 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.747 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.748 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.749 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.751 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.752 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.753 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.756 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.757 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.758 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-10T23:17:10.759 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.760 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.761 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.762 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.763 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.764 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.765 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.768 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.769 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.770 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.771 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.772 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.773 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.774 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.775 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.776 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-10T23:17:10.778 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-10T23:17:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-10T23:17:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.782 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.783 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.784 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.785 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-10T23:17:10.786 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-10T23:17:10.787 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.788 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-10T23:17:10.789 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.791 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.792 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.793 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.794 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.795 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.796 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.797 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.798 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.799 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.800 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.802 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.803 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.804 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.805 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.806 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.808 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.809 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.810 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.811 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.813 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.814 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.815 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.816 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.817 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.818 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.819 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "gid": 24137, 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "y", 2024-09-10T23:17:10.820 INFO:teuthology.orchestra.run.smithi120.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.821 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.822 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.823 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.824 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.825 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.826 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.827 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.828 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.829 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.831 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.832 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.833 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.834 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.835 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-10T23:17:10.836 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-10T23:17:10.837 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-10T23:17:10.838 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:17:10.839 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-10T23:17:10.840 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-10T23:17:10.841 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-10T23:17:10.842 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-10T23:17:10.843 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.844 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.845 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.846 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.847 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.848 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-10T23:17:10.849 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:10.850 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-10T23:17:10.851 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:17:10.852 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.853 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.854 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.855 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.856 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-10T23:17:10.857 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-10T23:17:10.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-10T23:17:10.859 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.860 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-10T23:17:10.861 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:17:10.862 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-10T23:17:10.863 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.864 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.865 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.866 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.867 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.868 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.870 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.871 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-10T23:17:10.872 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.873 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.876 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nginx": { 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nginx", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nginx container image", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.880 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.881 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.882 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-10T23:17:10.883 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba_metrics": { 2024-09-10T23:17:10.884 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.885 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.887 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.887 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.889 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.890 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.891 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.892 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.893 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.894 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-10T23:17:10.895 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-10T23:17:10.896 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-10T23:17:10.897 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-10T23:17:10.898 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.899 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.900 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.901 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.902 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.903 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.904 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:17:10.905 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:17:10.906 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-10T23:17:10.907 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.908 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.909 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-10T23:17:10.911 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.912 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-10T23:17:10.913 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-10T23:17:10.914 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-10T23:17:10.915 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-10T23:17:10.916 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-10T23:17:10.917 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-10T23:17:10.918 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.919 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-10T23:17:10.920 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-10T23:17:10.921 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.922 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:17:10.924 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.925 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.926 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.927 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.928 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.929 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.930 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.931 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.932 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.933 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.935 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.936 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.938 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.939 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.940 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.941 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.942 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.943 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.944 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.945 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.946 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.947 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.948 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.949 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.950 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.951 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.952 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.953 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.954 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.955 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.956 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "[]", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.957 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "{}", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.958 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.959 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.960 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.961 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.962 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.963 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.964 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.965 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.966 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.967 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.968 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.969 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.973 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.974 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:17:10.975 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.976 INFO:teuthology.orchestra.run.smithi120.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.977 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.978 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:17:10.979 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-10T23:17:10.980 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-10T23:17:10.981 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-10T23:17:10.982 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-10T23:17:10.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:10.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.985 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.986 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.987 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:10.988 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.989 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.991 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.992 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.993 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:10.994 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.995 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-10T23:17:10.996 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-10T23:17:10.997 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.998 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:10.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.000 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.001 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.002 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.005 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.006 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.007 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.009 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:17:11.010 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:17:11.011 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:17:11.012 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.013 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.014 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.015 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.016 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.017 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.018 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.019 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.020 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.021 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.022 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:17:11.025 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:17:11.026 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.027 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.029 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.030 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.031 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.032 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.033 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.034 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.035 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.038 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.039 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.041 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.042 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.043 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.044 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.045 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.046 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.047 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.048 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.049 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.050 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.051 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.053 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.056 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.057 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.058 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.061 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-10T23:17:11.062 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.063 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.065 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.066 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.067 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.068 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.069 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.071 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.072 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.073 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.074 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.075 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.076 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.077 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.078 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.079 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-10T23:17:11.080 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.081 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.083 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.084 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.085 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.086 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.087 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.088 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.089 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.090 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.091 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.092 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.093 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.095 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.096 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.097 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.098 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.099 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.100 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-10T23:17:11.101 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.102 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.103 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.104 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.105 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.106 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.107 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.108 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.110 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.111 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.113 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.114 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.115 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.116 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.117 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.119 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.120 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.121 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.122 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:11.123 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.124 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.125 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.127 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.128 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.129 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.130 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-10T23:17:11.131 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.132 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.133 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.135 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:17:11.136 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.137 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.138 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.139 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.140 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.141 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.142 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.143 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.144 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:17:11.145 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:17:11.146 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.147 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.148 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.149 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.150 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.152 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.153 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.154 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.155 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.157 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.158 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.159 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.160 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:11.161 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.162 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.163 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.164 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.165 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.166 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "max_requests": { 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_requests", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.167 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.168 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.169 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.170 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-10T23:17:11.171 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.172 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.173 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.174 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.175 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.176 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.177 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.178 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.179 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.180 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.181 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.182 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.183 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.184 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.185 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.186 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.187 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.188 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.189 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.190 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-10T23:17:11.191 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-10T23:17:11.192 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-10T23:17:11.193 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-10T23:17:11.194 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-10T23:17:11.195 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-10T23:17:11.196 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-10T23:17:11.197 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.198 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.199 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.200 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.202 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.203 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smb", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.204 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "internal_store_backend": { 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "internal_store_backend", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.206 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.207 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.208 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "update_orchestration": { 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "update_orchestration", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.232 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:17:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-10T23:17:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-10T23:17:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:17:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-10T23:17:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-10T23:17:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "modules": [ 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-10T23:17:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "dashboard", 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "iostat", 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "nfs", 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "restful" 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "available_modules": [ 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "alerts", 2024-09-10T23:17:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_destination": { 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_destination", 2024-09-10T23:17:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_from_name": { 2024-09-10T23:17:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_from_name", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "Ceph", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Email From: name", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_host": { 2024-09-10T23:17:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_host", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP server", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_password": { 2024-09-10T23:17:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_password", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Password to authenticate with", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_port": { 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_port", 2024-09-10T23:17:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "465", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP port", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_sender": { 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_sender", 2024-09-10T23:17:11.306 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_ssl": { 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_ssl", 2024-09-10T23:17:11.307 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "smtp_user": { 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smtp_user", 2024-09-10T23:17:11.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "User to authenticate as", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.309 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.310 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "balancer", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "active": { 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "active", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:17:11.311 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_time": { 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_time", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0000", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:11.312 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "begin_weekday": { 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "begin_weekday", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:11.313 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_max_iterations": { 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "25", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "250", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.314 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_metrics": { 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_metrics", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.315 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "crush_compat_step": { 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crush_compat_step", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.5", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0.001", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "0.999", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "end_time": { 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_time", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2359", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "end_weekday": { 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "end_weekday", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "0", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "6", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.319 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.320 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.321 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.322 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.323 INFO:teuthology.orchestra.run.smithi120.stdout: "min_score": { 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_score", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.324 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "upmap", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "crush-compat", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "none", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "read", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap", 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap-read" 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.325 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Balancer mode", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_ids": { 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_ids", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:17:11.326 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:17:11.327 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.328 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_deviation": { 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_deviation", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:17:11.329 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "upmap_max_optimizations": { 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.330 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm", 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_down_multiplier": { 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_down_multiplier", 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.331 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_refresh_rate": { 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_refresh_rate", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.332 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "20", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "agent_starting_port": { 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "agent_starting_port", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4721", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_ptrace": { 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_ptrace", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_interval": { 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_interval", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.335 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.336 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0.7", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm_log_destination": { 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cephadm_log_destination", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.337 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "file", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "file,syslog", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "syslog" 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "cgroups_split": { 2024-09-10T23:17:11.338 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cgroups_split", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "config_checks_enabled": { 2024-09-10T23:17:11.339 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_checks_enabled", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "config_dashboard": { 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_dashboard", 2024-09-10T23:17:11.340 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_alertmanager": { 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_alertmanager", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.341 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_base": { 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_base", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.342 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_elasticsearch": { 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.343 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "elasticsearch container image", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_grafana": { 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_grafana", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.344 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_haproxy": { 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_haproxy", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "HAproxy container image", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_agent": { 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:17:11.346 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_collector": { 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:17:11.347 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_jaeger_query": { 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:17:11.348 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Jaeger query container image", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_keepalived": { 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_keepalived", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:17:11.349 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Keepalived container image", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_loki": { 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_loki", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.350 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Loki container image", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nginx": { 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nginx", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.351 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nginx container image", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_node_exporter": { 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_node_exporter", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.352 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_nvmeof": { 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_nvmeof", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:17:11.353 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Nvme-of container image", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.354 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_prometheus": { 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_prometheus", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.355 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Prometheus container image", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_promtail": { 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_promtail", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.356 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Promtail container image", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba": { 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.357 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_samba_metrics": { 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.358 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "container_image_snmp_gateway": { 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:17:11.359 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "container_init": { 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "container_init", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:17:11.360 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "daemon_cache_timeout": { 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.361 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "900", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.362 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "default_registry": { 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "default_registry", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "docker.io", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.363 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "device_cache_timeout": { 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_cache_timeout", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1800", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.364 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "device_enhanced_scan": { 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_enhanced_scan", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.365 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "facts_cache_timeout": { 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "facts_cache_timeout", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.366 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "grafana_dashboards_path": { 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.367 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "host_check_interval": { 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "host_check_interval", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.368 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "hw_monitoring": { 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hw_monitoring", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.369 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "inventory_list_all": { 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "inventory_list_all", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.370 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.371 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "log_refresh_metadata": { 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_refresh_metadata", 2024-09-10T23:17:11.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.373 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.374 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.375 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.376 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:17:11.377 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "*", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "max_count_per_host": { 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_count_per_host", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.379 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max_osd_draining_count": { 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_osd_draining_count", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.380 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "migration_current": { 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "migration_current", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "internal - do not modify", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.381 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "mode": { 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mode", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "root", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm-package", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "root" 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:17:11.382 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "oob_default_addr": { 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "oob_default_addr", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "169.254.1.1", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:17:11.383 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "prometheus_alerts_path": { 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.384 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_insecure": { 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_insecure", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.385 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_password": { 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_password", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.386 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_url": { 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_url", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.387 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "registry_username": { 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "registry_username", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.388 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "secure_monitoring_stack": { 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.389 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "service_discovery_port": { 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "service_discovery_port", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8765", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.390 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.391 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "ssh_config_file": { 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssh_config_file", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.392 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "use_agent": { 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_agent", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "use_repo_digest": { 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "use_repo_digest", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_failed_host_check": { 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.395 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_daemons": { 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.396 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_on_stray_hosts": { 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.397 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crash", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.398 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.399 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.401 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.402 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "retain_interval": { 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retain_interval", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "31536000", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.403 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.404 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_recent_interval": { 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_recent_interval", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1209600", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.405 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dashboard", 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.406 INFO:teuthology.orchestra.run.smithi120.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:17:11.407 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.408 INFO:teuthology.orchestra.run.smithi120.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.409 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:17:11.410 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.412 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:17:11.413 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:17:11.414 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:17:11.415 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.416 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:17:11.417 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:17:11.418 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:17:11.419 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:17:11.420 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.421 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_URL": { 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.427 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.428 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "[]", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "{}", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.430 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.432 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.433 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:17:11.434 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:17:11.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.436 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.438 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.440 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.441 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.442 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.443 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "45", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.444 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "admin", 2024-09-10T23:17:11.445 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.446 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.447 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.448 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.449 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.450 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.451 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "cross_origin_url": { 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cross_origin_url", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.452 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "crt_file": { 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "crt_file", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.453 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "debug": { 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "debug", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:17:11.454 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "jwt_token_ttl": { 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "jwt_token_ttl", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "28800", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.455 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.456 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.457 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.458 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.459 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "motd": { 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "motd", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "The message of the day", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.464 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8080", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.466 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl_server_port": { 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl_server_port", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8443", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.468 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "redirect", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "redirect" 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.469 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.470 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "url_prefix": { 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url_prefix", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.471 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_monitoring": { 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_monitoring", 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.472 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "monitor device health metrics", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.473 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.474 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.475 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.476 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.477 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "mark_out_threshold": { 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mark_out_threshold", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.478 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "2419200", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "pool_name": { 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pool_name", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "device_health_metrics", 2024-09-10T23:17:11.479 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "retention_period": { 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "retention_period", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15552000", 2024-09-10T23:17:11.480 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_frequency": { 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_frequency", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "self_heal": { 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "self_heal", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.482 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.483 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.484 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "warn_threshold": { 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "warn_threshold", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7257600", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.485 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "diskprediction_local", 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.486 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.487 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.488 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.489 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.490 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "predict_interval": { 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predict_interval", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.491 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "86400", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "predictor_model": { 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "predictor_model", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.492 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "prophetstor", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.493 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "600", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.494 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "influx", 2024-09-10T23:17:11.495 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": false, 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "batch_size": { 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "batch_size", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5000", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.496 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "database": { 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "database", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "ceph", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.497 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "hostname": { 2024-09-10T23:17:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "hostname", 2024-09-10T23:17:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.498 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.499 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.500 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "30", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "5", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.501 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.503 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.504 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.505 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "password": { 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "password", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.506 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "port": { 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "port", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "8086", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.507 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "InfluxDB server port", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.508 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "ssl": { 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ssl", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "false", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.509 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "threads": { 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threads", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "32", 2024-09-10T23:17:11.510 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "username": { 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "username", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.511 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "verify_ssl": { 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "verify_ssl", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "true", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.512 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "insights", 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.513 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.514 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.515 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.516 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.517 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "iostat", 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.519 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.520 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.522 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.523 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.524 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.525 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "k8sevents", 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_event_retention_days": { 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.526 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "7", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "config_check_secs": { 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "config_check_secs", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.527 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "10", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.528 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.529 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.531 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.532 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.533 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "localpool", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.534 INFO:teuthology.orchestra.run.smithi120.stdout: "failure_domain": { 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "failure_domain", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "host", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.535 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.536 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.537 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.538 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "min_size": { 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "min_size", 2024-09-10T23:17:11.540 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "num_rep": { 2024-09-10T23:17:11.541 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "num_rep", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_num": { 2024-09-10T23:17:11.542 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_num", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "128", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "prefix": { 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prefix", 2024-09-10T23:17:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "subtree": { 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "subtree", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.545 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "rack", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.546 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mds_autoscaler", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.547 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.548 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.549 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.550 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.552 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirroring", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.553 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.555 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.556 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.557 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.558 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "nfs", 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.559 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.560 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.561 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.562 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.563 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.564 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:11.565 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "fail_fs": { 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "fail_fs", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.566 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.567 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.568 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.569 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.570 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator": { 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "orchestrator", 2024-09-10T23:17:11.571 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "cephadm", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "rook", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "test_orchestrator" 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Orchestrator backend", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.572 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.573 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_perf_query", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.574 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.575 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.577 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.578 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.579 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "osd_support", 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.580 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.581 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.582 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.583 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.585 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "pg_autoscaler", 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.586 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.587 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.588 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.589 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.591 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.592 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.593 INFO:teuthology.orchestra.run.smithi120.stdout: "threshold": { 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "threshold", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "3.0", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1.0", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "scaling threshold", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "progress", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_pg_recovery_event": { 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.595 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.596 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.597 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.598 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.600 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.601 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "max_completed_events": { 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_completed_events", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "50", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.602 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "sleep_interval": { 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sleep_interval", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.604 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "prometheus", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "cache": { 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "cache", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.605 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "exclude_perf_counters": { 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "exclude_perf_counters", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.606 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.607 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.610 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.611 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools": { 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.612 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "300", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.613 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "scrape_interval": { 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "scrape_interval", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15.0", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.614 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "::", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.615 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "9283", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.616 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.617 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "stale_cache_strategy": { 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stale_cache_strategy", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "log", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_behaviour": { 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_behaviour", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "default", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.619 INFO:teuthology.orchestra.run.smithi120.stdout: "default", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "error" 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "standby_error_status_code": { 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "standby_error_status_code", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.620 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "400", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "599", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.621 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.622 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.623 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.624 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.626 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_snap_create": { 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.627 INFO:teuthology.orchestra.run.smithi120.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.628 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "trash_purge_schedule": { 2024-09-10T23:17:11.629 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "trash_purge_schedule", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "restful", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "enable_auth": { 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enable_auth", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.631 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "key_file": { 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "key_file", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.632 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.633 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.634 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.635 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.636 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.637 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max_requests": { 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_requests", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "500", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.638 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "server_addr": { 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_addr", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "server_port": { 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "server_port", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.640 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.641 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rgw", 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.642 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.644 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.645 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.646 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "5", 2024-09-10T23:17:11.647 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.648 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rook", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.649 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.650 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.651 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.652 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.653 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "storage_class": { 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "storage_class", 2024-09-10T23:17:11.655 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "local", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "selftest", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.657 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.658 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.659 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.660 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.661 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "roption1": { 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption1", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.662 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "roption2": { 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "roption2", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "xyz", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.663 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption1": { 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption1", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.664 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption2": { 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption2", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption3": { 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption3", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "float", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.666 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption4": { 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption4", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.667 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption5": { 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption5", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.668 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption6": { 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption6", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.669 INFO:teuthology.orchestra.run.smithi120.stdout: "rwoption7": { 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rwoption7", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "1", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "42", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.670 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "testkey": { 2024-09-10T23:17:11.671 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testkey", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "testlkey": { 2024-09-10T23:17:11.672 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testlkey", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "testnewline": { 2024-09-10T23:17:11.673 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "testnewline", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.674 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "smb", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "internal_store_backend": { 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "internal_store_backend", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.675 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.676 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.677 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.678 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.679 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.680 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.681 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "update_orchestration": { 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "update_orchestration", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:17:11.682 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snap_schedule", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "allow_m_granularity": { 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "allow_m_granularity", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.683 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "dump_on_update": { 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "dump_on_update", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.684 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.685 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.686 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.687 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.688 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.689 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.690 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "stats", 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.691 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.692 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.693 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.694 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.695 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.696 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.697 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "status", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.698 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.699 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.700 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.701 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.702 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.703 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telegraf", 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "address": { 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "address", 2024-09-10T23:17:11.704 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.705 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "15", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.706 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.707 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.708 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.709 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.710 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.711 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "telemetry", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_basic": { 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_basic", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:17:11.713 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_crash": { 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_crash", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:17:11.714 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_device": { 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_device", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:17:11.715 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_ident": { 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_ident", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.716 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "channel_perf": { 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "channel_perf", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.717 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "contact": { 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "contact", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.718 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "description": { 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "description", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.719 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "device_url": { 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "device_url", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.720 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "enabled": { 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "enabled", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "24", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "8", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "last_opt_revision": { 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "last_opt_revision", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "1", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.723 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard": { 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.724 INFO:teuthology.orchestra.run.smithi120.stdout: "leaderboard_description": { 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "leaderboard_description", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.728 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "organization": { 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "organization", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.730 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "proxy": { 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "proxy", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.731 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.732 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "url": { 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "url", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.733 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "test_orchestrator", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.735 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.736 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.737 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.738 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.739 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.740 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.741 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.742 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.743 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.744 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.745 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "max_concurrent_clones": { 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "max_concurrent_clones", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.746 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "4", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "periodic_async_work": { 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "periodic_async_work", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.747 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Periodically check for async work", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_delay": { 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.748 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.749 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "True", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.750 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "can_run": true, 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "error_string": "", 2024-09-10T23:17:11.751 INFO:teuthology.orchestra.run.smithi120.stdout: "module_options": { 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "discovery_interval": { 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "discovery_interval", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "uint", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "100", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "identifier": { 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "identifier", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "interval": { 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "interval", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "secs", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "60", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.754 INFO:teuthology.orchestra.run.smithi120.stdout: "log_level": { 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_level", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.755 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster": { 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.756 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_cluster_level": { 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_cluster_level", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "info", 2024-09-10T23:17:11.757 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [ 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "critical", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "debug", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "error", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "info", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "warning" 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "log_to_file": { 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "log_to_file", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "bool", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "False", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "sqlite3_killpoint": { 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "dev", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 1, 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "0", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_host": { 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_host", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_port": { 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_port", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "int", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "10051", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "zabbix_sender": { 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "zabbix_sender", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "str", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "level": "advanced", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "flags": 0, 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "min": "", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "max": "", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "enum_allowed": [], 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "desc": "", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "long_desc": "", 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "tags": [], 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "see_also": [] 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "services": { 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "dashboard": "https://172.21.15.120:8443/" 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "always_on_modules": { 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "octopus": [ 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-10T23:17:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "pacific": [ 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-10T23:17:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "quincy": [ 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "reef": [ 2024-09-10T23:17:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "squid": [ 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "balancer", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "crash", 2024-09-10T23:17:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "devicehealth", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "orchestrator", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "pg_autoscaler", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "progress", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "rbd_support", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "status", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "telemetry", 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "volumes" 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "force_disabled_modules": {}, 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "last_failure_osd_epoch": 3, 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "active_clients": [ 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "devicehealth", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1997299028 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "libcephsqlite", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-10T23:17:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 2795140245 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "rbd_support", 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 1977369792 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-09-10T23:17:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "name": "volumes", 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "addrvec": [ 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type": "v2", 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "addr": "172.21.15.120:0", 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "nonce": 148579461 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-09-10T23:17:11.771 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 msgr2=0x7fdad00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 0x7fdad00b77a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdad4005d00 tx=0x7fdad4005ed0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 msgr2=0x7fdae406bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009a50 tx=0x7fdae00042f0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 shutdown_connections 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fdad00b52e0 0x7fdad00b77a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41033a0 0x7fdae41a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae41029e0 0x7fdae419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 --2- 172.21.15.120:0/932170801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4101890 0x7fdae406bbd0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 >> 172.21.15.120:0/932170801 conn(0x7fdae40fd150 msgr2=0x7fdae4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.333+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 shutdown_connections 2024-09-10T23:17:11.773 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:10.334+0000 7fdaeb2be640 1 -- 172.21.15.120:0/932170801 wait complete. 2024-09-10T23:17:11.780 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T23:17:11.931 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T23:17:11.931 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T23:17:11.931 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:17:11.932 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/235923607' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:17:11.932 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:10 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/932170801' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:17:11.933 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:10 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/235923607' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:17:11.933 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:10 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/932170801' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:17:11.933 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:10 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/235923607' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:17:11.933 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:10 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/932170801' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:17:11.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:11 smithi120 ceph-mon[31247]: pgmap v1089: 97 pgs: 97 active+clean; 177 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2024-09-10T23:17:11.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:11 smithi153 ceph-mon[34951]: pgmap v1089: 97 pgs: 97 active+clean; 177 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2024-09-10T23:17:11.952 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-09-10T23:17:11.952 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:17:12.012 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T23:17:12.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:11 smithi176 ceph-mon[34982]: pgmap v1089: 97 pgs: 97 active+clean; 177 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2024-09-10T23:17:12.310 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.919+0000 7f65da446640 1 -- 172.21.15.120:0/3623132718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 msgr2=0x7f65d4075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.919+0000 7f65da446640 1 --2- 172.21.15.120:0/3623132718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d4075920 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009900 tx=0x7f65c402a410 comp rx=0 tx=0).stop 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.919+0000 7f65da446640 1 -- 172.21.15.120:0/3623132718 shutdown_connections 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.919+0000 7f65da446640 1 --2- 172.21.15.120:0/3623132718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4113b20 0x7f65d4115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.919+0000 7f65da446640 1 --2- 172.21.15.120:0/3623132718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4075e60 0x7f65d41135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:12.957 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 --2- 172.21.15.120:0/3623132718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d4075920 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- 172.21.15.120:0/3623132718 >> 172.21.15.120:0/3623132718 conn(0x7f65d40fe830 msgr2=0x7f65d4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- 172.21.15.120:0/3623132718 shutdown_connections 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- 172.21.15.120:0/3623132718 wait complete. 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 Processor -- start 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- start start 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 0x7f65d419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4113b20 0x7f65d41a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4119460 con 0x7f65d4075540 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f65d41192e0 con 0x7f65d4075e60 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.920+0000 7f65da446640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f65d41195e0 con 0x7f65d4113b20 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:12.958 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:48828/0 (socket says 172.21.15.120:48828) 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 -- 172.21.15.120:0/3591650508 learned_addr learned my addr 172.21.15.120:0/3591650508 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d89e3640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4113b20 0x7f65d41a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d37fe640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 0x7f65d419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 -- 172.21.15.120:0/3591650508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4113b20 msgr2=0x7f65d41a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4113b20 0x7f65d41a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 -- 172.21.15.120:0/3591650508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 msgr2=0x7f65d419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 0x7f65d419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 -- 172.21.15.120:0/3591650508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d41a59e0 con 0x7f65d4075540 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d37fe640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 0x7f65d419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d3fff640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005ab0 tx=0x7f65c4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c403f070 con 0x7f65d4075540 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d41a5c70 con 0x7f65d4075540 2024-09-10T23:17:12.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c4031d60 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4035030 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.921+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d41ac390 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.922+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d4075920 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.924+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f65c403b580 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.924+0000 7f65d17fa640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 0x7f65bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.924+0000 7f65d37fe640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 0x7f65bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.924+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f65c4106da0 con 0x7f65d4075540 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.925+0000 7f65d37fe640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 0x7f65bc0b77a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:12.960 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:12.927+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65c40cdc90 con 0x7f65d4075540 2024-09-10T23:17:13.121 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.121+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65d41ac650 con 0x7f65d4075540 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.121+0000 7f65d17fa640 1 -- 172.21.15.120:0/3591650508 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+599 (secure 0 0 0) 0x7f65c40cd590 con 0x7f65d4075540 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6800/3131521405 2024-09-11T22:43:14.065833+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/1124015461 2024-09-11T22:43:14.065833+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/959921448 2024-09-11T22:43:14.065833+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/1170023971 2024-09-11T22:43:14.065833+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6801/872559236 2024-09-11T22:42:50.195302+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6800/872559236 2024-09-11T22:42:50.195302+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:6801/3131521405 2024-09-11T22:43:14.065833+0000 2024-09-10T23:17:13.122 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/2219003248 2024-09-11T22:42:50.195302+0000 2024-09-10T23:17:13.123 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/2465372008 2024-09-11T22:42:50.195302+0000 2024-09-10T23:17:13.123 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120:0/1252718387 2024-09-11T22:42:50.195302+0000 2024-09-10T23:17:13.123 INFO:teuthology.orchestra.run.smithi120.stderr:listed 10 entries 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 msgr2=0x7f65bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 0x7f65bc0b77a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8008040 comp rx=0 tx=0).stop 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 msgr2=0x7f65d419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005ab0 tx=0x7f65c4003cd0 comp rx=0 tx=0).stop 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 shutdown_connections 2024-09-10T23:17:13.124 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f65bc0b52e0 0x7f65bc0b77a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65d4113b20 0x7f65d41a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65d4075e60 0x7f65d419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 --2- 172.21.15.120:0/3591650508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65d4075540 0x7f65d419f930 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.123+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 >> 172.21.15.120:0/3591650508 conn(0x7f65d40fe830 msgr2=0x7f65d40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.124+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 shutdown_connections 2024-09-10T23:17:13.125 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:13.124+0000 7f65da446640 1 -- 172.21.15.120:0/3591650508 wait complete. 2024-09-10T23:17:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:13 smithi120 ceph-mon[31247]: pgmap v1090: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-09-10T23:17:13.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:13 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3591650508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:17:13.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:13 smithi153 ceph-mon[34951]: pgmap v1090: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-09-10T23:17:13.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:13 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3591650508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:17:14.007 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T23:17:14.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:17:14.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T23:17:14.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:13 smithi176 ceph-mon[34982]: pgmap v1090: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-09-10T23:17:14.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:13 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3591650508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:17:15.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T23:17:15.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:17:15.640 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:17:15.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:17:15.694 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-09-10T23:17:15.744 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:17:15.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:17:15.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:17:15.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:17:15.814 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-09-10T23:17:15.859 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:15 smithi120 ceph-mon[31247]: pgmap v1091: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-09-10T23:17:15.884 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T23:17:15.896 INFO:tasks.cephadm:Teardown begin 2024-09-10T23:17:15.896 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:17:15.896 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:15.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:15 smithi153 ceph-mon[34951]: pgmap v1091: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-09-10T23:17:16.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:15 smithi176 ceph-mon[34982]: pgmap v1091: 97 pgs: 97 active+clean; 98 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 254 op/s 2024-09-10T23:17:16.264 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.960+0000 7f7d367f3640 1 -- 172.21.15.120:0/2537837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30101250 msgr2=0x7f7d30101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.960+0000 7f7d367f3640 1 --2- 172.21.15.120:0/2537837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30101250 0x7f7d30101630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009900 tx=0x7f7d2002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 -- 172.21.15.120:0/2537837128 shutdown_connections 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 --2- 172.21.15.120:0/2537837128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d30105f40 0x7f7d3010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 --2- 172.21.15.120:0/2537837128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d301055a0 0x7f7d30105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 --2- 172.21.15.120:0/2537837128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30101250 0x7f7d30101630 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009900 tx=0x7f7d2002a3f0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 -- 172.21.15.120:0/2537837128 >> 172.21.15.120:0/2537837128 conn(0x7f7d30078140 msgr2=0x7f7d300ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 -- 172.21.15.120:0/2537837128 shutdown_connections 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.961+0000 7f7d367f3640 1 -- 172.21.15.120:0/2537837128 wait complete. 2024-09-10T23:17:16.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.962+0000 7f7d367f3640 1 Processor -- start 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.962+0000 7f7d367f3640 1 -- start start 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 0x7f7d301125b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30105f40 0x7f7d30112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 0x7f7d30118340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30103630 con 0x7f7d30105f40 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301034b0 con 0x7f7d30117e90 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d367f3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301037b0 con 0x7f7d301055a0 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30105f40 0x7f7d30112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 0x7f7d301125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:16.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 0x7f7d301125b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.120:44222/0 (socket says 172.21.15.120:44222) 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d2ffff640 1 -- 172.21.15.120:0/1532646012 learned_addr learned my addr 172.21.15.120:0/1532646012 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 0x7f7d30118340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 -- 172.21.15.120:0/1532646012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 msgr2=0x7f7d301125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 0x7f7d301125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 -- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30105f40 msgr2=0x7f7d30112af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30105f40 0x7f7d30112af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.963+0000 7f7d34d90640 1 -- 172.21.15.120:0/1532646012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d30118a70 con 0x7f7d30117e90 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d34d90640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 0x7f7d30118340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d28005fd0 tx=0x7f7d2800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:16.968 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800cd80 con 0x7f7d30117e90 2024-09-10T23:17:16.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3011d020 con 0x7f7d30117e90 2024-09-10T23:17:16.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3011d560 con 0x7f7d30117e90 2024-09-10T23:17:16.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28005020 con 0x7f7d30117e90 2024-09-10T23:17:16.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.964+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28010690 con 0x7f7d30117e90 2024-09-10T23:17:16.969 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.965+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3006a820 con 0x7f7d30117e90 2024-09-10T23:17:16.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.967+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7d280054d0 con 0x7f7d30117e90 2024-09-10T23:17:16.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.968+0000 7f7d2d7fa640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 0x7f7d180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:16.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.969+0000 7f7d2ffff640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 0x7f7d180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:16.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.969+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7f7d280e6180 con 0x7f7d30117e90 2024-09-10T23:17:16.970 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.969+0000 7f7d2ffff640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 0x7f7d180b7820 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7d20031440 tx=0x7f7d20005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:16.973 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:16.973+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d280ad010 con 0x7f7d30117e90 2024-09-10T23:17:17.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.193+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d30101630 con 0x7f7d30117e90 2024-09-10T23:17:17.194 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.194+0000 7f7d2d7fa640 1 -- 172.21.15.120:0/1532646012 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7f7d30101630 con 0x7f7d30117e90 2024-09-10T23:17:17.195 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:17.195 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:17:17.195 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 msgr2=0x7f7d180b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 0x7f7d180b7820 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7d20031440 tx=0x7f7d20005a40 comp rx=0 tx=0).stop 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 msgr2=0x7f7d30118340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 0x7f7d30118340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7d28005fd0 tx=0x7f7d2800ee60 comp rx=0 tx=0).stop 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 shutdown_connections 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f7d180b5360 0x7f7d180b7820 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d30117e90 0x7f7d30118340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:17.197 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d30105f40 0x7f7d30112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:17.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 --2- 172.21.15.120:0/1532646012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d301055a0 0x7f7d301125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:17.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 >> 172.21.15.120:0/1532646012 conn(0x7f7d30078140 msgr2=0x7f7d301065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:17.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 shutdown_connections 2024-09-10T23:17:17.198 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:17.196+0000 7f7d367f3640 1 -- 172.21.15.120:0/1532646012 wait complete. 2024-09-10T23:17:17.822 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:17 smithi120 ceph-mon[31247]: pgmap v1092: 97 pgs: 97 active+clean; 139 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.3 MiB/s wr, 365 op/s 2024-09-10T23:17:17.823 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:17 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1532646012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:17.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:17:17.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:17 smithi153 ceph-mon[34951]: pgmap v1092: 97 pgs: 97 active+clean; 139 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.3 MiB/s wr, 365 op/s 2024-09-10T23:17:17.949 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:17 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1532646012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:17 smithi176 ceph-mon[34982]: pgmap v1092: 97 pgs: 97 active+clean; 139 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.3 MiB/s wr, 365 op/s 2024-09-10T23:17:18.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:17 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1532646012' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:18.102 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1762894682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c8108020 msgr2=0x7fb1c810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1762894682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c8108020 0x7fb1c810e820 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009ca0 tx=0x7fb1c002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1762894682 shutdown_connections 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1762894682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c8108020 0x7fb1c810e820 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1762894682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c81076a0 0x7fb1c8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1762894682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1762894682 >> 172.21.15.120:0/1762894682 conn(0x7fb1c80fd6e0 msgr2=0x7fb1c80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:18.585 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1762894682 shutdown_connections 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.572+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1762894682 wait complete. 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 Processor -- start 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 -- start start 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c8108020 0x7fb1c819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111ce0 con 0x7fb1c81076a0 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111b60 con 0x7fb1c8108020 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cf52b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111e60 con 0x7fb1c8101c90 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47172/0 (socket says 172.21.15.120:47172) 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 -- 172.21.15.120:0/1310064103 learned_addr learned my addr 172.21.15.120:0/1310064103 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:18.586 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cdac8640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c8108020 0x7fb1c819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cd2c7640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 -- 172.21.15.120:0/1310064103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 msgr2=0x7fb1c8199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 -- 172.21.15.120:0/1310064103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c8108020 msgr2=0x7fb1c819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c8108020 0x7fb1c819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1ccac6640 1 -- 172.21.15.120:0/1310064103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c819f2d0 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.573+0000 7fb1cd2c7640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1ccac6640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c920 tx=0x7fb1b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8007c70 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c819f5c0 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b8007e10 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8018820 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.574+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81a5c80 con 0x7fb1c81076a0 2024-09-10T23:17:18.587 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.575+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c8102070 con 0x7fb1c81076a0 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.576+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb1b800f450 con 0x7fb1c81076a0 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.577+0000 7fb1be7fc640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 0x7fb1b40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.577+0000 7fb1cd2c7640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 0x7fb1b40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.577+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7fb1b80e5f10 con 0x7fb1c81076a0 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.577+0000 7fb1cd2c7640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 0x7fb1b40b77a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40058c0 tx=0x7fb1c4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:18.588 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.580+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1b8014030 con 0x7fb1c81076a0 2024-09-10T23:17:18.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.790+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c819a350 con 0x7fb1c81076a0 2024-09-10T23:17:18.792 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.791+0000 7fb1be7fc640 1 -- 172.21.15.120:0/1310064103 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11865 (secure 0 0 0) 0x7fb1b80ace90 con 0x7fb1c81076a0 2024-09-10T23:17:18.792 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:18.792 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":26,"btime":"2024-09-10T22:57:58:349570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14673,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/1392508310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1392508310},{"type":"v1","addr":"172.21.15.120:6835","nonce":1392508310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3319509867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":3319509867},{"type":"v1","addr":"172.21.15.120:6837","nonce":3319509867}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/987205566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":987205566},{"type":"v1","addr":"172.21.15.120:6839","nonce":987205566}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6841/420929141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6840","nonce":420929141},{"type":"v1","addr":"172.21.15.120:6841","nonce":420929141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/811375010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":811375010},{"type":"v1","addr":"172.21.15.153:6833","nonce":811375010}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1790379992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1790379992},{"type":"v1","addr":"172.21.15.153:6835","nonce":1790379992}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1099927074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1099927074},{"type":"v1","addr":"172.21.15.176:6833","nonce":1099927074}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/1833265650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":1833265650},{"type":"v1","addr":"172.21.15.153:6837","nonce":1833265650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/2945283653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":2945283653},{"type":"v1","addr":"172.21.15.153:6839","nonce":2945283653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/4233369048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":4233369048},{"type":"v1","addr":"172.21.15.176:6835","nonce":4233369048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:50:27.621402+0000","modified":"2024-09-10T22:57:58.349546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24550":{"gid":24550,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/3141658633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":3141658633},{"type":"v1","addr":"172.21.15.176:6839","nonce":3141658633}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25921":{"gid":25921,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/458342810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":458342810},{"type":"v1","addr":"172.21.15.176:6837","nonce":458342810}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24550,"qdb_cluster":[24550]},"id":1}]} 2024-09-10T23:17:18.793 INFO:teuthology.orchestra.run.smithi120.stderr:dumped fsmap epoch 26 2024-09-10T23:17:18.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 msgr2=0x7fb1b40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:18.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 0x7fb1b40b77a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40058c0 tx=0x7fb1c4005d30 comp rx=0 tx=0).stop 2024-09-10T23:17:18.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 msgr2=0x7fb1c8199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:18.794 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c920 tx=0x7fb1b800cde0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 shutdown_connections 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7fb1b40b52e0 0x7fb1b40b77a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1c8108020 0x7fb1c819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81076a0 0x7fb1c8199760 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 --2- 172.21.15.120:0/1310064103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1c8101c90 0x7fb1c8199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 >> 172.21.15.120:0/1310064103 conn(0x7fb1c80fd6e0 msgr2=0x7fb1c806b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 shutdown_connections 2024-09-10T23:17:18.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:18.794+0000 7fb1cf52b640 1 -- 172.21.15.120:0/1310064103 wait complete. 2024-09-10T23:17:19.453 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T23:17:19.702 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:19.944 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:19 smithi120 ceph-mon[31247]: pgmap v1093: 97 pgs: 97 active+clean; 127 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-09-10T23:17:19.944 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:19 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1310064103' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:19.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:19 smithi153 ceph-mon[34951]: pgmap v1093: 97 pgs: 97 active+clean; 127 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-09-10T23:17:19.948 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:19 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1310064103' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:20.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:19 smithi176 ceph-mon[34982]: pgmap v1093: 97 pgs: 97 active+clean; 127 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 9.6 MiB/s wr, 313 op/s 2024-09-10T23:17:20.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:19 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1310064103' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:17:20.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 -- 172.21.15.120:0/2301000072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 msgr2=0x7efc001020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 --2- 172.21.15.120:0/2301000072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001020b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009900 tx=0x7efbf802a410 comp rx=0 tx=0).stop 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 -- 172.21.15.120:0/2301000072 shutdown_connections 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 --2- 172.21.15.120:0/2301000072 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 --2- 172.21.15.120:0/2301000072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 0x7efc00107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 --2- 172.21.15.120:0/2301000072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001020b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 -- 172.21.15.120:0/2301000072 >> 172.21.15.120:0/2301000072 conn(0x7efc000fd700 msgr2=0x7efc000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 -- 172.21.15.120:0/2301000072 shutdown_connections 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 -- 172.21.15.120:0/2301000072 wait complete. 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.155+0000 7efc05033640 1 Processor -- start 2024-09-10T23:17:20.177 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 -- start start 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 0x7efc00199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112e40 con 0x7efc00101cd0 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112cc0 con 0x7efc001076e0 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efc05033640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7efc00112fc0 con 0x7efc00108060 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47188/0 (socket says 172.21.15.120:47188) 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 -- 172.21.15.120:0/2616282416 learned_addr learned my addr 172.21.15.120:0/2616282416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfe59c640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 0x7efc00199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:20.178 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbff59e640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 -- 172.21.15.120:0/2616282416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 msgr2=0x7efc0019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 -- 172.21.15.120:0/2616282416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 msgr2=0x7efc00199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 0x7efc00199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbfed9d640 1 -- 172.21.15.120:0/2616282416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0019f3a0 con 0x7efc00101cd0 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.156+0000 7efbff59e640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:17:20.179 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efbfed9d640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005940 tx=0x7efbf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf803f030 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0019f630 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf8003cb0 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8035070 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.157+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc001a5d50 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.158+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc001020b0 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.159+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7efbf803bb70 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.160+0000 7efbe7fff640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 0x7efbe80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.160+0000 7efbfe59c640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 0x7efbe80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.160+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7efbf81067d0 con 0x7efc00101cd0 2024-09-10T23:17:20.180 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.160+0000 7efbfe59c640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 0x7efbe80b77a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efc00106e80 tx=0x7efbf4009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:20.181 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.164+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efbf80cd6c0 con 0x7efc00101cd0 2024-09-10T23:17:20.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.345+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc001098b0 con 0x7efc00101cd0 2024-09-10T23:17:20.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.346+0000 7efbe7fff640 1 -- 172.21.15.120:0/2616282416 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22746 (secure 0 0 0) 0x7efbf80d4e20 con 0x7efc00101cd0 2024-09-10T23:17:20.347 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-09-10T23:17:20.347 INFO:teuthology.orchestra.run.smithi120.stdout:{"epoch":81,"fsid":"d2838936-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:42:26.976696+0000","modified":"2024-09-10T22:57:58.168156+0000","last_up_change":"2024-09-10T22:49:39.201724+0000","last_in_change":"2024-09-10T22:49:16.857274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:45:56.375659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:50:23.901262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:50:25.937324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1918891-2bc5-4422-b3ff-eaf029ff3161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6803","nonce":2370065006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6805","nonce":2370065006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6809","nonce":2370065006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":2370065006},{"type":"v1","addr":"172.21.15.120:6807","nonce":2370065006}]},"public_addr":"172.21.15.120:6803/2370065006","cluster_addr":"172.21.15.120:6805/2370065006","heartbeat_back_addr":"172.21.15.120:6809/2370065006","heartbeat_front_addr":"172.21.15.120:6807/2370065006","state":["exists","up"]},{"osd":1,"uuid":"8cb0107b-f7e1-48bd-8916-bd98f0075444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6801","nonce":2556493584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6803","nonce":2556493584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6807","nonce":2556493584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2556493584},{"type":"v1","addr":"172.21.15.153:6805","nonce":2556493584}]},"public_addr":"172.21.15.153:6801/2556493584","cluster_addr":"172.21.15.153:6803/2556493584","heartbeat_back_addr":"172.21.15.153:6807/2556493584","heartbeat_front_addr":"172.21.15.153:6805/2556493584","state":["exists","up"]},{"osd":2,"uuid":"fcf71dbb-0b3e-4778-9735-44deb5142b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6801","nonce":1904007733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6803","nonce":1904007733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6807","nonce":1904007733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1904007733},{"type":"v1","addr":"172.21.15.176:6805","nonce":1904007733}]},"public_addr":"172.21.15.176:6801/1904007733","cluster_addr":"172.21.15.176:6803/1904007733","heartbeat_back_addr":"172.21.15.176:6807/1904007733","heartbeat_front_addr":"172.21.15.176:6805/1904007733","state":["exists","up"]},{"osd":3,"uuid":"422694c8-3c8d-4cae-987f-829b757f91eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6811","nonce":1845328035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6813","nonce":1845328035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6817","nonce":1845328035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":1845328035},{"type":"v1","addr":"172.21.15.120:6815","nonce":1845328035}]},"public_addr":"172.21.15.120:6811/1845328035","cluster_addr":"172.21.15.120:6813/1845328035","heartbeat_back_addr":"172.21.15.120:6817/1845328035","heartbeat_front_addr":"172.21.15.120:6815/1845328035","state":["exists","up"]},{"osd":4,"uuid":"b5b35613-0bf7-4f22-8640-e73ac586d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6809","nonce":2098592997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6811","nonce":2098592997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6815","nonce":2098592997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":2098592997},{"type":"v1","addr":"172.21.15.153:6813","nonce":2098592997}]},"public_addr":"172.21.15.153:6809/2098592997","cluster_addr":"172.21.15.153:6811/2098592997","heartbeat_back_addr":"172.21.15.153:6815/2098592997","heartbeat_front_addr":"172.21.15.153:6813/2098592997","state":["exists","up"]},{"osd":5,"uuid":"c3110d3e-1def-43cf-afae-30ca5fb29647","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6809","nonce":2804384590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6811","nonce":2804384590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6815","nonce":2804384590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2804384590},{"type":"v1","addr":"172.21.15.176:6813","nonce":2804384590}]},"public_addr":"172.21.15.176:6809/2804384590","cluster_addr":"172.21.15.176:6811/2804384590","heartbeat_back_addr":"172.21.15.176:6815/2804384590","heartbeat_front_addr":"172.21.15.176:6813/2804384590","state":["exists","up"]},{"osd":6,"uuid":"8062f8c3-d12f-4df5-8ef5-4286a273c161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6819","nonce":2500547812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6821","nonce":2500547812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6825","nonce":2500547812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2500547812},{"type":"v1","addr":"172.21.15.120:6823","nonce":2500547812}]},"public_addr":"172.21.15.120:6819/2500547812","cluster_addr":"172.21.15.120:6821/2500547812","heartbeat_back_addr":"172.21.15.120:6825/2500547812","heartbeat_front_addr":"172.21.15.120:6823/2500547812","state":["exists","up"]},{"osd":7,"uuid":"44778377-474e-4621-bd97-1cb74a97869f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6817","nonce":1702194586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6819","nonce":1702194586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6823","nonce":1702194586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":1702194586},{"type":"v1","addr":"172.21.15.153:6821","nonce":1702194586}]},"public_addr":"172.21.15.153:6817/1702194586","cluster_addr":"172.21.15.153:6819/1702194586","heartbeat_back_addr":"172.21.15.153:6823/1702194586","heartbeat_front_addr":"172.21.15.153:6821/1702194586","state":["exists","up"]},{"osd":8,"uuid":"9699f84b-810d-46f9-b062-ea48812dafea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6817","nonce":2769671146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6819","nonce":2769671146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6823","nonce":2769671146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2769671146},{"type":"v1","addr":"172.21.15.176:6821","nonce":2769671146}]},"public_addr":"172.21.15.176:6817/2769671146","cluster_addr":"172.21.15.176:6819/2769671146","heartbeat_back_addr":"172.21.15.176:6823/2769671146","heartbeat_front_addr":"172.21.15.176:6821/2769671146","state":["exists","up"]},{"osd":9,"uuid":"6f143c9a-4955-4d2d-9670-42a73378a865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6827","nonce":283565809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6829","nonce":283565809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6833","nonce":283565809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":283565809},{"type":"v1","addr":"172.21.15.120:6831","nonce":283565809}]},"public_addr":"172.21.15.120:6827/283565809","cluster_addr":"172.21.15.120:6829/283565809","heartbeat_back_addr":"172.21.15.120:6833/283565809","heartbeat_front_addr":"172.21.15.120:6831/283565809","state":["exists","up"]},{"osd":10,"uuid":"7801ec3e-95f4-4dcb-9134-5d9dbf72145f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6825","nonce":718699262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6827","nonce":718699262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6831","nonce":718699262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":718699262},{"type":"v1","addr":"172.21.15.153:6829","nonce":718699262}]},"public_addr":"172.21.15.153:6825/718699262","cluster_addr":"172.21.15.153:6827/718699262","heartbeat_back_addr":"172.21.15.153:6831/718699262","heartbeat_front_addr":"172.21.15.153:6829/718699262","state":["exists","up"]},{"osd":11,"uuid":"bbced0d0-788b-48aa-9271-9cc721683b01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6825","nonce":2240071367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6827","nonce":2240071367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6831","nonce":2240071367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":2240071367},{"type":"v1","addr":"172.21.15.176:6829","nonce":2240071367}]},"public_addr":"172.21.15.176:6825/2240071367","cluster_addr":"172.21.15.176:6827/2240071367","heartbeat_back_addr":"172.21.15.176:6831/2240071367","heartbeat_front_addr":"172.21.15.176:6829/2240071367","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:03.083237+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:27.266397+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:45:52.747632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:18.992473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:46:43.398255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:07.598026+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:33.675362+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:47:58.765593+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:22.989889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:48:48.347402+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:13.414058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:49:37.803123+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6800/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1124015461":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/959921448":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/1170023971":"2024-09-11T22:43:14.065833+0000","172.21.15.120:6801/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6800/872559236":"2024-09-11T22:42:50.195302+0000","172.21.15.120:6801/3131521405":"2024-09-11T22:43:14.065833+0000","172.21.15.120:0/2219003248":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/2465372008":"2024-09-11T22:42:50.195302+0000","172.21.15.120:0/1252718387":"2024-09-11T22:42:50.195302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 msgr2=0x7efbe80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 0x7efbe80b77a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efc00106e80 tx=0x7efbf4009520 comp rx=0 tx=0).stop 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 msgr2=0x7efc001992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005940 tx=0x7efbf80042f0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 shutdown_connections 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7efbe80b52e0 0x7efbe80b77a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efc00108060 0x7efc0019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc001076e0 0x7efc00199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 --2- 172.21.15.120:0/2616282416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc00101cd0 0x7efc001992f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:20.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 >> 172.21.15.120:0/2616282416 conn(0x7efc000fd700 msgr2=0x7efc000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:20.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 shutdown_connections 2024-09-10T23:17:20.350 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:20.348+0000 7efc05033640 1 -- 172.21.15.120:0/2616282416 wait complete. 2024-09-10T23:17:21.072 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:20 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2616282416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:21.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:20 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2616282416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:21.101 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T23:17:21.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:20 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2616282416' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:17:21.349 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:21.759 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:21 smithi120 ceph-mon[31247]: pgmap v1094: 97 pgs: 97 active+clean; 65 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 9.4 MiB/s wr, 362 op/s 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.838+0000 7ffbf476a640 1 -- 172.21.15.120:0/1632263340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec103b70 msgr2=0x7ffbec103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.838+0000 7ffbf476a640 1 --2- 172.21.15.120:0/1632263340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec103b70 0x7ffbec103f50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009ca0 tx=0x7ffbdc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- 172.21.15.120:0/1632263340 shutdown_connections 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- 172.21.15.120:0/1632263340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec10d120 0x7ffbec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- 172.21.15.120:0/1632263340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 0x7ffbec10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- 172.21.15.120:0/1632263340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec103b70 0x7ffbec103f50 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- 172.21.15.120:0/1632263340 >> 172.21.15.120:0/1632263340 conn(0x7ffbec0fd700 msgr2=0x7ffbec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- 172.21.15.120:0/1632263340 shutdown_connections 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- 172.21.15.120:0/1632263340 wait complete. 2024-09-10T23:17:21.860 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 Processor -- start 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- start start 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec103b70 0x7ffbec199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 0x7ffbec199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec112e20 con 0x7ffbec10d120 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec112ca0 con 0x7ffbec103b70 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.839+0000 7ffbf476a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec112fa0 con 0x7ffbec104570 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:47204/0 (socket says 172.21.15.120:47204) 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 -- 172.21.15.120:0/2897742616 learned_addr learned my addr 172.21.15.120:0/2897742616 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2506640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec103b70 0x7ffbec199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf1d05640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 0x7ffbec199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:21.861 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 -- 172.21.15.120:0/2897742616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 msgr2=0x7ffbec199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 0x7ffbec199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 -- 172.21.15.120:0/2897742616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec103b70 msgr2=0x7ffbec199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec103b70 0x7ffbec199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 -- 172.21.15.120:0/2897742616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec19f4c0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf2d07640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009fd0 tx=0x7ffbe400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4011830 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec19f7b0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe4016070 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4010de0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.840+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec1a5ef0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.841+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbec103fd0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.843+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ffbe40119d0 con 0x7ffbec10d120 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.843+0000 7ffbeb7fe640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 0x7ffbd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:21.862 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.843+0000 7ffbf2506640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 0x7ffbd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:21.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.843+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8182+0+0 (secure 0 0 0) 0x7ffbe40e57b0 con 0x7ffbec10d120 2024-09-10T23:17:21.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.844+0000 7ffbf2506640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 0x7ffbd40b77a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0098d0 tx=0x7ffbdc009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:21.863 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:21.846+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffbe40ac6a0 con 0x7ffbec10d120 2024-09-10T23:17:22.057 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.057+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffbec02e5c0 con 0x7ffbec10d120 2024-09-10T23:17:22.060 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.059+0000 7ffbeb7fe640 1 -- 172.21.15.120:0/2897742616 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7ffbe40abfa0 con 0x7ffbec10d120 2024-09-10T23:17:22.060 INFO:teuthology.orchestra.run.smithi120.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:17:22.062 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 msgr2=0x7ffbd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 0x7ffbd40b77a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0098d0 tx=0x7ffbdc009900 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 msgr2=0x7ffbec19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009fd0 tx=0x7ffbe400b8e0 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 shutdown_connections 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7ffbd40b52e0 0x7ffbd40b77a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffbec10d120 0x7ffbec19ecf0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffbec104570 0x7ffbec199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 --2- 172.21.15.120:0/2897742616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbec103b70 0x7ffbec199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:22.063 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 >> 172.21.15.120:0/2897742616 conn(0x7ffbec0fd700 msgr2=0x7ffbec0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:22.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 shutdown_connections 2024-09-10T23:17:22.064 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:22.062+0000 7ffbf476a640 1 -- 172.21.15.120:0/2897742616 wait complete. 2024-09-10T23:17:22.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:21 smithi176 ceph-mon[34982]: pgmap v1094: 97 pgs: 97 active+clean; 65 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 9.4 MiB/s wr, 362 op/s 2024-09-10T23:17:22.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:21 smithi153 ceph-mon[34951]: pgmap v1094: 97 pgs: 97 active+clean; 65 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 9.4 MiB/s wr, 362 op/s 2024-09-10T23:17:22.355 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a[31243]: 2024-09-10T23:17:22.057+0000 7f0593be3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setuser ceph since I am not root 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: ignoring --setgroup ceph since I am not root 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mds[64841]: -- 172.21.15.176:0/953709152 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfa175f860 con 0x55cfa173d000 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: starting mds.c at 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: ignoring --setuser ceph since I am not root 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: ignoring --setgroup ceph since I am not root 2024-09-10T23:17:22.573 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mds[65142]: -- 172.21.15.176:0/1196115386 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55890babd860 con 0x55890ba9b000 2024-09-10T23:17:22.574 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: starting mds.i at 2024-09-10T23:17:22.711 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T23:17:22.946 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:22 smithi120 ceph-mon[31247]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T23:17:22.960 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:23.072 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:17:23.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:17:23.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:17:23.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:17:23.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T23:17:23.073 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:22 smithi176 ceph-mon[34982]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T23:17:23.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:17:23.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:17:23.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:17:23.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/2897742616' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:17:23.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T23:17:23.199 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:22 smithi153 ceph-mon[34951]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T23:17:23.435 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 -- 172.21.15.120:0/3566680603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8102de0 msgr2=0x7f2ad81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 --2- 172.21.15.120:0/3566680603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8102de0 0x7f2ad81031c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009900 tx=0x7f2ad002a410 comp rx=0 tx=0).stop 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 -- 172.21.15.120:0/3566680603 shutdown_connections 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 --2- 172.21.15.120:0/3566680603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad8109170 0x7f2ad810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 --2- 172.21.15.120:0/3566680603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad81087f0 0x7f2ad8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 --2- 172.21.15.120:0/3566680603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8102de0 0x7f2ad81031c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 -- 172.21.15.120:0/3566680603 >> 172.21.15.120:0/3566680603 conn(0x7f2ad80fe830 msgr2=0x7f2ad8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 -- 172.21.15.120:0/3566680603 shutdown_connections 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 -- 172.21.15.120:0/3566680603 wait complete. 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.433+0000 7f2adec5b640 1 Processor -- start 2024-09-10T23:17:23.436 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 -- start start 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad8102de0 0x7f2ad8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 0x7f2ad81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8112e30 con 0x7f2ad8109170 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8112cb0 con 0x7f2ad81087f0 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adec5b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8112fb0 con 0x7f2ad8102de0 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40186/0 (socket says 172.21.15.120:40186) 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 -- 172.21.15.120:0/1310601057 learned_addr learned my addr 172.21.15.120:0/1310601057 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2ad7fff640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 0x7f2ad81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2adc9f7640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad8102de0 0x7f2ad8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 -- 172.21.15.120:0/1310601057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad8102de0 msgr2=0x7f2ad8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:23.437 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad8102de0 0x7f2ad8199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 -- 172.21.15.120:0/1310601057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 msgr2=0x7f2ad81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 0x7f2ad81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 -- 172.21.15.120:0/1310601057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad819f330 con 0x7f2ad8109170 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2ad7fff640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 0x7f2ad81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.434+0000 7f2add1f8640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d750 tx=0x7f2ac000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.435+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0017070 con 0x7f2ad8109170 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.435+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac00054d0 con 0x7f2ad8109170 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.435+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad819f620 con 0x7f2ad8109170 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.435+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0004c60 con 0x7f2ad8109170 2024-09-10T23:17:23.438 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.435+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81a5ce0 con 0x7f2ad8109170 2024-09-10T23:17:23.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.436+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad81031c0 con 0x7f2ad8109170 2024-09-10T23:17:23.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.437+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2ac0004e00 con 0x7f2ad8109170 2024-09-10T23:17:23.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.438+0000 7f2ad5ffb640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 0x7f2ac40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:23.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.438+0000 7f2adc9f7640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 0x7f2ac40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:23.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.438+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8268+0+0 (secure 0 0 0) 0x7f2ac00e5620 con 0x7f2ad8109170 2024-09-10T23:17:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.438+0000 7f2adc9f7640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 0x7f2ac40b7820 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0006010 tx=0x7f2ad0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:23.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.442+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2ac00ac4c0 con 0x7f2ad8109170 2024-09-10T23:17:23.653 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:23.653+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ad802e5c0 con 0x7f2ad8109170 2024-09-10T23:17:24.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.071+0000 7f2ad5ffb640 1 -- 172.21.15.120:0/1310601057 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f2ac00abdc0 con 0x7f2ad8109170 2024-09-10T23:17:24.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.073+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 msgr2=0x7f2ac40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:24.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.073+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 0x7f2ac40b7820 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0006010 tx=0x7f2ad0000bc0 comp rx=0 tx=0).stop 2024-09-10T23:17:24.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 msgr2=0x7f2ad819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d750 tx=0x7f2ac000dc10 comp rx=0 tx=0).stop 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 shutdown_connections 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f2ac40b5360 0x7f2ac40b7820 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ad8109170 0x7f2ad819eb60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ad81087f0 0x7f2ad81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 --2- 172.21.15.120:0/1310601057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ad8102de0 0x7f2ad8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 >> 172.21.15.120:0/1310601057 conn(0x7f2ad80fe830 msgr2=0x7f2ad8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 shutdown_connections 2024-09-10T23:17:24.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:24.074+0000 7f2adec5b640 1 -- 172.21.15.120:0/1310601057 wait complete. 2024-09-10T23:17:24.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: pgmap v1096: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 9.7 MiB/s wr, 358 op/s 2024-09-10T23:17:24.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6837/3957683355,v1:172.21.15.176:6839/3957683355] up:boot 2024-09-10T23:17:24.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: mds.? [v2:172.21.15.176:6836/1740526815,v1:172.21.15.176:6838/1740526815] up:boot 2024-09-10T23:17:24.397 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T23:17:24.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:17:24.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:17:24.398 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:24 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:17:24.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: pgmap v1096: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 9.7 MiB/s wr, 358 op/s 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6837/3957683355,v1:172.21.15.176:6839/3957683355] up:boot 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: mds.? [v2:172.21.15.176:6836/1740526815,v1:172.21.15.176:6838/1740526815] up:boot 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:17:24.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:24 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:17:24.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: pgmap v1096: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 9.7 MiB/s wr, 358 op/s 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6837/3957683355,v1:172.21.15.176:6839/3957683355] up:boot 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: mds.? [v2:172.21.15.176:6836/1740526815,v1:172.21.15.176:6838/1740526815] up:boot 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: from='mgr.14150 172.21.15.120:0/3806030352' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:17:24.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:24 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:17:24.836 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:17:25.081 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:25.359 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:25 smithi120 ceph-mon[31247]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:17:25.359 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:25 smithi120 ceph-mon[31247]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:25 smithi120 ceph-mon[31247]: Cluster is now healthy 2024-09-10T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:25 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:25 smithi120 ceph-mon[31247]: fsmap 12 up:standby 2024-09-10T23:17:25.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:25 smithi153 ceph-mon[34951]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:17:25.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:25 smithi153 ceph-mon[34951]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:17:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:25 smithi153 ceph-mon[34951]: Cluster is now healthy 2024-09-10T23:17:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:25 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:17:25.449 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:25 smithi153 ceph-mon[34951]: fsmap 12 up:standby 2024-09-10T23:17:25.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:25 smithi176 ceph-mon[34982]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:17:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:25 smithi176 ceph-mon[34982]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:17:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:25 smithi176 ceph-mon[34982]: Cluster is now healthy 2024-09-10T23:17:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:25 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/1310601057' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:17:25.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:25 smithi176 ceph-mon[34982]: fsmap 12 up:standby 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.560+0000 7f0dd3e70640 1 -- 172.21.15.120:0/530797243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 msgr2=0x7f0dcc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.560+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/530797243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc1031a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009ca0 tx=0x7f0dbc02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- 172.21.15.120:0/530797243 shutdown_connections 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/530797243 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc109150 0x7f0dcc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/530797243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc1087d0 0x7f0dcc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/530797243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc1031a0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- 172.21.15.120:0/530797243 >> 172.21.15.120:0/530797243 conn(0x7f0dcc0fe830 msgr2=0x7f0dcc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:25.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- 172.21.15.120:0/530797243 shutdown_connections 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- 172.21.15.120:0/530797243 wait complete. 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 Processor -- start 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- start start 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc1087d0 0x7f0dcc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc109150 0x7f0dcc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc112e10 con 0x7f0dcc102dc0 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc112c90 con 0x7f0dcc109150 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd3e70640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc112f90 con 0x7f0dcc1087d0 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.561+0000 7f0dd1c0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40200/0 (socket says 172.21.15.120:40200) 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 -- 172.21.15.120:0/657215255 learned_addr learned my addr 172.21.15.120:0/657215255 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:25.579 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd240d640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc109150 0x7f0dcc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd140b640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc1087d0 0x7f0dcc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 -- 172.21.15.120:0/657215255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc1087d0 msgr2=0x7f0dcc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc1087d0 0x7f0dcc1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 -- 172.21.15.120:0/657215255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc109150 msgr2=0x7f0dcc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc109150 0x7f0dcc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 -- 172.21.15.120:0/657215255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc19f310 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd1c0c640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc02ade0 tx=0x7f0dbc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc003d60 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc19f5a0 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc0044f0 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc02fd10 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.562+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc1a5d40 con 0x7f0dcc102dc0 2024-09-10T23:17:25.580 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.563+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc1031a0 con 0x7f0dcc102dc0 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.565+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0dbc004040 con 0x7f0dcc102dc0 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.565+0000 7f0dcaffd640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 0x7f0db40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.565+0000 7f0dd140b640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 0x7f0db40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.565+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8268+0+0 (secure 0 0 0) 0x7f0dbc107130 con 0x7f0dcc102dc0 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.566+0000 7f0dd140b640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 0x7f0db40b77a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0005e10 tx=0x7f0dc0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:25.581 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.569+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0dbc0cdfd0 con 0x7f0dcc102dc0 2024-09-10T23:17:25.746 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:25.746+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0dcc10af90 con 0x7f0dcc102dc0 2024-09-10T23:17:26.086 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.086+0000 7f0dcaffd640 1 -- 172.21.15.120:0/657215255 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f0dbc0cd8d0 con 0x7f0dcc102dc0 2024-09-10T23:17:26.087 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 msgr2=0x7f0db40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 0x7f0db40b77a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0005e10 tx=0x7f0dc0005da0 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 msgr2=0x7f0dcc199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc02ade0 tx=0x7f0dbc003970 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 shutdown_connections 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f0db40b52e0 0x7f0db40b77a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dcc109150 0x7f0dcc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0dcc1087d0 0x7f0dcc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 --2- 172.21.15.120:0/657215255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dcc102dc0 0x7f0dcc199260 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:26.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 >> 172.21.15.120:0/657215255 conn(0x7f0dcc0fe830 msgr2=0x7f0dcc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:26.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 shutdown_connections 2024-09-10T23:17:26.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:26.088+0000 7f0dd3e70640 1 -- 172.21.15.120:0/657215255 wait complete. 2024-09-10T23:17:26.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:26 smithi120 ceph-mon[31247]: pgmap v1097: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 9.5 MiB/s wr, 353 op/s 2024-09-10T23:17:26.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:26 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:26 smithi153 ceph-mon[34951]: pgmap v1097: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 9.5 MiB/s wr, 353 op/s 2024-09-10T23:17:26.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:26 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:26.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:26 smithi176 ceph-mon[34982]: pgmap v1097: 97 pgs: 97 active+clean; 27 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 9.5 MiB/s wr, 353 op/s 2024-09-10T23:17:26.607 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:26 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:26.824 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid d2838936-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:17:27.069 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:17:27.103 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:27 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:27.103 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:27 smithi120 ceph-mon[31247]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T23:17:27.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:27 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:27.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:27 smithi153 ceph-mon[34951]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T23:17:27.572 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:27 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/657215255' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:27.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:27 smithi176 ceph-mon[34982]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T23:17:27.614 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 -- 172.21.15.120:0/3599849459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520103ca0 msgr2=0x7f4520104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 --2- 172.21.15.120:0/3599849459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520103ca0 0x7f4520104080 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4508009900 tx=0x7f450802a410 comp rx=0 tx=0).stop 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 -- 172.21.15.120:0/3599849459 shutdown_connections 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 --2- 172.21.15.120:0/3599849459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f452010d100 0x7f452010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 --2- 172.21.15.120:0/3599849459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45201045c0 0x7f452010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 --2- 172.21.15.120:0/3599849459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520103ca0 0x7f4520104080 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 -- 172.21.15.120:0/3599849459 >> 172.21.15.120:0/3599849459 conn(0x7f45200fd700 msgr2=0x7f45200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 -- 172.21.15.120:0/3599849459 shutdown_connections 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 -- 172.21.15.120:0/3599849459 wait complete. 2024-09-10T23:17:27.615 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.612+0000 7f4526efd640 1 Processor -- start 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 -- start start 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4520103ca0 0x7f45201992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45201045c0 0x7f45201997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112d70 con 0x7f452010d100 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112bf0 con 0x7f45201045c0 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4526efd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112ef0 con 0x7f4520103ca0 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40216/0 (socket says 172.21.15.120:40216) 2024-09-10T23:17:27.616 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 -- 172.21.15.120:0/3790862337 learned_addr learned my addr 172.21.15.120:0/3790862337 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f451ffff640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45201045c0 0x7f45201997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f4524c99640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4520103ca0 0x7f45201992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 -- 172.21.15.120:0/3790862337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4520103ca0 msgr2=0x7f45201992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4520103ca0 0x7f45201992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 -- 172.21.15.120:0/3790862337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45201045c0 msgr2=0x7f45201997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45201045c0 0x7f45201997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 -- 172.21.15.120:0/3790862337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452019f350 con 0x7f452010d100 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f452549a640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f451800d950 tx=0x7f451800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.613+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518017030 con 0x7f452010d100 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.614+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518004020 con 0x7f452010d100 2024-09-10T23:17:27.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.614+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452019f640 con 0x7f452010d100 2024-09-10T23:17:27.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.614+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518003b50 con 0x7f452010d100 2024-09-10T23:17:27.618 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.614+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a5d00 con 0x7f452010d100 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.615+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4520104080 con 0x7f452010d100 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.617+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4518003cf0 con 0x7f452010d100 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.618+0000 7f451dffb640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 0x7f450c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.618+0000 7f4524c99640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 0x7f450c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.618+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7743+0+0 (secure 0 0 0) 0x7f45180e5720 con 0x7f452010d100 2024-09-10T23:17:27.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.619+0000 7f4524c99640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 0x7f450c0b77a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4508031440 tx=0x7f4508005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:17:27.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.620+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4518021070 con 0x7f452010d100 2024-09-10T23:17:27.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:27.799+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f452019a3d0 con 0x7f452010d100 2024-09-10T23:17:28.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.105+0000 7f451dffb640 1 -- 172.21.15.120:0/3790862337 <== mon.0 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f452019a3d0 con 0x7f452010d100 2024-09-10T23:17:28.107 INFO:teuthology.orchestra.run.smithi120.stderr:pool 'cephfs_data' removed 2024-09-10T23:17:28.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.109+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 msgr2=0x7f450c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:28.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.109+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 0x7f450c0b77a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4508031440 tx=0x7f4508005b50 comp rx=0 tx=0).stop 2024-09-10T23:17:28.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.109+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 msgr2=0x7f452019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:17:28.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.109+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f451800d950 tx=0x7f451800de10 comp rx=0 tx=0).stop 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 shutdown_connections 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:6800/578504839,v1:172.21.15.120:6801/578504839] conn(0x7f450c0b52e0 0x7f450c0b77a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010d100 0x7f452019eb80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45201045c0 0x7f45201997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 --2- 172.21.15.120:0/3790862337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4520103ca0 0x7f45201992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 >> 172.21.15.120:0/3790862337 conn(0x7f45200fd700 msgr2=0x7f45200fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 shutdown_connections 2024-09-10T23:17:28.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-09-10T23:17:28.110+0000 7f4526efd640 1 -- 172.21.15.120:0/3790862337 wait complete. 2024-09-10T23:17:28.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:28 smithi120 ceph-mon[31247]: pgmap v1099: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:17:28.415 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:28 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:28 smithi153 ceph-mon[34951]: pgmap v1099: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:17:28.448 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:28 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:28 smithi176 ceph-mon[34982]: pgmap v1099: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:17:28.573 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:28 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:17:28.836 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:17:28.862 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:17:28.895 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:17:28.993 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T23:17:28.993 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:17:29.008 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:17:29.023 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:17:29.049 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T23:17:29.049 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T23:17:29.050 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a 2024-09-10T23:17:29.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:29 smithi153 ceph-mon[34951]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:29.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:29 smithi153 ceph-mon[34951]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T23:17:29.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:29 smithi176 ceph-mon[34982]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:29.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:29 smithi176 ceph-mon[34982]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T23:17:29.337 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 systemd[1]: Stopping Ceph mon.a for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:29.337 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 ceph-mon[31247]: from='client.? 172.21.15.120:0/3790862337' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:17:29.338 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 ceph-mon[31247]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T23:17:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a[31243]: 2024-09-10T23:17:29.335+0000 7f05993ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a[31243]: 2024-09-10T23:17:29.335+0000 7f05993ee640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T23:17:29.696 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 podman[123514]: 2024-09-10 23:17:29.471418209 +0000 UTC m=+0.224529117 container died 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T23:17:30.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 podman[123514]: 2024-09-10 23:17:29.845662636 +0000 UTC m=+0.598773498 container cleanup 065319f19c34d917b1414ff8ac956c6baf3fb9b759a269cb898ee28a59d9ed46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:17:30.155 INFO:journalctl@ceph.mon.a.smithi120.stdout:Sep 10 23:17:29 smithi120 bash[123514]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-a 2024-09-10T23:17:30.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T23:17:30.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:30.378 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T23:17:30.378 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T23:17:30.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.b 2024-09-10T23:17:30.698 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:30 smithi153 systemd[1]: Stopping Ceph mon.b for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-b[34947]: 2024-09-10T23:17:30.773+0000 7f771f6ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-b[34947]: 2024-09-10T23:17:30.773+0000 7f771f6ef640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T23:17:31.198 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:30 smithi153 podman[71040]: 2024-09-10 23:17:30.907149362 +0000 UTC m=+0.337559283 container died 380d42ee9192e0d9a3cda51bcd4107ce17173433fdc7be485203468fa965e669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:17:31.643 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:31 smithi153 podman[71040]: 2024-09-10 23:17:31.425341307 +0000 UTC m=+0.855751227 container cleanup 380d42ee9192e0d9a3cda51bcd4107ce17173433fdc7be485203468fa965e669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T23:17:31.644 INFO:journalctl@ceph.mon.b.smithi153.stdout:Sep 10 23:17:31 smithi153 bash[71040]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-b 2024-09-10T23:17:31.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T23:17:31.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:31.901 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T23:17:31.901 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T23:17:31.901 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.c 2024-09-10T23:17:32.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:31 smithi176 systemd[1]: Stopping Ceph mon.c for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:32.606 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-c[34978]: 2024-09-10T23:17:32.339+0000 7f6887f0f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:32.606 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-c[34978]: 2024-09-10T23:17:32.339+0000 7f6887f0f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T23:17:32.607 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:32 smithi176 podman[72642]: 2024-09-10 23:17:32.605984481 +0000 UTC m=+0.415173318 container died bdd517694a843533252ed2b0dee3a700578e8042032dcf4f5f2a101e6dadbf7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:17:33.322 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:33 smithi176 podman[72642]: 2024-09-10 23:17:33.098773196 +0000 UTC m=+0.907962038 container cleanup bdd517694a843533252ed2b0dee3a700578e8042032dcf4f5f2a101e6dadbf7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T23:17:33.323 INFO:journalctl@ceph.mon.c.smithi176.stdout:Sep 10 23:17:33 smithi176 bash[72642]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mon-c 2024-09-10T23:17:33.504 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T23:17:33.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:33.551 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T23:17:33.551 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T23:17:33.551 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x 2024-09-10T23:17:33.905 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 23:17:33 smithi120 systemd[1]: Stopping Ceph mgr.x for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:34.160 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 23:17:33 smithi120 podman[123637]: 2024-09-10 23:17:33.905919198 +0000 UTC m=+0.158248364 container died cbfc6d50a4ef89faeca222bda3492d8aa498700b60869eade594e6290cd6d959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T23:17:34.795 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 23:17:34 smithi120 podman[123637]: 2024-09-10 23:17:34.493084007 +0000 UTC m=+0.745413171 container cleanup cbfc6d50a4ef89faeca222bda3492d8aa498700b60869eade594e6290cd6d959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T23:17:34.795 INFO:journalctl@ceph.mgr.x.smithi120.stdout:Sep 10 23:17:34 smithi120 bash[123637]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-x 2024-09-10T23:17:34.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T23:17:35.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:35.006 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T23:17:35.006 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T23:17:35.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.y 2024-09-10T23:17:35.326 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 23:17:35 smithi153 systemd[1]: Stopping Ceph mgr.y for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:35.631 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 23:17:35 smithi153 podman[71164]: 2024-09-10 23:17:35.326303796 +0000 UTC m=+0.123192385 container died ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:17:35.948 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 23:17:35 smithi153 podman[71164]: 2024-09-10 23:17:35.730862312 +0000 UTC m=+0.527750901 container cleanup ad94726c6825ee73be29d8d5706b872fd221a686985ec969ef900cd2bd5f0fe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T23:17:35.949 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Sep 10 23:17:35 smithi153 bash[71164]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-y 2024-09-10T23:17:36.097 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T23:17:36.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:36.132 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T23:17:36.132 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T23:17:36.132 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.z 2024-09-10T23:17:36.572 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 23:17:36 smithi176 systemd[1]: Stopping Ceph mgr.z for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:36.896 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 23:17:36 smithi176 podman[72768]: 2024-09-10 23:17:36.576098985 +0000 UTC m=+0.248602076 container died 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T23:17:37.322 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 23:17:37 smithi176 podman[72768]: 2024-09-10 23:17:37.020569741 +0000 UTC m=+0.693072823 container cleanup 7e60e763bb8d21b42864688cd9bce24db0909968cc2f69d1cf3061a1beaa1b74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:17:37.323 INFO:journalctl@ceph.mgr.z.smithi176.stdout:Sep 10 23:17:37 smithi176 bash[72768]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mgr-z 2024-09-10T23:17:37.463 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T23:17:37.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:37.499 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T23:17:37.499 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T23:17:37.499 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.0 2024-09-10T23:17:37.824 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:37 smithi120 systemd[1]: Stopping Ceph osd.0 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:38.196 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T23:17:37.821+0000 7f2f4c736640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:38.196 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T23:17:37.821+0000 7f2f4c736640 -1 osd.0 84 *** Got signal Terminated *** 2024-09-10T23:17:38.196 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0[48029]: 2024-09-10T23:17:37.821+0000 7f2f4c736640 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:17:43.690 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:43 smithi120 podman[123769]: 2024-09-10 23:17:43.404379147 +0000 UTC m=+5.720415744 container died 6fa927c931cbe948774fbafec7f861bf026ea2f95da330d83160a1ac5cd83fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:17:43.946 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:43 smithi120 podman[123769]: 2024-09-10 23:17:43.789701632 +0000 UTC m=+6.105738226 container cleanup 6fa927c931cbe948774fbafec7f861bf026ea2f95da330d83160a1ac5cd83fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T23:17:43.946 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:43 smithi120 bash[123769]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0 2024-09-10T23:17:44.446 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123786]: 2024-09-10 23:17:44.091707308 +0000 UTC m=+0.681688676 container remove 6fa927c931cbe948774fbafec7f861bf026ea2f95da330d83160a1ac5cd83fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T23:17:44.796 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123874]: 2024-09-10 23:17:44.36225828 +0000 UTC m=+0.016337712 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:17:44.796 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123874]: 2024-09-10 23:17:44.546920931 +0000 UTC m=+0.201000408 container create ce6b0868397601055a2228dd00ea4880fae893e2953194cadda9916fc5073c60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:17:45.100 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123874]: 2024-09-10 23:17:44.912127458 +0000 UTC m=+0.566206890 container init ce6b0868397601055a2228dd00ea4880fae893e2953194cadda9916fc5073c60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T23:17:45.100 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123874]: 2024-09-10 23:17:44.91559471 +0000 UTC m=+0.569674150 container start ce6b0868397601055a2228dd00ea4880fae893e2953194cadda9916fc5073c60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T23:17:45.100 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:44 smithi120 podman[123874]: 2024-09-10 23:17:44.995156771 +0000 UTC m=+0.649236214 container attach ce6b0868397601055a2228dd00ea4880fae893e2953194cadda9916fc5073c60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T23:17:45.399 INFO:journalctl@ceph.osd.0.smithi120.stdout:Sep 10 23:17:45 smithi120 podman[123874]: 2024-09-10 23:17:45.100326328 +0000 UTC m=+0.754405770 container died ce6b0868397601055a2228dd00ea4880fae893e2953194cadda9916fc5073c60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T23:17:45.723 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T23:17:45.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:45.765 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T23:17:45.765 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T23:17:45.765 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.1 2024-09-10T23:17:45.948 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:45 smithi153 systemd[1]: Stopping Ceph osd.1 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:46.448 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T23:17:46.159+0000 7f2b75fd6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:46.449 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T23:17:46.159+0000 7f2b75fd6640 -1 osd.1 84 *** Got signal Terminated *** 2024-09-10T23:17:46.449 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1[39402]: 2024-09-10T23:17:46.159+0000 7f2b75fd6640 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:17:51.948 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:51 smithi153 podman[71304]: 2024-09-10 23:17:51.656818657 +0000 UTC m=+5.703803646 container died bba203d55b4f47ce9117fc90c23409aa8627c4673998819c517c3716111ead23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:17:52.244 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71304]: 2024-09-10 23:17:52.042522179 +0000 UTC m=+6.089507170 container cleanup bba203d55b4f47ce9117fc90c23409aa8627c4673998819c517c3716111ead23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T23:17:52.244 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 bash[71304]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1 2024-09-10T23:17:52.578 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71319]: 2024-09-10 23:17:52.243933466 +0000 UTC m=+0.581015740 container remove bba203d55b4f47ce9117fc90c23409aa8627c4673998819c517c3716111ead23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:17:52.915 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71410]: 2024-09-10 23:17:52.578338441 +0000 UTC m=+0.112740636 container create 9475215f0a0efa7b0539cb864291c007d1b44f71832359b1974b477353bb154a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T23:17:52.915 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71410]: 2024-09-10 23:17:52.481494955 +0000 UTC m=+0.015897161 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:17:52.915 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71410]: 2024-09-10 23:17:52.822655034 +0000 UTC m=+0.357057229 container init 9475215f0a0efa7b0539cb864291c007d1b44f71832359b1974b477353bb154a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T23:17:52.915 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71410]: 2024-09-10 23:17:52.825846956 +0000 UTC m=+0.360249152 container start 9475215f0a0efa7b0539cb864291c007d1b44f71832359b1974b477353bb154a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T23:17:53.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:52 smithi153 podman[71410]: 2024-09-10 23:17:52.914316805 +0000 UTC m=+0.448719009 container attach 9475215f0a0efa7b0539cb864291c007d1b44f71832359b1974b477353bb154a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:17:53.199 INFO:journalctl@ceph.osd.1.smithi153.stdout:Sep 10 23:17:53 smithi153 podman[71410]: 2024-09-10 23:17:53.012537992 +0000 UTC m=+0.546940187 container died 9475215f0a0efa7b0539cb864291c007d1b44f71832359b1974b477353bb154a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T23:17:53.578 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T23:17:53.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:17:53.618 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T23:17:53.618 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T23:17:53.618 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.2 2024-09-10T23:17:53.933 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:53 smithi176 systemd[1]: Stopping Ceph osd.2 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:17:54.322 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T23:17:53.931+0000 7f4ae017b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:17:54.323 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T23:17:53.931+0000 7f4ae017b640 -1 osd.2 84 *** Got signal Terminated *** 2024-09-10T23:17:54.323 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2[39928]: 2024-09-10T23:17:53.931+0000 7f4ae017b640 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:17:59.822 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:59 smithi176 podman[72903]: 2024-09-10 23:17:59.546385532 +0000 UTC m=+5.725881940 container died fb9c48a8a5968ad27e1fe454e89bb4d875d1dff8aca36ccff2567534c7532e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-10T23:18:00.167 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:59 smithi176 podman[72903]: 2024-09-10 23:17:59.956734998 +0000 UTC m=+6.136231399 container cleanup fb9c48a8a5968ad27e1fe454e89bb4d875d1dff8aca36ccff2567534c7532e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T23:18:00.167 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:17:59 smithi176 bash[72903]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2 2024-09-10T23:18:00.459 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[72918]: 2024-09-10 23:18:00.166733531 +0000 UTC m=+0.615675133 container remove fb9c48a8a5968ad27e1fe454e89bb4d875d1dff8aca36ccff2567534c7532e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:00.778 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[73006]: 2024-09-10 23:18:00.459121774 +0000 UTC m=+0.098632935 container create a2c0a386264eae2f07a9abb08224985cf40395b589bb64401a07e7cbd10eb9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:00.778 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[73006]: 2024-09-10 23:18:00.376224792 +0000 UTC m=+0.015735967 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:00.778 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[73006]: 2024-09-10 23:18:00.694713959 +0000 UTC m=+0.334225122 container init a2c0a386264eae2f07a9abb08224985cf40395b589bb64401a07e7cbd10eb9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T23:18:00.778 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[73006]: 2024-09-10 23:18:00.698106686 +0000 UTC m=+0.337617862 container start a2c0a386264eae2f07a9abb08224985cf40395b589bb64401a07e7cbd10eb9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:00.778 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:00 smithi176 podman[73006]: 2024-09-10 23:18:00.777955287 +0000 UTC m=+0.417466461 container attach a2c0a386264eae2f07a9abb08224985cf40395b589bb64401a07e7cbd10eb9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T23:18:01.322 INFO:journalctl@ceph.osd.2.smithi176.stdout:Sep 10 23:18:01 smithi176 podman[73006]: 2024-09-10 23:18:01.05388297 +0000 UTC m=+0.693394147 container died a2c0a386264eae2f07a9abb08224985cf40395b589bb64401a07e7cbd10eb9be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:01.656 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T23:18:01.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:01.696 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T23:18:01.696 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T23:18:01.696 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.3 2024-09-10T23:18:02.018 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:01 smithi120 systemd[1]: Stopping Ceph osd.3 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:02.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:02 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:02.018+0000 7f1eeac4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:02.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:02 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:02.018+0000 7f1eeac4b640 -1 osd.3 84 *** Got signal Terminated *** 2024-09-10T23:18:02.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:02 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:02.018+0000 7f1eeac4b640 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:04.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:04 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:04.092+0000 7f1ee7260640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.829881+0000 front 2024-09-10T23:17:38.829850+0000 (oldest deadline 2024-09-10T23:18:03.529996+0000) 2024-09-10T23:18:04.823 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:04.327+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:05.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:04 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:04.900+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:05.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:05 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:05.132+0000 7f1ee7260640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.829881+0000 front 2024-09-10T23:17:38.829850+0000 (oldest deadline 2024-09-10T23:18:03.529996+0000) 2024-09-10T23:18:05.654 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:05.331+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:06.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:05.653+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:06.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:05 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:05.937+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:06.446 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:06 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3[54777]: 2024-09-10T23:18:06.167+0000 7f1ee7260640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.829881+0000 front 2024-09-10T23:17:38.829850+0000 (oldest deadline 2024-09-10T23:18:03.529996+0000) 2024-09-10T23:18:06.696 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:06.344+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:07.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:06.695+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:07.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:06 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:06.959+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:07.199 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:06 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:06.720+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:07.665 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:07.341+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:07.793 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:07 smithi120 podman[123977]: 2024-09-10 23:18:07.518731666 +0000 UTC m=+5.613080835 container died cef1dcd187f3ddad1e8394f53b97507588fa2d8ffd9b49f93423f0858d4529ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T23:18:07.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:07 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:07.697+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:08.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:07.664+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:08.196 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:07 smithi120 podman[123977]: 2024-09-10 23:18:07.976030839 +0000 UTC m=+6.070396583 container cleanup cef1dcd187f3ddad1e8394f53b97507588fa2d8ffd9b49f93423f0858d4529ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:18:08.196 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:07 smithi120 bash[123977]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3 2024-09-10T23:18:08.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:07 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:07.995+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:08.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:08.300+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:08.696 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:08 smithi120 podman[124005]: 2024-09-10 23:18:08.386756463 +0000 UTC m=+0.865181808 container remove cef1dcd187f3ddad1e8394f53b97507588fa2d8ffd9b49f93423f0858d4529ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T23:18:08.696 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:08 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:08.535+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:08.697 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:08 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:08.370+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:08.968 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:08 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:08.717+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:09.065 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:08 smithi120 podman[124094]: 2024-09-10 23:18:08.640184088 +0000 UTC m=+0.015829264 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:09.065 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:08 smithi120 podman[124094]: 2024-09-10 23:18:08.754571793 +0000 UTC m=+0.130216972 container create eab9e89dd0dd0ad47f67225f93c1dc4dcfe9056dd3689ace8115eb247b037e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:09.065 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:08 smithi120 podman[124094]: 2024-09-10 23:18:08.965309534 +0000 UTC m=+0.340954710 container init eab9e89dd0dd0ad47f67225f93c1dc4dcfe9056dd3689ace8115eb247b037e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:18:09.066 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:08 smithi120 podman[124094]: 2024-09-10 23:18:08.968727888 +0000 UTC m=+0.344373059 container start eab9e89dd0dd0ad47f67225f93c1dc4dcfe9056dd3689ace8115eb247b037e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T23:18:09.066 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:09 smithi120 podman[124094]: 2024-09-10 23:18:09.065371431 +0000 UTC m=+0.441016608 container attach eab9e89dd0dd0ad47f67225f93c1dc4dcfe9056dd3689ace8115eb247b037e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T23:18:09.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:08.646+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:09.338 INFO:journalctl@ceph.osd.3.smithi120.stdout:Sep 10 23:18:09 smithi120 podman[124094]: 2024-09-10 23:18:09.158919714 +0000 UTC m=+0.534564891 container died eab9e89dd0dd0ad47f67225f93c1dc4dcfe9056dd3689ace8115eb247b037e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:09.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:08 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:08.968+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:09.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:09.299+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:09.696 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:09 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:09.491+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:09.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:09 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:09.337+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:09.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T23:18:09.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:09.808 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T23:18:09.808 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T23:18:09.808 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.4 2024-09-10T23:18:09.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:09 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:09.685+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:09.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:09 smithi153 systemd[1]: Stopping Ceph osd.4 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:10.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:09.658+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:10.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:09 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:09.950+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:10.449 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:10 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:10.103+0000 7f7f0ea50640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:10.449 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:10 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:10.103+0000 7f7f0ea50640 -1 osd.4 84 *** Got signal Terminated *** 2024-09-10T23:18:10.449 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:10 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:10.103+0000 7f7f0ea50640 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:10.649 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:10 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:10.332+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:10.696 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:10 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:10.541+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:10.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:10 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:10.383+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:10.935 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:10 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:10.667+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:11.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:10 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:10.648+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:11.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:10 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:10.934+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:11.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:11 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:11.289+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:11.696 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:11 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:11.585+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:11.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:11 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:11.405+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:11.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:11 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:11.670+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:12.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:11 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:11.638+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:12.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:11 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:11.984+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:12.631 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:12 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:12.326+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:12.696 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:12 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:12.623+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:12.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:12 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:12.418+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:12.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:12 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:12.418+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:12.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:12 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:12.633+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:13.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:12 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:12.630+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:13.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:12 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:12.955+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:13.587 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:13 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:13.326+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:13.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:13 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:13.594+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:13.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:13 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:13.456+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:13.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:13 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:13.456+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:13.948 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:13 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:13.651+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:14.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:13 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:13.586+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:14.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:13 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:13.733+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:14.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:13 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:13.971+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:14.626 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:14 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:14.339+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:14.722 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:14 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4[46116]: 2024-09-10T23:18:14.670+0000 7f7f0b065640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.021300+0000 front 2024-09-10T23:17:42.021398+0000 (oldest deadline 2024-09-10T23:18:06.121263+0000) 2024-09-10T23:18:14.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:14 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:14.616+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:14.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:14 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:14.616+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:14.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:14 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:14.477+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:14.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:14 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:14.477+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:15.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:14 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:14.626+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:15.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:14 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:14.699+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:15.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:14 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:14.937+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:15.608 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:15 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:15.324+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:15.897 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:15 smithi153 podman[71513]: 2024-09-10 23:18:15.621550452 +0000 UTC m=+5.617265823 container died bacdf0805b3a289e68a9a05a97b88f43f518da6f95610c2e6eb598709f0071a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:15.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:15 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:15.654+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:15.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:15 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:15.654+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:15.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:15 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:15.468+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:15.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:15 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:15.468+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:16.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:15 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:15.708+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:16.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:15 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:15.607+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:16.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:15 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:15.607+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:16.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:15 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:15.902+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:16.199 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:15 smithi153 podman[71513]: 2024-09-10 23:18:15.987053053 +0000 UTC m=+5.982768418 container cleanup bacdf0805b3a289e68a9a05a97b88f43f518da6f95610c2e6eb598709f0071a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T23:18:16.199 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:15 smithi153 bash[71513]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4 2024-09-10T23:18:16.528 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71542]: 2024-09-10 23:18:16.21327426 +0000 UTC m=+0.585709378 container remove bacdf0805b3a289e68a9a05a97b88f43f518da6f95610c2e6eb598709f0071a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T23:18:16.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:16 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:16.312+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:16.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:16 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:16.312+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:16.832 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.430407476 +0000 UTC m=+0.015615187 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:16.832 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.530762937 +0000 UTC m=+0.115970648 container create f42e3bcf9d9aeb623836f386569732bf0df1fe17d1950e9a611dc2e79120eaef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:16.832 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.758520737 +0000 UTC m=+0.343728449 container init f42e3bcf9d9aeb623836f386569732bf0df1fe17d1950e9a611dc2e79120eaef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T23:18:16.832 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.76172727 +0000 UTC m=+0.346934992 container start f42e3bcf9d9aeb623836f386569732bf0df1fe17d1950e9a611dc2e79120eaef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:18:16.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:16.470+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:16.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:16.470+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:16.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:16.686+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:16.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:16 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:16.686+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:17.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:16 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:16.674+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:17.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:16 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:16.651+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:17.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:16 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:16.651+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:17.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:16 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:16.830+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:17.199 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.850022865 +0000 UTC m=+0.435230587 container attach f42e3bcf9d9aeb623836f386569732bf0df1fe17d1950e9a611dc2e79120eaef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:17.199 INFO:journalctl@ceph.osd.4.smithi153.stdout:Sep 10 23:18:16 smithi153 podman[71631]: 2024-09-10 23:18:16.956153652 +0000 UTC m=+0.541361366 container died f42e3bcf9d9aeb623836f386569732bf0df1fe17d1950e9a611dc2e79120eaef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T23:18:17.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:16 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:16.912+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:17.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:16 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:16.912+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:17.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:17.288+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:17.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:17.288+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:17.589 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T23:18:17.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:17.629 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T23:18:17.629 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T23:18:17.629 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.5 2024-09-10T23:18:17.926 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:17 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:17.817+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:17.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:17 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:17.697+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:17.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:17 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:17.697+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:17.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:17 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:17.506+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:17.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:17 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:17.506+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:17.959 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 systemd[1]: Stopping Ceph osd.5 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:17.960 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:17.697+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:17.960 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:17.662+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:17.960 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:17.662+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:18.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:17 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:17.926+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:18.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:17 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:17.926+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:18.269 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:17.959+0000 7f4ace794640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:18.269 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:17.959+0000 7f4ace794640 -1 osd.5 84 *** Got signal Terminated *** 2024-09-10T23:18:18.269 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:17 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:17.959+0000 7f4ace794640 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:18.269 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:18.268+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:18.269 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:18.268+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:18.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:18.715+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:18.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:18.715+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:18.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:18.556+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:18.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:18.556+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:19.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:18.703+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:19.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:18.616+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:19.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:18 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:18.616+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:19.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:18 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:18.790+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:19.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:18 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:18.951+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:19.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:18 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:18.951+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:19.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:19 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:19.222+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:19.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:19 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:19.222+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:19.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:19.571+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:19.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:19.571+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:19.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:19.711+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:19.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:19 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:19.711+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:20.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:19 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:19.716+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:20.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:19 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:19.622+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:20.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:19 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:19.622+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:20.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:19 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:19.833+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:20.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:19 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:19.833+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:20.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:19 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:19.973+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:20.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:19 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:19.973+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:20.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:20 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:20.196+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:20.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:20 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:20.196+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:20.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:20 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:20.705+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:20.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:20 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:20.705+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:20.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:20 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:20.555+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:20.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:20 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:20.555+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:21.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:20 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:20.760+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:21.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:20 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:20.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:21.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:20 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:20.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:21.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:20 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:20.816+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:21.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:20 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:20.816+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:21.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:20 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:20.944+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:21.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:20 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:20.944+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:21.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:21.200+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:21.572 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:21.200+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:21.573 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:21.200+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.270472+0000 front 2024-09-10T23:17:56.270414+0000 (oldest deadline 2024-09-10T23:18:20.970547+0000) 2024-09-10T23:18:21.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:21 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:21.673+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:21.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:21 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:21.673+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:21.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:21 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:21.549+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:21.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:21 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:21.549+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:21.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:21 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:21.549+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:22.072 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:21.662+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:22.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:21.662+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:22.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:21.758+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:22.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:21 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:21.758+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:22.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:21 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:21.781+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:22.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:21 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:21.781+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:22.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:21 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:21.970+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:22.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:21 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:21.970+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:22.576 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:22.242+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:38.867710+0000 front 2024-09-10T23:17:38.867833+0000 (oldest deadline 2024-09-10T23:18:03.567653+0000) 2024-09-10T23:18:22.576 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:22.242+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:51.069793+0000 front 2024-09-10T23:17:51.069886+0000 (oldest deadline 2024-09-10T23:18:16.270124+0000) 2024-09-10T23:18:22.576 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5[46720]: 2024-09-10T23:18:22.242+0000 7f4acada9640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.270472+0000 front 2024-09-10T23:17:56.270414+0000 (oldest deadline 2024-09-10T23:18:20.970547+0000) 2024-09-10T23:18:22.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:22.695+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:22.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:22.695+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:22.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:22.546+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:22.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:22.546+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:22.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:22 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:22.546+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:23.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:22.792+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:23.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:22.792+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:23.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:22.644+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:23.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:22 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:22.644+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:23.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:22 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:22.788+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:23.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:22 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:22.788+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:23.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:22 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:22.953+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:23.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:22 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:22.953+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:23.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:22 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:22.953+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:23.801 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:23 smithi176 podman[73120]: 2024-09-10 23:18:23.545880954 +0000 UTC m=+5.718854072 container died 78728c6dded3dce893c93cedd94be45109ed1ded31f034cd14bc181b7b8b6245 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:18:23.802 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:23 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:23.682+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:23.802 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:23 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:23.682+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:23.802 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:23 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:23.778+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:23.802 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:23 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:23.778+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:23.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:23.679+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:23.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:23.679+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:23.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:23.679+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:23.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:23.567+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:23.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:23.567+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:23.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:23.567+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:24.072 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:23 smithi176 podman[73120]: 2024-09-10 23:18:23.90038566 +0000 UTC m=+6.073358778 container cleanup 78728c6dded3dce893c93cedd94be45109ed1ded31f034cd14bc181b7b8b6245 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T23:18:24.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:23 smithi176 bash[73120]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5 2024-09-10T23:18:24.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:23 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:23.811+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:24.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:23 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:23.811+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:24.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:23 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:23.992+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:24.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:23 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:23.992+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:24.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:23 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:23.992+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:24.526 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73135]: 2024-09-10 23:18:24.21814064 +0000 UTC m=+0.672745615 container remove 78728c6dded3dce893c93cedd94be45109ed1ded31f034cd14bc181b7b8b6245 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:24.788 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:24.685+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:24.788 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:24.685+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:24.788 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:24.685+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:24.788 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.429084524 +0000 UTC m=+0.015712174 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:24.788 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.53549307 +0000 UTC m=+0.122120779 container create be3f5371ef0358fe2bdcb7ceed90a048cc5f5df91b1e50921e39b45eb2bf888a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:24.789 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.787988833 +0000 UTC m=+0.374616483 container init be3f5371ef0358fe2bdcb7ceed90a048cc5f5df91b1e50921e39b45eb2bf888a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-10T23:18:24.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:24.550+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:24.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:24.550+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:24.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:24.550+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:24.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:24.646+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:24.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:24.646+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:24.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:24 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:24.646+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:25.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:24.787+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:25.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:24 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:24.787+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:25.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.791336119 +0000 UTC m=+0.377963769 container start be3f5371ef0358fe2bdcb7ceed90a048cc5f5df91b1e50921e39b45eb2bf888a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T23:18:25.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.846396928 +0000 UTC m=+0.433024608 container attach be3f5371ef0358fe2bdcb7ceed90a048cc5f5df91b1e50921e39b45eb2bf888a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:18:25.073 INFO:journalctl@ceph.osd.5.smithi176.stdout:Sep 10 23:18:24 smithi176 podman[73224]: 2024-09-10 23:18:24.98139114 +0000 UTC m=+0.568018791 container died be3f5371ef0358fe2bdcb7ceed90a048cc5f5df91b1e50921e39b45eb2bf888a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T23:18:25.190 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:24 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:24.800+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:25.190 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:24 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:24.800+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:25.190 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.031+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:25.190 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.031+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:25.190 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.031+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:25.613 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T23:18:25.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:25.654 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T23:18:25.654 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T23:18:25.654 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.6 2024-09-10T23:18:25.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:25.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:25.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:25.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:25.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:25.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:25.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.610+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:25.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.610+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:25.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.610+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:25.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 systemd[1]: Stopping Ceph osd.6 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:26.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:25 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:25.804+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:26.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:25 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:25.804+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:26.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:25 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:25.684+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:26.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:25 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:25.684+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:26.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:25 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:25.684+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:26.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:25.763+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:26.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:25.763+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:26.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.982+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:26.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.982+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:26.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:25 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:25.982+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:26.446 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.953+0000 7fa860564640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:26.446 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.953+0000 7fa860564640 -1 osd.6 84 *** Got signal Terminated *** 2024-09-10T23:18:26.446 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:25 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:25.953+0000 7fa860564640 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:26.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:26.603+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:26.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:26.603+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:26.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:26.603+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:26.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:26.500+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:26.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:26.500+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:26.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:26.500+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:27.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:26 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:26.814+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:27.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:26 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:26.814+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:27.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:26 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:26.729+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:27.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:26 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:26.729+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:27.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:26 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:26.729+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:27.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:26.772+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:27.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:26.772+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:27.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:26.976+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:27.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:26.976+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:27.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:26 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:26.976+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:27.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:27.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:27.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:27.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:27.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:27.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:27.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:27.502+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:27.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:27.502+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:27.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:27 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:27.502+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:28.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:27 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:27.814+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:28.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:27 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:27.814+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:28.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:27 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:27.711+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:28.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:27 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:27.711+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:28.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:27 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:27.711+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:28.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:27 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:27.795+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:28.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:27 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:27.795+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:28.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:28 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:28.003+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:28.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:28 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:28.003+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:28.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:28 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:28.003+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:28.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:28.532+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:28.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:28.532+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:28.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:28.532+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:28.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:28.628+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:28.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:28.628+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:28.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:28.628+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:28.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:28 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:28.628+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:03.600978+0000 front 2024-09-10T23:18:03.601009+0000 (oldest deadline 2024-09-10T23:18:27.701133+0000) 2024-09-10T23:18:29.051 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:28 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:28.769+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:29.052 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:28 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:28.769+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:29.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:28 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:28.767+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:29.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:28 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:28.767+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:29.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:28 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:28.679+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:29.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:28 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:28.679+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:29.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:28 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:28.679+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:29.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:29.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:29.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:29.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:29.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:29.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:29.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:29.553+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:29.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:29.553+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:29.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:29.553+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:29.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:29.587+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:29.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:29.587+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:29.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:29.587+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:29.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:29 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:29.587+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:03.600978+0000 front 2024-09-10T23:18:03.601009+0000 (oldest deadline 2024-09-10T23:18:27.701133+0000) 2024-09-10T23:18:30.040 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:29.745+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:30.040 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:29 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:29.745+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:30.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:29.800+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:30.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:29.800+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:30.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:29.633+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:30.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:29.633+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:30.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:29.633+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:30.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:29 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:29.633+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:30.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:30.039+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:30.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:30.039+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:30.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:30.039+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:30.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:30.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:30.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:30.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:30.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:30.548+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:30.946 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:30.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.597793+0000 front 2024-09-10T23:17:42.597769+0000 (oldest deadline 2024-09-10T23:18:08.497980+0000) 2024-09-10T23:18:30.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:30.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:48.498658+0000 front 2024-09-10T23:17:48.498529+0000 (oldest deadline 2024-09-10T23:18:14.398414+0000) 2024-09-10T23:18:30.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:30.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.499494+0000 front 2024-09-10T23:17:55.499579+0000 (oldest deadline 2024-09-10T23:18:23.100020+0000) 2024-09-10T23:18:30.947 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:30 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6[61694]: 2024-09-10T23:18:30.595+0000 7fa858b71640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:03.600978+0000 front 2024-09-10T23:18:03.601009+0000 (oldest deadline 2024-09-10T23:18:27.701133+0000) 2024-09-10T23:18:31.051 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:30.758+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:31.051 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:30 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:30.758+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:31.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:30.840+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:30.840+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:30.840+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:30.667+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:30.667+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:30.667+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:31.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:30 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:30.667+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:31.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:31.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:31.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:31.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:31.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:31.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:31.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:31.050+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:31.739 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:31 smithi120 podman[124196]: 2024-09-10 23:18:31.447115397 +0000 UTC m=+5.597371242 container died e07b7d07d5ef2b7544b7b89269e409d1733cfce325c459f4b3cbf7b9a7687f75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T23:18:31.739 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:31.541+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:31.740 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:31.541+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:31.740 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:31.541+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:31.740 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:31 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:31.541+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:32.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:31.798+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:31.798+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:31.798+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:31.638+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:31.638+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:31.638+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:32.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:31 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:31.638+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:32.173 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:31 smithi120 podman[124196]: 2024-09-10 23:18:31.963350157 +0000 UTC m=+6.113605990 container cleanup e07b7d07d5ef2b7544b7b89269e409d1733cfce325c459f4b3cbf7b9a7687f75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T23:18:32.173 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:31 smithi120 bash[124196]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:31.790+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:31.790+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:31.790+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.014+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.014+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.014+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:32.201 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.014+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:32.446 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124211]: 2024-09-10 23:18:32.173388662 +0000 UTC m=+0.726311121 container remove e07b7d07d5ef2b7544b7b89269e409d1733cfce325c459f4b3cbf7b9a7687f75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T23:18:32.813 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:32 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:32.524+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:32.813 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:32 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:32.524+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:32.813 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:32 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:32.524+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:32.813 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:32 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:32.524+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:32.814 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124302]: 2024-09-10 23:18:32.482693183 +0000 UTC m=+0.016096399 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:32.814 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124302]: 2024-09-10 23:18:32.599142783 +0000 UTC m=+0.132545982 container create ec72b2dc627b507e1e7b0001835eb8820e2c0167893c0beb5a73aff519cef85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T23:18:32.814 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124302]: 2024-09-10 23:18:32.81018329 +0000 UTC m=+0.343586490 container init ec72b2dc627b507e1e7b0001835eb8820e2c0167893c0beb5a73aff519cef85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T23:18:33.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:32.842+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:32.842+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:32.842+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:32.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:32.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:32.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:33.073 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:32 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:32.656+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:33.196 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124302]: 2024-09-10 23:18:32.814018849 +0000 UTC m=+0.347422049 container start ec72b2dc627b507e1e7b0001835eb8820e2c0167893c0beb5a73aff519cef85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T23:18:33.196 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:32 smithi120 podman[124302]: 2024-09-10 23:18:32.901930108 +0000 UTC m=+0.435333316 container attach ec72b2dc627b507e1e7b0001835eb8820e2c0167893c0beb5a73aff519cef85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T23:18:33.196 INFO:journalctl@ceph.osd.6.smithi120.stdout:Sep 10 23:18:33 smithi120 podman[124302]: 2024-09-10 23:18:33.005048812 +0000 UTC m=+0.538452022 container died ec72b2dc627b507e1e7b0001835eb8820e2c0167893c0beb5a73aff519cef85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T23:18:33.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:32.838+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:33.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:32.838+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:33.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:32.838+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:33.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.985+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:33.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.985+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:33.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.985+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:33.200 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:32 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:32.985+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:33.617 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T23:18:33.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:33.656 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T23:18:33.656 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T23:18:33.656 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.7 2024-09-10T23:18:33.875 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:33.617+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:33.876 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:33.617+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:33.876 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:33.617+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:33.876 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:33.617+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:33.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:33 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:33.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:33.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:33 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:33.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:33.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:33 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:33.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:33.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:33 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:33.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:33.948 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 systemd[1]: Stopping Ceph osd.7 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:33.948 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.848+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:33.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.848+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:33.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.848+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:33.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.947+0000 7fe143533640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:33.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.947+0000 7fe143533640 -1 osd.7 84 *** Got signal Terminated *** 2024-09-10T23:18:33.949 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:33 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:33.947+0000 7fe143533640 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:34.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:33.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:34.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:33.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:34.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:33 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:33.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:34.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:34.016+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:34.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:34.016+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:34.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:34.016+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:34.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:34.016+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:34.890 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:34.576+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:34.891 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:34.576+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:34.891 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:34.576+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:34.891 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:34.576+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:34.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:34.585+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:34.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:34.585+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:34.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:34.585+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:34.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:34 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:34.585+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:35.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:34.888+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:35.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:34.888+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:35.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:34 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:34.888+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:35.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:35.012+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:35.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:35.012+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:35.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:35.012+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:35.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:35.012+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:35.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:34.890+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:35.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:34.890+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:35.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:34 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:34.890+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:35.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:35.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:35.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:35.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:35.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:35.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:35.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:35.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:35.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:35 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:35.582+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:35.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:35 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:35.582+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:35.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:35 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:35.582+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:35.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:35 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:35.582+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:36.198 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:35.934+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:35.934+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:35 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:35.934+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:36.019+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:36.019+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:36.019+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:36.019+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:36.019+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:36.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:35.899+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:36.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:35.899+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:36.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:35 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:35.899+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:36.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:36.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:36.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:36.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:36.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:36.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:36.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:36.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:36.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:36 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:36.619+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:36.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:36 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:36.619+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:36.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:36 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:36.619+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:36.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:36 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:36.619+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:37.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:36.884+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:37.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:36.884+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:37.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:36.884+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:37.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:36 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:36.884+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:37.448 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:36.954+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:36.954+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:36.954+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:37.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:37.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:37.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:37.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:37.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:37.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:37.595 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:37.546+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:37.595 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:37.546+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:37.595 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:37.546+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:37.595 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:37.546+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:37.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:37.600+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:37.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:37.600+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:37.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:37.600+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:37.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:37 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:37.600+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:38.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:37.856+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:38.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:37.856+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:38.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:37.856+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:38.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:37 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:37.856+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:38.448 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:37.987+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.792412+0000 front 2024-09-10T23:17:50.792535+0000 (oldest deadline 2024-09-10T23:18:16.692603+0000) 2024-09-10T23:18:38.448 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:37.987+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.693132+0000 front 2024-09-10T23:17:56.692984+0000 (oldest deadline 2024-09-10T23:18:19.593223+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:37 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7[52587]: 2024-09-10T23:18:37.987+0000 7fe13fb48640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.494195+0000 front 2024-09-10T23:18:05.494174+0000 (oldest deadline 2024-09-10T23:18:31.394203+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:38 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:38.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:38 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:38.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:38 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:38.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:38 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:38.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:38 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:38.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:38.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:38.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:38.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:38.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:38.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:38.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:38.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:38.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:38.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:38.540+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:38.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:38.594+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:38.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:38.594+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:38.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:38.594+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:38.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:38.594+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:38.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:38 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:38.594+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:39.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:38.875+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:39.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:38.875+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:39.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:38.875+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:39.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:38 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:38.875+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:39.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:39 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:39.069+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:39.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:39 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:39.069+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:39.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:39 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:39.069+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:39.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:39 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:39.069+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:39.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:39 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:39.069+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:39.757 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:39 smithi153 podman[71734]: 2024-09-10 23:18:39.493113593 +0000 UTC m=+5.646116209 container died 368742f6335d8930b399b3609d6e7a959b12e6196ac9932bc409faf7b96ee9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:18:39.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:39.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:39.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:39.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:39.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:39.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:39.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:39.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:39.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:39.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:39.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:39 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:39.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:39.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:39 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:39.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:39.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:39 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:39.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:39.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:39 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:39.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:39.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:39 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:39.557+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:40.107 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:39 smithi153 podman[71734]: 2024-09-10 23:18:39.864498191 +0000 UTC m=+6.017500796 container cleanup 368742f6335d8930b399b3609d6e7a959b12e6196ac9932bc409faf7b96ee9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T23:18:40.107 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:39 smithi153 bash[71734]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7 2024-09-10T23:18:40.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:39.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:40.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:39.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:40.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:39.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:40.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:39 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:39.874+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71750]: 2024-09-10 23:18:40.107347234 +0000 UTC m=+0.613768964 container remove 368742f6335d8930b399b3609d6e7a959b12e6196ac9932bc409faf7b96ee9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:40 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:40.106+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:40 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:40.106+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:40 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:40.106+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:40 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:40.106+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:40.417 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:40 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:40.106+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:40.698 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.416654484 +0000 UTC m=+0.105566359 container create a14ce13534b8ad45594405fbbf8e3f0796392b04d8e7a2a0bcf9d1d4b00a678a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:40.698 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.327020515 +0000 UTC m=+0.015932358 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:40.699 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.627472553 +0000 UTC m=+0.316384383 container init a14ce13534b8ad45594405fbbf8e3f0796392b04d8e7a2a0bcf9d1d4b00a678a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T23:18:40.699 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.630772143 +0000 UTC m=+0.319683982 container start a14ce13534b8ad45594405fbbf8e3f0796392b04d8e7a2a0bcf9d1d4b00a678a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:18:40.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:40.524+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:40.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:40.524+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:40.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:40.524+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:40.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:40.524+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:40.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:40.524+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:40.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:40 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:40.596+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:40.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:40 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:40.596+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:40.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:40 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:40.596+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:40.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:40 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:40.596+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:40.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:40 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:40.596+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:41.060 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.720669251 +0000 UTC m=+0.409581100 container attach a14ce13534b8ad45594405fbbf8e3f0796392b04d8e7a2a0bcf9d1d4b00a678a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:18:41.060 INFO:journalctl@ceph.osd.7.smithi153.stdout:Sep 10 23:18:40 smithi153 podman[71839]: 2024-09-10 23:18:40.823164519 +0000 UTC m=+0.512076352 container died a14ce13534b8ad45594405fbbf8e3f0796392b04d8e7a2a0bcf9d1d4b00a678a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T23:18:41.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:40.888+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:41.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:40.888+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:41.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:40.888+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:41.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:40 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:40.888+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:41.447 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:41.059+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:41.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:41.059+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:41.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:41.059+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:41.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:41.059+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:41.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:41.059+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:41.472 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T23:18:41.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:41.512 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T23:18:41.512 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T23:18:41.512 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.8 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 systemd[1]: Stopping Ceph osd.8 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:41.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:41.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:41.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:41.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:41.819 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:41.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:41.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:41 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:41.558+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:41.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:41 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:41.558+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:41.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:41 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:41.558+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:41.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:41 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:41.558+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:41.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:41 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:41.558+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:42.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.818+0000 7f27f1deb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.818+0000 7f27f1deb640 -1 osd.8 84 *** Got signal Terminated *** 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.818+0000 7f27f1deb640 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.872+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.872+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.872+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:42.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:41 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:41.872+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:42.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:42.096+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:42.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:42.096+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:42.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:42.096+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:42.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:42.096+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:42.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:42 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:42.096+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:42.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:42.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:42.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:42.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:42.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:42.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:42.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:42.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:42.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:42.510+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:42.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:42 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:42.580+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:42.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:42 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:42.580+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:42.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:42 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:42.580+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:42.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:42 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:42.580+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:42.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:42 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:42.580+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:43.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:42.879+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:43.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:42.879+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:43.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:42.879+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:43.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:42 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:42.879+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:43.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:43 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:43.115+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:43.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:43 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:43.115+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:43.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:43 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:43.115+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:43.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:43 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:43.115+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:43.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:43 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:43.115+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:43.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:43.551+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:43.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:43.551+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:43.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:43.551+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:43.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:43.551+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:43.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:43.551+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:43.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:43 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:43.577+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:43.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:43 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:43.577+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:43.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:43 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:43.577+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:43.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:43 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:43.577+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:43.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:43 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:43.577+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:44.322 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:43.834+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:44.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:43.834+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:44.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:43.834+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:44.323 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:43 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:43.834+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:44.350 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:44.081+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:44.350 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:44.081+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:44.350 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:44.081+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:44.350 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:44.081+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:44.350 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:44.081+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:44.818+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:44.818+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:44.818+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:44.818+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:44.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:44.824 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:44 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:44.570+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:44.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:44 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:44.608+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:44.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:44 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:44.608+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:44.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:44 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:44.608+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:44.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:44 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:44.608+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:44.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:44 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:44.608+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:45.366 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:45 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:45.105+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:45.366 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:45 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:45.105+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:45.366 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:45 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:45.105+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:45.366 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:45 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:45.105+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:45.366 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:45 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:45.105+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:45.817 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:45.549+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:45.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:45.624+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:45.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:45.624+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:45.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:45.624+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:45.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:45.624+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:45.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:45 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:45.624+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:46.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:45.816+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:46.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:45.816+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:46.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:45.816+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:46.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:45.816+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:46.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:45 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:45.816+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:19.886134+0000 front 2024-09-10T23:18:19.886154+0000 (oldest deadline 2024-09-10T23:18:45.186276+0000) 2024-09-10T23:18:46.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:46.080+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:46.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:46.080+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:46.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:46.080+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:46.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:46.080+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:46.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:46 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:46.080+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:46.807 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:46.553+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:46.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:46 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:46.667+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:46.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:46 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:46.667+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:46.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:46 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:46.667+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:46.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:46 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:46.667+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:46.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:46 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:46.667+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:47.072 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:46.806+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.281532+0000 front 2024-09-10T23:17:47.281562+0000 (oldest deadline 2024-09-10T23:18:13.181460+0000) 2024-09-10T23:18:47.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:46.806+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.382902+0000 front 2024-09-10T23:17:55.382883+0000 (oldest deadline 2024-09-10T23:18:21.282935+0000) 2024-09-10T23:18:47.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:46.806+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.183846+0000 front 2024-09-10T23:18:04.183973+0000 (oldest deadline 2024-09-10T23:18:30.584179+0000) 2024-09-10T23:18:47.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:46.806+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.485232+0000 front 2024-09-10T23:18:13.485201+0000 (oldest deadline 2024-09-10T23:18:36.385194+0000) 2024-09-10T23:18:47.073 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:46 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8[53189]: 2024-09-10T23:18:46.806+0000 7f27ea3f8640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:19.886134+0000 front 2024-09-10T23:18:19.886154+0000 (oldest deadline 2024-09-10T23:18:45.186276+0000) 2024-09-10T23:18:47.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:47 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:47.129+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:47.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:47 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:47.129+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:47.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:47 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:47.129+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:47.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:47 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:47.129+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:47.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:47 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:47.129+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:47.533 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:47 smithi176 podman[73339]: 2024-09-10 23:18:47.273322109 +0000 UTC m=+5.562942652 container died 788a69201c0785d7430ed791cf40c96d1a9443728d0ab3c17ca1cb1ac714d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:18:47.822 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:47 smithi176 podman[73339]: 2024-09-10 23:18:47.617753527 +0000 UTC m=+5.907374113 container cleanup 788a69201c0785d7430ed791cf40c96d1a9443728d0ab3c17ca1cb1ac714d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:47 smithi176 bash[73339]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:47.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:47 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:47.532+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:48.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:47.711+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:48.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:47.711+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:48.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:47.711+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:48.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:47.711+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:48.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:47 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:47.711+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:48.228 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:47 smithi176 podman[73356]: 2024-09-10 23:18:47.919167279 +0000 UTC m=+0.645935174 container remove 788a69201c0785d7430ed791cf40c96d1a9443728d0ab3c17ca1cb1ac714d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T23:18:48.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:48.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:48.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:48.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:48.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:48.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:48 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:48.084+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:48.487 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.228187038 +0000 UTC m=+0.102567899 container create 8aa100d4db3a446dc94ae9e7125cd492cfbf9ab1fd72c17d32b6d6fa4f7b9169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T23:18:48.487 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.14137872 +0000 UTC m=+0.015759541 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:48.488 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.446961072 +0000 UTC m=+0.321341892 container init 8aa100d4db3a446dc94ae9e7125cd492cfbf9ab1fd72c17d32b6d6fa4f7b9169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T23:18:48.488 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.450200505 +0000 UTC m=+0.324581325 container start 8aa100d4db3a446dc94ae9e7125cd492cfbf9ab1fd72c17d32b6d6fa4f7b9169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.58898911 +0000 UTC m=+0.463369934 container attach 8aa100d4db3a446dc94ae9e7125cd492cfbf9ab1fd72c17d32b6d6fa4f7b9169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.8.smithi176.stdout:Sep 10 23:18:48 smithi176 podman[73439]: 2024-09-10 23:18:48.642094382 +0000 UTC m=+0.516475202 container died 8aa100d4db3a446dc94ae9e7125cd492cfbf9ab1fd72c17d32b6d6fa4f7b9169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:48.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:48.486+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:49.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:49.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:49.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:49.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:49.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:49.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:48 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:48.730+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:49.208 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T23:18:49.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:49.248 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T23:18:49.248 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T23:18:49.248 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.9 2024-09-10T23:18:49.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:49.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:49.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:49.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:49.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:49.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:49 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:49.077+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:49.558 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 systemd[1]: Stopping Ceph osd.9 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:49.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:49.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:49.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:49.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:49.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:49.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:49 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:49.465+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:49.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.557+0000 7f7fe2710640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:49.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.557+0000 7f7fe2710640 -1 osd.9 84 *** Got signal Terminated *** 2024-09-10T23:18:49.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.557+0000 7f7fe2710640 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:49.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:49.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:49.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:49.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:49.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:49.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:49 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:49.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:50.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:50.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:50.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:50.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:50.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:50.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:50 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:50.066+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:50.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:50.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:50.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:50.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:50.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:50.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:50 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:50.424+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:51.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.701+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:51.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:51.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:51.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:51.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:51.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:50 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:50.702+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:51.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:51.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:51.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:51.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:51.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:51.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:51 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:51.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:51.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:51.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:51.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:51.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:51.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:51.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:51.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:52.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:52.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:52.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:52.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:52.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:52.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:51 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:51.751+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:52.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:52.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:52.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:52.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:52.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:52.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:52 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:52.007+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:52.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:52.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:52.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:52.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:52.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:52.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:52 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:52.416+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:53.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:53.196 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:53.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:53.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:53.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:53.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:53.197 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:52 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:52.714+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:29.357410+0000 front 2024-09-10T23:18:29.357455+0000 (oldest deadline 2024-09-10T23:18:52.257488+0000) 2024-09-10T23:18:53.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:53.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:53.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:53.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:53.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:53.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:53.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:53 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:53.029+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:53.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:53.897 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:53.897 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:53.897 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:53.897 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:53.897 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:53 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:53.388+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:53.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:42.050625+0000 front 2024-09-10T23:17:42.050869+0000 (oldest deadline 2024-09-10T23:18:07.950760+0000) 2024-09-10T23:18:53.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:47.951334+0000 front 2024-09-10T23:17:47.951202+0000 (oldest deadline 2024-09-10T23:18:12.051287+0000) 2024-09-10T23:18:53.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:55.552169+0000 front 2024-09-10T23:17:55.552120+0000 (oldest deadline 2024-09-10T23:18:21.452155+0000) 2024-09-10T23:18:53.946 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:05.453461+0000 front 2024-09-10T23:18:05.453435+0000 (oldest deadline 2024-09-10T23:18:30.753540+0000) 2024-09-10T23:18:53.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:14.854796+0000 front 2024-09-10T23:18:14.854900+0000 (oldest deadline 2024-09-10T23:18:38.354845+0000) 2024-09-10T23:18:53.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:22.956624+0000 front 2024-09-10T23:18:22.956641+0000 (oldest deadline 2024-09-10T23:18:48.256666+0000) 2024-09-10T23:18:53.947 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:53 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9[68648]: 2024-09-10T23:18:53.688+0000 7f7fdad1d640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:29.357410+0000 front 2024-09-10T23:18:29.357455+0000 (oldest deadline 2024-09-10T23:18:52.257488+0000) 2024-09-10T23:18:54.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:54.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:54 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:54.046+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:54.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:54.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:54.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:54.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:54.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:54.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:54 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:54.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:55.374 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 podman[124415]: 2024-09-10 23:18:55.019177685 +0000 UTC m=+5.574074823 container died 3b24c2a068a358b353fec78dd18341ee3b7b49ff1bb1e0cd41e2a0aa7b0af1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T23:18:55.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:55.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:55.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:55.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:55.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:55.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:55.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:55 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:55.001+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:55.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 podman[124415]: 2024-09-10 23:18:55.374184823 +0000 UTC m=+5.929082030 container cleanup 3b24c2a068a358b353fec78dd18341ee3b7b49ff1bb1e0cd41e2a0aa7b0af1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:18:55.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 bash[124415]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9 2024-09-10T23:18:55.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 podman[124431]: 2024-09-10 23:18:55.558980903 +0000 UTC m=+0.533625516 container remove 3b24c2a068a358b353fec78dd18341ee3b7b49ff1bb1e0cd41e2a0aa7b0af1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T23:18:55.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:55.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:55.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:55.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:55.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:55.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:55.433+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:56.270 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 podman[124516]: 2024-09-10 23:18:55.976552293 +0000 UTC m=+0.111761100 container create a81a1958437dbbcb5c25bc7ca9d3722fd9f0f959f9cbdd953ebf73ad54996a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T23:18:56.271 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:55 smithi120 podman[124516]: 2024-09-10 23:18:55.880665718 +0000 UTC m=+0.015874490 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:18:56.271 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:56 smithi120 podman[124516]: 2024-09-10 23:18:56.204065067 +0000 UTC m=+0.339273828 container init a81a1958437dbbcb5c25bc7ca9d3722fd9f0f959f9cbdd953ebf73ad54996a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:18:56.271 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:56 smithi120 podman[124516]: 2024-09-10 23:18:56.20734681 +0000 UTC m=+0.342555563 container start a81a1958437dbbcb5c25bc7ca9d3722fd9f0f959f9cbdd953ebf73ad54996a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T23:18:56.271 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:56 smithi120 podman[124516]: 2024-09-10 23:18:56.270442934 +0000 UTC m=+0.405651696 container attach a81a1958437dbbcb5c25bc7ca9d3722fd9f0f959f9cbdd953ebf73ad54996a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:56.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:56 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:56.048+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:56.696 INFO:journalctl@ceph.osd.9.smithi120.stdout:Sep 10 23:18:56 smithi120 podman[124516]: 2024-09-10 23:18:56.397246302 +0000 UTC m=+0.532455074 container died a81a1958437dbbcb5c25bc7ca9d3722fd9f0f959f9cbdd953ebf73ad54996a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:56.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:56 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:56.386+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:57.118 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T23:18:57.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:18:57.156 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T23:18:57.157 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T23:18:57.157 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.10 2024-09-10T23:18:57.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.056+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:57.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 systemd[1]: Stopping Ceph osd.10 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.382+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.382+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.382+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.382+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.382+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:57.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:57 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:57.383+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:57.948 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.448+0000 7fedbd205640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:18:57.948 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.448+0000 7fedbd205640 -1 osd.10 84 *** Got signal Terminated *** 2024-09-10T23:18:57.948 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:57 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:57.448+0000 7fedbd205640 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:18:58.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:58.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:58 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:58.083+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:58.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:58 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:58.346+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:18:59.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:18:59.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:18:59 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:18:59.060+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:18:59.822 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:18:59.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:18:59.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:18:59.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:18:59.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:18:59.823 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:18:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:18:59.344+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:19:00.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:19:00.452 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:00 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:00.088+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:19:00.572 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:00.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:00.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:00.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:00.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:00.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:00 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:00.297+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:01.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:19:01.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:19:01.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:19:01.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:19:01.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:19:01.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:19:01.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:01 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:01.082+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:19:01.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:01.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:01.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:01.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:01.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:01.574 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:01 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:01.307+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:02.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.754919+0000 front 2024-09-10T23:17:40.754799+0000 (oldest deadline 2024-09-10T23:18:04.854740+0000) 2024-09-10T23:19:02.448 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.655988+0000 front 2024-09-10T23:17:50.656015+0000 (oldest deadline 2024-09-10T23:18:16.556203+0000) 2024-09-10T23:19:02.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:56.556861+0000 front 2024-09-10T23:17:56.556722+0000 (oldest deadline 2024-09-10T23:18:22.456747+0000) 2024-09-10T23:19:02.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.758023+0000 front 2024-09-10T23:18:04.758008+0000 (oldest deadline 2024-09-10T23:18:30.657891+0000) 2024-09-10T23:19:02.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:12.358913+0000 front 2024-09-10T23:18:12.358959+0000 (oldest deadline 2024-09-10T23:18:35.859096+0000) 2024-09-10T23:19:02.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.660462+0000 front 2024-09-10T23:18:21.660523+0000 (oldest deadline 2024-09-10T23:18:47.460914+0000) 2024-09-10T23:19:02.449 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10[59408]: 2024-09-10T23:19:02.058+0000 7fedb981a640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.120:6822 osd.6 since back 2024-09-10T23:18:30.361898+0000 front 2024-09-10T23:18:30.362021+0000 (oldest deadline 2024-09-10T23:18:52.661839+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:02.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:02 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:02.280+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 podman[71953]: 2024-09-10 23:19:02.942084548 +0000 UTC m=+5.588338291 container stop ba4f35ac7d962ab0fdfc0c75a58c60a05abcbacb217114d81115070cf05b88e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:19:03.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:02 smithi153 podman[71953]: 2024-09-10 23:19:02.967861954 +0000 UTC m=+5.614115687 container died ba4f35ac7d962ab0fdfc0c75a58c60a05abcbacb217114d81115070cf05b88e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T23:19:03.500 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:03 smithi153 podman[71953]: 2024-09-10 23:19:03.323604596 +0000 UTC m=+5.969858326 container cleanup ba4f35ac7d962ab0fdfc0c75a58c60a05abcbacb217114d81115070cf05b88e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T23:19:03.500 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:03 smithi153 bash[71953]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10 2024-09-10T23:19:03.572 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:03.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:03 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:03.286+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:03.930 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:03 smithi153 podman[71968]: 2024-09-10 23:19:03.499825994 +0000 UTC m=+0.551720668 container remove ba4f35ac7d962ab0fdfc0c75a58c60a05abcbacb217114d81115070cf05b88e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:03 smithi153 podman[72060]: 2024-09-10 23:19:03.832572474 +0000 UTC m=+0.015921530 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:03 smithi153 podman[72060]: 2024-09-10 23:19:03.952790477 +0000 UTC m=+0.136139526 container create ab00ef6b92468e98eed5ddcf49a8601c861de868c8d613d49f9986e6fcc25ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:19:04.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:04 smithi153 podman[72060]: 2024-09-10 23:19:04.153287477 +0000 UTC m=+0.336636522 container init ab00ef6b92468e98eed5ddcf49a8601c861de868c8d613d49f9986e6fcc25ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:19:04.199 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:04 smithi153 podman[72060]: 2024-09-10 23:19:04.157020488 +0000 UTC m=+0.340369542 container start ab00ef6b92468e98eed5ddcf49a8601c861de868c8d613d49f9986e6fcc25ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:04.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:04 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:04.290+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:04.615 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:04 smithi153 podman[72060]: 2024-09-10 23:19:04.261755518 +0000 UTC m=+0.445104578 container attach ab00ef6b92468e98eed5ddcf49a8601c861de868c8d613d49f9986e6fcc25ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:19:04.616 INFO:journalctl@ceph.osd.10.smithi153.stdout:Sep 10 23:19:04 smithi153 podman[72060]: 2024-09-10 23:19:04.344602078 +0000 UTC m=+0.527951140 container died ab00ef6b92468e98eed5ddcf49a8601c861de868c8d613d49f9986e6fcc25ac8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T23:19:04.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T23:19:04.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:04.950 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T23:19:04.950 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T23:19:04.950 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.11 2024-09-10T23:19:05.252 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 systemd[1]: Stopping Ceph osd.11 for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:05.252 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:05.252 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.251+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.272+0000 7f083afc3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.272+0000 7f083afc3640 -1 osd.11 84 *** Got signal Terminated *** 2024-09-10T23:19:05.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:05 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:05.272+0000 7f083afc3640 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:19:06.474 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:06.474 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:06.474 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:06.474 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:06.474 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:06.475 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:06.475 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:06 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:06.206+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:07.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:07 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:07.239+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:08.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:08 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:08.267+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6820 osd.8 since back 2024-09-10T23:18:43.250999+0000 front 2024-09-10T23:18:43.250970+0000 (oldest deadline 2024-09-10T23:19:07.951076+0000) 2024-09-10T23:19:09.572 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6806 osd.0 since back 2024-09-10T23:17:40.043460+0000 front 2024-09-10T23:17:40.043385+0000 (oldest deadline 2024-09-10T23:18:04.743353+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2024-09-10T23:17:50.044372+0000 front 2024-09-10T23:17:50.044323+0000 (oldest deadline 2024-09-10T23:18:15.344519+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6804 osd.2 since back 2024-09-10T23:17:58.845462+0000 front 2024-09-10T23:17:58.845714+0000 (oldest deadline 2024-09-10T23:18:24.145626+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.120:6814 osd.3 since back 2024-09-10T23:18:04.146135+0000 front 2024-09-10T23:18:04.146157+0000 (oldest deadline 2024-09-10T23:18:29.346621+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2024-09-10T23:18:13.447675+0000 front 2024-09-10T23:18:13.447661+0000 (oldest deadline 2024-09-10T23:18:37.547635+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6812 osd.5 since back 2024-09-10T23:18:21.048524+0000 front 2024-09-10T23:18:21.048576+0000 (oldest deadline 2024-09-10T23:18:44.548608+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2024-09-10T23:18:37.950530+0000 front 2024-09-10T23:18:37.950596+0000 (oldest deadline 2024-09-10T23:19:03.250523+0000) 2024-09-10T23:19:09.573 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:09 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11[59741]: 2024-09-10T23:19:09.276+0000 7f08375d8640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.176:6820 osd.8 since back 2024-09-10T23:18:43.250999+0000 front 2024-09-10T23:18:43.250970+0000 (oldest deadline 2024-09-10T23:19:07.951076+0000) 2024-09-10T23:19:10.936 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:10 smithi176 podman[73556]: 2024-09-10 23:19:10.664277915 +0000 UTC m=+5.506949303 container died f2696778bdd3357cab219863937064e561db168d945f155aafe7239ee2844f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T23:19:11.303 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73556]: 2024-09-10 23:19:11.035380871 +0000 UTC m=+5.878052303 container cleanup f2696778bdd3357cab219863937064e561db168d945f155aafe7239ee2844f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:19:11.303 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 bash[73556]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11 2024-09-10T23:19:11.572 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73585]: 2024-09-10 23:19:11.303218144 +0000 UTC m=+0.637127122 container remove f2696778bdd3357cab219863937064e561db168d945f155aafe7239ee2844f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T23:19:11.948 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73671]: 2024-09-10 23:19:11.514316398 +0000 UTC m=+0.015643491 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:19:11.948 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73671]: 2024-09-10 23:19:11.645747179 +0000 UTC m=+0.147074267 container create 2604d383cc531b1557f6fb81ba13258d3a51682d3acd52bde9630402a6fe424e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T23:19:11.948 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73671]: 2024-09-10 23:19:11.848311766 +0000 UTC m=+0.349638854 container init 2604d383cc531b1557f6fb81ba13258d3a51682d3acd52bde9630402a6fe424e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T23:19:11.948 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73671]: 2024-09-10 23:19:11.851747672 +0000 UTC m=+0.353074761 container start 2604d383cc531b1557f6fb81ba13258d3a51682d3acd52bde9630402a6fe424e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T23:19:11.948 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:11 smithi176 podman[73671]: 2024-09-10 23:19:11.948018303 +0000 UTC m=+0.449345403 container attach 2604d383cc531b1557f6fb81ba13258d3a51682d3acd52bde9630402a6fe424e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:19:12.323 INFO:journalctl@ceph.osd.11.smithi176.stdout:Sep 10 23:19:12 smithi176 podman[73671]: 2024-09-10 23:19:12.046504686 +0000 UTC m=+0.547831767 container died 2604d383cc531b1557f6fb81ba13258d3a51682d3acd52bde9630402a6fe424e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T23:19:12.603 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T23:19:12.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:12.642 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T23:19:12.642 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T23:19:12.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.a 2024-09-10T23:19:12.943 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:12 smithi120 systemd[1]: Stopping Ceph mds.a for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:13.196 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:12 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a[75513]: 2024-09-10T23:19:12.940+0000 7f7294e82640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:13.196 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:12 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a[75513]: 2024-09-10T23:19:12.940+0000 7f7294e82640 -1 mds.a *** got signal Terminated *** 2024-09-10T23:19:17.236 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:16 smithi120 podman[124620]: 2024-09-10 23:19:16.950234977 +0000 UTC m=+4.110436691 container died a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:19:17.601 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:17 smithi120 podman[124620]: 2024-09-10 23:19:17.316802822 +0000 UTC m=+4.477004529 container cleanup a339a6377d0d11eea375b44fe3b0aa719c832edd7a0a75ea444b35696a8c28bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:19:17.601 INFO:journalctl@ceph.mds.a.smithi120.stdout:Sep 10 23:19:17 smithi120 bash[124620]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-a 2024-09-10T23:19:17.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T23:19:17.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:17.784 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T23:19:17.784 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T23:19:17.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.d 2024-09-10T23:19:18.196 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:17 smithi120 systemd[1]: Stopping Ceph mds.d for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:18.196 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d[74332]: 2024-09-10T23:19:18.103+0000 7f875b335640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:18.197 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:18 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d[74332]: 2024-09-10T23:19:18.103+0000 7f875b335640 -1 mds.d *** got signal Terminated *** 2024-09-10T23:19:22.196 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:21 smithi120 podman[124751]: 2024-09-10 23:19:21.834874043 +0000 UTC m=+3.834213129 container died 928e6f5136890a98c211f4afd58b643007d430fd007960f12c2e37e18aca5b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T23:19:22.581 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:22 smithi120 podman[124751]: 2024-09-10 23:19:22.287640155 +0000 UTC m=+4.286979249 container cleanup 928e6f5136890a98c211f4afd58b643007d430fd007960f12c2e37e18aca5b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T23:19:22.581 INFO:journalctl@ceph.mds.d.smithi120.stdout:Sep 10 23:19:22 smithi120 bash[124751]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-d 2024-09-10T23:19:22.717 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T23:19:22.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:22.751 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T23:19:22.751 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T23:19:22.752 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.g 2024-09-10T23:19:23.196 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:22 smithi120 systemd[1]: Stopping Ceph mds.g for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:23.197 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g[75099]: 2024-09-10T23:19:23.075+0000 7fcd22ef3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:23.197 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:23 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g[75099]: 2024-09-10T23:19:23.075+0000 7fcd22ef3640 -1 mds.g *** got signal Terminated *** 2024-09-10T23:19:25.925 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:25 smithi120 podman[124871]: 2024-09-10 23:19:25.479942 +0000 UTC m=+2.508596519 container died 0c133ff96da924ecd8c0783cae8cb0b7b712f6dfeecf9b42cbbc57abed6934b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:19:26.196 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 podman[124871]: 2024-09-10 23:19:26.100454824 +0000 UTC m=+3.129109374 container cleanup 0c133ff96da924ecd8c0783cae8cb0b7b712f6dfeecf9b42cbbc57abed6934b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:19:26.196 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 bash[124871]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g 2024-09-10T23:19:26.576 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T23:19:26.598 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 podman[124885]: 2024-09-10 23:19:26.327267126 +0000 UTC m=+0.838630442 container remove 0c133ff96da924ecd8c0783cae8cb0b7b712f6dfeecf9b42cbbc57abed6934b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T23:19:26.598 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 systemd[1]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.g.service: Deactivated successfully. 2024-09-10T23:19:26.598 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 systemd[1]: Stopped Ceph mds.g for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:19:26.598 INFO:journalctl@ceph.mds.g.smithi120.stdout:Sep 10 23:19:26 smithi120 systemd[1]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.g.service: Consumed 1.124s CPU time. 2024-09-10T23:19:26.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:26.609 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T23:19:26.609 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T23:19:26.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.j 2024-09-10T23:19:26.930 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:26 smithi120 systemd[1]: Stopping Ceph mds.j for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:27.196 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j[73916]: 2024-09-10T23:19:26.929+0000 7fd6cfe61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:27.196 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:26 smithi120 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j[73916]: 2024-09-10T23:19:26.929+0000 7fd6cfe61640 -1 mds.j *** got signal Terminated *** 2024-09-10T23:19:30.621 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:30 smithi120 podman[124993]: 2024-09-10 23:19:30.351245976 +0000 UTC m=+3.521630252 container died 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T23:19:30.946 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:30 smithi120 podman[124993]: 2024-09-10 23:19:30.804582898 +0000 UTC m=+3.974967173 container cleanup 121daa7bfd4d0c02c8756802ac30381e1917abad0b3c1563310c2d378d35c169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:19:30.946 INFO:journalctl@ceph.mds.j.smithi120.stdout:Sep 10 23:19:30 smithi120 bash[124993]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-j 2024-09-10T23:19:31.234 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T23:19:31.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:31.267 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T23:19:31.267 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T23:19:31.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.b 2024-09-10T23:19:31.574 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:31 smithi153 systemd[1]: Stopping Ceph mds.b for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:31.948 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b[65047]: 2024-09-10T23:19:31.573+0000 7eff8d75c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:31.948 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:31 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b[65047]: 2024-09-10T23:19:31.573+0000 7eff8d75c640 -1 mds.b *** got signal Terminated *** 2024-09-10T23:19:35.866 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:35 smithi153 podman[72163]: 2024-09-10 23:19:35.586183321 +0000 UTC m=+4.125552077 container died fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T23:19:36.198 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:35 smithi153 podman[72163]: 2024-09-10 23:19:35.939715296 +0000 UTC m=+4.479084061 container cleanup fc0eed164bbfdd37467b2d49022abf42af5dc489029f83b755b96bbd45e1766d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T23:19:36.198 INFO:journalctl@ceph.mds.b.smithi153.stdout:Sep 10 23:19:35 smithi153 bash[72163]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-b 2024-09-10T23:19:36.365 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T23:19:36.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:36.402 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T23:19:36.402 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T23:19:36.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.e 2024-09-10T23:19:36.948 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:36 smithi153 systemd[1]: Stopping Ceph mds.e for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:36.948 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e[64442]: 2024-09-10T23:19:36.718+0000 7f3e1932f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:36.948 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:36 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e[64442]: 2024-09-10T23:19:36.718+0000 7f3e1932f640 -1 mds.e *** got signal Terminated *** 2024-09-10T23:19:40.695 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:40 smithi153 podman[72297]: 2024-09-10 23:19:40.321558198 +0000 UTC m=+3.704869079 container died 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T23:19:40.948 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:40 smithi153 podman[72297]: 2024-09-10 23:19:40.776288987 +0000 UTC m=+4.159599867 container cleanup 3573f74b2e939a4c995ed084fef05b3670d762855d63a2628afa9b39e1d37c2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T23:19:40.949 INFO:journalctl@ceph.mds.e.smithi153.stdout:Sep 10 23:19:40 smithi153 bash[72297]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-e 2024-09-10T23:19:41.259 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T23:19:41.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:41.293 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T23:19:41.293 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T23:19:41.293 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.h 2024-09-10T23:19:41.623 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:41 smithi153 systemd[1]: Stopping Ceph mds.h for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:41.948 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h[64747]: 2024-09-10T23:19:41.622+0000 7f670e646640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:41.948 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:41 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h[64747]: 2024-09-10T23:19:41.622+0000 7f670e646640 -1 mds.h *** got signal Terminated *** 2024-09-10T23:19:43.939 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:43 smithi153 podman[72418]: 2024-09-10 23:19:43.658178797 +0000 UTC m=+2.151144955 container died 862be588a002a0ac3e91ca1a1086e6261381e7dd31541bd39e47c5edbb47e49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T23:19:44.198 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:44 smithi153 podman[72418]: 2024-09-10 23:19:44.03060798 +0000 UTC m=+2.523574137 container cleanup 862be588a002a0ac3e91ca1a1086e6261381e7dd31541bd39e47c5edbb47e49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T23:19:44.198 INFO:journalctl@ceph.mds.h.smithi153.stdout:Sep 10 23:19:44 smithi153 bash[72418]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-h 2024-09-10T23:19:44.450 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T23:19:44.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:44.483 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T23:19:44.483 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T23:19:44.483 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.k 2024-09-10T23:19:44.948 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:44 smithi153 systemd[1]: Stopping Ceph mds.k for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:44.948 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k[64132]: 2024-09-10T23:19:44.775+0000 7f41820d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:44.949 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:44 smithi153 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k[64132]: 2024-09-10T23:19:44.775+0000 7f41820d5640 -1 mds.k *** got signal Terminated *** 2024-09-10T23:19:47.785 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:47 smithi153 podman[72539]: 2024-09-10 23:19:47.464333359 +0000 UTC m=+2.762655114 container died 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:19:48.085 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:47 smithi153 podman[72539]: 2024-09-10 23:19:47.859288805 +0000 UTC m=+3.157610566 container cleanup 11c10d3cdf2ce05959f131d98e5adc0533618a1292ad63aeadd2e45ab57da10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-10T23:19:48.085 INFO:journalctl@ceph.mds.k.smithi153.stdout:Sep 10 23:19:47 smithi153 bash[72539]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-k 2024-09-10T23:19:48.206 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T23:19:48.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:48.238 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T23:19:48.238 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T23:19:48.238 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.c 2024-09-10T23:19:48.542 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:48 smithi176 systemd[1]: Stopping Ceph mds.c for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:48.807 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: 2024-09-10T23:19:48.540+0000 7ff07dd95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:48.807 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:48 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c[64837]: 2024-09-10T23:19:48.540+0000 7ff07dd95640 -1 mds.c *** got signal Terminated *** 2024-09-10T23:19:50.459 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:50 smithi176 podman[73775]: 2024-09-10 23:19:50.170908562 +0000 UTC m=+1.732787231 container died ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:19:50.818 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:50 smithi176 podman[73775]: 2024-09-10 23:19:50.566665615 +0000 UTC m=+2.128544268 container cleanup ec03888c56e8af7ed0bcb47f8b258a656ddb6d9a9ce85ace58b1318a7757e73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T23:19:50.818 INFO:journalctl@ceph.mds.c.smithi176.stdout:Sep 10 23:19:50 smithi176 bash[73775]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-c 2024-09-10T23:19:50.941 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T23:19:50.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:50.977 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T23:19:50.977 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T23:19:50.978 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.f 2024-09-10T23:19:51.072 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:51 smithi176 systemd[1]: Stopping Ceph mds.f for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:51.572 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f[64531]: 2024-09-10T23:19:51.279+0000 7fd82571a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:51.572 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:51 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f[64531]: 2024-09-10T23:19:51.279+0000 7fd82571a640 -1 mds.f *** got signal Terminated *** 2024-09-10T23:19:54.996 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:54 smithi176 podman[73908]: 2024-09-10 23:19:54.630839227 +0000 UTC m=+3.442063305 container died a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:19:55.305 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 podman[73908]: 2024-09-10 23:19:55.10369631 +0000 UTC m=+3.914920344 container cleanup a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T23:19:55.305 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 bash[73908]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f 2024-09-10T23:19:55.533 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T23:19:55.556 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 podman[73922]: 2024-09-10 23:19:55.30488969 +0000 UTC m=+0.626037295 container remove a0730dd0bf86c2869c152b1185df491c8df269f3dfadae269f7d08719bd8c781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T23:19:55.556 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 systemd[1]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.f.service: Deactivated successfully. 2024-09-10T23:19:55.556 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 systemd[1]: Stopped Ceph mds.f for d2838936-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:19:55.556 INFO:journalctl@ceph.mds.f.smithi176.stdout:Sep 10 23:19:55 smithi176 systemd[1]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.f.service: Consumed 1.114s CPU time. 2024-09-10T23:19:55.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:55.566 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T23:19:55.566 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T23:19:55.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.i 2024-09-10T23:19:55.823 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:55 smithi176 systemd[1]: Stopping Ceph mds.i for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:56.322 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: 2024-09-10T23:19:55.940+0000 7fa3db5b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:56.322 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:55 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i[65138]: 2024-09-10T23:19:55.940+0000 7fa3db5b8640 -1 mds.i *** got signal Terminated *** 2024-09-10T23:19:58.516 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:58 smithi176 podman[74030]: 2024-09-10 23:19:58.163058215 +0000 UTC m=+2.384354765 container died b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T23:19:58.822 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:58 smithi176 podman[74030]: 2024-09-10 23:19:58.615319969 +0000 UTC m=+2.836616514 container cleanup b5fbd3613a2fb50970e9c63ddba129d8f75049c9acc59f87acfbfcea161ea997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:19:58.822 INFO:journalctl@ceph.mds.i.smithi176.stdout:Sep 10 23:19:58 smithi176 bash[74030]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-i 2024-09-10T23:19:59.038 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T23:19:59.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:19:59.071 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T23:19:59.071 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T23:19:59.071 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.l 2024-09-10T23:19:59.272 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:19:59 smithi176 systemd[1]: Stopping Ceph mds.l for d2838936-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:19:59.572 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:19:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l[64231]: 2024-09-10T23:19:59.411+0000 7f0f244ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:19:59.573 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:19:59 smithi176 ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l[64231]: 2024-09-10T23:19:59.411+0000 7f0f244ed640 -1 mds.l *** got signal Terminated *** 2024-09-10T23:20:01.300 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:20:00 smithi176 podman[74151]: 2024-09-10 23:20:00.901860102 +0000 UTC m=+1.595544269 container died c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:20:02.028 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:20:01 smithi176 podman[74151]: 2024-09-10 23:20:01.644906312 +0000 UTC m=+2.338590521 container cleanup c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T23:20:02.028 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:20:01 smithi176 bash[74151]: ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l 2024-09-10T23:20:02.028 INFO:journalctl@ceph.mds.l.smithi176.stdout:Sep 10 23:20:01 smithi176 podman[74162]: 2024-09-10 23:20:01.837365595 +0000 UTC m=+0.734731553 container remove c6f065d6ac54558b2be30efeacb9bfcef11165d49fb729b57217167f6b466aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-d2838936-6fc5-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T23:20:02.045 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d2838936-6fc5-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T23:20:02.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:20:02.079 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T23:20:02.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:20:02.285 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:20:04.881 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:20:05.087 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:20:07.586 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:20:07.796 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:20:10.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:20:10.264 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:20:10.297 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:20:10.323 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T23:20:10.325 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi120/crash 2024-09-10T23:20:10.326 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:20:10.457 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:20:10.457 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:20:10.459 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi153/crash 2024-09-10T23:20:10.460 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:20:10.578 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:20:10.578 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:20:10.580 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi176/crash 2024-09-10T23:20:10.581 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:20:10.624 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:20:10.624 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:20:10.625 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T23:20:10.625 DEBUG:teuthology.orchestra.run.smithi120:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T23:20:10.665 INFO:tasks.cephadm:Compressing logs... 2024-09-10T23:20:10.666 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:20:10.708 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:20:10.711 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:20:10.734 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T23:20:10.734 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-09-10T23:20:10.735 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T23:20:10.736 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:20:10.736 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:20:10.736 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T23:20:10.736 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:20:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T23:20:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T23:20:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log: 83.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:20:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T23:20:10.738 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T23:20:10.740 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T23:20:10.741 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:20:10.743 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T23:20:10.743 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:20:10.743 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:20:10.745 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:20:10.745 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T23:20:10.747 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:20:10.747 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:20:10.747 INFO:teuthology.orchestra.run.smithi120.stderr: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T23:20:10.748 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T23:20:10.749 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T23:20:10.749 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T23:20:10.750 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T23:20:10.751 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log: 2024-09-10T23:20:10.751 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log: 95.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T23:20:10.752 INFO:teuthology.orchestra.run.smithi153.stderr: 95.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T23:20:10.753 INFO:teuthology.orchestra.run.smithi120.stderr: 87.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:20:10.755 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T23:20:10.755 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:20:10.755 INFO:teuthology.orchestra.run.smithi153.stderr: 91.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:20:10.756 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T23:20:10.756 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T23:20:10.761 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T23:20:10.762 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:20:10.763 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T23:20:10.765 INFO:teuthology.orchestra.run.smithi153.stderr: 93.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log: 87.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:20:10.766 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/cephadm.log: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T23:20:10.769 INFO:teuthology.orchestra.run.smithi120.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:20:10.770 INFO:teuthology.orchestra.run.smithi153.stderr: 95.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T23:20:10.771 INFO:teuthology.orchestra.run.smithi153.stderr: 95.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T23:20:10.773 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/cephadm.log: 91.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:20:10.794 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 83.4% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:20:10.843 INFO:teuthology.orchestra.run.smithi176.stderr: 93.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T23:20:10.885 INFO:teuthology.orchestra.run.smithi176.stderr: 90.5% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:20:10.936 INFO:teuthology.orchestra.run.smithi176.stderr: 87.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:20:10.972 INFO:teuthology.orchestra.run.smithi176.stderr: 91.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:20:11.030 INFO:teuthology.orchestra.run.smithi120.stderr: 90.6% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T23:20:11.147 INFO:teuthology.orchestra.run.smithi176.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:20:11.151 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T23:20:11.202 INFO:teuthology.orchestra.run.smithi153.stderr: 91.7% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T23:20:11.398 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T23:20:15.449 INFO:teuthology.orchestra.run.smithi120.stderr: 92.5% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T23:20:27.825 INFO:teuthology.orchestra.run.smithi153.stderr: 94.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T23:20:28.878 INFO:teuthology.orchestra.run.smithi120.stderr: 94.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T23:20:28.905 INFO:teuthology.orchestra.run.smithi153.stderr: 93.9% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T23:20:31.116 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T23:20:32.000 INFO:teuthology.orchestra.run.smithi120.stderr: 94.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T23:20:32.348 INFO:teuthology.orchestra.run.smithi153.stderr: 94.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T23:20:33.765 INFO:teuthology.orchestra.run.smithi176.stderr: 91.6% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T23:20:34.755 INFO:teuthology.orchestra.run.smithi153.stderr: 94.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T23:20:34.757 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-09-10T23:20:34.757 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m24.031s 2024-09-10T23:20:34.757 INFO:teuthology.orchestra.run.smithi153.stderr:user 1m18.231s 2024-09-10T23:20:34.758 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m2.961s 2024-09-10T23:20:34.809 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T23:20:34.811 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-09-10T23:20:34.811 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m24.088s 2024-09-10T23:20:34.811 INFO:teuthology.orchestra.run.smithi120.stderr:user 1m25.492s 2024-09-10T23:20:34.811 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m3.271s 2024-09-10T23:23:37.036 INFO:teuthology.orchestra.run.smithi176.stderr: 94.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T23:25:19.376 INFO:teuthology.orchestra.run.smithi176.stderr: 94.0% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T23:25:50.169 INFO:teuthology.orchestra.run.smithi176.stderr: 94.2% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T23:26:04.311 INFO:teuthology.orchestra.run.smithi176.stderr: 94.3% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T23:27:05.878 INFO:teuthology.orchestra.run.smithi176.stderr: 94.5% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T23:40:32.648 INFO:teuthology.orchestra.run.smithi176.stderr: 93.1% -- replaced with /var/log/ceph/d2838936-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T23:40:32.757 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-10T23:40:32.757 INFO:teuthology.orchestra.run.smithi176.stderr:real 20m22.031s 2024-09-10T23:40:32.757 INFO:teuthology.orchestra.run.smithi176.stderr:user 14m43.022s 2024-09-10T23:40:32.757 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m35.284s 2024-09-10T23:40:32.758 INFO:tasks.cephadm:Archiving logs... 2024-09-10T23:40:32.759 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi120/log 2024-09-10T23:40:32.760 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:40:44.932 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi153/log 2024-09-10T23:40:44.933 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:40:56.642 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi176/log 2024-09-10T23:40:56.642 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:43:14.818 INFO:tasks.cephadm:Removing cluster... 2024-09-10T23:43:14.834 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:43:15.037 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:43:15.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:43:15.602 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:43:15.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d2838936-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:43:16.743 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: d2838936-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:43:17.734 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T23:43:17.734 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:43:17.754 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:43:17.774 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:43:17.790 INFO:tasks.cephadm:Teardown complete 2024-09-10T23:43:17.790 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T23:43:17.803 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T23:43:17.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:43:17.806 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:43:17.817 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:43:17.978 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T23:43:17.988 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T23:43:17.988 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:43:17.992 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:43:17.994 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:43:18.006 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:43:18.009 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:43:18.009 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -325us[ -328us] +/- 50ms 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +356us[ +354us] +/- 80ms 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +148us[ +145us] +/- 64ms 2024-09-10T23:43:18.012 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -329us[ -301us] +/- 50ms 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 +334us[ +334us] +/- 80ms 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +129us[ +129us] +/- 64ms 2024-09-10T23:43:18.014 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:43:18.084 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:43:18.084 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-09-10T23:43:18.084 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -261us[ -261us] +/- 50ms 2024-09-10T23:43:18.085 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 +376us[ +376us] +/- 80ms 2024-09-10T23:43:18.085 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +161us[ +161us] +/- 64ms 2024-09-10T23:43:18.085 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:43:18.098 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T23:43:18.108 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T23:43:18.108 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T23:43:18.118 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:43:18.208 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:43:18.284 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:43:18.380 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:43:18.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:18.418 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:43:18.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:18.455 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:43:18.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:18.495 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T23:43:19.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T23:43:19.616 INFO:teuthology.task.internal:Duration was 4070.865153 seconds 2024-09-10T23:43:19.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T23:43:19.700 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T23:43:19.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:43:19.703 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:43:19.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:43:19.747 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:43:19.750 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:43:19.767 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:43:20.240 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T23:43:20.240 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-09-10T23:43:20.240 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:43:20.263 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-09-10T23:43:20.263 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:43:20.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-09-10T23:43:20.287 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:43:20.323 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T23:43:20.323 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:43:20.326 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:43:20.329 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:43:20.395 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T23:43:20.395 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:43:20.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:43:20.401 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:43:21.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T23:43:21.996 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T23:43:21.996 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:43:22.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:43:22.052 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:43:22.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T23:43:22.145 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.150 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.168 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-09-10T23:43:22.175 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-09-10T23:43:22.189 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-09-10T23:43:22.208 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:22.236 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:22.256 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:43:22.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:43:22.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T23:43:22.285 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T23:43:22.286 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi120 2024-09-10T23:43:22.286 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:43:22.353 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi153 2024-09-10T23:43:22.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:43:22.397 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899745/remote/smithi176 2024-09-10T23:43:22.397 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:43:22.438 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T23:43:22.438 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:43:22.441 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:43:22.443 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:43:22.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T23:43:22.503 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T23:43:22.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T23:43:22.514 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T23:43:22.514 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:43:22.516 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:43:22.519 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:43:22.529 INFO:teuthology.orchestra.run.smithi120.stdout: 266030 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:43 /home/ubuntu/cephtest 2024-09-10T23:43:22.533 INFO:teuthology.orchestra.run.smithi153.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:43 /home/ubuntu/cephtest 2024-09-10T23:43:22.547 INFO:teuthology.orchestra.run.smithi176.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:43 /home/ubuntu/cephtest 2024-09-10T23:43:22.548 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T23:43:22.559 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T23:43:22.585 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 4070.865152835846 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T23:43:22.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:43:22.745 INFO:teuthology.run:pass